Issue
I have an image that requires root privilege to start.
Now I'm trying to deploy it on OpenShift.
this is the deployment yaml I used to deploy it
apiVersion: apps/v1
kind: Deployment
metadata:
name: xyz
annotations:
k8s.v1.cni.cncf.io/networks: macvlan-conf
spec:
selector:
matchLabels:
name: xyz
template:
metadata:
labels:
name: xyz
spec:
containers:
- name: xyz
image: 172.30.1.1:5000/myproject/xyz@sha256:bf3d219941ec0de7f52f6babbca23e03cc2611d327552b08f530ead9ec627ec2
imagePullPolicy: Always
securityContext:
capabilities:
add:
- ALL
privileged: false
allowPrivilegeEscalation: false
runAsUser: 0
serviceAccount: runasanyuid
serviceAccountName: runasanyuid
hostNetwork: true
resources:
limits:
memory: "12000Mi"
requests:
memory: "6000Mi"
ports:
- containerPort: 2102
command:
- /usr/sbin/sshd -D
please note that I already created a SCC called 'scc-admin' to run the pods in the project I'm working on with any UID, as I know that OpenShift doesn't allow pods to start with root privilege by default.
kind: SecurityContextConstraints
apiVersion: v1
metadata:
name: scc-admin
allowPrivilegedContainer: true
runAsUser:
type: RunAsAny
seLinuxContext:
type: RunAsAny
fsGroup:
type: RunAsAny
supplementalGroups:
type: RunAsAny
users:
- developer
groups:
- developer
that's what I found on the internet as a solution for my issue, but I guess it didn't work as well :(
[root@centos72_base ~]# oc get scc
NAME PRIV CAPS SELINUX RUNASUSER FSGROUP SUPGROUP PRIORITY READONLYROOTFS VOLUMES
anyuid true [] MustRunAs RunAsAny RunAsAny RunAsAny 10 false [configMap downwardAPI emptyDir hostPath persistentVolumeClaim projected secret]
hostaccess false [] MustRunAs MustRunAsRange MustRunAs RunAsAny <none> false [configMap downwardAPI emptyDir hostPath persistentVolumeClaim projected secret]
hostmount-anyuid false [] MustRunAs RunAsAny RunAsAny RunAsAny <none> false [configMap downwardAPI emptyDir hostPath nfs persistentVolumeClaim projected secret]
hostnetwork false [] MustRunAs MustRunAsRange MustRunAs MustRunAs <none> false [configMap downwardAPI emptyDir persistentVolumeClaim projected secret]
nonroot false [] MustRunAs MustRunAsNonRoot RunAsAny RunAsAny <none> false [configMap downwardAPI emptyDir persistentVolumeClaim projected secret]
privileged true [*] RunAsAny RunAsAny RunAsAny RunAsAny <none> false [*]
restricted false [] MustRunAs MustRunAsRange MustRunAs RunAsAny <none> false [configMap downwardAPI emptyDir persistentVolumeClaim projected secret]
scc-admin true [] RunAsAny RunAsAny RunAsAny RunAsAny <none> false [awsElasticBlockStore azureDisk azureFile cephFS cinder configMap downwardAPI emptyDir fc flexVolume flocker gcePersistentDisk gitRepo glusterfs iscsi nfs persistentVolumeClaim photonPersistentDisk portworxVolume projected quobyte rbd scaleIO secret storageOS vsphere]
[root@centos72_base ~]#
please also note that this image works fine with docker using the below command
docker run -d --network host --privileged --cap-add=ALL --security-opt seccomp=unconfined --name xyz 172.30.1.1:5000/myproject/xyz /usr/sbin/sshd -D
[root@centos72_base ~]# docker ps | grep xyz
793e339ff732 172.30.1.1:5000/myproject/xyz "/usr/sbin/sshd -D" About a minute ago Up About a minute xyz
and on OpenShift i get these errors with the deployment file I provided above
Error creating: pods "xyz-7966f58588-" is forbidden: unable to validate against any security context constraint: [spec.containers[0].securityContext.securityContext.runAsUser: Invalid value: 0: must be in the ranges: [1000140000, 1000149999] capabilities.add: Invalid value: "ALL": capability may not be added]
which means that i have to remove
capabilities:
add:
- ALL
and
runAsUser: 0
to start the pod
and when I remove them from the yaml file, I get a crash loopback error from the pod
so can anyone please help me with that
Solution
The SCC you created currently applies to the User developer, and the group developer.
Your deployment uses a ServiceAccount runasanyuid.
You need to edit your SecurityContextConstraint, allowing that ServiceAccount:
[...]
users:
- developer
- system:serviceaccount:<namespace-for-your-deployment>:runasanyuid
groups:
- developer
As a side note, instead of creating your own SCC, if you just need to run a privileged container, OpenShift ships with the anyuid one that you could have re-used - just adding your SA into its users list.
While the best practice would be to avoid privileged containers. Unless there is a good reason, you should not run processes as root - which has been true, long before OCI.
If your application needs to write stuff some place, you could use some emptyDir volumes. If your application tries to bind on privileged ports, you should be able to reconfigure it. If your application complains about not being able to resolve an username for its UID, you could look into nsswrapper. Granting privileges with SecurityContextConstraints or PodSecurityPolicies isn't the norm, they're meant to be exceptions, carefully thought.
If your Pod crashes when running as non-root, try to check its logs. If that doesn't help, try to oc debug -n <namespace> pod/<podname>, which should start a new Pod, opening a shell into it, where you would be able to execute its entrypoint yourself, maybe try another set of options, slightly change your configuration, ... until you would get it right. You could even try from your workstation, docker run --user=12435679 xxx: pass it a random UID and see how your container/application deals with it.
Answered By - SYN Answer Checked By - Mary Flores (WPSolving Volunteer)