Kubernetes Plugin: Option to run pod as different user (e.g. as root)

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Kubernetes Plugin: Option to run pod as different user (e.g. as root)

Torsten Gippert

Hello everybody,


this question is about implementing an option to run a build pod (using the kubernetes plugin from Carlos Sanchez) as a different user.

@Carlos: Great work, we love the ability to define our own pod templates!


Software versions:

  • Jenkins version: 2.89.2

  • Kubernetes Plugin version: 1.1.2

  • Durable Task Plugin Version: 1.17


First, let's assume the following scenario:

One JNLP container and 2 containers with different images in a podTemplate.


  • jnlp container - image jenkins/jnlp-slave:alpine - running as user 'jenkins' with uid 10000 (ten thousand)

  • node container - image node:9.3-alpine - running as root user with uid 0

  • gradle container - image gradle:4.4-alpine - running as user 'gradle' with uid 1000 (one thousand).


podTemplate from Jenkinsfile:

podTemplate(label: nodeLabel, containers: [

  containerTemplate(name: 'node', image: 'node:9.3-alpine', ttyEnabled: true, command: 'cat'),

  containerTemplate(name: 'gradle', image: 'gradle:4.4-alpine', ttyEnabled: true, command: 'cat')

])


Switching into container 'gradle' is not possible due to permission denied errors (seems to be caused by the durable task plugin, but I am not sure about that).
Root cause seems to be that the jnlp container runs with uid 10000 (yes, ten thousand - the 'old' image jenkins/jnlp-slave:2.62 ran with uid 1000) and therefore the working directory has file/directory permissions that prevent other users except uid 10000 and root (uid 0) from accessing the working directory.



I also added a pod yaml file (see attachment pod-permission-denied.yaml) that shows the "problem":
1. You can run that pod (kubectl apply -f pod-permission-denied.yaml) and
2. start a shell in the different containers (kubectl exec -ti -c [jnlp|node|gradle] permission-denied sh) to

3. run some commands like ‘id’ or ‘whoami’ within a container to show you the different user-ids the containers are running with.


See attachments

  • Jenkinsfile-permission-denied.groovy   (contains Jenkinsfile to reproduce the “problem”)

  • Jenkinsfile-permission-denied_output.txt   (contains the stdout of the Jenkins build job)

  • pod-permission-denied.yaml (simplified k8s pod declaration)


Suggested solution:

Add options to set 'runAsUser' and 'fsGroup' at podTemplate level to be able to get rid of permission problems.


'runAsUser' and 'fsGroup' stand for the user id (uid) and match the options from podSecurityContext at pod level in kubernetes pod specification (see https://kubernetes.io/docs/tasks/configure-pod-container/security-context/)


suggested podTemplate from Jenkinsfile:

podTemplate(label: nodeLabel, runAsUser: 0, fsGroup: 0,

containers: [

  containerTemplate(name: 'node', image: 'node:9.3-alpine', ttyEnabled: true, command: 'cat'),

  containerTemplate(name: 'gradle', image: 'gradle:4.4-alpine', ttyEnabled: true, command: 'cat')

])






The pod ‘run-as-root’ (see attachment pod-run-as-root.yaml) is a minimal modified version of the pod ‘permission-denied’ (see above) with a different pod security context:


securityContext:

  runAsUser: 0

  fsGroup: 0


1. You can run that pod (kubectl apply -f pod-run-as-root.yaml) and
2. start a shell in the different containers (kubectl exec -ti -c [jnlp|node|gradle] run-as-root sh) to

3. run some commands like ‘id’ or ‘whoami’ within a container to show you that each container is running as root now.

Therefore no permission denied problems
should occur if this will be implemented in the kubernetes plugin.

See attachments

  • pod-run-as-root.yaml (simplified k8s pod declaration with podSecurityContext set to root user)


What do you think? Could this be a proper way to “override” the the container specific default users?


--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/e06a9c7b-79bd-43cc-9428-498432f0fcfd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Jenkinsfile-permission-denied.groovy (1K) Download Attachment
Jenkinsfile-permission-denied_output.txt (3K) Download Attachment
pod-permission-denied.yaml (2K) Download Attachment
pod-run-as-root.yaml (2K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Kubernetes Plugin: Option to run pod as different user (e.g. as root)

Carlos Sanchez
Yes runAsUser is what you need but it's not yet supported as a field in the plugin

This will hopefully be supported soon, working on a way to directly import Pod yaml




On Tue, Jan 9, 2018 at 4:20 PM, Torsten Gippert <[hidden email]> wrote:

Hello everybody,


this question is about implementing an option to run a build pod (using the kubernetes plugin from Carlos Sanchez) as a different user.

@Carlos: Great work, we love the ability to define our own pod templates!


Software versions:

  • Jenkins version: 2.89.2

  • Kubernetes Plugin version: 1.1.2

  • Durable Task Plugin Version: 1.17


First, let's assume the following scenario:

One JNLP container and 2 containers with different images in a podTemplate.


  • jnlp container - image jenkins/jnlp-slave:alpine - running as user 'jenkins' with uid 10000 (ten thousand)

  • node container - image node:9.3-alpine - running as root user with uid 0

  • gradle container - image gradle:4.4-alpine - running as user 'gradle' with uid 1000 (one thousand).


podTemplate from Jenkinsfile:

podTemplate(label: nodeLabel, containers: [

  containerTemplate(name: 'node', image: 'node:9.3-alpine', ttyEnabled: true, command: 'cat'),

  containerTemplate(name: 'gradle', image: 'gradle:4.4-alpine', ttyEnabled: true, command: 'cat')

])


Switching into container 'gradle' is not possible due to permission denied errors (seems to be caused by the durable task plugin, but I am not sure about that).
Root cause seems to be that the jnlp container runs with uid 10000 (yes, ten thousand - the 'old' image jenkins/jnlp-slave:2.62 ran with uid 1000) and therefore the working directory has file/directory permissions that prevent other users except uid 10000 and root (uid 0) from accessing the working directory.



I also added a pod yaml file (see attachment pod-permission-denied.yaml) that shows the "problem":
1. You can run that pod (kubectl apply -f pod-permission-denied.yaml) and
2. start a shell in the different containers (kubectl exec -ti -c [jnlp|node|gradle] permission-denied sh) to

3. run some commands like ‘id’ or ‘whoami’ within a container to show you the different user-ids the containers are running with.


See attachments

  • Jenkinsfile-permission-denied.groovy   (contains Jenkinsfile to reproduce the “problem”)

  • Jenkinsfile-permission-denied_output.txt   (contains the stdout of the Jenkins build job)

  • pod-permission-denied.yaml (simplified k8s pod declaration)


Suggested solution:

Add options to set 'runAsUser' and 'fsGroup' at podTemplate level to be able to get rid of permission problems.


'runAsUser' and 'fsGroup' stand for the user id (uid) and match the options from podSecurityContext at pod level in kubernetes pod specification (see https://kubernetes.io/docs/tasks/configure-pod-container/security-context/)


suggested podTemplate from Jenkinsfile:

podTemplate(label: nodeLabel, runAsUser: 0, fsGroup: 0,

containers: [

  containerTemplate(name: 'node', image: 'node:9.3-alpine', ttyEnabled: true, command: 'cat'),

  containerTemplate(name: 'gradle', image: 'gradle:4.4-alpine', ttyEnabled: true, command: 'cat')

])






The pod ‘run-as-root’ (see attachment pod-run-as-root.yaml) is a minimal modified version of the pod ‘permission-denied’ (see above) with a different pod security context:


securityContext:

  runAsUser: 0

  fsGroup: 0


1. You can run that pod (kubectl apply -f pod-run-as-root.yaml) and
2. start a shell in the different containers (kubectl exec -ti -c [jnlp|node|gradle] run-as-root sh) to

3. run some commands like ‘id’ or ‘whoami’ within a container to show you that each container is running as root now.

Therefore no permission denied problems
should occur if this will be implemented in the kubernetes plugin.

See attachments

  • pod-run-as-root.yaml (simplified k8s pod declaration with podSecurityContext set to root user)


What do you think? Could this be a proper way to “override” the the container specific default users?


--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/e06a9c7b-79bd-43cc-9428-498432f0fcfd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CALHFn6OfDpOv_qjwuP6JtGTPWpdXvSuLUkkh5MZ4Pxyeke-2mA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.