Kubernetes: how to set VolumeMount user group and file permissions

I'm running a Kubernetes cluster on AWS using kops. I've mounted an EBS volume onto a container and it is visible from my application but it's read only because my application does not run as root. How can I mount a PersistentVolumeClaim as a user other than root? The VolumeMount does not seem to have any options to control the user, group or file permissions of the mounted path.

Here is my Deployment yaml file:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: notebook-1
spec:
replicas: 1
template:
metadata:
labels:
app: notebook-1
spec:
volumes:
- name: notebook-1
persistentVolumeClaim:
claimName: notebook-1
containers:
- name: notebook-1
image: jupyter/base-notebook
ports:
- containerPort: 8888
volumeMounts:
- mountPath: "/home/jovyan/work"
name: notebook-1
195151 次浏览
小开
最佳答案

Pod Security Context 支持设置 fsGroup，它允许您设置拥有卷的组 ID，以及谁可以写入卷。文档中的例子:

apiVersion: v1
kind: Pod
metadata:
name: hello-world
spec:
containers:
# specification of the pod's containers
# ...
securityContext:
fsGroup: 1234

这里是 给你

小开

对于 k8s 版本1.10 + ，增加了 runAsGroup，它类似于 fsGroup，但工作原理不同。

这里可以跟踪实现: https://github.com/kubernetes/features/issues/213

小开

最后，我使用与主容器具有相同 volumeMount的 initContainer来为自定义 Grafana 图像设置适当的权限。

当吊舱中的容器作为 root以外的用户运行并且需要对挂载的卷具有写权限时，这是必需的。

initContainers:
- name: take-data-dir-ownership
image: alpine:3
# Give `grafana` user (id 472) permissions a mounted volume
# https://github.com/grafana/grafana-docker/blob/master/Dockerfile
command:
- chown
- -R
- 472:472
- /var/lib/grafana
volumeMounts:
- name: data
mountPath: /var/lib/grafana

更新: 注意，在没有 -R(递归)标志的情况下运行 chown可能就足够了，因为无论 pod 重新启动与否，权限通常都会保持在卷本身内。如果卷中有大量的文件，这将是可取的，因为处理所有这些文件将需要时间(取决于为 initContainer设置的 resources限制)。

更新2: 在 Kubernetes v1.23中，ABC0和 securityContext.fsGroupChangePolicy的特性进入了 GA/稳定状态。更多信息请参见 另一个答案

为 Pods 配置卷权限和所有权更改策略的特性在1.23中移到了 GA。这允许用户跳过对挂载的递归权限更改，并加快吊舱启动时间。

小开

当您必须以非 root 用户的身份在容器中运行流程时，这是 Kubernetes Deployments/StatuseSet 面临的挑战之一。但是，当您挂载一个卷到一个吊舱，它总是得到安装的权限 root:root。

因此，非 root 用户必须能够访问它想要读写数据的文件夹。

Please follow the below steps for the same.

  1. 在 Dockerfile 创建用户组并分配组 ID。
  2. Create user with user ID and add to the group in Dockerfile.
  3. change ownership recursively for the folders the user process wants to read/write.

  4. 在 pod spec上下文中添加 Deployment/StatuseSet 中的以下行。

    spec:
securityContext:
runAsUser: 1099
runAsGroup: 1099
fsGroup: 1099

runAsUser

Specifies that for any Containers in the Pod, all processes run with user ID 1099.

runAsGroup

Specifies the primary group ID of 1099 for all processes within any containers of the Pod.

If this field is omitted, the primary group ID of the containers will be root(0).

Any files created will also be owned by user 1099 and group 1099 when runAsGroup is specified.

fsGroup

Specifies the owner of any volume attached will be owner by group ID 1099.

Any files created under it will be having permission of nonrootgroup:nonrootgroup.

小开

若要更改文件系统权限，请在实际容器启动之前运行 initcontainer

这里的例子弹性搜索吊舱

initContainers:
- command:
- sh
- -c
- chown -R 1000:1000 /usr/share/elasticsearch/data
- sysctl -w vm.max_map_count=262144
- chgrp 1000 /usr/share/elasticsearch/data
image: busybox:1.29.2
imagePullPolicy: IfNotPresent
name: set-dir-owner
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:                         #Volume mount path
- mountPath: /usr/share/elasticsearch/data
name: elasticsearch-data

更改容器中的用户组

spec:
containers:
securityContext:
privileged: true
runAsUser: 1000
小开

请参阅这期: https://github.com/kubernetes/kubernetes/issues/2630

如果它是一个 emptydir，spec中的 securityContext可以使用:

spec:
securityContext:
runAsUser: 1000
fsGroup: 1000
containers: ...

如果该卷是 hostpath，则 initContainer可用于该卷中的 chown路径:

initContainers:
- name: example-c
image: busybox:latest
command: ["sh","-c","mkdir -p /vol-path && chown -R 1000:1000 /vol-path"]
resources:
limits:
cpu: "1"
memory: 1Gi
volumeMounts:
- name: vol-example
mountPath: /vol-path
小开

经过几次迭代之后，我最终使用

\{\{- $root := . }}
...
initContainers:
- name: volume-mount-hack
image: busybox
command: ["sh", "-c", "find /data -user root -exec chown 33:33 {} \\;"]
volumeMounts:
\{\{- range $key,$val := .Values.persistence.mounts }}
- name: data
mountPath: /data/\{\{ $key }}
subPath: \{\{ $root.Values.projectKey }}/\{\{ $key }}
\{\{- end }}

与其他解决方案相比，它更加清洁和可配置。此外，这也是 再快点的方式—— find 命令只更改实际属于 root 用户的文件/目录的权限。

当您使用大量文件安装卷时，这可能会对容器的启动/加载时间(几秒钟甚至几分钟!)产生重大影响.

尝试比较

chown www-data:www-data ./ -R

还有

find /data -user root -exec chown 33:33 {} \\;

you may be surprised!

小开
  • 在 minikube 中，通过设置 runAsUser: 0，它在作为 根用户运行 initContainers之后工作
    initContainers:
- name: change-ownership-container
image: busybox
command: ["/bin/chown","-R","1000:1000", "/home/jovyan/work"]
securityContext:
runAsUser: 0
privileged: true
volumeMounts:
- name: notebook-data
mountPath: /home/jovyan/work

所以整个 Yaml 文件看起来是这样的

apiVersion: apps/v1
kind: StatefulSet
metadata:
name: jupyter
labels:
release: jupyter
spec:
replicas:
updateStrategy:
type: RollingUpdate
serviceName: jupyter-headless
podManagementPolicy: Parallel
selector:
matchLabels:
release: jupyter
template:
metadata:
labels:
release: jupyter
annotations:
spec:
restartPolicy: Always
terminationGracePeriodSeconds: 30
securityContext:
runAsUser: 1000
fsGroup: 1000
containers:
- name: jupyter
image: "jupyter/base-notebook:ubuntu-20.04"
imagePullPolicy: IfNotPresent
ports:
- name: http
containerPort: 8888
protocol: TCP
- name: blockmanager
containerPort: 7777
protocol: TCP
- name: driver
containerPort: 2222
protocol: TCP
volumeMounts:
- name: notebook-data
mountPath: /home/jovyan/work
resources:
limits:
cpu: 200m
memory: 300Mi
requests:
cpu: 100m
memory: 200Mi
initContainers:
- name: change-ownership-container
image: busybox
command: ["/bin/chown","-R","1000:1000", "/home/jovyan/work"]
securityContext:
runAsUser: 0
privileged: true
volumeMounts:
- name: notebook-data
mountPath: /home/jovyan/work
volumes:
- name: notebook-data
persistentVolumeClaim:
claimName: jupyter-pvc
小开

在我的示例中，我使用 scratch作为基本映像，并将用户设置为65543。我需要一个目录的书面许可。我用的是 emptyDir音量,

spec:
containers:
...
volumeMounts:
- mountPath: /tmp
name: tmp
# readOnly: true
volumes:
- name: tmp
emptyDir: {}
小开

对于在 pod 中使用 configmap 作为文件的用户

我正在从一个配置图中加载数据，作为飞船容器内的文件，这里是我的舱单:

#./script-cm.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: script-cm
labels:
app: script
data:
data-script: |
#!/bin/bash
set -e
echo "some script commands"

#./deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name:  script
namespace: default
labels:
app:  script
spec:
selector:
matchLabels:
app: script
replicas: 1
template:
metadata:
labels:
app:  script
spec:
restartPolicy: Always
containers:
- name:  script-container
image:  ubuntu:20.04
resources: {}
volumeMounts:
- name: influxdb-provisioning
mountPath: /docker-entrypoint-initdb.d/data.sh
subPath: data.sh


volumes:
- name: script-bind
configMap:
name: script-cm
items:
- key: data-script
path: data.sh
mode: 0777

正如你所看到的，我正在跟随 K8s 文件将一个配置映射绑定到 pod 中，mode: 0777允许我在那个特定的文件上给出 execution permissions，你也可以运行以下命令使用 kubectl explain获得更好的想法:

kubectl explain deployment.spec.template.spec.volumes.configMap.items.mode

确保使用正确的权限而不是0777，因为不推荐使用它，特别是在敏感数据上！

小开

现在还有可以在安全上下文中指定的 fsGroupChangePolicy


提交答案