Docker镜像无法挂载宿主机目录排查&解决

问题描述

由于交付需要，需要给客户交付镜像，QA测试过程中发现有一台机器无法正常运行服务，经过排查发现，是由于服务需要挂载宿主机的目录进行读写文件(/tmp), 和宿主机的client进行交互，但容器启动后并没有成功挂载。

问题结论

snap安装的docker存在诸多限制，改用官方安装的版本即可。

排查经过

我是通过compose文件启动容器的，里面增加的配置如下：

volumes:
  - /dev:/dev
  - /tmp:/tmp

未成功挂载，先通过如下命令验证是否我的镜像有问题

sudo docker run -it --rm -v /tmp:/tmp alpine ls /tmp

打印结果中无宿主机/tmp目录下的文件。

检查AppArmor

sudo aa-status
apparmor module is loaded.
128 profiles are loaded.
40 profiles are in enforce mode.
   /snap/snapd/24792/usr/lib/snapd/snap-confine
   /snap/snapd/24792/usr/lib/snapd/snap-confine//mount-namespace-capture-helper
   /usr/bin/man
   /usr/lib/NetworkManager/nm-dhcp-client.action
   /usr/lib/NetworkManager/nm-dhcp-helper
   /usr/lib/connman/scripts/dhclient-script
   /usr/lib/snapd/snap-confine
   /usr/lib/snapd/snap-confine//mount-namespace-capture-helper
   /{,usr/}sbin/dhclient
   docker-default
   lsb_release
   man_filter
   man_groff
   nvidia_modprobe
   nvidia_modprobe//kmod
   plasmashell
   plasmashell//QtWebEngineProcess
   rsyslogd
   snap-update-ns.docker
   snap.docker.compose
   snap.docker.docker
   snap.docker.dockerd
   snap.docker.hook.configure
   snap.docker.hook.connect-plug-graphics-core22
   snap.docker.hook.install
   snap.docker.hook.post-refresh
   snap.docker.nvidia-container-toolkit
   tcpdump
   ubuntu_pro_apt_news
   ubuntu_pro_esm_cache
   ubuntu_pro_esm_cache//apt_methods
   ubuntu_pro_esm_cache//apt_methods_gpgv
   ubuntu_pro_esm_cache//cloud_id
   ubuntu_pro_esm_cache//dpkg
   ubuntu_pro_esm_cache//ps
   ubuntu_pro_esm_cache//ubuntu_distro_info
   ubuntu_pro_esm_cache_systemctl
   ubuntu_pro_esm_cache_systemd_detect_virt
   unix-chkpwd
   unprivileged_userns
0 profiles are in complain mode.
0 profiles are in prompt mode.
0 profiles are in kill mode.
88 profiles are in unconfined mode.
   1password
   Discord
   MongoDB Compass
   QtWebEngineProcess
   brave
   buildah
   busybox
   cam
   ch-checkns
   ch-run
   chrome
   crun
   devhelp
   element-desktop
   epiphany
   evolution
   firefox
   flatpak
   geary
   github-desktop
   goldendict
   ipa_verify
   kchmviewer
   keybase
   lc-compliance
   libcamerify
   linux-sandbox
   loupe
   lxc-attach
   lxc-create
   lxc-destroy
   lxc-execute
   lxc-stop
   lxc-unshare
   lxc-usernsexec
   mmdebstrap
   msedge
   nautilus
   notepadqq
   obsidian
   opam
   opera
   pageedit
   podman
   polypane
   privacybrowser
   qcam
   qmapshack
   qutebrowser
   rootlesskit
   rpm
   rssguard
   runc
   sbuild
   sbuild-abort
   sbuild-adduser
   sbuild-apt
   sbuild-checkpackages
   sbuild-clean
   sbuild-createchroot
   sbuild-destroychroot
   sbuild-distupgrade
   sbuild-hold
   sbuild-shell
   sbuild-unhold
   sbuild-update
   sbuild-upgrade
   scide
   signal-desktop
   slack
   slirp4netns
   steam
   stress-ng
   surfshark
   systemd-coredump
   thunderbird
   toybox
   trinity
   tup
   tuxedo-control-center
   userbindmount
   uwsgi-core
   vdens
   virtiofsd
   vivaldi-bin
   vpnns
   vscode
   wpcom
16 processes have profiles defined.
16 processes are in enforce mode.
   /bin/node_exporter (3457) docker-default
   /usr/sbin/rsyslogd (2546) rsyslogd
   /snap/docker/3265/bin/dockerd (2922) snap.docker.dockerd
   /snap/docker/3265/bin/containerd (3016) snap.docker.dockerd
   /snap/docker/3265/bin/containerd-shim-runc-v2 (3408) snap.docker.dockerd
   /snap/docker/3265/bin/containerd-shim-runc-v2 (3409) snap.docker.dockerd
   /snap/docker/3265/bin/docker-proxy (3616) snap.docker.dockerd
   /snap/docker/3265/bin/docker-proxy (3633) snap.docker.dockerd
   /snap/docker/3265/bin/docker-proxy (3773) snap.docker.dockerd
   /snap/docker/3265/bin/docker-proxy (3779) snap.docker.dockerd
   /snap/docker/3265/bin/containerd-shim-runc-v2 (1584848) snap.docker.dockerd
   /snap/docker/3265/bin/containerd-shim-runc-v2 (1607800) snap.docker.dockerd
   /snap/docker/3265/bin/docker-proxy (1608117) snap.docker.dockerd
   /snap/docker/3265/bin/docker-proxy (1608126) snap.docker.dockerd
   /snap/docker/3265/bin/docker-proxy (1608133) snap.docker.dockerd
   /snap/docker/3265/bin/docker-proxy (1608140) snap.docker.dockerd
0 processes are in complain mode.
0 processes are in prompt mode.
0 processes are in kill mode.
0 processes are unconfined but have a profile defined.
0 processes are in mixed mode.

无问题。

查看挂载过程

sudo strace -f -e mount docker run --rm -v /tmp:/tmp alpine ls /tmp 2>&1 | grep tmp

无输出。

目前查到资料说snap安装的docker会有很多限制，开始排查

什么是 Snap 版 Docker？

Snap 是 Canonical (Ubuntu 公司) 开发的软件打包和部署系统，特点如下：

• 沙盒环境：每个 Snap 应用在受限的环境中运行

• 自动更新：后台自动更新到最新版本

• 跨发行版：相同的 Snap 包可在多个 Linux 发行版运行

• 依赖打包：所有依赖打包在一个文件中

Snap 版 Docker 是通过 Ubuntu Snap 商店安装的 Docker 版本：

Snap 版 Docker 的常见问题

文件系统访问限制

• 沙盒隔离：Snap 应用默认只能访问 /home 和 /media 等特定目录

• 系统目录限制：访问 /tmp、/dev 等系统目录需要特殊权限

• 挂载传播限制：无法修改挂载传播设置

使用如下命令卸载docker，并启动官方安装的docker，问题解决

sudo snap remove --purge docker
# 删除残留配置
sudo rm -rf /var/snap/docker
sudo rm -rf ~/snap/docker

#本机实际已经安装，只是被snap安装的覆盖掉了
#sudo apt update
#sudo apt install -y docker-ce docker-ce-cli containerd.io
sudo systemctl start docker

至此，问题解决。