Ouyang's Blog

Working on ML Infra

0%

Docker notes

Posted on Edited on In Disqus:

Run and exec

  • docker run --ipc=host --gpus all --entrypoint=/bin/bash -it -d -v /home/name:/home/name -e TZ=Asia/Beijing --name ouyang tensorflow/tensorflow:2.4.1-gpu
  • run flags:
    • --pid=host: Use it if you want your container to share the host’s process namespace, basically allowing processes within the container to see all of the processes on the system.
    • --ipc=private|shareable|host|container:name-or-id: IPC namespace provides separation of named shared memory segments, semaphores and message queues. Shared memory segments is commenly used by databases and C/OpenMPI, C++/using boost libraries and etc.
  • docker exec -it ${container_name} /bin/bash
  • docker exec -it -w /home/name ouyang sh -c “tmux a” /bin/bash

System

  • docker system df
  • docker system df -v
  • sudo service docker start/stop

Save and load

  • Save images: docker save busybox > busybox.tar
  • Load images: docker load < busybox.tar
  • docker tag 75h34h257889e docker.xxx.com/ouyang/cuda:11.1-cudnn8-devel-ubuntu18.04
  • docker login docker.xxx.com --username *** --password **
  • docker push docker.xxx.net/ouyang/cuda:tag

Build

  • Template: docker build --network host -t docker.xxx.com/project project
    • The “project” is folder which contain a Dockerfile script and some other things(for the usage of COPY and etc).
  • Another template: docker build . -f DockerfileNvidia --network host -t docker.xxx.com/project:latest
    • In this case, DockerfileNvidia is a Dockerfile script but it’s not named Dockerfile.
    • The period means the context is under current working directory.
    • --network host: Could be used for proxy purpose.
  • Specifying target build stage (–target)
    • When building a Dockerfile with multiple build stages, –target can be used to specify an intermediate build stage by name as a final stage for the resulting image. Commands after the target stage will be skipped.
      1
      2
      3
      4
      FROM debian AS build-env
      ...
      FROM alpine AS production-env
      ... 
      1
      docker build -t mybuildimage --target build-env .
  • Set build-time variables (–build-arg)