Environment variables
Compose CLI environment variables
These configure the docker-compose command line execution.
COMPOSE_PROJECT_NAME
Sets the project name. This value is prepended along with the service name to the container on start up.
ex : COMPOSE_PROJECT_NAME=fooapp
If the template declares two services db and web, containers will be fooapp_db_1 and fooapp_web_1.
COMPOSE_FILE
Specify the path to the Compose file instead of docker-compose.yml in the current directory.
ex: COMPOSE_FILE: foo-docker/foo-docker-compose.yml
DOCKER_HOST
Sets the URL of the docker daemon instead of default to unix:///var/run/docker.sock
ex: DOCKER_HOST: foodomain:2375
Environment variables in compose template
Substitute environment variables in Compose files :
web: image: "webapp:${TAG}" |
These env variables may be either present on the current shell or may be passed dynamically with the flag --env-file
such as :
docker-compose --env-file ./config/.env.dev ...
Set environment variables in a container :
web: environment: - DEBUG=1 |
Pass environment variables to a container :
web: environment: - DEBUG |
Here the variable value comes from the variable in the shell in which Compose is run.
The “.env” file :
It sets default values for any environment variables referenced in the Compose file in an environment file named .env
.
Example :
$ cat .env TAG=v1.5 $ cat docker-compose.yml version: '3' services: web: image: "webapp:${TAG}" |
Flags before subcommand
Docker-like flags
-H FOO_HOST
: to execute the command on the specified docker host.
Prefix of containers/services
By default, all docker object created by docker-compose (services, volume, network…) are prefixed with the current directory name such as currentDir_.
To override that prefix :
docker-compose -p otherPrefix
When we use that process to run a docker-compose project, we also need to specify that same suffix for other commands related to that project such as :
docker-compose -p otherPrefix ps
to show current services.
Basic commands
docker-compose build
Usage : build [options] [--build-arg key=val...] [SERVICE...]
* Build or rebuild services declared in the docker-compose file.
By default, all declared services are built. But we could also specify exactly those to build.
Helpful flags :
--no-cache Do not use cache when building the image. --no-rm Do not remove intermediate containers after a successful build. --parallel Build images in parallel. --progress string Set type of progress output (auto, plain, tty). EXPERIMENTAL flag for native builder. |
docker-compose up
Usage : up [options] [--scale SERVICE=NUM...] [SERVICE...]
* Create and start containers for declared services declared in the docker-compose file.
It does that for not yet existing containers and that re-create and re-start them in case of existing containers which the image or the docker-compose configuration has changed.
Be aware : by default, that doesn’t build the images.
By default, all declared services are handled. But we could also specify exactly those to handle.
Helpful flags :
-d, --detach Detached mode (deamon) --build Build images before starting containers --no-deps Don't start linked services. --force-recreate Recreate containers even if their configuration and image haven't changed --abort-on-container-exit Stops all containers if any container was stopped. Incompatible with -d. --exit-code-from SERVICE Return the exit code of the selected service container. Implies --abort-on-container-exit. |
Examples :
* Build, create if needed and start in detached mode all services declared in the docker-compose file :
docker-compose up -d --build
* Build, create if needed and start a specific service declared in the docker-compose file :
docker-compose up --build my-service
* Build, force the recreation and start a specific service declared in the docker-compose file :
docker-compose up --build --force-recreate my-service
Combine docker-compose build and up to get a full build and a restart of containers
Sometimes, we changed things in multiple places : the applications, the DockerImages, the configuration/docker-compose templates.
In that case, a full build (no cache) and a stop/restart of all containers makes sense :
docker-compose build --no-cache && docker-compose up -d --force-recreate
Note that here the subcommand up --build
is not enough because the build still uses the image layers cache. Only the subcommand build --no-cache
will do that.
And my really favorite combination command is the same and by following logs :
docker-compose build --no-cache && docker-compose up -d --force-recreate && docker-compose logs -f
docker-compose pull : useful for snapshot images
Latest and development docker image tags are going to be overwritten.
To ensure docker compose always uses the last version of our images, we can execute first docker-compose pull
.
Finally quite similar to what we do with docker pull.
docker-compose down
Usage: down [options]
Stops containers and clean them, that is removes containers, networks, volumes, and images created by up
.
By default, the only things removed are:
– Containers for services defined in the Compose file
– Networks defined in the `networks` section of the Compose file
– The default network, if one is used
Helpful flags :
-v or –volumes : remove names and anonymous volumes attached to containers.
docker-compose run
Usage:
run [options] [-v VOLUME...] [-p PORT...] [-e KEY=VAL...] [-l KEY=VALUE...] SERVICE [COMMAND] [ARGS...]
Run a one-off command on a service.
By default, linked services will be started, unless they are already
running.
That command can be seen as a swiss-knife. That allow to execute a command on a service but not only.
That is also a way to start and run a specific service with further options that the up
command
doesn’t provide.
In that use, we can see that as the symmetric command to docker run
in the context of docker-compose.
Note that by default, ports are not mapped to the host even if the docker-compose template declare that.
Helpful flags (almost all of them) :
-d, --detach Detached mode: Run container in the background, print new container name. --name NAME Assign a name to the container --entrypoint CMD Override the entrypoint of the image. -e KEY=VAL Set an environment variable (can be used multiple times) -u, --user="" Run as specified username or uid --no-deps Don't start linked services. --rm Remove container after run. Ignored in detached mode. -p, --publish=[] Publish a container's port(s) to the host --service-ports Run command with the service's ports enabled and mapped to the host. -v, --volume=[] Bind mount a volume (default []) -T Disable pseudo-tty allocation. By default `docker-compose run` allocates a TTY. -w, --workdir="" Working directory inside the container |
Examples :
* Run a service by enabling the port mapping, disabling the dependencies services start and by changing the entrypoint of the image by bash :
docker-compose run --no-deps --service-ports --entrypoint bash fooservice
Declare a named volume
For example for a volume with as directory /app/data
on the image/container and mounted/named
app-data
on the host :
version: '2.3' services: my-app: ... volumes: - app-data:/app/data volumes: app-data: |
Use a user-bridge and a none network
You can rely on that.
Handle multiple environments
My general idea
– don’t rely too much on the .env
file because that is not specific to a service or to an environment because that way binds all properties declared in into every containers and you cannot have an .env
file by environment.
– the .env file
that we can add in the directory of the docker-compose template and the service: env_file
yaml property that we define in the docker-compose.yml are very distinct things.
The .env file
allows to provide a properties file (with key=value) that are usable both in the build and the execution time for every containers while the service: env_file
yaml property allows to provide a properties file (with key=value) that are usable only at the execution time and for a specific (or more) container
– Use the service: env_file
yaml property to define runtime properties needed for a container
– Use the service: build: args
yaml property to define buildtime properties needed for a container
– Define a base docker-compose.yml template and define a specific docker-compose.yml for each environment with only the things that differ from the base template.
Define a docker-compose.yml template base and define a custom template by environment
Here a simple use case : we have 2 backend services that communicate and 2 environments to containerize : integration, production.
1) We define the base docker-compose.yml template that is common for any environment (while we may still override that).
version: '3.5' services: quizz-spring-boot: build: context: ../quizz/quizz-backend dockerfile: docker/Dockerfile-quizz-spring-boot volumes: - quizz-data:/usr/quizz/db networks: - network-quizz-app hall-of-fame-spring-boot: build: context: ../hall-of-fame/hall-of-fame-backend dockerfile: docker/Dockerfile-hall-of-fame-spring-boot volumes: - quizz-data:/usr/quizz/db networks: - network-quizz-app volumes: quizz-data: networks: network-quizz-app: driver: bridge |
2) We define in the same directory a docker-compose.override.yml template for the integration environment.
Docker override docker-compose.override.yml with that file by default if it is found.
We could add new properties or override these defined in the base template. Here we only add new properties, that is rather a good practice because in a general way, we should avoid to move in the base template things that are overridden for about each environment template : that’s makes things unclear and error-prone.
In the example, we can see some interesting things :
– the ports are specified (makes sense : it depends on the actual environment)
– in quizz-spring-boot, we define an argument for the build of the quizz-spring-boot service : APPLICATION_PROPERTIES_FILENAME=application-integration.properties
.
That is a simple way to define a set of properties usable in the build of a specific container/service and for a specific environment, here the quizz-spring-boot service and the integration environment.
– in hall-of-frame-spring-boot, we provide environment properties coming from the integration.env file : ../hall-of-fame/hall-of-fame-backend/docker/envs/integration.env
usable at runtime of that specific container and for a specific environment, here the quizz-spring-boot service and the integration environment.
version: '3.5' services: quizz-spring-boot: build: args: - APPLICATION_PROPERTIES_FILENAME=application-integration.properties ports: - "8282:8080" hall-of-fame-spring-boot: ports: - "8283:8080" env_file: - ../hall-of-fame/hall-of-fame-backend/docker/envs/integration.env |
The application-integration.properties is a classical properties file that we use with spring boot. We could pass it in the application such as.
While the integration.env is also a properties file but with a more general way to name properties since here we will usethe environment variables directly in the Dockerfile.
Here is a snippet of these.
integration.env :
LOGGING_LEVEL_ROOT=INFO LOGGING_LEVEL_SECURITY=DEBUG |
application-integration.properties :
.... logging.level.root=INFO spring.datasource.url = jdbc:postgresql:database .... |
Here how we use at build time in the Dockerfile the arg passed :
RUN echo "application-properties used : $APPLICATION_PROPERTIES_FILENAME" ARG APPLICATION_PROPERTIES_FILENAME RUN test -n "$APPLICATION_PROPERTIES_FILENAME" COPY config/$APPLICATION_PROPERTIES_FILENAME ./ |
The code declares the arg to be able to get its value, then it checks that the variable is valued (string length > 0) and then we copy the file on the image currently built.
And here how we use at runtime in the Dockerfile the environment properties passed :
CMD java -cp /app:/app/lib/* mypackage.myApplication \ --logging.level.root=$LOGGING_LEVEL_ROOT \ --logging.level.org.springframework.security=$LOGGING_LEVEL_SECURITY \ |
3) We define another docker-compose template for the production environment :
version: '3.5' services: quizz-spring-boot: build: args: - APPLICATION_PROPERTIES_FILENAME=application-production.properties ports: - "8282:8080" hall-of-fame-spring-boot: ports: - "8283:8080" env_file: - ../hall-of-fame/hall-of-fame-backend/docker/envs/production.env |
We can see that the template is very close of the integration template but with env properties and args specific to the production.
Of course, even if not done here, we could have overridden properties defined in the base template as Dockerfile or anything…
4) Commands to execute the docker-compose according to the environment.
* Build and run for the production env (need to declare base template and then the production template) :
docker-compose -f docker-compose.yml -f docker-compose.production.yml up --build -d
* Build and run for the production env (need to declare nothing if we are in the right directory) :
docker-compose up --build -d
Dependency between services
Express dependency between services
We do it by adding a depends_on
attribute in the service declaration that depends on another service.
The idea behind is that docker-compose will create services in the defined order, and ensure that up
and stop
sub-commands honor that relationship.
Example :
version: "2.4" services: web: build: . depends_on: - db - redis redis: image: redis db: image: postgres |
Go further : express a dependency condition for starting a services
The thing is that in the above example, the services are created in the defined order but the web service doesn’t wait for db and redis to be ready, they only wait for these to be started.
To set that constraint, we need two things :
– specify the condition: service_healthy
in the service declaration that depends on another service
By default the value is condition: service_started
. That’s the behavior that we had previously.
– specify a healthcheck
attribute under the service that is defined as a dependency.
service_healthy
condition relies on that healthcheck.
Previous example refined :
web service starts when redis is started (as previously) and db is healthy.
version: "2.4" services: web: build: . depends_on: db: condition: service_healthy redis: condition: service_started redis: image: redis db: image: postgres healthcheck: test: "exit 0" |