Using the host ip in docker-compose

小开

Is there a better solution?

Absolutely! You don't need the host ip at all for communication between containers. If you link containers in your docker-compose.yaml file, you will have access to a number of environment variables that you can use to discover the ip addresses of your services.

Consider, for example, a docker-compose configuration with two containers: one using consul, and one running some service that needs to talk to consul.

consul:
image: progrium/consul
command: -server -bootstrap
webserver:
image: larsks/mini-httpd
links:
- consul

First, by starting consul with just -server -bootstrap, consul figures out it's own advertise address, for example:

consul_1    | ==> Consul agent running!
consul_1    |          Node name: 'f39ba7ef38ef'
consul_1    |         Datacenter: 'dc1'
consul_1    |             Server: true (bootstrap: true)
consul_1    |        Client Addr: 0.0.0.0 (HTTP: 8500, HTTPS: -1, DNS: 53, RPC: 8400)
consul_1    |       Cluster Addr: 172.17.0.4 (LAN: 8301, WAN: 8302)
consul_1    |     Gossip encrypt: false, RPC-TLS: false, TLS-Incoming: false
consul_1    |              Atlas: <disabled>

In the webserver container, we find the following environment variables available to pid 1:

CONSUL_PORT=udp://172.17.0.4:53
CONSUL_PORT_8300_TCP_START=tcp://172.17.0.4:8300
CONSUL_PORT_8300_TCP_ADDR=172.17.0.4
CONSUL_PORT_8300_TCP_PROTO=tcp
CONSUL_PORT_8300_TCP_PORT_START=8300
CONSUL_PORT_8300_UDP_END=udp://172.17.0.4:8302
CONSUL_PORT_8300_UDP_PORT_END=8302
CONSUL_PORT_53_UDP=udp://172.17.0.4:53
CONSUL_PORT_53_UDP_ADDR=172.17.0.4
CONSUL_PORT_53_UDP_PORT=53
CONSUL_PORT_53_UDP_PROTO=udp
CONSUL_PORT_8300_TCP=tcp://172.17.0.4:8300
CONSUL_PORT_8300_TCP_PORT=8300
CONSUL_PORT_8301_TCP=tcp://172.17.0.4:8301
CONSUL_PORT_8301_TCP_ADDR=172.17.0.4
CONSUL_PORT_8301_TCP_PORT=8301
CONSUL_PORT_8301_TCP_PROTO=tcp
CONSUL_PORT_8301_UDP=udp://172.17.0.4:8301
CONSUL_PORT_8301_UDP_ADDR=172.17.0.4
CONSUL_PORT_8301_UDP_PORT=8301
CONSUL_PORT_8301_UDP_PROTO=udp
CONSUL_PORT_8302_TCP=tcp://172.17.0.4:8302
CONSUL_PORT_8302_TCP_ADDR=172.17.0.4
CONSUL_PORT_8302_TCP_PORT=8302
CONSUL_PORT_8302_TCP_PROTO=tcp
CONSUL_PORT_8302_UDP=udp://172.17.0.4:8302
CONSUL_PORT_8302_UDP_ADDR=172.17.0.4
CONSUL_PORT_8302_UDP_PORT=8302
CONSUL_PORT_8302_UDP_PROTO=udp
CONSUL_PORT_8400_TCP=tcp://172.17.0.4:8400
CONSUL_PORT_8400_TCP_ADDR=172.17.0.4
CONSUL_PORT_8400_TCP_PORT=8400
CONSUL_PORT_8400_TCP_PROTO=tcp
CONSUL_PORT_8500_TCP=tcp://172.17.0.4:8500
CONSUL_PORT_8500_TCP_ADDR=172.17.0.4
CONSUL_PORT_8500_TCP_PORT=8500
CONSUL_PORT_8500_TCP_PROTO=tcp

There is a set of variables for each port EXPOSEd by the consul image. For example, in that second image, we could interact with the consul REST API by connecting to:

http://${CONSUL_PORT_8500_TCP_ADDR}:8500/

小开

Environment variables, as suggested in the earlier solution, are created by Docker when containers are linked. But the env vars are not automatically updated if the container is restarted. So, it is not recommended to use environment variables in production.

Docker, in addition to creating the environment variables, also updates the host entries in /etc/hosts file. In fact, Docker documentation recommends using the host entries from etc/hosts instead of the environment variables.

Reference: https://docs.docker.com/userguide/dockerlinks/

Unlike host entries in the /etc/hosts file, IP addresses stored in the environment variables are not automatically updated if the source container is restarted. We recommend using the host entries in /etc/hosts to resolve the IP address of linked containers.

小开

With the new version of Docker Compose (1.4.0) you should be able to do something like this:

docker-compose.yml

consul:
image: progrium/consul
command: -server -advertise HOSTIP -bootstrap

bash

$ sed -e "s/HOSTIP/${HOSTIP}/g" docker-compose.yml | docker-compose --file - up

This is thanks to the new feature:

Compose can now read YAML configuration from standard input, rather than from a file, by specifying - as the filename. This makes it easier to generate configuration dynamically:
$ echo 'redis: {"image": "redis"}' | docker-compose --file - up

小开

docker-compose allows to use environment variables from the environment running the compose command.

See documentation at https://docs.docker.com/compose/compose-file/#variable-substitution

Assuming you can create a wrapper script, like @balver suggested, you can set an environment variable called EXTERNAL_IP that will include the value of $(docker-machine ip).

Example:

#!/bin/sh
export EXTERNAL_IP=$(docker-machine ip)
exec docker-compose $@

and

# docker-compose.yml
version: "2"
services:
consul:
image: consul
environment:
- "EXTERNAL_IP=${EXTERNAL_IP}"
command: agent -server -advertise ${EXTERNAL_IP} -bootstrap

Unfortunately if you are using random port assignment, there is no way to add EXTERNAL_PORT, so the ports must be linked statically.

PS: Something very similar is enabled by default in HashiCorp Nomad, also includes mapped ports. Doc: https://www.nomadproject.io/docs/jobspec/interpreted.html#interpreted_env_vars

小开

I've used docker internal network IP that seems to be static: 172.17.0.1

小开

extra_hosts works, it's hard coded into docker-compose.yml but for my current static setup, at this moment that's all I need.

version: '3'
services:
my_service:
container_name: my-service
image: my-service:latest
extra_hosts:
- "myhostname:192.168.0.x"


...


networks:
- host
networks:
host:

小开

Create a script to set, every boot, your host IP in an environment variable.

sudo vi /etc/profile.d/docker-external-ip.sh

Then copy inside this code:

export EXTERNAL_IP=$(hostname -I | awk '{print $1}')

Now you can use it in your docker-compose.yml file:

version: '3'
services:
my_service:
container_name: my-service
image: my-service:latest
environment:
- EXTERNAL_IP=${EXTERNAL_IP}
extra_hosts:
- my.external-server.net:${EXTERNAL_IP}


...

environment --> to set as system environment var in your docker container
extra_hosts --> to add these hosts to your docker container