Docker networking and Stacks

Explaining Docker Networking:

Docker networking is basically used to establish communication between the docker containers and the outside world via host machine or you can say it is a communication passage through which all the isolated containers communicate with each other in various situations to perform the required actions.

• User-defined bridge networks are best when you need multiple containers to communicate on the same Docker host.
• Host networks are best when the network stack should not be isolated from the Docker host, but you want other aspects of the container to be isolated.
• Overlay networks are best when you need containers running on different Docker hosts to communicate, or when multiple applications work together using swarm services.
• Macvlan networks are best when you are migrating from a VM setup or need your containers to look like physical hosts on your network, each with a unique MAC address.
• Third-party network plugins allow you to integrate Docker with specialized network stacks.

Overlay Networking:

-bash-4.2$ docker network create --driver overlay mydrupal
p6wlckz3tpbsvrxg7vkscb3o5
-bash-4.2$ docker network ls
NETWORK ID          NAME                DRIVER              SCOPE
b7fad9e8721f        bridge              bridge              local
630c7e7be1df        docker_gwbridge     bridge              local
8df84c6fceb2        host                host                local
oy1sf7s64vkh        ingress             overlay             swarm
p6wlckz3tpbs        mydrupal            overlay             swarm
913d04ce1d13        none                null                local

-bash-4.2$ docker service create --name psql --network mydrupal -e POSTGRES_PASSWORD=mypass postgres
7cxim9tpkhs528l10mvnkqvwg
overall progress: 1 out of 1 tasks
1/1: running   [==================================================>]
verify: Service converged

-bash-4.2$ docker service ps msql
no such service: msql

-bash-4.2$ docker service ps psql
ID                  NAME                IMAGE               NODE                DESIRED STATE       CURRENT STATE            ERROR               PORTS
4aj11siw8gv3        psql.1              postgres:latest     worker2             Running             Running 17 seconds ago
-bash-4.2$ docker service ls
ID                  NAME                MODE                REPLICAS            IMAGE               PORTS
7cxim9tpkhs5        psql                replicated          1/1                 postgres:latest
-bash-4.2$ docker service create --name drupal --network mydrupal -p 80:80 drupal
hyr5o861bdzk50s45bu8z52tv
overall progress: 1 out of 1 tasks
1/1: running
verify: Service converged
-bash-4.2$ docker service ls
ID                  NAME                MODE                REPLICAS            IMAGE               PORTS
hyr5o861bdzk        drupal              replicated          1/1                 drupal:latest       *:80->80/tcp
7cxim9tpkhs5        psql                replicated          1/1                 postgres:latest
-bash-4.2$ docker service ps drupal
ID                  NAME                IMAGE               NODE                DESIRED STATE       CURRENT STATE                ERROR               PORTS
t79l6b5y6han        drupal.1            drupal:latest       manager             Running             Running about a minute ago
-bash-4.2$

Routing Mesh: Global Traffic Router

1. Route ingress (incoming) packets for a service to proper Task.
2. Spans all nodes in Swarm
3. Uses IPVS from Linux Kernel.
4. Load balances Swarm Services across their Tasks.
5. Two ways this works:

i. Container to Container in a overlay network (Uses VIP).
ii. External traffic incoming to published ports (all nodes listen).

6. It is a stateless load balancing.
7. this LB is at OSI layer 3 (TCP), not layer 4 (DNS).
8. Both the limitations can overcome with:

Nginx or HAProxy LB proxy, or:
Docker Enterprise edition, which come with built-in L4 web proxy.


-bash-4.2$ docker service create --name search --replicas 3 -p 9200:9200 elasticsearch:2
rqvmbrn5d4rlsfir9sb2hv1w0
overall progress: 3 out of 3 tasks
1/3: running   [==================================================>]
2/3: running   [==================================================>]
3/3: running   [==================================================>]
verify: Service converged
-bash-4.2$ docker service ls
ID                  NAME                MODE                REPLICAS            IMAGE               PORTS
hyr5o861bdzk        drupal              replicated          1/1                 drupal:latest       *:80->80/tcp
7cxim9tpkhs5        psql                replicated          1/1                 postgres:latest
rqvmbrn5d4rl        search              replicated          3/3                 elasticsearch:2     *:9200->9200/tcp
-bash-4.2$ docker service ps search
ID                  NAME                IMAGE               NODE                DESIRED STATE       CURRENT STATE                ERROR               PORTS
yrxkktj47j05        search.1            elasticsearch:2     manager             Running             Running about a minute ago
fhvst8z2b8vo        search.2            elasticsearch:2     worker2             Running             Running 53 seconds ago
jwu0fld4om1t        search.3            elasticsearch:2     worker2             Running             Running 53 seconds ago
-bash-4.2$ ip r
default via 192.168.29.1 dev ens33 proto static metric 100
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1
172.18.0.0/16 dev docker_gwbridge proto kernel scope link src 172.18.0.1
192.168.29.0/24 dev ens33 proto kernel scope link src 192.168.29.10 metric 100
-bash-4.2$
-bash-4.2$

-bash-4.2$ curl -v 192.168.29.10:9200
* About to connect() to 192.168.29.10 port 9200 (#0)
*   Trying 192.168.29.10...
* Connected to 192.168.29.10 (192.168.29.10) port 9200 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 192.168.29.10:9200
> Accept: */*
>
< HTTP/1.1 200 OK
< Content-Type: application/json; charset=UTF-8
< Content-Length: 368
<
{
  "name" : "Maxwell Dillon",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "2iyfSDGNRq2EdxE1vWXHTA",
  "version" : {
    "number" : "2.4.6",
    "build_hash" : "5376dca9f70f3abef96a77f4bb22720ace8240fd",
    "build_timestamp" : "2017-07-18T12:17:44Z",
    "build_snapshot" : false,
    "lucene_version" : "5.5.4"
  },
  "tagline" : "You Know, for Search"
}
* Connection #0 to host 192.168.29.10 left intact
-bash-4.2$ curl -v 192.168.29.11:9200
* About to connect() to 192.168.29.11 port 9200 (#0)
*   Trying 192.168.29.11...
* Connected to 192.168.29.11 (192.168.29.11) port 9200 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 192.168.29.11:9200
> Accept: */*
>
< HTTP/1.1 200 OK
< Content-Type: application/json; charset=UTF-8
< Content-Length: 368
<
{
  "name" : "Maxwell Dillon",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "2iyfSDGNRq2EdxE1vWXHTA",
  "version" : {
    "number" : "2.4.6",
    "build_hash" : "5376dca9f70f3abef96a77f4bb22720ace8240fd",
    "build_timestamp" : "2017-07-18T12:17:44Z",
    "build_snapshot" : false,
    "lucene_version" : "5.5.4"
  },
  "tagline" : "You Know, for Search"
}
* Connection #0 to host 192.168.29.11 left intact
-bash-4.2$ curl -v 192.168.29.12:9200
* About to connect() to 192.168.29.12 port 9200 (#0)
*   Trying 192.168.29.12...
* Connected to 192.168.29.12 (192.168.29.12) port 9200 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 192.168.29.12:9200
> Accept: */*
>
< HTTP/1.1 200 OK
< Content-Type: application/json; charset=UTF-8
< Content-Length: 369
<
{
  "name" : "George Tarleton",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "0sXPGfTOQCCscT57rYUxxw",
  "version" : {
    "number" : "2.4.6",
    "build_hash" : "5376dca9f70f3abef96a77f4bb22720ace8240fd",
    "build_timestamp" : "2017-07-18T12:17:44Z",
    "build_snapshot" : false,
    "lucene_version" : "5.5.4"
  },
  "tagline" : "You Know, for Search"
}
* Connection #0 to host 192.168.29.12 left intact
-bash-4.2$

Stacks:

1. In 1.13 docker adds a new layer of abstraction to swarm called Stacks
2. Stacks accept Compose files as their declarative definition for services, networks, volumes.
3. We can "docker stack deploy" rather than "docker service create".
4. Stack manages all those objects for us, including overlay n/w per stack. Add stack name to start of their name.
5. new "deploy": key in compose file. Can't do build.
6. compose ignores "deploy"; swarm ignores "build".
7. docker-compose cli not needed on swarm server.


-bash-4.2$ git clone https://github.com/shaan2212/example-voting-app.git
fatal: could not create work tree dir 'example-voting-app'.: Permission denied
-bash-4.2$ sudo git clone https://github.com/shaan2212/example-voting-app.git
[sudo] password for dockmgr:
Cloning into 'example-voting-app'...
remote: Enumerating objects: 832, done.
remote: Total 832 (delta 0), reused 0 (delta 0), pack-reused 832
Receiving objects: 100% (832/832), 950.35 KiB | 356.00 KiB/s, done.
Resolving deltas: 100% (293/293), done.
-bash-4.2$ ls -l
total 4
drwxr-xr-x. 2 root root   40 Nov 30 08:21 customimage
drwxr-xr-x. 8 root root 4096 Dec  4 23:14 example-voting-app
-bash-4.2$
-bash-4.2$
-bash-4.2$ cd example-voting-app/
-bash-4.2$ ls -l
total 128
-rw-r--r--. 1 root root 54824 Dec  4 23:14 architecture.png
-rw-r--r--. 1 root root   808 Dec  4 23:14 docker-compose-javaworker.yml
-rw-r--r--. 1 root root   517 Dec  4 23:14 docker-compose-k8s.yml
-rw-r--r--. 1 root root   400 Dec  4 23:14 docker-compose-simple.yml
-rw-r--r--. 1 root root  1107 Dec  4 23:14 docker-compose-windows-1809.yml
-rw-r--r--. 1 root root   994 Dec  4 23:14 docker-compose-windows.yml
-rw-r--r--. 1 root root   821 Dec  4 23:14 docker-compose.yml
-rw-r--r--. 1 root root  1476 Dec  4 23:14 docker-stack-simple.yml
-rw-r--r--. 1 root root  1037 Dec  4 23:14 docker-stack-windows-1809.yml
-rw-r--r--. 1 root root  1284 Dec  4 23:14 docker-stack-windows.yml
-rw-r--r--. 1 root root  1707 Dec  4 23:14 docker-stack.yml
-rw-r--r--. 1 root root  2058 Dec  4 23:14 ExampleVotingApp.sln
-rw-r--r--. 1 root root  1182 Dec  4 23:14 Jenkinsfile
drwxr-xr-x. 2 root root   250 Dec  4 23:14 k8s-specifications
-rw-r--r--. 1 root root  3248 Dec  4 23:14 kube-deployment.yml
-rw-r--r--. 1 root root 10758 Dec  4 23:14 LICENSE
-rw-r--r--. 1 root root   185 Dec  4 23:14 MAINTAINERS
-rw-r--r--. 1 root root  3574 Dec  4 23:14 README.md
drwxr-xr-x. 6 root root   193 Dec  4 23:14 result
drwxr-xr-x. 5 root root   107 Dec  4 23:14 vote
drwxr-xr-x. 6 root root   149 Dec  4 23:14 worker
-bash-4.2$ vi docker-stack.yml
-bash-4.2$
-bash-4.2$ docker stack deploy -c docker-stack.yml voteapp
Creating network voteapp_frontend
Creating network voteapp_backend
Creating network voteapp_default
Creating service voteapp_result
Creating service voteapp_worker

Creating service voteapp_visualizer
Creating service voteapp_redis
Creating service voteapp_db
Creating service voteapp_vote
-bash-4.2$
-bash-4.2$ docker stack ls
NAME                SERVICES            ORCHESTRATOR
voteapp             6                   Swarm
-bash-4.2$ docker stack ps voteapp
ID                  NAME                   IMAGE                                          NODE                DESIRED STATE       CURRENT STATE              ERROR               PORTS
m3w89adon80k        voteapp_vote.1         dockersamples/examplevotingapp_vote:before     worker2             Running             Preparing 14 seconds ago
q9yhao2gjndd        voteapp_db.1           postgres:9.4                                   manager             Running             Preparing 24 seconds ago
uku8r4t3ryfu        voteapp_redis.1        redis:alpine                                   worker2             Running             Running 7 seconds ago
i333b3yqlkh7        voteapp_visualizer.1   dockersamples/visualizer:stable                manager             Running             Preparing 34 seconds ago
nqrffe2jyqcq        voteapp_worker.1       dockersamples/examplevotingapp_worker:latest   manager             Running             Preparing 42 seconds ago
718s5harqvxf        voteapp_result.1       dockersamples/examplevotingapp_result:before   manager             Running             Running 5 seconds ago
0s84riq1vmj8        voteapp_vote.2         dockersamples/examplevotingapp_vote:before     manager             Running             Preparing 14 seconds ago
-bash-4.2$ docker container ls
CONTAINER ID        IMAGE                                          COMMAND                  CREATED             STATUS              PORTS                NAMES
6dea2cc3e13b        dockersamples/examplevotingapp_result:before   "node server.js"         17 seconds ago      Up 15 seconds       80/tcp               voteapp_result.1.718s5harqvxfsqn7lumiuvsoz
ccf165bd59ee        elasticsearch:2                                "/docker-entrypoint.…"   22 minutes ago      Up 22 minutes       9200/tcp, 9300/tcp   search.1.yrxkktj47j05sqiwi9alyvrdg
954b383b5f72        drupal:latest                                  "docker-php-entrypoi…"   47 minutes ago      Up 47 minutes       80/tcp               drupal.1.t79l6b5y6hanwuv1f0ggfu1ep

-bash-4.2$ docker network ls
NETWORK ID          NAME                DRIVER              SCOPE
b7fad9e8721f        bridge              bridge              local
630c7e7be1df        docker_gwbridge     bridge              local
8df84c6fceb2        host                host                local
oy1sf7s64vkh        ingress             overlay             swarm
p6wlckz3tpbs        mydrupal            overlay             swarm
913d04ce1d13        none                null                local
6uud58ijvggy        voteapp_backend     overlay             swarm
jrsobkuwnb7o        voteapp_default     overlay             swarm
ty2yifkoawhr        voteapp_frontend    overlay             swarm


-bash-4.2$ docker stack services voteapp
ID                  NAME                 MODE                REPLICAS            IMAGE                                          PORTS
6gq4smbw5sw4        voteapp_worker       replicated          1/1                 dockersamples/examplevotingapp_worker:latest
j2nbudo31jhd        voteapp_visualizer   replicated          1/1                 dockersamples/visualizer:stable                *:8080->8080/tcp
qkgi2jp3yqy4        voteapp_redis        replicated          1/1                 redis:alpine                           
rg04xelmvf39        voteapp_db           replicated          1/1                 postgres:9.4                           
tujsar2ocfhz        voteapp_result       replicated          1/1                 dockersamples/examplevotingapp_result:before   *:5001->80/tcp
tzejc82b0urb        voteapp_vote         replicated          2/2                 dockersamples/examplevotingapp_vote:before     *:5000->80/tcp

updated the replicas value to 5 from 2 and re-deployed it:

-bash-4.2$ docker stack deploy -c docker-stack.yml voteapp
Updating service voteapp_result (id: tujsar2ocfhzuh144944qnq4p)
Updating service voteapp_worker (id: 6gq4smbw5sw4dyvl5uroy1jpl)
Updating service voteapp_visualizer (id: j2nbudo31jhd6kyjt0b9pizn4)
Updating service voteapp_redis (id: qkgi2jp3yqy459bjsosivilpw)
Updating service voteapp_db (id: rg04xelmvf39sn85gga54gx0f)
Updating service voteapp_vote (id: tzejc82b0urb73vahqyx85ggu)


-bash-4.2$ docker stack services voteapp
ID                  NAME                 MODE                REPLICAS            IMAGE                                          PORTS
6gq4smbw5sw4        voteapp_worker       replicated          1/1                 dockersamples/examplevotingapp_worker:latest
j2nbudo31jhd        voteapp_visualizer   replicated          1/1                 dockersamples/visualizer:stable                *:8080->8080/tcp
qkgi2jp3yqy4        voteapp_redis        replicated          1/1                 redis:alpine                           
rg04xelmvf39        voteapp_db           replicated          1/1                 postgres:9.4                           
tujsar2ocfhz        voteapp_result       replicated          1/1                 dockersamples/examplevotingapp_result:before   *:5001->80/tcp
tzejc82b0urb        voteapp_vote         replicated          5/5                 dockersamples/examplevotingapp_vote:before     *:5000->80/tcp
-bash-4.2$