DOCKER CHEAT SHEET

 

BEST PRACTICES

This is where general Docker best practices and war stories go:

• Docker Documentation
• Docker Cheat Sheet
• Best practices for writing Dockerfiles
• Dockerfile Best Practices
• Dockerfile Best Practices 2
• 
  
• Building Good Docker images
• Write Excellent Dockerfiles
• 
  
• 15-docker-tips-in-5-minutes
• Everyday Hacks Docker
•  
• Docker Image Security Best Practices
• Docker Bench Security
• Docker Engine Security Cheat Sheet
• Docker Security Best Practices for Your Vessel And Container
•  
• Docker tips tricks and tutorials  

DOCKER CHEATSHEET

Docker Registries & Repositories
Login to a Registry
docker login
docker login localhost:8080
Logout from a Registry
docker logout
docker logout localhost:8080
Searching an Image
docker search nginx
docker search --filter stars=3 --no-trunc nginx
Pulling an Image
docker image pull nginx
docker image pull eon01/nginx localhost:5000/myadmin/nginx
Pushing an Image
docker image push eon01/nginx
docker image push eon01/nginx localhost:5000/myadmin/nginx

Running Containers
Create and Run a Simple Container

• Start an ubuntu:latest image
• Bind the port 80 from the CONTAINER to port 3000 on the HOST
• Mount the current directory to /data on the CONTAINER
• Note: on windows you have to change -v ${PWD}:/data to -v "C:\Data":/data

docker container run --name infinite -it -p 3000:80 -v ${PWD}:/data ubuntu:latest
Creating a Container
docker container create -t -i eon01/infinite --name infinite
Running a Container
docker container run -it --name infinite -d eon01/infinite
Renaming a Container
docker container rename infinite infinity
Removing a Container
docker container rm infinite
Updating a Container
docker container update --cpu-shares 512 -m 300M infinite

Starting & Stopping Containers
Starting
docker container start nginx
Stopping
docker container stop nginx
Restarting
docker container restart nginx
Pausing
docker container pause nginx
Unpausing
docker container unpause nginx
Blocking a Container
docker container wait nginx
Sending a SIGKILL
docker container kill nginx
Sending another signal
docker container kill -s HUP nginx
Connecting to an Existing Container
docker container attach nginx

Getting Information about Containers
Running Containers
docker container ls
docker container ls -a
Container Logs
docker logs infinite
Follow Container Logs
docker container logs infinite -f
Inspecting Containers
docker container inspect infinite
docker container inspect --format '{{ .NetworkSettings.IPAddress }}' $(docker ps -q)
Containers Events
docker system events infinite
Public Ports
docker container port infinite
Running Processes
docker container top infinite
Container Resource Usage
docker container stats infinite
Inspecting changes to files or directories on a container’s filesystem
docker container diff infinite

Manipulating Images
Listing Images
docker image ls
Building Images
docker build .
docker build github.com/creack/docker-firefox
docker build - < Dockerfile
docker build - < context.tar.gz
docker build -t eon/infinite .
docker build -f myOtherDockerfile .
curl example.com/remote/Dockerfile | docker build -f - .
Removing an Image
docker image rm nginx
Loading a Tarred Repository from a File
docker image load < ubuntu.tar.gz
docker image load --input ubuntu.tar
Save an Image to a Tar Archive
docker image save busybox > ubuntu.tar
Showing the History of an Image
docker image history
Creating an Image From a Container
docker container commit nginx
Tagging an Image
docker image tag nginx eon01/nginx
Pushing an Image
docker image push eon01/nginx

Networking
Creating Networks

• docker network create -d overlay MyOverlayNetwork
• docker network create -d bridge MyBridgeNetwork
• docker network create -d overlay \ --subnet=192.168.0.0/16 \ --subnet=192.170.0.0/16 \ --gateway=192.168.0.100 \ --gateway=192.170.0.100 \ --ip-range=192.168.1.0/24 \ --aux-address="my-router=192.168.1.5" --aux-address="my-switch=192.168.1.6" \ --aux-address="my-printer=192.170.1.5" --aux-address="my-nas=192.170.1.6" \ MyOverlayNetwork

Removing a Network
docker network rm MyOverlayNetwork
Listing Networks
docker network ls
Getting Information About a Network
docker network inspect MyOverlayNetwork
Connecting a Running Container to a Network
docker network connect MyOverlayNetwork nginx
Connecting a Container to a Network When it Starts
docker container run -it -d --network=MyOverlayNetwork nginx
Disconnecting a Container from a Network
docker network disconnect MyOverlayNetwork nginx
Exposing Ports
Using Dockerfile, you can expose a port on the container using:
EXPOSE
You can also map the container port to a host port using:
e.g.
docker run -p $HOST_PORT:$CONTAINER_PORT --name infinite -t infinite

Security
Guidelines for building secure Docker images

• Prefer minimal base images
• Dedicated user on the image as the least privileged user
• Sign and verify images to mitigate MITM attacks
• Find, fix and monitor for open source vulnerabilities
• Don’t leak sensitive information to docker images
• Use fixed tags for immutability
• Use COPY instead of ADD
• Use labels for metadata
• Use multi-stage builds for small secure images
• Use a linter

More detailed information on Snyk's 10 Docker Image Security Best Practices blog

Cleaning Docker
Removing a Running Container
docker container rm nginx
Removing a Container and its Volume
docker container rm -v nginx
Removing all Exited Containers
docker container rm $(docker container ls -a -f status=exited -q)
Removing All Stopped Containers
docker container rm `docker container ls -a -q`
Removing a Docker Image
docker image rm nginx
Removing Dangling Images
docker image rm $(docker image ls -f dangling=true -q)
Removing all Images
docker image rm $(docker image ls -a -q)
Removing all untagged images
docker image rm -f $(docker image ls | grep "^" | awk "{print $3}")
Stopping & Removing all Containers
docker container stop $(docker container ls -a -q) && docker container rm $(docker container ls -a -q)
Removing Dangling Volumes
docker volume rm $(docker volume ls -f dangling=true -q)
Removing all unused (containers, images, networks and volumes)
docker system prune -f
Clean all
docker system prune -a

Docker Swarm
Initializing the Swarm
docker swarm init --advertise-addr 192.168.10.1
Getting a Worker to Join the Swarm
docker swarm join-token worker
Getting a Manager to Join the Swarm
docker swarm join-token manager
Listing Services
docker service ls
Listing nodes
docker node ls
Creating a Service
docker service create --name vote -p 8080:80 instavote/vote
Listing Swarm Tasks
docker service ps
Scaling a Service
docker service scale vote=3
Updating a Service
docker service update --image instavote/vote:movies vote
docker service update --force --update-parallelism 1 --update-delay 30s nginx
docker service update --update-parallelism 5--update-delay 2s --image instavote/vote:indent vote
docker service update --limit-cpu 2 nginx
docker service update --replicas=5 nginx
Notes:
This work was first published in Painless Docker Course

Dockerfile Instructions

FROM Usage: FROM FROM : FROM @ Information: FROM must be the first non-comment instruction in the Dockerfile. FROM can appear multiple times within a single Dockerfile in order to create multiple images. Simply make a note of the last image ID output by the commit before each new FROM command. The tag or digest values are optional. If you omit either of them, the builder assumes a latest by default. The builder returns an error if it cannot match the tag value. Reference - Best Practices

MAINTAINER Usage: MAINTAINER The MAINTAINER instruction allows you to set the Author field of the generated images. Reference

RUN Usage: RUN (shell form, the command is run in a shell, which by default is /bin/sh -c on Linux or cmd /S /C on Windows) RUN ["", "", ""] (exec form) Information: The exec form makes it possible to avoid shell string munging, and to RUN commands using a base image that does not contain the specified shell executable. The default shell for the shell form can be changed using the SHELL command. Normal shell processing does not occur when using the exec form. For example, RUN ["echo", "$HOME"] will not do variable substitution on $HOME. Reference - Best Practices

CMD Usage: CMD ["","",""] (exec form, this is the preferred form) CMD ["",""] (as default parameters to ENTRYPOINT) CMD (shell form) Information: The main purpose of a CMD is to provide defaults for an executing container. These defaults can include an executable, or they can omit the executable, in which case you must specify an ENTRYPOINT instruction as well. There can only be one CMD instruction in a Dockerfile. If you list more than one CMD then only the last CMD will take effect. If CMD is used to provide default arguments for the ENTRYPOINT instruction, both the CMD and ENTRYPOINT instructions should be specified with the JSON array format. If the user specifies arguments to docker run then they will override the default specified in CMD. Normal shell processing does not occur when using the exec form. For example, CMD ["echo", "$HOME"] will not do variable substitution on $HOME. Reference - Best Practices

LABEL Usage: LABEL = [= ...] Information: The LABEL instruction adds metadata to an image. To include spaces within a LABEL value, use quotes and backslashes as you would in command-line parsing. Labels are additive including LABELs in FROM images. If Docker encounters a label/key that already exists, the new value overrides any previous labels with identical keys. To view an image’s labels, use the docker inspect command. They will be under the "Labels" JSON attribute. Reference - Best Practices

EXPOSE Usage: EXPOSE [ ...] Information: Informs Docker that the container listens on the specified network port(s) at runtime. EXPOSE does not make the ports of the container accessible to the host. Reference - Best Practices

ENV Usage: ENV ENV = [= ...] Information: The ENV instruction sets the environment variable to the value . The value will be in the environment of all “descendant” Dockerfile commands and can be replaced inline as well. The environment variables set using ENV will persist when a container is run from the resulting image. The first form will set a single variable to a value with the entire string after the first space being treated as the - including characters such as spaces and quotes. Reference - Best Practices

ADD Usage: ADD [ ...] ADD ["", ... ""] (this form is required for paths containing whitespace) Information: Copies new files, directories, or remote file URLs from and adds them to the filesystem of the image at the path . may contain wildcards and matching will be done using Go’s filepath.Match rules. If is a file or directory, then they must be relative to the source directory that is being built (the context of the build). is an absolute path, or a path relative to WORKDIR. If doesn’t exist, it is created along with all missing directories in its path. Reference - Best Practices

COPY Usage: COPY [ ...] COPY ["", ... ""] (this form is required for paths containing whitespace) Information: Copies new files or directories from and adds them to the filesystem of the image at the path . may contain wildcards and matching will be done using Go’s filepath.Match rules. must be relative to the source directory that is being built (the context of the build). is an absolute path, or a path relative to WORKDIR. If doesn’t exist, it is created along with all missing directories in its path. Reference - Best Practices

ENTRYPOINT Usage: ENTRYPOINT ["", "", ""] (exec form, preferred) ENTRYPOINT (shell form) Information: Allows you to configure a container that will run as an executable. Command line arguments to docker run will be appended after all elements in an exec form ENTRYPOINT and will override all elements specified using CMD. The shell form prevents any CMD or run command line arguments from being used, but the ENTRYPOINT will start via the shell. This means the executable will not be PID 1 nor will it receive UNIX signals. Prepend exec to get around this drawback. Only the last ENTRYPOINT instruction in the Dockerfile will have an effect. Reference - Best Practices

VOLUME Usage: VOLUME ["", ...] VOLUME [ ...] Creates a mount point with the specified name and marks it as holding externally mounted volumes from native host or other containers. Reference - Best Practices

USER Usage: USER The USER instruction sets the user name or UID to use when running the image and for any RUN, CMD and ENTRYPOINT instructions that follow it in the Dockerfile. Reference - Best Practices

WORKDIR Usage: WORKDIR

Information:

• Sets the working directory for any RUN, CMD, ENTRYPOINT, COPY, and ADD instructions that follow it.
• It can be used multiple times in the one Dockerfile. If a relative path is provided, it will be relative to the path of the previous WORKDIR instruction.

Reference - Best Practices

ARG

Usage:

• ARG [=]

Information:

• Defines a variable that users can pass at build-time to the builder with the docker build command using the --build-arg = flag.
• Multiple variables may be defined by specifying ARG multiple times.
• It is not recommended to use build-time variables for passing secrets like github keys, user credentials, etc. Build-time variable values are visible to any user of the image with the docker history command.
• Environment variables defined using the ENV instruction always override an ARG instruction of the same name.
• Docker has a set of predefined ARG variables that you can use without a corresponding ARG instruction in the Dockerfile.
  • HTTP_PROXY and http_proxy
  • HTTPS_PROXY and https_proxy
  • FTP_PROXY and ftp_proxy
  • NO_PROXY and no_proxy

Reference

ONBUILD

Usage:

• ONBUILD

Information:

• Adds to the image a trigger instruction to be executed at a later time, when the image is used as the base for another build. The trigger will be executed in the context of the downstream build, as if it had been inserted immediately after the FROM instruction in the downstream Dockerfile.
• Any build instruction can be registered as a trigger.
• Triggers are inherited by the "child" build only. In other words, they are not inherited by "grand-children" builds.
• The ONBUILD instruction may not trigger FROM, MAINTAINER, or ONBUILD instructions.

Reference - Best Practices

STOPSIGNAL

Usage:

• STOPSIGNAL

The STOPSIGNAL instruction sets the system call signal that will be sent to the container to exit. This signal can be a valid unsigned number that matches a position in the kernel’s syscall table, for instance 9, or a signal name in the format SIGNAME, for instance SIGKILL.
Reference

HEALTHCHECK

Usage:

• HEALTHCHECK [] CMD (check container health by running a command inside the container)
• HEALTHCHECK NONE (disable any healthcheck inherited from the base image)

Information:

• Tells Docker how to test a container to check that it is still working
• Whenever a health check passes, it becomes healthy. After a certain number of consecutive failures, it becomes unhealthy.
• The that can appear are...
  • --interval= (default: 30s)
  • --timeout= (default: 30s)
  • --retries= (default: 3)
• The health check will first run interval seconds after the container is started, and then again interval seconds after each previous check completes. If a single run of the check takes longer than timeout seconds then the check is considered to have failed. It takes retries consecutive failures of the health check for the container to be considered unhealthy.
• There can only be one HEALTHCHECK instruction in a Dockerfile. If you list more than one then only the last HEALTHCHECK will take effect.
• can be either a shell command or an exec JSON array.
• The command's exit status indicates the health status of the container.
  • 0: success - the container is healthy and ready for use
  • 1: unhealthy - the container is not working correctly
  • 2: reserved - do not use this exit code
• The first 4096 bytes of stdout and stderr from the are stored and can be queried with docker inspect.
• When the health status of a container changes, a health_status event is generated with the new status.

Reference

SHELL

Usage:

• SHELL ["", "", ""]

Information:

• Allows the default shell used for the shell form of commands to be overridden.
• Each SHELL instruction overrides all previous SHELL instructions, and affects all subsequent instructions.
• Allows an alternate shell be used such as zsh, csh, tcsh, powershell, and others.

Reference

VM vs Docker Containers

Docker Commands

Set Environment Settings:

• DOCKER_CERT_PATH=C:\Users\jini\.docker\machine\certs
• DOCKER_HOST=tcp://192.168.99.100:2376
• DOCKER_TLS_VERIFY=1
• DOCKER_TOOLBOX_INSTALL_PATH= C:\Program Files\Docker Toolbox

Docker Lifecycle:

• docker run creates a container.
• docker stop stops it.
• docker start will start it again.
• docker restart restarts a container.
• docker rm deletes a container.
• docker kill sends a SIGKILL to a container.
• docker attach will connect to a running container.
• docker wait blocks until container stops.

If you want to run and then interact with a container, docker start then docker attach to get in.
If you want to poke around in an image, docker run -t -i to open a tty.

Docker Info:

• docker ps -a shows running and stopped containers.
• docker inspect looks at all the info on a container (including IP address).
• docker logs gets logs from container.
• docker events gets events from container.
• docker port shows public facing port of container.
• docker top shows running processes in container.
• docker diff shows changed files in the container’s FS.

Docker Images/Container Lifecycle:

• docker images shows all images.
• docker import creates an image from a tarball.
• docker build creates image from Dockerfile.
• docker commit creates image from a container.
• docker rmi removes an image.
• docker insert inserts a file from URL into image. (kind of odd, you’d think images would be immutable after create)
• docker load loads an image from a tar archive as STDIN, including images and tags (as of 0.7).
• docker save saves an image to a tar archive stream to STDOUT with all parent layers, tags & versions (as of 0.7).

Info

• docker history shows history of image.
• docker tag tags an image to a name (local or registry).

Docker Compose

Define and run multi-container applications with Docker.

• docker-compose --help
• create docker-compose.yml
version: '3'
services:
  eureka:
    restart: always
    build: ./micro1-eureka-server
    ports:
      - "8761:8761"
• docker-compose stop
docker-compose rm -f
docker-compose build
docker-compose up -d
docker-compose start
docker-compose ps
• Scaling containers running a given service
docker-compose scale eureka=3
• Healing, i.e., re-running containers that have stopped
docker-compose up --no-recreate

Docker Hub

Docker.io hosts its own index to a central registry which contains a large number of repositories.

• docker login to login to a registry.
• docker search searches registry for image.
• docker pull pulls an image from registry to local machine.
• docker push pushes an image to the registry from local machine.

Dockerfile

Instructions

•     .dockerignore
•     FROM Sets the Base Image for subsequent instructions.
•     MAINTAINER (deprecated - use LABEL instead)
•     RUN execute any commands in a new layer on top of the current image
•     CMD provide defaults for an executing container.
•     EXPOSE informs Docker that the container listens on the specified network ports at runtime.
•     ENV sets environment variable.
•     ADD copies new files, directories or remote file to container. Invalidates caches.
      Avoid ADD and use COPY instead.
•     COPY copies new files or directories to container. By default this copies as root regardless    of the USER/WORKDIR settings. Use --chown=: to give ownership to another user/group. (Same for ADD.)
•     ENTRYPOINT configures a container that will run as an executable.
•     VOLUME creates a mount point for externally mounted volumes or other containers.
•     USER sets the user name for following RUN / CMD / ENTRYPOINT commands.
•     WORKDIR sets the working directory.
•     ARG defines a build-time variable.
•     ONBUILD adds a trigger instruction when the image is used as the base for another build.
•     STOPSIGNAL sets the system call signal that will be sent to the container to exit.
•     LABEL apply key/value metadata to your images, containers, or daemons.

Tutorial: Flux7’s Dockerfile Tutorial

Examples: Examples

Best Practices: Best to look at http://github.com/wsargent/docker-devenv and the best practices / take 2 for more details.

Volumes:

Docker volumes are free-floating filesystems. They don’t have to be connected to a particular container.

Volumes are useful in situations where you can’t use links (which are TCP/IP only). For instance, if you need to have two docker instances communicate by leaving stuff on the filesystem.

You can mount them in several docker containers at once, using docker run -volume-from

Get Environment Settings
docker run --rm ubuntu env
Delete old containers
docker ps -a | grep 'weeks ago' | awk '{print $1}' | xargs docker rm
Delete stopped containers
docker rm `docker ps -a -q`
Show image dependencies
docker images -viz | dot -Tpng -o docker.png
Original
https://github.com/wsargent/docker-cheat-sheet/blob/master/README.md

Portainer and UI for Docker infra

Installation Portainer:

• You need as pre-requisite to have docker and docker swarm installed
• Then just execute the following command to install the Portainer container
  docker run -d -p 9000:9000 portainer/portainer
• To run portainer on a local instance of the docker engine use the following command :
  docker run -d -p 9000:9000 -v /var/run/docker.sock:/var/run/docker.sock  portainer/portainer
• A public demo is available at => demo.portainer.ioMore deployments options:
  username admin, password tryportainer
  After the password definition, you get a login where you enter the new password.

The next step is, to define the first endpoint, which is guided:

• Here you provide a name (which is shown in the UI), the IP-Address of the docker-machine with the port (you get it from the $env:DOCKER_HOST environment variable).
• The docker-machine is secured by TLS, so you have to provide the certificates.
  The files can be found under your home directory:
  C:\Users\\.docker\machine\certs

• With this UI, you can do a lot of basic stuff on the docker-machine like starting/stopping/killing containers. Pulling Images, maintaining the network part and a lot more.

Admiral - Management UI for Containers (Docker):
VMware Admiral is a highly scalable and very lightweight Container Management platform for deploying and managing container based applications. It is designed to have a small footprint and boot extremely quickly. Admiral is intended to provide automated deployment and life-cycle management of containers.

• Rule-based resource management - Setup your deployment preferences to let Admiral manage container placement.
• Live state updates - Provides a live view of your system.
• Efficient multi-container template management - Enables logical multi-container application deployments.

Installation & Configuration:
Run following command
docker run -d -p 8282:8282 --name admiral vmware/admiral

Click on Add a Host
Before you add host please ensure Remote API is enabled for the host.
Please refer my previous blog - http://bit.ly/2kRQqOc


Fill in details and add the host

• Address – http://:2375
• Placement Zone – Choose default-placementzone
• Login Credentials – you will need to create new and choose that

Now you can manage containers or create new if required.
Following the Admiral Container which we created

For more information refer official Github link - Click Here

UI's for Docker:

As you can see, at the top we have the Docker Engine which would be running on a single host.

You then have your orchestration/scheduler interacting, with typically, several Docker Engines.

• Green; this is a command line tool / interface.
• Blue; Web Interface, this is what we are going to be covering in the post.
• Purple; Desktop App, I have included this as it is typically part of peoples journey into using Docker.

Docker Swarm Tutorial

Docker Swarm is the Docker native clustering solution, which can turn a group of distributed,

docker hosts into a single large virtual server.

• Docker Swarm provides the standard Docker API, and it can communicate with any tool that already works with Docker daemon allowing easy scaling to multiple hosts
• In docker swarm you create one or more managers and worker machines in the cluster .. the manager(s) take care of the orchestration of your deployed services (e.g., Creation/Replication/Assigning tasks to nodes/load balancing/service discovery )
• docker run --rm swarm --help

Step 1: Create cluster machines using docker-machine

docker-machine create --driver virtualbox --virtualbox-memory "3000" master
docker-machine create --driver virtualbox --virtualbox-memory "3000" worker1
docker-machine create --driver virtualbox --virtualbox-memory "3000" worker2

Note: Docker machine can also allow you create these machines on digital ocean, AWS, and other third party cloud services .. they have other drivers .

PS C:\Program Files\Docker Toolbox> docker-machine ls
NAME      ACTIVE   DRIVER       STATE     URL                         SWARM   DOCKER        ERRORS
master    -        virtualbox   Stopped                                       Unknown
worker1   -        virtualbox   Stopped                                       Unknown
worker2   -        virtualbox   Stopped                                       Unknown

Step 2: Start/Show machine IP/SSH into machine:

PS C:\Program Files\Docker Toolbox> docker-machine start master
PS C:\Program Files\Docker Toolbox> docker-machine ip master
192.168.99.101
PS C:\Program Files\Docker Toolbox> docker-machine ssh  master
                        ##         .
                  ## ## ##        ==
               ## ## ## ## ##    ===
           /"""""""""""""""""\___/ ===
      ~~~ {~~ ~~~~ ~~~ ~~~~ ~~~ ~ /  ===- ~~~
           \______ o           __/
             \    \         __/
              \____\_______/
 _                 _   ____     _            _
| |__   ___   ___ | |_|___ \ __| | ___   ___| | _____ _ __
| '_ \ / _ \ / _ \| __| __) / _` |/ _ \ / __| |/ / _ \ '__|
| |_) | (_) | (_) | |_ / __/ (_| | (_) | (__|   <  __/ |
|_.__/ \___/ \___/ \__|_____\__,_|\___/ \___|_|\_\___|_|
Boot2Docker version 17.05.0-ce, build HEAD : 5ed2840 - Fri May  5 21:04:09 UTC 2017
Docker version 17.05.0-ce, build 89658be
docker@master:~$ exit

Step 3: Init the swarm:You have to run this command on a manager machine.

PS C:\Program Files\Docker Toolbox> docker-machine ssh master
docker@master:~$ docker swarm init --advertise-addr 192.168.99.101

Step 4: Configure the worker machines to join the swarm
The below swarm join command is always generated after init swarm command above (include token).
In order to run on both workers I will have to SSH to each one run the command then exit.

docker-machine ssh worker1
docker swarm join \
       --token SWMTKN-1-41e2y83z8t462u4039lk0ugfwnuot5vqzbiv82ug6eadtbnbhl-2lfjygo3ic0nrnhjijffxhy47 
       192.168.99.101:2377
exit

docker-machine ssh worker2
docker swarm join \
       --token SWMTKN-1-41e2y83z8t462u4039lk0ugfwnuot5vqzbiv82ug6eadtbnbhl-2lfjygo3ic0nrnhjijffxhy47 
       192.168.99.101:2377
exit

To add a manager to this swarm, run 'docker swarm join-token manager' and follow
Step 5: Display all cluster nodes configured

PS C:\Program Files\Docker Toolbox> docker-machine ssh master
docker@master:~$ docker node ls
ID                            HOSTNAME            STATUS              AVAILABILITY        MANAGER STATUS
63izegp4x3to7cgoao1j9uqqg *   master              Ready               Active              Leader
svamypaqkur7ai3je5ob2z78t     worker2             Ready               Active
whopzq6ywt2eakrend8ji3hb8     worker1             Ready               Active

Notice the master is configured as leader and the * is because we are currently connected to it
Step 6: Create a network to make our services visible to each other

docker@master:~$ docker network create -d overlay my_network

Step 7: Create services and publish its ports
I've pushed both images to my docker hub account and now I will pull/run them on the cluster.

docker tag bipin:eureka bipingupta007/eureka
docker push bipingupta007/eureka
docker tag bipin:config bipingupta007/config
docker push bipingupta007/config

docker@master:~$ docker service create -p 8761:8761 --name eureka --network my_network bipingupta007/eureka 
docker@master:~$ docker service create -p 8888:8888 --name config --network my_network bipingupta007/config

• Here I've just deployed the services with 1 replicas .. however I could have used --replicas parameter to set the number of replicas of the service.
• You can also see that I've exposed the ports and also specified the network to make sure they can access each other using the service name

Step 8: Display all services

docker@master:~$ docker service ls
ID                  NAME                MODE                REPLICAS            IMAGE                         PORTS
5t42nzl8bvo0        eureka              replicated          2/2                 bipingupta007/eureka:latest   *:8761->8761/tcp
uffvcped9b5o        config              replicated          1/1                 bipingupta007/config:latest   *:8888->8888/tcp

This will display the services and show you many of the replicas has been started and how many are still being prepared .. it should show something like this:
Step 9: Display tasks of a service

docker@master:~$ docker service ps eureka
ID                  NAME                IMAGE                         NODE                DESIRED STATE       CURRENT STATE           ERROR               PORTS
cvppczf7wgfw        eureka.1            bipingupta007/eureka:latest   worker1             Running             Running 2 minutes ago

Step 10: Scale a service
As I've mentioned before I could have set the number of replicas of a service at creation time .. now since I've already started it lets scale one of them to run 5 replicas.

docker@master:~$ docker service scale eureka=3
eureka scaled to 3

docker@master:~$ docker service ls
ID                  NAME                MODE                REPLICAS            IMAGE                         PORTS
5t42nzl8bvo0        eureka              replicated          3/3                 bipingupta007/eureka:latest   *:8761->8761/tcp

docker@master:~$ docker service ps eureka
ID                  NAME                IMAGE                         NODE                DESIRED STATE       CURRENT STATE                ERROR               PORTS
cvppczf7wgfw        eureka.1            bipingupta007/eureka:latest   worker1             Running             Running 8 minutes ago
86vuhj694evg        eureka.2            bipingupta007/eureka:latest   master              Running             Running about a minute ago
a6oi3gfq9pcv        eureka.3            bipingupta007/eureka:latest   worker2             Running             Running about a minute ago

Of course you can either scale up or down.

Step 11: Remove the service

docker@master:~$ docker service rm eureka
docker@master:~$ docker service ls
docker@master:~$ docker ps

and verify that the service was removed.
Step 12: Let's try calling the service
You can actually specify any of the cluster machine IPs not necessarily the manager.
http://192.168.99.101:8761/

Continuous Delivery with Docker on Mesos and Marathon

Full functional development and continuous integration environments

• Here is the full docker-compose.yml file that includes all parts of the system.
• In addition to Jenkins and Docker registry we have Mesos master, single Mesos slave, Mesosphere Marathon and Zookeeper for internal Mesos communication.

docker-compose.yml
===============================================
version: '3'
services:
  zookeeper:
    image: garland/zookeeper
    hostname: "zookeeper"
    container_name: zookeeper
    ports:
      - "2181:2181"
      - "2888:2888"
      - "3888:3888"
  mesos-master:
    image: garland/mesosphere-docker-mesos-master
    hostname: "mesos-master"
    container_name: mesos-master
    environment:
      - MESOS_QUORUM=1
      - MESOS_ZK=zk://192.168.99.100:2181/mesos
      - MESOS_REGISTRY=in_memory
      - MESOS_LOG_DIR=C:/var/log/mesos
      - MESOS_WORK_DIR=C:/var/lib/mesos
    links:
      - zookeeper:zk
    ports:
      - "5050:5050"
  mesos-slave:
    image: garland/mesosphere-docker-mesos-master:latest
    hostname: "mesos-slave-1"
    container_name: mesos-slave-1
    environment:
      - MESOS_HOSTNAME=192.168.99.100
      - MESOS_MASTER=zk://192.168.99.100:2181/mesos
      - MESOS_LOG_DIR=C:/var/log/mesos
      - MESOS_LOGGING_LEVEL=INFO
    entrypoint: mesos-slave
    links:
      - zookeeper:zk
      - mesos-master:master
    ports:
      - "5151:5151"
  marathon:
    image: garland/mesosphere-docker-marathon
    hostname: "marathon"
    container_name: marathon
    environment:
      - MARATHON_MASTER=zk://192.168.99.100:2181/mesos
      - MARATHON_ZK=zk://192.168.99.100:2181/marathon
    command: --master zk://zk:2181/mesos --zk zk://zk:2181/marathon
    links:
      - zookeeper:zk
      - mesos-master:master
    ports:
      - "8080:8080"
###Environment 
===============================================


It is important to note that the Jenkins container now includes a link to Marathon. This is required to be able to post the requests from the Jenkins container to the Marathon container.

Now we can restart the system and see it all up and running:
docker-compose up

A quick introduction to Apache Mesos

Apache Mesos is a centralised fault-tolerant cluster manager. It’s designed for distributed computing environments to provide resource isolation and management across a cluster of slave nodes.
In some ways, Mesos provides the opposite to virtualization:

• Virtualization splits a single physical resource into multiple virtual resources
• Mesos joins multiple physical resources into a single virtual resource

It schedules CPU and memory resources across the cluster in much the same way the Linux Kernel schedules local resources.

A Mesos cluster is made up of four major components:

• ZooKeeper
• Mesos masters
• Mesos slaves
• Frameworks

ZooKeeper
Apache ZooKeeper is a centralized configuration manager, used by distributed applications such as Mesos to coordinate activity across a cluster. Mesos uses ZooKeeper to elect a leading master and for slaves to join the cluster.

Mesos masters
A Mesos master is a Mesos instance in control of the cluster.A cluster will typically have multiple Mesos masters to provide fault-tolerance, with one instance elected the leading master.

Mesos slaves
A Mesos slave is a Mesos instance which offers resources to the cluster.
They are the ‘worker’ instances - tasks are allocated to the slaves by the Mesos master.

Frameworks:

• On its own, Mesos only provides the basic “kernel” layer of your cluster. It lets other applications request resources in the cluster to perform tasks, but does nothing itself.
• Frameworks bridge the gap between the Mesos layer and your applications. They are higher level abstractions which simplify the process of launching tasks on the cluster.

1. Chronos
   
   Chronos is a cron-like fault-tolerant scheduler for a Mesos cluster. You can use it to schedule jobs, receive failure and completion notifications, and trigger other dependent jobs.
2. Marathon
   
   Marathon is the equivalent of the Linux upstart or init daemons, designed for long-running applications. You can use it to start, stop and scale applications across the cluster.

Others :

There are a few other frameworks:

• Aurora - service scheduler
• Hadoop - data processing
• Jenkins - Jenkins slave manager
• Spark - data processing
• Torque - resource manager

The quick start guide

Using Vagrant

Vagrant and the vagrant-mesos Vagrant file can help you quickly build:

• a standalone Mesos instance
• a multi-machine Mesos cluster of ZooKeepers, masters and slaves

Unfortunately, the network configuration is a bit difficult to work with - it uses a private network between the VMs, and SSH tunnelling to provide access to the cluster.
Using Mesosphere and Amazon Web Services

Mesosphere provide Elastic Mesosphere, which can quickly launch a Mesos cluster using Amazon EC2.

Vagrant files to build individual components of a Mesos cluster. It’s a work in progress, but it can already build a working Mesos cluster without the networking issues. It uses bridged networking, with dynamically assigned IPs, so all instances can be accessed directly through your local network.

You’ll need the following GitHub repositories:
ian-kent/vagrant-zookeeper
ian-kent/vagrant-mesos-master
ian-kent/vagrant-mesos-slave

At the moment, a cluster is limited to one ZooKeeper, but can support multiple Mesos masters and slaves.

Each of the instances is also built with Serf to provide decentralized service discovery. You can use serf members from any instance to list all other instances.

To help test deployments, there’s also a MongoDB build with Serf installed:
ian-kent/vagrant-mongodb

Like the ZooKeeper instances, the MongoDB instance joins the same Serf cluster but isn’t part of the Mesos cluster.

Once your cluster is running

You’ll need to install a framework.
Mesosphere lets you choose to install Marathon on Amazon EC2, so that could be a good place to start.
With Marathon or Aurora, you can even run other frameworks in the Mesos cluster for scalability and fault-tolerance.

WHY DOCKER ?

Why Docker ?

COMMUNITY SUPPORT

DOCKER on Single Host

DOCKER-DEVELOPMENT-ENVIRONMENT

 

DOCKER PLATFORM & Related Tools

DOCKER-FLOW