Are you happy with your logging solution? Would you help us out by taking a 30-second survey? Click here

deimos

Mesos containerizer hooks for Docker

Subscribe to updates I use deimos


Statistics on deimos

Number of watchers on Github 249
Number of open issues 10
Average time to close an issue 19 days
Main language Python
Average time to merge a PR 2 days
Open pull requests 0+
Closed pull requests 2+
Last commit about 5 years ago
Repo Created over 5 years ago
Repo Last Updated over 1 year ago
Size 2.66 MB
Organization / Authormesosphere
Contributors16
Page Updated
Do you use deimos? Leave a review!
View open issues (10)
View deimos activity
View on github
Fresh, new opensource launches 🚀🚀🚀
Trendy new open source projects in your inbox! View examples

Subscribe to our mailing list

Evaluating deimos for your project? Score Explanation
Commits Score (?)
Issues & PR Score (?)

======

deimos

Deimos is a Docker plugin for Mesos, providing external containerization as described in MESOS-816_.

NOTE: Mesos 0.20.0 shipped with built in Docker support and this project is no longer actively maintained. More info on the Docker in Mesos 0.20.0_ docs page.


Installation

For a complete installation walkthrough, see this article_.

Deimos can be installed from the Cheeseshop_.

.. code-block:: bash

pip install deimos

Passing Parameters to Docker

In Mesos, every successful resource offer is ultimately followed up with a TaskInfo that describes the work to be done. Within the TaskInfo is a CommandInfo and within the CommandInfo there is a ContainerInfo (following MESOS-816_). The ContainerInfo structure allows specification of an image URL and container options. For example:

.. code-block:: c

{
  container = ContainerInfo {
    image = "docker:///ubuntu"
    options = ["-c", "10240"]
  }
}

Deimos handles image URLs beginning with docker:/// by stripping the prefix and using the remainder as the image name. The container options are passed to docker run when the task is launched. If a // is found in the options list, all the following arguments will be append to the the end of the run command. This is useful when using an image with an entrypoint defined. For example:

.. code-block:: c

{
  container = ContainerInfo {
    image = "docker:///flynn/slugrunner"
    options = ["//", "start", "web"]
  }
}

If no ContainerInfo is present in a task, Deimos will still containerize it, by using the --default_container_image passed to the slave, or taking a reasonable guess based on the host's distribution and release.

Some options for Docker, like -H, do not apply only to docker run. These options should be set in the Deimos configuration file.

Deimos recognizes Mesos resources that specify ports, CPUs and memory and translates them to appropriate Docker options.


Passing Parameters through Marathon

Marathon has a REST api to submit JSON-formatted requests to run long-running commands.

From this JSON object, the following keys are used by Deimos:

  • container A nested object with details about what Docker image to run

    • image What Docker image to run, it may have a custom registry but must have a version tag
    • options A list of extra options to add to the Docker invocation
  • cmd What command to run with Docker inside the image. Deimos automatically adds /bin/sh -c to the front

  • env Extra environment variables to pass to the Docker image

  • cpus How many CPU shares to give to the container, can be fractional, gets multiplied by 1024 and added with docker run -c

  • mem How much memory to give to the container, in megabytes

.. code-block:: bash

curl -v -X POST http://mesos1.it.corp:8080/v2/apps \
        -H Content-Type:application/json -d '{
    "id": "marketing",
    "container": {
      "image": "docker:///registry.int/marketing:latest",
      "options": ["-v", "/srv:/srv"]
    },
    "cmd": "/webapp/script/start.sh",
    "env": {"VAR":"VALUE"},
    "cpus": 2,
    "mem": 768.0,
    "instances": 2
}'

This turns into a Docker execution line similar to this:

.. code-block:: bash

docker run --sig-proxy --rm \
           --cidfile /tmp/deimos/mesos/10330424-95c2-4119-b2a5-df8e1d1eead9/cid \
           -w /tmp/mesos-sandbox \
           -v /tmp/deimos/mesos/10330424-95c2-4119-b2a5-df8e1d1eead9/fs:/tmp/mesos-sandbox \
           -v /srv:/srv -p 31014:3000 \
           -c 2048 -m 768m \
           -e PORT=31014 -e PORT0=31014 -e PORTS=31014 -e VAR=VALUE \
           registry.int/marketing:latest sh -c "/webapp/script/start.sh"

Logging

Deimos logs to the console when run interactively and to syslog when run in the background. You can configure logging explicitly in the Deimos configuration file.


Configuration

There is an example configuration file in example.cfg which documents all the configuration options. The two config sections that are likely to be most important in production are:

  • [docker]: global Docker options (--host)

  • [log]: logging settings

Configuration files are searched in this order:

.. code-block:: bash

./deimos.cfg
~/.deimos
/etc/deimos.cfg
/usr/etc/deimos.cfg
/usr/local/etc/deimos.cfg

Only one configuration file -- the first one found -- is loaded. To see what Deimos thinks its configuration is, run deimos config.


The State Directory

Deimos creates a state directory for each container, by default under /tmp/deimos, where it tracks the container's status, start time and PID. File locks are maintained for each container to coordinate invocations of Deimos that start, stop and probe the container.

To clean up state directories belonging to exited containers, invoke Deimos as follows:

.. code-block:: bash

deimos state --rm

This task can be run safely from Cron at a regular interval. In the future, Deimos will not require separate invocation of the state subcommand for regular operation.


Configuring Mesos To Use Deimos

Only the slave needs to be configured. Set these options:

.. code-block:: bash

--containerizer_path=/usr/local/bin/deimos --isolation=external

The packaged version of Mesos can also load these options from files:

.. code-block:: bash

echo /usr/local/bin/deimos    >    /etc/mesos-slave/containerizer_path
echo external                 >    /etc/mesos-slave/isolation

.. _from the Cheeseshop: https://pypi.python.org/pypi/deimos

.. _MESOS-816: https://issues.apache.org/jira/browse/MESOS-816

.. _this article: https://mesosphere.io/learn/run-docker-on-mesosphere/

.. _Docker in Mesos 0.20.0: http://mesos.apache.org/documentation/latest/docker-containerizer/

deimos open issues Ask a question     (View All Issues)
  • about 5 years Customize docker run arguments
  • about 5 years Deimos should not override workdir
  • about 5 years Question: How does port pairing work?
  • about 5 years Container flaps between 'Staging' and 'Running' => can't download a package
  • about 5 years More than just docker?
  • about 5 years Mesos environment variables not set when task_info specified
  • over 5 years Mesos with Spark trying to download non-existent libmesos/ubuntu:14.04
  • over 5 years Error when trying to start mesos-slave with bad path for containerizer_path
  • over 5 years Python 2.6 support
  • over 5 years Deimos `state` should perform integrity checks
  • over 5 years Deimos does not GC the state directory
deimos questions on Stackoverflow (View All Questions)
  • Deimos Expecting 'MESOS_SLAVE_PID' in environment variables
deimos list of languages used
Other projects in Python