아재

# Question

I'm looking for some pros and cons of whether to go with Marathon and Chronos, Docker Swarm or Kubernetes when running Docker containers on DC/OS.

For example, when is it better to use Marathon/Chronos than Kubernetes and vice versa?

Right now I'm mostly into experimenting but hopefully we'll start using one of these services in production after the summer. This may disqualify Docker Swarm since I'm not sure if it'll be production ready by then.

What I like about Docker Swarm is that it's essentially just "Docker commands" and you don't have to learn something completely new. We're already using docker-compose and that will work out of the box with Docker Swarm (at least in theory) so that would be a big plus. My main concern with Docker Swarm is if it'll cover all use cases required to run a system in production.

# Answer

I'll try to break down the unique aspects of each container orchestration framework on Mesos.

Use Docker Swarm if:

• You want to use the familiar Docker API to launch Docker containers on Mesos.
• Swarm may eventually provide an API to talk to Kubernetes (even K8s-Mesos) too.
• See: http://www.techrepublic.com/article/docker-and-mesos-like-peanut-butter-and-jelly/

Use Kubernetes-Mesos if:

• You want to launch K8s Pods, which are groups of containers co-scheduled and co-located together, sharing resources.
• You want to launch a service alongside one or more sidekick containers (e.g. log archiver, metrics monitor) that live next to the parent container.
• You want to use the K8s label-based service-discovery, load-balancing, and replication control.
• See http://kubernetesio.blogspot.com/2015/04/kubernetes-and-mesosphere-dcos.html

Use Marathon if:

• You want to launch Docker or non-Docker long-running apps/services.
• You want to use Mesos attributes for constraint-based scheduling.
• You want to use Application Groups and Dependencies to launch, scale, or upgrade related services.
• You want to use health checks to automatically restart unhealthy services or rollback unhealthy deployments/upgrades.
• You want to integrate HAProxy or Consul for service discovery.
• You want to launch and monitor apps through a web UI or REST API.
• You want to use a framework built from the start with Mesos in mind.

Use Chronos if:

• You want to launch Docker or non-Docker tasks that are expected to exit.
• You want to schedule a task to run at a specific time/schedule (a la cron).
• You want to schedule a DAG workflow of dependent tasks.
• You want to launch and monitor jobs through a web UI or REST API.
• You want to use a framework built from the start with Mesos in mind.

source: https://stackoverflow.com/questions/29198840/marathon-vs-kubernetes-vs-docker-swarm-on-dc-os-with-docker-containers/29815033#29815033

# Question

What exactly is the difference between Apache's Mesos and Google's Kubernetes? I understand both are server cluster management software. Can anyone elaborate where the main differences are - when would which framework be preferred?

Why would you want to use Kubernetes on top of Mesosphere?

# Answer

Kubernetes is an open source project that brings 'Google style' cluster management capabilities to the world of virtual machines, or 'on the metal' scenarios. It works very well with modern operating system environments (like CoreOS or Red Hat Atomic) that offer up lightweight computing 'nodes' that are managed for you. It is written in Golang and is lightweight, modular, portable and extensible. We (the Kubernetes team) are working with a number of different technology companies (including Mesosphere who curate the Mesos open source project) to establish Kubernetes as the standard way to interact with computing clusters. The idea is to reproduce the patterns that we see people needing to build cluster applications based on our experience at Google. Some of these concepts include:

• pods — a way to group containers together
• replication controllers — a way to handle the lifecycle of containers
• labels — a way to find and query containers, and
• services — a set of containers performing a common function.

So with Kubernetes alone you will have something that is simple, easy to get up-and-running, portable and extensible that adds 'cluster' as a noun to the things that you manage in the lightest weight manner possible. Run an application on a cluster, and stop worrying about an individual machine. In this case, cluster is a flexible resource just like a VM. It is a logical computing unit. Turn it up, use it, resize it, turn it down quickly and easily.

With Mesos, there is a fair amount of overlap in terms of the basic vision, but the products are at quite different points in their lifecycle and have different sweet spots. Mesos is a distributed systems kernel that stitches together a lot of different machines into a logical computer. It was born for a world where you own a lot of physical resources to create a big static computing cluster. The great thing about it is that lots of modern scalable data processing application run well on Mesos (Hadoop, Kafka, Spark) and it is nice because you can run them all on the same basic resource pool, along with your new age container packaged apps. It is somewhat more heavy weight than the Kubernetes project, but is getting easier and easier to manage thanks to the work of folks like Mesosphere.

Now what gets really interesting is that Mesos is currently being adapted to add a lot of the Kubernetes concepts and to support the Kubernetes API. So it will be a gateway to getting more capabilities for your Kubernetes app (high availability master, more advanced scheduling semantics, ability to scale to a very large number of nodes) if you need them, and is well suited to run production workloads (Kubernetes is still in an alpha state).

When asked, I tend to say:

1. Kubernetes is a great place to start if you are new to the clustering world; it is the quickest, easiest and lightest way to kick the tires and start experimenting with cluster oriented development. It offers a very high level of portability since it is being supported by a lot of different providers (Microsoft, IBM, Red Hat, CoreOs, MesoSphere, VMWare, etc).

2. If you have existing workloads (Hadoop, Spark, Kafka, etc), Mesos gives you a framework that let's you interleave those workloads with each other, and mix in a some of the new stuff including Kubernetes apps.

3. Mesos gives you an escape valve if you need capabilities that are not yet implemented by the community in the Kubernetes framework.

source: https://stackoverflow.com/questions/26705201/whats-the-difference-between-apaches-mesos-and-googles-kubernetes/26789308#26789308

# Question

I use Ubuntu for development and deployment and have a need for creating an isolated environment.

I am considering either Vagrant or Docker for this purpose. What are the pros and cons, or how do these solutions compare?

# Answer

If your purpose is the isolation, I think Docker is what you want.

Vagrant is a virtual machine manager. It allows you to script the virtual machine configuration as well as the provisioning. However, it is still a virtual machine depending on VirtualBox (or others) with a huge overhead. It requires you to have a hard drive file that can be huge, it takes a lot of ram, and performance may be not very good.

Docker on the other hand uses kernel cgroup and namespacing via LXC. It means that you are using the same kernel as the host and the same file system. You can use Dockerfile with the docker build command in order to handle the provisioning and configuration of your container. You have an example at docs.docker.com on how to make your Dockerfile; it is very intuitive.

The only reason you could want to use Vagrant is if you need to do BSD, Windows or other non-Linux development on your Ubuntu box. Otherwise, go for Docker.

source: https://stackoverflow.com/questions/16647069/should-i-use-vagrant-or-docker-for-creating-an-isolated-environment/21314566#21314566

[Question]

I keep rereading the Docker documentation to try to understand the difference between Docker and a full VM. How does it manage to provide a full filesystem, isolated networking environment, etc. without being as heavy?

Why is deploying software to a docker image (if that's the right term) easier than simply deploying to a consistent production environment?

[Answer]

Docker originally used LinuX Containers (LXC), but later switched to runC (formerly known as libcontainer), which runs in the same operating system as its host. This allows it to share a lot of the host operating system resources. Also, it uses a layered filesystem (AuFS) and manages networking.

AuFS is a layered file system, so you can have a read only part and a write part which are merged together. One could have the common parts of the operating system as read only (and shared amongst all of your containers) and then give each container its own mount for writing.

So, let's say you have a 1 GB container image; if you wanted to use a full VM, you would need to have 1 GB times x number of VMs you want. With Docker and AuFS you can share the bulk of the 1 GB between all the containers and if you have 1000 containers you still might only have a little over 1 GB of space for the containers OS (assuming they are all running the same OS image).

A full virtualized system gets its own set of resources allocated to it, and does minimal sharing. You get more isolation, but it is much heavier (requires more resources). With Docker you get less isolation, but the containers are lightweight (require fewer resources). So you could easily run thousands of containers on a host, and it won't even blink. Try doing that with Xen, and unless you have a really big host, I don't think it is possible.

A full virtualized system usually takes minutes to start, whereas Docker/LXC/runC containers take seconds, and often even less than a second.

There are pros and cons for each type of virtualized system. If you want full isolation with guaranteed resources, a full VM is the way to go. If you just want to isolate processes from each other and want to run a ton of them on a reasonably sized host, then Docker/LXC/runC seems to be the way to go.

For more information, check out this set of blog posts which do a good job of explaining how LXC works.

Why is deploying software to a docker image (if that's the right term) easier than simply deploying to a consistent production environment?

Deploying a consistent production environment is easier said than done. Even if you use tools like Chef and Puppet, there are always OS updates and other things that change between hosts and environments.

Docker gives you the ability to snapshot the OS into a shared image, and makes it easy to deploy on other Docker hosts. Locally, dev, qa, prod, etc.: all the same image. Sure you can do this with other tools, but not nearly as easily or fast.

This is great for testing; let's say you have thousands of tests that need to connect to a database, and each test needs a pristine copy of the database and will make changes to the data. The classic approach to this is to reset the database after every test either with custom code or with tools like Flyway - this can be very time-consuming and means that tests must be run serially. However, with Docker you could create an image of your database and run up one instance per test, and then run all the tests in parallel since you know they will all be running against the same snapshot of the database. Since the tests are running in parallel and in Docker containers they could run all on the same box at the same time and should finish much faster. Try doing that with a full VM.

From comments...

Interesting! I suppose I'm still confused by the notion of "snapshot[ting] the OS". How does one do that without, well, making an image of the OS?

Well, let's see if I can explain. You start with a base image, and then make your changes, and commit those changes using docker, and it creates an image. This image contains only the differences from the base. When you want to run your image, you also need the base, and it layers your image on top of the base using a layered file system: as mentioned above, Docker uses AUFS. AUFS merges the different layers together and you get what you want; you just need to run it. You can keep adding more and more images (layers) and it will continue to only save the diffs. Since Docker typically builds on top of ready-made images from a registry, you rarely have to "snapshot" the whole OS yourself

Source: https://stackoverflow.com/questions/16047306/how-is-docker-different-from-a-virtual-machine