Mesos Cluster

The INDIGO-DataCloud PaaS relies on Apache Mesos for:

• managed service deployment

• user applications execution

The instantiation of the high-available Mesos cluster is managed by the INDIGO Orchestrator in a fully automated way as soon as a user request described by a TOSCA template is submitted. Once the cluster is up and running, it can be re-used for successive requests.

Mesos is able to manage cluster resources (cpu, mem) providing isolation and sharing across distributed applications (frameworks)

Marathon and Chronos are two powerful frameworks that can be deployed on top of a Mesos Cluster.

Sophisticated two-level scheduling and efficient resource isolation are the key-features of the Mesos middleware that are exploited in the INDIGO PaaS, in order to run different workloads (long-running services, batch jobs, etc) on the same resources while preserving isolation and prioritizing their execution.

INDIGO PaaS uses:

• Marathon to deploy, monitor and scale Long-Running services, ensuring that they are always up and running.

• Chronos to run user applications (jobs), taking care of fetching input data, handling dependencies among jobs, rescheduling

  failed jobs.

Summary:

• Release Notes

  • What's new

  • Supported platforms

  • Deployment Notes

  • Artifacts

  • Documentation

  • Support

Release Notes

What's New

• Support for Ubuntu 16.04

• Mesos upgraded to 1.1.0

• Marathon upgraded to 1.4.1

• Chronos upgraded to 3.0.2

• marathon-consul, mesos-consul, haproxy-consul replaced with marathon-lb

• Support for persistent storage (using rex-ray driver)

• New TOSCA templates for Mesos Cluster:

  • added cluster elasticity

  • deployment on AWS

The supported platforms*

Operating System: Ubuntu 14.04, CentOS 7, Ubuntu 16.04 Cloud Management Frameworks - any

Deployment Notes

• Mesos Clusters can be automatically deployed using the available TOSCA templates. The deployment is performed through ansible recipes

Artifacts

• List of the affected packages and/or containers:*

• Updated ansible roles:

  • indigo-dc.zookeeper

  • indigo-dc.consul

  • indigo-dc.mesos

  • indigo-dc.chronos

  • indigo-dc.marathon

  • indigo-dc.keepalived

• New ansible roles:

  • indigo-dc.marathon-lb](https://github.com/indigo-dc/ansible-role-marathon-lb)

• Deprecated ansible roles:

  • indigo-dc.haproxy-consul

• Updated docker images:

  • indigodatacloud/mesos-master:1.1.0

  • indigodatacloud/marathon:1.4.1

  • indigodatacloud/chronos:3.0.2

Documentation

• Link to the GitBook documentation: https://www.gitbook.com/book/indigo-dc/mesos-cluster/details

Support

• Please use the [INDIGO - DataCloud CatchAll GGUS Support Unit](

  https://wiki.egi.eu/wiki/GGUS:INDIGO_DataCloud_Catch-all_FAQ)