Demo

The RAD Lab Demo

The Demo project allows you to run a scala console with access to some special objects and methods that allow you to launch an EC2 cluster (via deploylib) and run a variety of RAD Lab systems software such as a Mesos master and slaves.

Using the "demo" project (e.g. the demo console)

The file SCADS_HOME/demo/src/main/scala/DemoSetup.scala contains the objects/methods that are constitute the demo project. Each component that needs to be run (e.g. Mesos, meta scheduler, Scads director, Webapp director, Rain, Spark) should eventually be runnable via method such as startScadr. In order to run the demo console (and get access to these commands), do the following:

1. Clone the SCADS git repository
2. Download and install SBT
3. Build a "demo" console, use the following commands:
   1. sbt (run this from inside the root SCADS directory, opens an sbt console which puts you at a ">" prompt)
   2. project demo
   3. write-packaged-classpath
4. Open the demo console by:
   1. exiting sbt (or opening a new shell window and cd-ing to SCADS root directory again)
   2. from the SCADS root directory, run bin/mvnconsole
5. Now you should be able to interact with any demo clusters being run by other RAD Labians or set up your own. See the next section for an overview of the commands you can use in the demo console (all the demo project includes deploylib so you have all of deploylib at your disposal in this terminal, plus some).

Deploylib commands

See the deploylib page (especially the section titled "Using deploylib with Mesos") for a list of commands available in deploylib, such as starting mesos masters, slaves, etc.

Demo commands

In order to demonstrate the various software systems/artifacts of the RAD Lab research projects interoperating together, we have created a scala repl environment built on deploylib that makes it easy to launch clusters containing the RAD Lab software stack (Mesos, SCADS, Rain, Chukwa, Spark, deploylib, ...) and then demonstrate their functionality. The following is a sorted list of commands that you should be able to run to setup and interact with the RAD Lab demo.

NOTE: some of these are deploylib commands, they are marked with an *.

1. setupMesosMaster() - Start a mesos master and make it the primary for the demo. Only needs to be run by one person.
2. addSlaves(<NUM_SLAVES>)* - Add at least 6-10 slaves for the various applications (SCADr Director, Rain, the web app, ...)
3. startScadrDirector() - start the SCADS director for the SCADr web app.
4. startScadr() - run the SCADr web app as a Mesos web framework.

Other demo commands/scala objects that you might need while running the demo

• restartServiceScheduler() - Restart the service meta-scheduler on the mesos master. Note this should only be run by the cluster owner, and it will kill all running jobs. Note: If you change code that is part of the ScadrDirector mesos framework or the WebAppDirector framework, you should run this command (and then resetScads also) after you run write-packaged-classpath to get the newest version of your code running on the cluster.
• DemoConfig - Object with val members that hold the hard coded config settings for your demo. E.g. DemoConfig.serviceSchedule
  • DemoConfig.serviceScheduler - Show the actor ID of the running Mesos master.
• resetScads() - Delete the namespaces that ScadrDirector creates in zookeeper so that you can re-run the experiment.

Running your own cluster

All configuration / coordination for the demo is done relative to a location in zookeeper. Thus if you want to set up a separate cluster to experiment with, all you need to do it change zooKeeperRoot in the DemoConfig object to point to /home/<username>/demo instead of /demo. You can then use the setup commands listed above.

Notes on demo integration

• Demo MySQL Logging Logging metrics to MySQL