Run tasks natively on Docker

[valeriano-manassero]

Hi everybody,
there's a way to run BOINC2DOCKER apps natively on Linux using Docker without installing Virtualbox?

Thanks.

[davidpanderson]

Not currently. We'd need to write a "Docker wrapper". This would be a good thing to do, if someone has time.

[marius311]

Hi Valeriano,

(author of boinc2docker here)

Currently no native Docker Linux version exists, although I've thought about it. I actually think it has more negatives that positives to be honest, mainly

• Requires getting users to install Docker (whereas a large number already have VBox)
• Running Docker containers effectively requires root, I think most users wouldn't want to be running BOINC apps with root permissions
• Docker does not natively have an option to checkpoint or suspend jobs to disk. With Docker running in VBox, we get this automatically via VBox snapshots.

Of course there are advantages,

• Better performance. However, virtualization in my experience is really less of a performance hit than most people think, only about 10% for our jobs on Cosmology@Home for example.
• Quicker startup time. The VBox Docker jobs take ~10 seconds to boot, its true, there's no way around this with the way the current setup works.

Anyway, for these reasons a native Docker version is not near the top of my todo list, although if anyone wanted to I'd be happy to help incorporate it into boinc2docker.

[marius311]

That said, I left out one pretty cool reason to want native Docker, and that's to be able to run on ARM architecture. Given the number of people I see on Twitter, etc.. showing off their Raspberry Pi clusters running Docker, it'd be a pretty nice "market" to tap into.

[valeriano-manassero]

Hi marius311,
first, thank you for your interest and your work.

I understand that a large number of users have Virtualbox installed but I'm thinking about users like me that have headless linux boxes.

For example, it's easier to install Docker via yum or apt for me (and Windows/Mac users can install Docker 1.12 easly on ther platform).

Since I know well Docker platform, I agree with you on root permissions for Docker deamon while I think volumes can persist jobs to disk (I don't know if they can be used as "checkpoints").

Maybe I can help to write a "Docker Wrapper" so I will dive deep into some docs. If you have any hints, I'm here.

Thanks.

[marius311]

Great, give it a go and don't hesitate to contact me.

The basic structure would be to write a native BOINC app that uses the Docker API to talk to a Docker daemon and set up / run / suspend / etc... the containers, in the same way that vboxwrapper uses the VBox API to do the same for VMs. This should be helpful in deciding what language to write this in / how to talk to the Docker daemon, probably Go or C++ is the way to go.

It'd be ideal if eventually this app ("dockerwrapper"?) could take input files in the exact same format as boinc2docker jobs, which use one BOINC input file per Docker container layer. Although to start it might just be easier to have the users Docker daemon pull the necessary images (this is how boinc2docker worked early on, but its more prone to errors / wastes users' resources).

Other than that there's quite a lot of various resource management type things that vboxwrapper does that would have to be replicated here, but again that's probably not necessary right away.

(Re: checkpointing, you're right these jobs will be able to use volume to checkpoint, but the point with the VBox is jobs is that they can be checkpointed without a single extra line of code from the app-writer, since VM can just be suspended. That won't be the case for these Docker jobs, granted I've seen discussion of native Docker "checkpointing" so maybe we'll get that for free in the future, who knows)

[ChristianBeer]

The dockerwrapper also needs to talk to BOINC so you really are restricted to C++. I think vboxwrapper is a good starting point to also see the BOINC interaction that is required by dockerwrapper. The code of vboxwrapper is here: https://github.com/BOINC/boinc/tree/master/samples/vboxwrapper the documentation is here: http://boinc.berkeley.edu/trac/wiki/VboxApps

It would be very nice and convenient to have the same naming conventions for both. So for example the configuration should go into a docker_job.xml file which contains XML similar to vbox_job.xml.

[Ageless93]

How about what this person wrote? https://boinc.berkeley.edu/dev/forum_thread.php?id=11756

[marius311]

Those are just Docker containers running the BOINC client (which would be great to have something official nevertheless). This issue was about deliverying Docker workunits that would run natively on a volunteers computer who had Docker installed, as opposed to via Virtualbox as is currently possible.

[AenBleidd]

@valeriano-manassero, why closing this? This still looks like a nice feature

[valeriano-manassero]

@valeriano-manassero, why closing this? This still looks like a nice feature

Sorry I misclicked while cleaning some stale issues on my list. Unfortunately this on is old and I never got enough time to investigate in in deep but yes, it should be nice to have it 👍

[AenBleidd]

Closing this in favor of #5512