Initial planning phase: efficient data management for persistent workers

[wlandau]

Following up on #561 (comment). My thoughts so far:

• Minimize the data shuffling required to load the dependencies of each target on each persistent worker.
• Maybe assign each target to the worker with the most dependency data already loaded. Here, there are tradeoffs with target runtimes when it comes to load balancing.
• Maybe implement a mechanism that shares memory across workers. Maybe POSIX threads for local parallelism?

@mschubert, is this different from what you have in mind?

[wlandau]

Through #800 and related issues, I am reducing the size of the common data sent to the workers. We could also reduce the data sent over with each individual target and use existing worker data to come up with sensible worker affinities. @mschubert, were you thinking about something similar for #561 (comment)?

[wlandau]

Looks like this is a major focus of clustermq going forward: #933 (comment), mschubert/clustermq#151, mschubert/clustermq#154, mschubert/clustermq#160, mschubert/clustermq#161.

[wlandau]

Will reopen if anything needs to happen in drake.