Add support for session partitioning of dataflow, support for folder-based multiplexing of processing.

[obsh]

We can have a list of documents with links to visualizations on the index page. Or just render the newest one and provide dropdown to select another document.

[allenday]

Let's keep it simple. Multiple direct links to single reports.

…

On Wed, Feb 13, 2019, 10:15 Alexander Bushkovsky ***@***.***> wrote: We can have a list of documents with links to visualizations on the index page. Or just render the newest one and provide dropdown to select another document. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#57>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAanP0RjVv-uDLFWrpvRUHdiEc2qm5maks5vM3UxgaJpZM4a4cv-> .

[Firedrops]

Related to this, how does sessioning work, if at all, currently? So far our tests have all dumped into the same datastore document, unless we stopped the job and re-deployed it.

[allenday]

currently no user sessions or support for folder-based multiplexing.

[allenday]

multiplexing meaning that we add an additional GroupByKey for the folder name at the beginning of the pipe. This way each of the samples stays as a group, but many samples can run together, then finally get split apart again when written to different firestore collections (based on folder)

[Firedrops]

I see that latest commit has a new index.html with hard-coded links to previous runs. Is it possible to automate the generation of these links?

[Firedrops]

I have realised that index.html always returns Error 502: Bad Gateway nginx/1.14.0 (Ubuntu).

I have also made an attempt to read the firebase documents, in a .js, but I'm not sure how to use it from sunburst.html. So far, this code should render document ids like resultDocument--2019-02-18T02-21-45UTC in plaintext, although it is likely that I have made errors since I could not really test.

const firebaseDocuments = document.querySelector('#firebase_documents')

// create element and render file

function renderUrl(doc){
  let li = document.createElement('li');

  li.setAttribute('doc-id', doc.id);

  firebaseDocuments.appendChild(li);
}

db.collection('new_scanning_species_sequences_statistic').get().then((snapshot) => {
  snapshot.docs.forEach(doc => {
    renderUrl(doc);
  })
})