Fix error when pickling HTMLExporter

[kycutler]

Argh. Turns out the HTML exporter had a similar issue to the TemplateExporter (in #1398) which causes it to fail to pickle. This PR fixes the HTML exporter as well.

I checked the other exporters manually and none appeared to have nested functions, but I didn't write tests for all of them.

[MSeal]

Thanks for helping dig into those pickling issues more. It does look like it's still not hope with pickling those functions in the exporter though based on that stack trace

[kycutler]

Yeah my bad, I missed the failed test in Python3.7 on my local. I fixed those errors (including in Python3.6) but now it looks like the cannot pickle 'weakref' object errors may be caused by something in Jinja... I'm wondering if there is a way to refactor these three resources to avoid this, but I'm afraid I'm out of my depth there.

For my specific use case, I only need the output, which is feasible with these current changes:

import nbconvert, nbformat
from concurrent.futures import ProcessPoolExecutor

def do_export(exporter, notebook):
    (output, resources) = exporter.from_notebook_node(notebook)
    return output

if __name__ == "__main"":
  executor = ProcessPoolExecutor()
  exporter = nbconvert.exporters.HTMLExporter()
  nb = nbformat.read('example.ipynb', as_version=4)

  output = executor.submit(do_export, exporter, nb).result() # succeeds

Can this be considered "good enough"? Or do you have any ideas about fixing the resource pickling?

[MSeal]

Try moving those resource functions outside of the class entirely and accept an env argument. Then wrap the resource assignments with a lambda / partial function that supplies the self.environment field as the first argument. I think that would avoid having the full self weak reference present but have the environment and name still be dynamic.

[kycutler]

Hmm, that didn't quite work. It looks like there are references to the exporter within the environment -- for example environment.filters["filter_data_type"].parent, or any of the other filters that inherit from LoggingConfigurable and keep a reference to the parent.