Supporting Data Packages

[rufuspollock]

I always like these nice and simple approaches - back in the day i did a hack with only JS + HTML https://github.com/okfn/datacatalog.js (no longer active). (This sort of morphed into the node-based effort for http://data.okfn.org/ - https://github.com/okfn/data.okfn.org).

I also should shout out to a python equivalent of this project done by some other folks: https://github.com/centraldedados/datacentral - http://okfnlabs.org/blog/2014/08/19/datacentral.html

Anyway, other than the props one thing I wanted to suggest was utilizing data packages in an appropriate manner. For example, generating a nice datapackage.json in each datasets directory to turn it into a a proper Data Package :-)

[timwis]

Thanks for the kind words @rgrp, and for those other resources. Regarding data packages, I'd love to figure out a way to support them in JKAN. There's not an obvious solution for it because Jekyll only allows you to output/render collection items once, and currently we're outputting them as HTML files (the dataset pages themselves). A few options come to mind:

1. Take Jekyll out of the picture entirely per Discussion: Is Jekyll necessary? #50
2. Instead of using .md files with YAML front-matter in the _datasets directory, use data packages JSON files as the source. A bit more difficult for manual editing than YAML of course.
3. Implement a more complex build process per this comment so we can use a Jekyll plugin
4. We could pretty easily generate a single JSON file containing an array of all of the datapackage.json objects, just as we did with data.json
5. The dataset HTML page template can actually render the datapackage.json inside a <script> tag...it may be possible to add a download button, or at least a way to copy & paste the json

[Floppy]

I'm loading and rendering info from a datapackage in https://github.com/SomethingNewUK/somethingnewuk.github.io/blob/master/about/finances/index.md at the moment, if that helps at all. See http://somethingnew.org.uk/about/finances/ for the rendered version.

I do want to turn some of this stuff into plugins or common includes at least, to make it a bit tidier.

[loleg]

With Datacentral no longer supported, I do not see any stable alternatives to JKAN at the moment for hosting a DMS statically. It would be great to see progress on this issue, so: bump

[rufuspollock]

@loleg you can host a DMS statically with PortalJS https://portaljs.org. See e.g. this example https://github.com/datopian/portaljs/tree/main/examples/fivethirtyeight

PS: if you have questions come chat on the DataHub discord server https://discord.gg/xfFDMPU9dC

[loleg]

I was not aware that fully static sites (ie. GitHub Pages) are possible with Next.js? The docs specify a NodeJS server requirement.

Also, is it possible to add more than just a single Data Package? The example does not make it clear, and I can't find a doc to that effect.

I'll post my questions on Discord if you wish to chat.

[rufuspollock]

@loleg sorry for the slow reply - i got behind on github notifications.

I was not aware that fully static sites (ie. GitHub Pages) are possible with Next.js? The docs specify a NodeJS server requirement.

Yes it's called SSG (static site generation) and is totally possible with NextJS - in fact it is a basis for portaljs and flowershow https://flowershow.app

Also, is it possible to add more than just a single Data Package? The example does not make it clear, and I can't find a doc to that effect.

Yes, you can just do a single data package - and you can also publish single data packages with the new DataHub Cloud https://datahub.io

That example is about doing a catalog.

I'll post my questions on Discord if you wish to chat.

That would be great - in datahub channel!