Tag policies based on comparison spreadsheet

[gregjd]

Use the policy-comparison spreadsheet to find the exact quotes from policies, match them to the documents live on this site, and tag those sections as such.

A simpler level of this concept would be to have each policy have in its metadata a list of the guidelines it makes use of, using the comparison spreadsheet to see which cells have text in them.

[gregjd]

Complication: Some cells have info filled in that represents the opposite of having satisfied that recommendation. Makes things more difficult.

There's also the old problem that what counts as satisfying the recommendations has varied depending on who inputted the policy and when.

Given these, it's unclear if it's worth the human effort for this step, versus just using that human effort to tag the actual sections. But, it may be possible to do a "lighter" version that doesn't require as much human effort (besides writing the code 😉) that will take values containing "n/a" or "opposite" and count that as not satisfying the recommendation.

[gregjd]

Update: I more strongly think it's not worth doing the intermediate step of just saying whether or not a policy meets a guideline based on some script that read info from the comparison spreadsheet. Instead, partially coming out of the conversation with @katyaabaz this morning, I think we should do these steps:

1. Comparison of list of policies: Compare (via script or manually) OpenDataPolicies.org to the comparison spreadsheet to see if there are any policies that are in one but not the other. Any that are in the spreadsheet but not the site should be added to the site. (Ones that are on the site but not in the spreadsheet should be noted but don't need immediate action.) Also, we should grab any extra metadata that may be in the spreadsheet.
2. Automatic tagging: Write a script to grab the information from the comparison spreadsheet and hopefully tag many of the matching sections.
3. Manual tagging: Because of the way the spreadsheet has been filled out, some entries will not match with a section of the policy on the site (for example, if additional comments from us were also put in the entry, so not just the quote), and so these will need to be tagged manually. The script could generate a list of the non-matching entries, and then people could go tag.
4. Creation of a new comparison page: Create a page on OpenDataPolicies.org that mimics or replaces the functionality of the comparison spreadsheet, using the tagged policies on the site as a source.

After these are done, the comparison spreadsheet would cease to be useful, so we'd stop updating it, put a prominent notice on it with a link to the new site, and update our hyperlinks in the various places we link to it.

I think we can do at least 1, 2, and 3 by mid-June.

We also need to discuss whether or not we are updating our Guidelines any time soon. If so, that may impact both the timing of this and also how we go about it. (I already think each guideline should have a keyword and the matching sections in policies should be tagged with that guidelines' keyword and not the guideline's number.)