-
Notifications
You must be signed in to change notification settings - Fork 2
Decision Log
Gabe Sawhney edited this page Apr 8, 2023
·
2 revisions
- DECISION: Google Drive folder naming: My Drive / data / [docid]
- other fields (scrape timestamp, ID of host) are saved as metadata
- DECISION: scraper will send data to database thru POST JSON file with the metadata from newly downloaded RFPs. Basic authentication is required. Data includes public URLs for all files.
- DECISION: scraper will save Ariba html to each call folder, unless it's identical to the previous one (in which case we delete it). It will also save XML chunk as a separate file in the folder, also only if it's different.
- DECISION: we call individual records "calls". Because it's the term the city uses.
- DECISION: the scraper runs weekly
- DECISION: we'll run the scraper, database and API on Azure, in a Docker container.
- DECISION: design documentation will live on github, on the wiki.
- DECISION: design proposals should be submitted as issues, since that offers a way for people to comment.