Skip to content

Latest commit

 

History

History
64 lines (43 loc) · 3.56 KB

digital.gov-scan-upload.js.md

File metadata and controls

64 lines (43 loc) · 3.56 KB
Raw

Digital.gov Site Scanning program - Google Sheet Highlights

This Node.js script downloads a CSV file from a specified URL (weekly-snapshot.csv), renames it with the current date, and uploads its contents to a Google Sheets document. The Google Sheets API is used to handle authentication and data operations.

By presenting the CSV data as a Google Sheet, it is:

  • Easier to see the data aggregated here than it is in a CSV file. Making it more consumable for everyone.
  • A check against changes to the data. Errors may show up if there are changes in the data structure.
  • Useful to highlight those agencies which are falling behind. Making this more visible may encourage improvements. If it isn't monitored it doesn't matter.

Overview

The script performs the following steps:

  1. Downloads a CSV file from the Site Scanning page.
  2. Renames the file with the current date.
  3. Authenticates with Google Sheets using provided credentials.
  4. Uploads the CSV data to a new sheet in the specified Google Sheets document.
  5. Updates an “Introduction” sheet in the Google Sheets document with the title of the newly created sheet.

Advantages

  • Data Aggregation: The data is easier to consume in a Google Sheets format than in a CSV file.
  • Data Integrity: Changes in the data structure can be detected, allowing for error checking. digital.gov-scan-upload.js.mdAccountability: Highlighting agencies that are falling behind can encourage improvements. Monitoring and visibility are key.

Google Sheets Document

The processed data is uploaded to the following Google Sheets document: Google Sheets Link

Installation Instructions

Prerequisites

  • Node.js (v16 or later recommended)
  • npm (Node Package Manager)
  • Google Cloud project with Google Sheets API enabled
  • Credentials JSON file for Google Sheets API

Install

  1. Download the digital.gov-scan-upload.js file
  2. Setup a Google Cloud project
  3. Enable the Google Sheets API
  4. Save your OAuth Config
  5. Ensure you have a recent version of node running
  6. Run node digital.gov-scan-upload.js.md

Maintainer

The maintainer of this script is Mike Gifford. Bugs, features and comments can be left in the issue queue.

Todo

  • Incorporate history for: CMS, Web Metrics
  • Remove older CSV file sheets
  • Create Summary history for all sheets

License

This project is licensed under the GNU General Public License v3.0. You can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program. If not, see https://www.gnu.org/licenses/.

This README file includes installation instructions, a brief overview of the script, the advantages of using the script, and the necessary setup and scheduling details.