Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

QA testing: Try scraping other Legistar cities #53

Open
2 tasks
krammy19 opened this issue Nov 8, 2021 · 0 comments
Open
2 tasks

QA testing: Try scraping other Legistar cities #53

krammy19 opened this issue Nov 8, 2021 · 0 comments

Comments

@krammy19
Copy link
Collaborator

krammy19 commented Nov 8, 2021

We know that our scraper works for San Jose, but we should begin testing it out on other cities that also use the Legistar platform. Please determine what, if any, different parameters / arguments need to be included in order to successfully scrape other cities that use Legistar.

Tasks:

  • Design a separate set of launch.json arguments to use the Legistar_Selenium scraper on Sunnyvale, Mountain View and any other cities that you want to try. See here for a list of all the Legistar cities in California.

  • Create a spreadsheet of any errors that occur during the scrape and / or issues.

Note: in order to scrape from cities besides San Jose, you will likely need to change the link text search in the Legistar_selenium scraper from "Memorandum" to "Staff Report". San Jose labels its staff memos as "Memorandum" whereas most other cities call them "Staff reports"

Line 142 in Legistar_Selenium:

memorandum = driver.find_element_by_link_text('Memorandum')

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant