Skip to content

Refactored the whole scraper

Latest
Compare
Choose a tag to compare
@emartinez-dev emartinez-dev released this 02 Dec 20:24
· 2 commits to master since this release

A lot of stuff was changed in the last commits, I will try to summarize:

  • The stations are persisted now through a JSON file.
  • The stations selection has fuzzy finding features now.
  • The scraper module has been rewritten entirely.
  • The scraping mechanism is not based on headless browser anymore, making it less flaky and lighter, as it only uses the requests library.
  • The bot module is a complete mess, but it works and now persists the user input using Pickle.