Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

added loader class and mapping for OEWS data #2

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

ming-afk
Copy link

resolves #1 . Scripts included here.

@ming-afk ming-afk requested a review from thomaslin2020 July 28, 2023 19:32
@trafico-bot trafico-bot bot added the 🔍 Ready for Review Pull Request is not reviewed yet label Jul 28, 2023
@thomaslin2020
Copy link
Collaborator

  • Please modify the code so that instead of reading the excel file locally, read it from the url. This also means that we don't need to add too many custom rules in .gitignore.
  • What's the problem with repetitions in OCC_TITLE? Would O_GROUP be important for our use cases?

@ming-afk
Copy link
Author

  • with the current method I manually downloaded the excel data. The data updates once a year, would it be unnecessary to read the url as we are good for one year now?
  • The data looks like below. For example the "Chief Executives" position has different O_GROUP but all other data being the same. Case where the O_GROUP and other data are both different is rare. But I'll check the meaning of O_GROUP further to ensure I understand it correctly.
Screen Shot 2023-07-28 at 10 20 25 PM

@thomaslin2020
Copy link
Collaborator

I think it's still good to have files downloaded through requests as data files aren't easy to share with github

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🔍 Ready for Review Pull Request is not reviewed yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Backend] Job and Salary Data Collection
2 participants