Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

What preprocessing is needed before feeding the text to the library? #56

Open
MarwaEssam opened this issue Sep 9, 2020 · 1 comment

Comments

@MarwaEssam
Copy link

Here is a link to a news article I am processing :https://www.washingtonpost.com/news/animalia/wp/2017/07/10/teen-camper-wakes-up-to-crunching-noise-and-discovers-his-head-is-inside-bears-mouth/

And this is the result I got from the library after giving it the paragraphs text (without any preprocessing) using the following code:
text = "Teen camper wak........."
title = "Teen camper......."
lead = " Asleep in the mountains....."
date_publish = '2017-07-10 16:17:00'
doc = Document(title, lead, text, date_publish)
doc = extractor.parse(doc)

Here are the results I got for the top answers:

Who-->Teen camper , 1.0 (Dylan 0.9077324478178369)
What-->wakes up to ‘ crunching noise ’ , 1.0
When-->A day later , 0.8240795304744271
Where-->Boulder , Colo. , 0.6813391706278147
Why-->Teen camper , 0.5860000000000001
how-->Asleep in the mountains northwest of Boulder , Colo. , , 1.0

A clear and concise description of your question.
May you please guide me on how to make this result better? What preprocessing is needed? Are there any parameters I can tune? How about the enhancer? I tried to use it as in the example but there is no enhancer package found in the code.
**Versions ** The latest

  • OS: Mac
  • Python Version 3.8
  • news-please Version didn't use it

I am trying to match documents based on the events they mention (event-based linking)

@MarwaEssam
Copy link
Author

What enhancer are you referring to and where is this example?

The one in this file : parse_documents_with_enhancer.py (check the code in the library)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant