Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Problems scraping bet365 #41

Open
DjarDjar opened this issue Mar 23, 2022 · 3 comments
Open

Problems scraping bet365 #41

DjarDjar opened this issue Mar 23, 2022 · 3 comments

Comments

@DjarDjar
Copy link

DjarDjar commented Mar 23, 2022

So I did all the ReadME instructions (installed the docker and ran it) and everything ran smoothly until I tried to call the api.competitions() method. What I got as a result of that is just: {}
I think this issue might be cloudflare related but I have no idea how to fix it. Is there anyone else facing the same problem? My code :

from soccerapi.api import ApiBet365
api = ApiBet365()
competitions = api.competitions()
print(competitions)
@DjarDjar
Copy link
Author

DjarDjar commented Mar 24, 2022

UPDATE
So I cloned a local copy of the repository and tried to understand how the code works. Turns out it actually is cloudflare-related. So in the competitions function in the ApiBet365 class I changed the code to only return the table_country variable. Like this: https://prnt.sc/ktqM0kbE2OP9 And what I get as a response is: https://prnt.sc/_ka0Pgpo-OEB So the issue is definitely caused by some type of cloudflare detection. @S1M0N38 maybe you know what could've broken?

@DjarDjar
Copy link
Author

DjarDjar commented Mar 26, 2022

UDATE 2
So looks like the problem is related to the x-net-sync-term. More specifically it looks like you can't have the same x-net-sync-term tokens for different requests. The method provided by the docker doesn't work because of that. Btw am I the only one experiencing this issue? I think If there was a way to check what headers a request has before sending it would easily fix the issue because that way we can detect whenever an x-net-sync term would be sent and just simply not send it but extract it and give it to the api.

@Jmqcooper1
Copy link

Did you ever get this fixed?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants