Added support for gifdeliverynetwork/redgifs in GfycatRipper and Added Soundgasm Ripper #1695
Conversation
- Added an additional check in the getVideosURL function to check for redgifs urls if initial check fails - This may need to change depending on if gfycat changes the way they handle the redirects in their document - Uses previous implemntation found in older releases of ripme
borderline232
changed the title
|
Per @CaptRussia it seems to have trouble with some old legacy gfycats that don't get transferred over to redgifs properly Atm the current implementation is able to scrape most of the newer vids that get transferred from gfycat to redgifs using the mp4Source found in the HTML of the redirected page (click gfycat -> redirects to gifdeliverynetwork -> view source -> find mp4Source). The problem is that sometimes the mp4Source is not valid, but the webmSource is and sometimes even that is not valid. The only reliable source in the HTML is the contentUrl in the script tag which usually links to a mobile version of the mp4 and happens to be lower resolution to the other's mentioned. So I was wondering which method you'd want to use? We can only do one because there really isn't a way to check the validity of each URL until we actually rip it (I believe). Keep mp4Source (mildly reliable + high res) |
|
I'm not familiar with the intricacies of the ripping process that Ripme uses, however, would it not be possible to fallback onto the different versions upon a failed ripme attempt? For instance, it would attempt the mp4 source. If that failed it would move onto attempting the webm source. And failing that, the mobile.mp4 source. Would it be possible to implement such a fallback feature? In theory, I think it would make sense, however, Java isn't my language and so I can't comment on the feasibility of implementing it. Failing that. I'd suggest adding a config option for the user to specify what option they would prefer. That's less than ideal though as it would be adding bloat to the config option that only one ripper would be utilising. For fear of beating a dead horse, I think the fallback implementation would be the best route. It would capture the best quality possible while also offering maximum reliability. Failing that. I'm partial to having maximum reliability and so if I had to I would vote for ripping the mobile.mp4. |
- Created new ripper to scrape soundgasm.net urls and obtain their audio - Added ripper to RipUtils so its deteced in the reddit ripper - Created test for basic and reddit urls and both passed
|
Accidentally forgot I still have a PR open but I added a soundgasm ripper which works fine. @cyian-1756 could you merge this in when you're free thanks |
borderline232
changed the title
- Added a dot in redgifs regex to account for them in usernames and search queries - Replaced fuzzyExists with better library efficient call that check the folder for existing file
|
put 2 of the commits onto the other one contained classes, and mixed 2 unrelated things into one commit. would you mind splitting it? |
|
took out "upport for redgifs support in GfycatRipper" as the tests were failing. |
|
@soloturn yo, you still want me to put the commit in the other fork? Also, to clarify my own fork which I used to create the pr (borderline232/ripme) is a little bit ahead and has bunch of new commits from the master branch of this one, so this PR just keeps adding the stuff that's on mine. |
|
@borderline232 @soloturn has this been fixed the gifdeliverynetwork issue ? If it has been can someone provide the compailed .jar file where it works ? Thanks. Just saw ur builds borderline it works there nice. |
|
@borderline232 i cannot make out which commit we should merge. would be great if you could create a pr for https://github.com/ripmeapp2/ripme/tree/main. |
- Added selfpost download functionality, currently it is fixed and cannot be disabled in the jar - Fixed Redgifs using mobile mp4 in its document and instead uses its api instead to fetch the hd version
- redgifs api blocking api calls so just replaced mobile string
- Fixed extracting comments using the Reddit Api more comments are negligent - Made some quick fixes for gfycat urls redirecting to redgifs, now ripping the mp4 instead of mobile
Category
This change is exactly one of the following (please change
[ ]to[x]) to indicate which:Description
Testing
Required verification:
mvn test(there are no new failures or errors).Optional but recommended: