drm free content batch downloader, widefrog

[Bruce236]

Attached - I am very confused, was I supposed to run code after the command window opens to install the software?

[Bruce236]

Here is how I have the folder setup for the program.

[2nHxWW6GkN1l916N3ayz8HQoi]

I am very confused

Please read (and reread) the first post (from first page) from top to bottom until you understand. I know it's long but it was written specifically like that so people won't ask about things that were detailed there.

[Bruce236]

This is where I am at

[2nHxWW6GkN1l916N3ayz8HQoi]

This is where I am at

It is expected of users to have some basic tech knowledge if they want to use this downloader (I'll be sure to add a disclaimer at the top of the first post). If you look before your comments, people were capable of downloading and figuring it out with the information I gave because they had at least some basic knowledge.

Judging from the fact that you pasted an http URL in the cmd and expected something magic to happen, something tells me you're not tech-savvy enough. In addition in the first post, I said:

I'd really appreciate it if this post remains only a support thread for any eventual relevant issues. For Python-related issues, you can see Stackoverflow, and for CDM issues the sticky threads of this forum are useful.

And unfortunately pip enters into that category. Use google especially if the message displayed is: "pip is not recognized as an internal or external command operable program or batch file".

[Lusiiky]

Hello !

I'm sorry if this has been explained somewhere but... I don't understand one thing: why does the config.json reset every time I try to use widefrog?
Am I missing something?

I only changed what I needed in it: indicated the location of a .wvd and tf1_fr credentials, but everything resets each time I run the command, and I get no results each time.
Did I need to change anything else?

Thanks a lot!
Lusiiky

[2nHxWW6GkN1l916N3ayz8HQoi]

Greetings. When you edit the json file you gotta make sure it remains a valid json file. That means any text remains between quotes. So you should have:

"tf1.fr": {
    "EMAIL": "email",
    "PASSWORD": "password"
}

Instead of:

"tf1.fr": {
    "EMAIL": email,
    "PASSWORD": password
}

I hope this was the issue.

Edit: a general rule, if you see something in double quotes, edit it with what is needed but leave it in the end in the same quotes

[Lusiiky]

I'm sorry if this has been explained somewhere but... I don't understand one thing: why does the config.json reset every time I try to use widefrog?
Am I missing something?

So, a quick update: I've restarted my PC... it's a simple solution, and the config.json no longer resets, so my credentials and the .wvd should work.
I've been to the tf1 website, and I can log in with the credentials I've been given in the config.json, so...

Problem: it still doesn't work, and the --debug doesn't seem to add any information...
Here are the logs:

> widefrog.exe --debug "https://www.tf1.fr/tf1/secret-story/videos/secret-story-2024-quotidienne-du-6-juin-2024-63640017.html"
[INFO] Starting the service initialization stage.
[USER_ERROR]/[APP_ERROR] Failed to download: https://www.tf1.fr/tf1/secret-story/videos/secret-story-2024-quotidienne-du-6-juin-2024-63640017.html. Reason: Failed to initialize the tf1_fr service. Solution: Check the service credentials in the app_files\config.json file. If you are sure everything is right then debug the service.
[INFO] Current progress: [1/1]
[INFO] Starting the collections extraction stage.
[INFO] Starting the media data extraction stage.
[INFO] Finished generating the download commands.
[WARNING] No download commands have been generated.

Do you have an idea, a solution?

Thank you!
Lusiiky

[Lusiiky]

Greetings. When you edit the json file you gotta make sure it remains a valid json file. That means any text remains between quotes. So you should have:

"tf1.fr": {
    "EMAIL": "email",
    "PASSWORD": "password"
}

Instead of:

"tf1.fr": {
    "EMAIL": email,
    "PASSWORD": password
}

I hope this was the issue.

My email/password is correctly quoted.
And my file_path is in the root, so it's only

"CDM_WVD_FILE_PATH": "device.wvd",

I didn't change anything else.

Lusiiky

[gwe202]

This is where I am at

Hi, you need add python to path in Windows

[2nHxWW6GkN1l916N3ayz8HQoi]

Do you have an idea, a solution?

Just making sure, are you using gmail/apple/facebook etc credentials when logging in (are you logging through other sites)? The tool supports only the classic email/password credentials that aren't affiliated with other big email providers. If you're doing this correctly, I'll send you a PM later to properly debug and find the issue.

[Lusiiky]

ust making sure, are you using gmail/apple/facebook etc credentials when logging in (are you logging through other sites)? The tool supports only the classic email/password credentials that aren't affiliated with other big email providers. If you're doing this correctly, I'll send you a PM later to properly debug and find the issue.

Hello!
Um... yes, I use the normal login with my Outlook email and password.
No 2FA, and no connection with Facebook, Google, etc..

[2nHxWW6GkN1l916N3ayz8HQoi]

Issue got solved for tf1.fr login. The fix will be included in the next version. Temporarily, if someone else is having the same issues go to services/tf1_fr.py and look for tf1_fr.LOGIN_URL . Replace that requests post with this updated one:

        response = json.loads(requests.post(
            tf1_fr.LOGIN_URL,
            data={
                "loginID": credentials["EMAIL"],
                "password": credentials["PASSWORD"],
                "APIKey": tf1_fr.API_KEY
            }
        ).content.decode())

Pay close attention to tab indents. The issue was that some specific emails or passwords were messing with the url encoding format. So JSON is used now instead to ignore any possible errors.

[2nHxWW6GkN1l916N3ayz8HQoi]

v1.9.0: + threenow.co.nz

Minor changes:
=> extended the plus.fifa.com service
- you can now download a list of videos that are the result of a filter search from https://www.plus.fifa.com/en/archive
- multi-audio track language is now supported, all the audio mpds are collected and merged into one single mpd file

=> extended the rakuten.tv service
- the popup URL of a show can now be grabbed from any context
- improved the merge mechanism, now multiple mpds with different base URLs can be downloaded at once as long as they have the same decryption keys

=> fixed the tf1.fr login bug

Some known problems:
- The merge mechanism for rakuten may fail for the rare video that has multiple decryption keys spread across multiple mpds (thanks @snake for finding this bug and explaining it). Apparently, this scenario is rare but you should make sure that your audio is properly decrypted if you download from a video with multiple audio tracks. If you get this problem feel free to leave a message. I'm curious how rare it is. I'll have to think of a way to handle this scenario.

By the way, is someone using this tool on a Linux distro and is struggling? I'm thinking of adding support for Linux but I don't know if it's worth it.

[pssh]

This is where I am at

Hi Bruce,

Please check the following guide one how to get Windows set-up:
(Please do not message me and instead ask in relevant thread.)
https://forum.videohelp.com/threads/414487

If you follow up all the instructions then the below will work for you.

Here is what worked for me (in PowerShell = WinKey+x +i):


#v1_9_0

mkdir $home\git\WideFrog_v1_9_0
$ProgressPreference = 'SilentlyContinue'
Invoke-WebRequest https://files.videohelp.com/u/308780/widefrog_v1_9_0.zip -OutFile $home\Downloads\widefrog_v1_9_0.zip
tar -xzvf $home\Downloads\widefrog_v1_9_0.zip -C $home\git\WideFrog_v1_9_0\

cd $home\git\WideFrog_v1_9_0
py -m venv env
.\env\Scripts\activate

pip install beautifulsoup4 html5lib browser_cookie3 numpy pycountry requests Unidecode xmltodict cf_clearance playwright librespot==0.0.9

pip install pywidevine==1.8.0

pip install protobuf==3.20.1

pip install playwright

pyinstaller --noconfirm --onefile --console --add-data "services;services/" --add-data "utils;utils/" --add-data "utils/tools;tools/" --hidden-import "browser_cookie3" --hidden-import "librespot" --hidden-import "bs4" --hidden-import "xmltodict" --hidden-import "pycountry" --hidden-import "html5lib" "widefrog.py"

python widefrog.py

python widefrog.py --help

python widefrog.py --config

#Copy you CDM inside "$home\git\WideFrog_v1_9_0" - ammend your CDM name accordingly..

cp $home\wvd-BackUp\google_aosp_on_ia_emulator_14.0.0_xxxxxxxx_4464_l3.wvd $home\git\WideFrog_v1_9_0\

#Edit "config.json" to add the CDM path

nano $home\git\WideFrog_v1_9_0\app_files\config.json

"CDM_WVD_FILE_PATH": "./google_aosp_on_ia_emulator_14.0.0_xxxxxxxx_4464_l3 .wvd",

python widefrog.py "https://35mm.online/vod/animacja/pani-twardowska"

[2nHxWW6GkN1l916N3ayz8HQoi]

Please check the following guide one how to get Windows set-up

Well done. Nice written guide

[stew]

i've used the dazn user-agent trick with yt-dlp up until now. it stopped working, so i thought i'd try this, but it also fails (at the media data extraction stage). can anyone confirm whether or not widefrog still works on that service?

[2nHxWW6GkN1l916N3ayz8HQoi]

can anyone confirm whether or not widefrog still works on that service?

You shouldn't expect any miracles when it comes to this downloader when a popular trick is kaput. Most of these tricks used by the downloader are based on forum posts made by others (that's why I always give credit where credit is due). You should try this, for me it works:
https://forum.videohelp.com/threads/414381-N_m3u8DL-RE-Not-Able-to-Download-Stream#post2738262

Well done with that find @chris. I honestly have no idea why you need a "magic" user agent to download this content from them. Or how you even get such values and from where. I'll add this in the next version for a dazn fix.

[stew]

can anyone confirm whether or not widefrog still works on that service?

You shouldn't expect any miracles when it comes to this downloader when a popular trick is kaput. Most of these tricks used by the downloader are based on forum posts made by others (that's why I always give credit where credit is due). You should try this, for me it works:
https://forum.videohelp.com/threads/414381-N_m3u8DL-RE-Not-Able-to-Download-Stream#post2738262

Well done with that find @chris. I honestly have no idea why you need a "magic" user agent to download this content from them. Or how you even get such values and from where. I'll add this in the next version for a dazn fix.

thank you both! that works.

i wasn't sure if the code in widefrog did some more tricks in addition to the user-agent trick, or if i had done a user-error.

i assume the user-agent is extracted from phone or tv apps. the user-agent mentioned WebOS; Linux/SmartTV which is the name of LG's smart tv OS / system.

[2nHxWW6GkN1l916N3ayz8HQoi]

i assume the user-agent is extracted from phone or tv apps. the user-agent mentioned WebOS; Linux/SmartTV which is the name of LG's smart tv OS / system.

Oh yeah, definitely. But:
- why is it working for something so specific when dazn is blocking a lot of other user agents
- why can they even block user agent when on your browser you can download that mpd (do you have to do other additional requests so that you could download it with N_m3u8DL-RE/yt-dlp using your real user agent?)
- where do you even start to look if the previous magic user agent stopped working, do you just go randomly on any device and try?

Lots of questions. But I guess beggars can't be choosers. It's good that people tell about these magic agents.

[2nHxWW6GkN1l916N3ayz8HQoi]

v1.10.0: + kijk.nl

Minor changes:
=> fixed the dazn.com service (kudos to @chrisl106)

=> fixed the rakuten.tv service (thanks @snake for testing)
- now all sets of decryption keys are grabbed and merged as well with all the mpds (apparently there's some hidden content for audio in some regions that can be obtained from their api and those had other license url calls with other keys)

[cmonSon]

v1.10.0: + kijk.nl

Appreciate the support for Dutch speaking streaming services!

Small request:

Would it be possible to add a login ability for NPO? Either by using credentials or cookies? The reason being that when being logged in with a Plus account you get access to a different .mpd that includes 720/1080p. The rest of the script would still work and would require no rewriting because NPO gives the same keys for both being logged in and out.

Would only need to be able to log in and download through the updated .mpd basically.

Thanks again for all your work.

[PSXman_uk]

[Attachment 79813 - Click to enlarge]

Got it running on my phone connected to NZ with vpn downloading well impressed thanks

[2nHxWW6GkN1l916N3ayz8HQoi]

Appreciate the support for Dutch speaking streaming services!

You're welcome.

Small request:

I specified in the first post that paid content isn't a focus of this downloader (that includes paid higher resolutions whether or not you have a paid account). I am curious if just the cookies are enough though. It would be funny if their API was that badly designed. Try this patch and overwrite the npo.nl service. You shouldn't expect a better attempt than this.
<url removed, the patch is included from v1.11.0> (zip will be deleted later after testing and confirming)

Got it running on my phone connected to NZ with vpn downloading well impressed thanks

Awesome. Thanks for testing! The downloads launching will be skipped because you're not running it on windows but it's nice to see that everything else works fine.

[cmonSon]

I specified in the first post that paid content isn't a focus of this downloader (that includes paid higher resolutions whether or not you have a paid account). I am curious if just the cookies are enough though. It would be funny if their API was that badly designed. Try this patch and overwrite the npo.nl service. You shouldn't expect a better attempt than this.
https://files.videohelp.com/u/308780/patch.zip (zip will be deleted later after testing and confirming)

Yes, I understand and appreciate you going out of your way but I was curious if simply running the script while logged in would produce the higher res.

I can indeed confirm with your updated patch it grabs the 1080p stream, downloads it and decrypts it successfully!

Thanks again.

[2nHxWW6GkN1l916N3ayz8HQoi]

Thanks again.

You're welcome. They kinda did this to themselves. Shouldn't have used the same API flow for both types of content where only the basic account cookies are relevant. Sloppy job. The downside to this is that I don't know what happens when those cookies expire because they clearly must have some kind of refresh mechanism.

Either the script crashes or their API defaults expired cookies to basic non-account flow. If you find me a free video where you need an account I can add the refresh mechanism after I check the behavior. If not, I'll add this patch in the next version and leave it like this until someone complains about random crashes (then I'll just remove it fully).

[cmonSon]

If you find me a free video where you need an account I can add the refresh mechanism after I check the behavior. If not, I'll add this patch in the next version and leave it like this until someone complains about random crashes (then I'll just remove it fully).

Not sure such a video exists. In my experience everything is playable as long you have a Dutch IP, regardless of being logged in or not.

Just need the Plus account to either watch it in HD or watch it early.

I can have a look around and if you don't hear anything then I couldn't find one.

[2nHxWW6GkN1l916N3ayz8HQoi]

v1.11.0: + play.tv3.lt with h265 support included for the few videos that have it from this site

Minor changes:
=> extended the nba.com service, now you can download lists of videos at once from specific URLs

[2nHxWW6GkN1l916N3ayz8HQoi]

v1.12.0: +viki.com

Minor changes:
=> extended the 35mm.online service, you can download pages of content from a filtered search result

[grzech67]

v1.12.0: +viki.com

Minor changes:
=> extended the 35mm.online service, you can download pages of content from a filtered search result

what command to use to use this new option ??