Devine - Modular Movie, TV, and Music Archival Software

[cyberschlumpf]

you will need chrome addon https://chromewebstore.google.com/detail/get-cookiestxt-locally/cclelndahbckbenkjhflpd...cc?hl=en&pli=1
name: default.txt put it in cookies folder (your own PATH/cookies/TUBI)
like that cookies

[Attachment 79306 - Click to enlarge]

There is a warning in Devine about the Chrome Add On. You can find the warning on the Devine GitHub website:

https://github.com/devine-dl/devine?tab=readme-ov-file

Scroll down the webpage and there is a section near the bottom that is Cookie File Format and Extensions. There are two alternatives, the chromium one seems just about the same. I have removed the extension and got the alternate version for Chrome. Hopefully the old extension did not do any harm. I felt a little uneasy about it yesterday so I decided to un enable it until I was ready to use it again. I did use it today before I read this but hopefully things will be fine.

I would like to ask for some support regarding the cookie export.
I am trying to download a free TV stream from DISCOVERYplus UK to test the service.

Created an account on DSCP, logged in, installed Chrome extension https://chromewebstore.google.com/detail/get-cookiestxt-locally/cclelndahbckbenkjhflpd...ce=ext_sidebar

Exported cookies to file as DSCP.txt into my cookies folder.
In contrast to the screenshot above there are 44 lines in the cookies export file.

Running devine afterwards gives:

(venv) F:\devine>devine dl -q 720 --s-lang en --sub-format srt DSCP https://www.discoveryplus.com/gb/channel/quest?pc=268

                       / __ \/ ____/ |  / /  _/ | / / ____/
                      / / / / __/  | | / // //  |/ / __/
                     / /_/ / /___  | |/ // // /|  / /___
                    /_____/_____/  |___/___/_/ |_/_____/
                    ⠀
                    v3.3.3 Copyright © 2019-2024 rlaphoenix
                      https://github.com/devine-dl/devine

     Service Config loaded
     Loaded Cdm Widevine CDM: 4445 (L3)
     Loaded 1 Vaults

  ────────────────────────────── Service: DSCP ───────────────────────────────

     Service has no Geofence
     - Invalid Token. Cookies are invalid or may have expired.

Also tried refreshing my session in the webpage, exporting again and retrying once more, within seconds. Same error "Invalid or expired". Browser happily plays the live stream!
Any idea?
There are only cookies of domain ".discoveryplus.com" exported in the file. Would I potentially need other sites in addition in the file?

Also tried changing the config from "downloader: aria2c" to "downloader: curl-impersonate" - still the same problem.

Verified the whole process with a TUBI stream and storing in cookies\TUBI.txt: Everything works as expected. So there should be no issue with the format of the cookies file, it's naming etc.

[SaintyUK]

I'm pretty sure attempting to download the live stream is not possible with the current version of service.
Only show and video:

TITLE_RE = r"^(?:https?://(?:www\.)?discoveryplus\.com(?:/[a-z]{2})?)?/(?P<type>show|video)/(?P<id>[a-z0-9-/]+)"

As for the cookies, the cookie 'st' is the only mandatory one:

# Netscape HTTP Cookie File
# http://curl.haxx.se/rfc/cookie_spec.html
# This is a generated file!  Do not edit.

eu1-prod-direct.discoveryplus.com	FALSE	/	TRUE		st	{VALUE}

Note: Replace {VALUE} with the st cookie value.

[cyberschlumpf]

Thought as well it might be due to the live stream (where should devine stop?) so I found another apparently FTA episode:

https://www.discoveryplus.com/gb/show/ainsleys-festive-flavours

Same issue here.

It seems that extension does indeed NOT export that particular subdomain. It's in my cookie view in the browser but doesn't get exported. Will add. Thank you! Which cookie extension is the actual recommended one? Do you have a direct link for me please?

[cyberschlumpf]

Affirmative! Adding the cookie for

eu1-prod-direct.discoveryplus.com

manually, makes everything work! (A empty line at the end of the file was NOT necessary). Can download the mentioned episode!

Thank you very much!

[ehurdler]

@billybanana:

Looking at the code, it seems like you need to use --movie for movies (pay attention to the placement of the option):

devine dl --no-folder -q 720 -sl en DSNP --movie entity-5850d3bf-947e-4265-8240-8ec13a87946d

Devine has been very cleverly written in that it leaves the door open for additional option parameters to be used if the service author wants/needs it. In this case, @TPD94 has added a --movie option to be used when requesting movies to make it easier to differentiate the type of request to the API.

Whenever an option has been added to a service like this, you need to use it after the service tag, i.e. DSNP --movie.

My DSNP service code and others are all pretty hot garbage. Crunchyroll is my best written one. Most of them I wrote because I was slap happy at finally figuring Devine out. Soon I’ll start rewriting some of them for better manifests.

I am still looking, tracing thru your DSNP and HULU services. They work great! But will only get episodes 1-10 of a season.

What may be a fix for that? I just haven't either figured it out yet, or won't but I am still looking

Example - Lost s01e01

devine dl -q 720 -w S01E10 DSNP entity-466b3994-b574-44f1-88bc-63707507a6cb

This does work for episodes 1-10 on any season, any show thus far. Anything past episode 10, will just process, then do nothing.


[Attachment 81972 - Click to enlarge]

devine dl -q 720 -w S01E11 DSNP entity-466b3994-b574-44f1-88bc-63707507a6cb

[Attachment 81973 - Click to enlarge]

Same goes for HULU, but now I will check Crunchyroll as you said that way your best one. Maybe that will help me.

Otherwise, any ideas?

[PunchDrunkL0ve]

Note: Replace {VALUE} with the st cookie value.

Where do I find the st cookie value? Ta

Edit: Found it!

[mickmars]

I'm getting quite a lot of this lately (I have a Nord subscription set up in my yaml)
Any suggestions to fix it ?
Thanks

❌ Download Failed...
HTTPSConnectionPool(host='vod-hls-uk-live.akamaized.net',
port=443): Max retries exceeded with url

[Tom Saurus]

I ran across today on Pluto TV U.S. that at anytime seasons can disappear of On Demand shows. But they can come back again at a different time in the month.

I am noticing a lot of the U.S. Pluto TV website just in not displaying things and you do a search and it says it is having problems. I tried it both Nord VPN and Windscribe Free 10 GB per month version. The Canadian version of the website is not having any such problems but it has a lot less to chose from. Does anyone else have any problems with the website? I am hoping it is a temporary thing.

[PunchDrunkL0ve]

Any suggestions on how to deal with DSCP constantly shutting the connection? --slow doesn't seem to help.

ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed
connection without response'))

Edit: seems to have sorted itself out -phew

[stabbedbybrick]

@TPD94 has updated the Gitea version where my devine services are hosted, so for those of you who don't use git clone can now download the project as a ZIP file by clicking the three dots in the top right corner. I'd still recommend learning git though, for obvious reasons.

As a side note, I've managed to add N_m3u8DL-RE as an option in my own version of Devine, which works really well so far. Not only is it significantly more stable and faster in most cases, it comes with some nice additional features:

• Certain manifests not supported by Devine can now be used with N_m3uDL-RE instead
• It can be used proxy-less, which lets you to use a free VPN/proxy for API calls while downloading with your system connection
• A fully working progress bar along with file size info
• Using Ctrl+c to stop a download actually works now, where it rarely worked properly before. At least on my end.
• It could potentially allow livestream functionality

[Attachment 82080 - Click to enlarge]

I can't make any promises, but since there seem to be some demand for it, I'll see if I can figure out a good way to make it available for people. Either by making a pull request or something else.

[sipho]

Chalk me down for the version that uses N_m3u8DL-RE please.

[SaintyUK]

Woah stabbedbybrick that sounds incredible!!!

It is a shame that devine-dl is not more actively developed. Have you considered forking and starting devine-brick?

[PunchDrunkL0ve]

@stabbedbybrick having the option to use N_m3u8DL-RE as a downloader would be amazing with fully working progress bar and ctrl-C. I'm up for it too.

[stabbedbybrick]

Have you considered forking and starting devine-brick?

That would actually be the easiest solution. Don't know about that name, though

But it's also tricky, because I really don't want to create a situation where people come to me with bug reports related to the main Devine.
We'll see. I'll figure something out.

[achilles]

Did something change on Tubi? I tried downloading an episode of something and got this error "no scheme supplied."

                       / __ \/ ____/ |  / /  _/ | / / ____/
                      / / / / __/  | | / // //  |/ / __/
                     / /_/ / /___  | |/ // // /|  / /___
                    /_____/_____/  |___/___/_/ |_/_____/
                    ⠀
                    v3.3.3 Copyright © 2019-2024 rlaphoenix
                      https://github.com/devine-dl/devine

     Service Config loaded
     Loaded Cdm Widevine CDM: 4445 (L3)
     Loaded 1 Vaults
     Loaded Hola: 43 Countries

  ────────────────────────────── Service: TUBI ───────────────────────────────

     Service is not Geoblocked in your region
     Authenticated with Service

  ────────────── Series: Leah Remini: It's All Relative (2015) ───────────────

     2 Seasons, 27 Episodes

  ─────── Leah Remini: It's All Relative S01E01 Welcome to the Family ────────


     1 Video
     └── [H.264, SDR] | en | 1280x720 @ 2563 kb/s, 29.970 FPS
         ⠼ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ • --:-- • FAILED
     1 Audio
     └── [AAC] | en | 2.0
         ⠼ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ • --:-- • FAILED
     1 Subtitle
     └── [SRT] | en
          ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ • 00:00 • Downloaded


     ❌ Download Failed...
        Invalid URL '11hcgcbk.mp4': No scheme supplied. Perhaps you meant
     https://11hcgcbk.mp4?

Basically, it's failing trying to get the init segment. It's missing the URL path. The M3U8 from tubi only has the filename in it, no path.
I fixed this by going into the tubi code and doing this in get_tracks:

track.url = os.path.dirname(self.manifest) + "/" + master.segments[0].uri

It was working previously so I'm not sure if something changed on tubi. Or if my edit is a right long-term solution.

stabbedbybrick, thougts?

Also, did the traceback disappear in devine 3.3.3? devine 3.3.2 gives me a traceback, but 3.3.3 does not. Do I have to turn it on somehow?

[stabbedbybrick]

Did something change on Tubi? I tried downloading an episode of something and got this error "no scheme supplied."


❌ Download Failed...
Invalid URL '11hcgcbk.mp4': No scheme supplied. Perhaps you meant
https://11hcgcbk.mp4?
[/code]

Basically, it's failing trying to get the init segment. It's missing the URL path. The M3U8 from tubi only has the filename in it, no path.
I fixed this by going into the tubi code and doing this in get_tracks:

track.url = os.path.dirname(self.manifest) + "/" + master.segments[0].uri

It was working previously so I'm not sure if something changed on tubi. Or if my edit is a right long-term solution.

stabbedbybrick, thougts?

Also, did the traceback disappear in devine 3.3.3? devine 3.3.2 gives me a traceback, but 3.3.3 does not. Do I have to turn it on somehow?

Yeah, you have the right idea. They've removed the base URI from the playlists for some reason. Your solution works, but might cause issues if a stream comes a long where the URI is not missing and you'll end up with an incorrect track.url. I just pushed a fix that should work in both scenarios.

As for the traceback, I'm not sure what the differences are to previous versions but it mostly depends on what the error is.

[Tom Saurus]

I have to do some heavy duty thinking and put some effort to update Devine. Somehow some month ago I was able to get TUBI TV working and was happy with that. I don't know how I did it but Tubi TV at least worked. Since I have so little confidence I didn't touch the program trying to update so there is only about four services in there. I look forward to this new Devine where it works with N_m3u8DL-RE and I never got that proxy with free Windscribe working in Freevine but I better figure it out as PlutoTV and NordVPN are not working together now. I find it works with Windscribe but 10 GB doesn't last very long.

[cyberschlumpf]

>> I usually throw in a "--slow" too, builds in a human timed gap between episodes.

Is this a new option? (--slow). My devine doesn't seem to know this option??

[pauli3]

As a side note, I've managed to add N_m3u8DL-RE as an option in my own version of Devine, which works really well so far. Not only is it significantly more stable and faster in most cases..

+1 for that too, still having to use freevine for some services cause of N_m3u8DL-RE; as the downloaders with devine are significantly slower

[stabbedbybrick]

Out of curiosity, those of you that report consistently slow speeds, are you using Hola proxy per chance? Have you experimented with the '--workers' option? Because while N_m3u8DL-RE is often faster, both Aria and Requests have treated me pretty well on basically all services. At least when it comes to speed. But if you have Hola added as a dependency, it will automatically load if the service require a proxy, and things will turn to syrup.

[Tom Saurus]

Some good news I went to stabbedbybrick devine services and copied the new .init file and config file from the TUBI Service. I found the correct folder this time which was in Users GIT and replaced the files in there and now TUBI TV is working for me and I am very happy about that. Maybe I can add in the other Services I am interested in there by downloading them and dropping them in that Services folder. I can give it a try. I know there is a better way to do all this but this worked.

As for Pluto TV for me when I use the Firefox Browser to look to see what is available at Pluto TV U.S. it will either give me the Canadian version or not display at all and this is with Nord VPN. Now if I use the Waterfox Browser and Nord VPN with a US IP it allows me to see what is available on the Pluto TV U.S. website. What I need to know is in Devine if I add the Pluto TV Service how exactly to configure things that Windscribe can Proxy things so that it can give Devine what it needs without using up my 10 GB free Windscribe. I don't mind using some of course but 10 GB would be gone pretty fast. I tried it in Freevine and it is says the proxy is activated but fails as it says Title None. Thanks in advance for any guidance.

My confidence is very low with Devine; and I feel very stupid and a pain in the butt and also ashamed for asking for guidance but remember there are others like me I assume.

[sipho]

Out of curiosity, those of you that report consistently slow speeds, are you using Hola proxy per chance? Have you experimented with the '--workers' option? Because while N_m3u8DL-RE is often faster, both Aria and Requests have treated me pretty well on basically all services. At least when it comes to speed. But if you have Hola added as a dependency, it will automatically load if the service require a proxy, and things will turn to syrup.

I have workers set to 16 in the devine.yaml file and this variable in my batch file.

set "_options=--slow --no-proxy --no-source -l all"

It just seems to take forever compared to freevine but perhaps I must do a test for content that is available (for me) on both platforms.

[cyberschlumpf]

Sorry, if this has been asked before, but is there a way to let devine reuse what's already there? For example, assume I have downloaded already the video part and decrypted it and then during the download of the audio track some problem occurs (network disconnect or whatever). When I now just restart the same devine command, devine will again also download the video track although this has already been perfectly done.

Or, on one installation I forgot to install shaka-packager so at the very end the whole workflow failed. When I added the binary and restarted the job it again started by downloaded the video and audio which took again 15+ minutes...

[shank99]

Those instances are frustrating! I believe it is an enhancement that has been requested, but hasn't been able to be implemented yet.

What you can do is request the missing track using the -V or -A option and then remux the result using MKVToolNix or ffmpeg, etc.