Devine - Modular Movie, TV, and Music Archival Software

[stabbedbybrick]

Thanks for last correction.
ITV problem this time - https://www.itv.com/watch/the-mummy/2a6561/2a6561a0001

83323

This issue lies with ITV. They've added subtitles that don't exist, and it doesn't look like they work in the web player either.

You can skip the subtitles with:

-V -A

[phased]

[

This issue lies with ITV. They've added subtitles that don't exist, and it doesn't look like they work in the web player either.

Thanks for the work-around. But you only find there are no subs after the download fails. Any way to trap and ignore the non-fetch of subs instead?

[SE14man]

This show from ITV https://www.itv.com/watch/daily-mirror-pride-of-britain-awards/L2604/L2604a0013 gave me this message: "No titles returned. Nothing to download".
I've tried different combinations

devine dl ITV https://www.itv.com/watch/daily-mirror-pride-of-britain-awards/L2604/L2604a0013

devine dl -w S00 ITV https://www.itv.com/watch/daily-mirror-pride-of-britain-awards/L2604

devine dl -w S00E00 ITV https://www.itv.com/watch/daily-mirror-pride-of-britain-awards/L2604

but without success.

It's doesn't have proper labels so the script currently fails to pick it up. I'll take a proper look at it when I have some time.

In the meantime, here's the 1080p manifest + keys (expires in ~6 hours):

https://itvpnpctv.blue.content.itv.com/L2604-0013-001/22/1/VAR038-V2/L2604-0013-001_22_1_VAR038-V2.ism/.mpd?filter=%28%28type%3D%3D%22video%22%26%26DisplayHeight%3C%3D1080%29%7C%7C%28type%21%3D%22video%22%29%29&hdnea=st%3D1730566839~exp%3D1730588439~acl%3D/L2604-0013-001/%2A~hmac%3D8d8890fda933dcaa233a77ebb8caf316b961813aaa6451a6656306c2bb162bc4

bebe3dd86a484024968c1065213f0840:0fba83e1e9903794e8387940ca300aee
5561910329d74581b9c209d5eab75901:ee6b85d6711a1c07754fb93b5de38035

Hey stabbed - Hope all is well its been a while!

Do you know if ITV is still L1 for 1080p or has there been a workaround found?

Cheers!

[stabbedbybrick]

Hey stabbed - Hope all is well its been a while!

Do you know if ITV is still L1 for 1080p or has there been a workaround found?

Cheers!

Heyo!

When I last checked, there was still a rare chance you could get 1080p with L3 by messing with the "featureset" parameters in the playlist request. And that was mostly for random episodes of old shows. If there's another trick to it, I unfortunately haven't found it.

[Tom Saurus]

I don't know how I ever got Devine to finally work for me so I get trepadacious to fool around with it. Well anyway I have been using it for TUBI and decided to add CBC because for whatever reason YT-DLP is failing for me when it comes to CBC. It downloads but fails at the putting the audio and video together stage aka muxing. So it seems I have CBC set up in Devine as I copied the CBC Service and put it in the correct folder and set up my credentials. I did a list titles line for the TV Show Heartland and that turned out fine but I keep getting an error when I input this line:

devine dl -q 540 CBC S18E04-S18E06 https://gem.cbc.ca/heartland

It gives me this error:

Error: Got unexpected extra argument (https://gem.cbc.ca/heartland)

Thank you in advance for giving me some guidance and for taking mercy on a twit such as me.

[the-equalizer]

you missed out the -w, haven't you?

devine dl -q 540 -w S18E04-S18E06 CBC https://gem.cbc.ca/heartland

I don't know how I ever got Devine to finally work for me so I get trepadacious to fool around with it. Well anyway I have been using it for TUBI and decided to add CBC because for whatever reason YT-DLP is failing for me when it comes to CBC. It downloads but fails at the putting the audio and video together stage aka muxing. So it seems I have CBC set up in Devine as I copied the CBC Service and put it in the correct folder and set up my credentials. I did a list titles line for the TV Show Heartland and that turned out fine but I keep getting an error when I input this line:

devine dl -q 540 CBC S18E04-S18E06 https://gem.cbc.ca/heartland

It gives me this error:

Error: Got unexpected extra argument (https://gem.cbc.ca/heartland)

Thank you in advance for giving me some guidance and for taking mercy on a twit such as me.

[Tom Saurus]

Thank you for your post. I did try with -w and Devine rejected it. I did go back a few pages and this works but it will be a 1080p file; something about the -q Devine is not liking.

devine dl -w S18e04 cbc https://gem.cbc.ca/heartland

Update: It downloaded a huge 1080p video file but failed with CC Extractor which I should probably replace with an earlier version.

Further update: Your line seems to be working but I am going to stop the download as I know it is going to ultimately fail. My CC Extractor when I select properties doesn't tell me what version it is; but it must be the wrong one.

[ehurdler]

Thank you for your post. I did try with -w and Devine rejected it. I did go back a few pages and this works but it will be a 1080p file; something about the -q Devine is not liking.

devine dl -w S18e04 cbc https://gem.cbc.ca/heartland

Update: It downloaded a huge 1080p video file but failed with CC Extractor which I should probably replace with an earlier version.

Further update: Your line seems to be working but I am going to stop the download as I know it is going to ultimately fail. My CC Extractor when I select properties doesn't tell me what version it is; but it must be the wrong one.

I am having the same issue with CBC with CCExtractor, which I have tried

devine dl -q 1080 -a DD+ -V -A -w S01E01 CBC https://gem.cbc.ca/plan-b

Which Outputs

devine dl -q 1080 -a DD+ -V -A -w S01E01 CBC https://gem.cbc.ca/plan-b

                       / __ \/ ____/ |  / /  _/ | / / ____/
                      / / / / __/  | | / // //  |/ / __/
                     / /_/ / /___  | |/ // // /|  / /___
                    /_____/_____/  |___/___/_/ |_/_____/
                    ⠀
                    v3.3.3 Copyright © 2019-2024 rlaphoenix
                      https://github.com/devine-dl/devine

     Service Config loaded
     Loaded Cdm Widevine CDM: 4464 (L3)
     Loaded 1 Vaults
     Loaded Hola: 43 Countries

  ─────────────────────────────── Service: CBC ───────────────────────────────

     Service is not Geoblocked in your region
      + Using cached login tokens
      + Using cached access tokens
     Authenticated with Service

  ────────────────────────────── Series: Plan B ──────────────────────────────

     2 Seasons, 12 Episodes

  ────────────────────────────── Plan B S01E01 ───────────────────────────────


     1 Video
     └── [H.264, SDR] | en | 1920x1080 @ 6811 kb/s, 29.970 FPS
          ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ • 02:55 • Downloaded
     1 Audio
     └── [DD+] | en | 6.0
          ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ • 04:42 • Downloaded

     Track downloads finished in 4m49s
╭───────────────────── Traceback (most recent call last) ──────────────────────╮
│ in _run_module_as_main:198                                                   │
│ in _run_code:88                                                              │
│                                                                              │
│ in <module>:7                                                                │
│                                                                              │
│ C:\Users\**user**\git\devine-333-pip\env\Lib\site-packages\click\core.py:11 │
│ 57 in __call__                                                               │
│                                                                              │
│ C:\Users\**user**\git\devine-333-pip\env\Lib\site-packages\click\core.py:10 │
│ 78 in main                                                                   │
│                                                                              │
│ C:\Users\**user**\git\devine-333-pip\env\Lib\site-packages\click\core.py:16 │
│ 88 in invoke                                                                 │
│                                                                              │
│ C:\Users\**user**\git\devine-333-pip\env\Lib\site-packages\click\core.py:16 │
│ 88 in invoke                                                                 │
│                                                                              │
│ C:\Users\**user**\git\devine-333-pip\env\Lib\site-packages\click\core.py:16 │
│ 57 in _process_result                                                        │
│                                                                              │
│ C:\Users\**user**\git\devine-333-pip\env\Lib\site-packages\click\core.py:78 │
│ 3 in invoke                                                                  │
│                                                                              │
│ C:\Users\**user**\git\devine-333-pip\env\Lib\site-packages\devine\commands\ │
│ dl.py:595 in result                                                          │
│                                                                              │
│   592 │   │   │   │   │   │   │   video_track = title.tracks.videos[video_tr │
│   593 │   │   │   │   │   │   │   track_id = f"ccextractor-{video_track.id}" │
│   594 │   │   │   │   │   │   │   cc_lang = title.language or video_track.la │
│ ❱ 595 │   │   │   │   │   │   │   cc = video_track.ccextractor(              │
│   596 │   │   │   │   │   │   │   │   track_id=track_id,                     │
│   597 │   │   │   │   │   │   │   │   out_path=config.directories.temp / con │
│   598 │   │   │   │   │   │   │   │   │   id=track_id,                       │
│                                                                              │
│ C:\Users\**user**\git\devine-333-pip\env\Lib\site-packages\devine\core\trac │
│ ks\video.py:300 in ccextractor                                               │
│                                                                              │
│   297 │   │   out_path = Path(out_path)                                      │
│   298 │   │                                                                  │
│   299 │   │   try:                                                           │
│ ❱ 300 │   │   │   subprocess.run([                                           │
│   301 │   │   │   │   binaries.CCExtractor,                                  │
│   302 │   │   │   │   "-trim",                                               │
│   303 │   │   │   │   "-nobom",                                              │
│                                                                              │
│ C:\Users\**user**\AppData\Local\Programs\Python\Python312\Lib\subprocess.py │
│ :571 in run                                                                  │
│                                                                              │
│    568 │   │   │   raise                                                     │
│    569 │   │   retcode = process.poll()                                      │
│    570 │   │   if check and retcode:                                         │
│ ❱  571 │   │   │   raise CalledProcessError(retcode, process.args,           │
│    572 │   │   │   │   │   │   │   │   │    output=stdout, stderr=stderr)    │
│    573 │   return CompletedProcess(process.args, retcode, stdout, stderr)    │
│    574                                                                       │
╰──────────────────────────────────────────────────────────────────────────────╯
CalledProcessError: Command
'[WindowsPath('C:/Users/**user**/bin/ccextractor.EXE'), '-trim', '-nobom',
'-noru', '-ru1', '-o',
WindowsPath('C:/Users/**user**/AppData/Local/Temp/devine/Subtitle_ccextractor-6d
7beeae_en.srt'),
WindowsPath('C:/Users/**user**/AppData/Local/Temp/devine/Video_6d7beeae_repack.m
p4')]' returned non-zero exit status 3221225477.

I do have ccextractor.EXE in C:/Users/**user**/bin/ccextractor.EXE as well as C:\Program Files (x86)\CCExtractor

Should this not be trying to exclude subs?

[PSXman_uk]

This is the version recommended

CCExtractor detailed version info
        Version: 0.93
        Git commit: c7ebd45d9fec39dcdd6dc2312f1a9c89838e2b85
        Compilation date: 2021-08-16
        File SHA256: e6a3667322aaa2243f30aa9fe781d259a550966d302595efd2289a44a4fd6f97
Libraries used by CCExtractor
        Tesseract Version: 4.00.00alpha
        Leptonica Version: leptonica-1.74 (Dec 31 2016, 10:56:23) [MSC v.1900 LIB Release x86]
        libGPAC Version: 1.0.1
        zlib: 1.2.11
        utf8proc Version: 2.4.0
        protobuf-c Version: 1.3.1
        libpng Version: 1.6.37
        FreeType
        libhash
        nuklear
        libzvbi

[Attachment 83353 - Click to enlarge]

[ehurdler]

This is the version recommended

CCExtractor detailed version info
        Version: 0.93
        Git commit: c7ebd45d9fec39dcdd6dc2312f1a9c89838e2b85
        Compilation date: 2021-08-16
        File SHA256: e6a3667322aaa2243f30aa9fe781d259a550966d302595efd2289a44a4fd6f97
Libraries used by CCExtractor
        Tesseract Version: 4.00.00alpha
        Leptonica Version: leptonica-1.74 (Dec 31 2016, 10:56:23) [MSC v.1900 LIB Release x86]
        libGPAC Version: 1.0.1
        zlib: 1.2.11
        utf8proc Version: 2.4.0
        protobuf-c Version: 1.3.1
        libpng Version: 1.6.37
        FreeType
        libhash
        nuklear
        libzvbi

[Attachment 83353 - Click to enlarge]

Ill give it a shot, after I posted, I do see I have CCExtractor 0.94.

[PSXman_uk]

This is the version recommended

CCExtractor detailed version info
        Version: 0.93
        Git commit: c7ebd45d9fec39dcdd6dc2312f1a9c89838e2b85
        Compilation date: 2021-08-16
        File SHA256: e6a3667322aaa2243f30aa9fe781d259a550966d302595efd2289a44a4fd6f97
Libraries used by CCExtractor
        Tesseract Version: 4.00.00alpha
        Leptonica Version: leptonica-1.74 (Dec 31 2016, 10:56:23) [MSC v.1900 LIB Release x86]
        libGPAC Version: 1.0.1
        zlib: 1.2.11
        utf8proc Version: 2.4.0
        protobuf-c Version: 1.3.1
        libpng Version: 1.6.37
        FreeType
        libhash
        nuklear
        libzvbi

[Attachment 83353 - Click to enlarge]

Ill give it a shot, after I posted, I do see I have CCExtractor 0.94.

https://files.videohelp.com/u/33133/ccextractor.zip

[Tom Saurus]

Thanks for the zip file. I am giving it a try now.

Update: It worked and the 540p file has the bonus of chapter marks which comes in handy. Thank you to all those who helped. Also I feel some confidence about adding in some other services.

[Tom Saurus]

It has been hit and miss with CBC. Some episodes are downloading completely and being muxed and turning out terrific. Others are stalling when it comes to downloading the audio and then just ending.

[ehurdler]

This is the version recommended

CCExtractor detailed version info
        Version: 0.93
        Git commit: c7ebd45d9fec39dcdd6dc2312f1a9c89838e2b85
        Compilation date: 2021-08-16
        File SHA256: e6a3667322aaa2243f30aa9fe781d259a550966d302595efd2289a44a4fd6f97
Libraries used by CCExtractor
        Tesseract Version: 4.00.00alpha
        Leptonica Version: leptonica-1.74 (Dec 31 2016, 10:56:23) [MSC v.1900 LIB Release x86]
        libGPAC Version: 1.0.1
        zlib: 1.2.11
        utf8proc Version: 2.4.0
        protobuf-c Version: 1.3.1
        libpng Version: 1.6.37
        FreeType
        libhash
        nuklear
        libzvbi

[Attachment 83353 - Click to enlarge]

Ill give it a shot, after I posted, I do see I have CCExtractor 0.94.

https://files.videohelp.com/u/33133/ccextractor.zip

This worked perfectly for me as well! Thank you for the direct link/zip as well!

[WhiskeyJack]

It has been hit and miss with CBC. Some episodes are downloading completely and being muxed and turning out terrific. Others are stalling when it comes to downloading the audio and then just ending.

The download process for CBC at default is, download video, extract CC subs from video, download audio
CBC has CC subs embedded in the video file, not as a separate stream the manifest can pull.
So, after the video is downloaded the cbc service instructs the usage of ccextractor, which has a bug in the most recent version.

To prove this, use the workers command and grab everything
--workers 4 --downloads 3

Roll back to version .93 and Devine will work correctly.

cextractor --version
CCExtractor 0.93, Carlos Fernandez Sanz, Volker Quetschke.
Teletext portions taken from Petr Kutalek's telxcc
--------------------------------------------------------------------------
CCExtractor detailed version info
        Version: 0.93
        Git commit: Unknown
        Compilation date: 2024-10-19
        File SHA256: Could not open file
Libraries used by CCExtractor
        libGPAC Version: 1.0.1
        zlib: 1.2.11
        utf8proc Version: 2.4.0
        protobuf-c Version: 1.3.1
        libpng Version: 1.6.37
        FreeType 
        libhash
        nuklear
        libzvbi

[Tom Saurus]

It has been hit and miss with CBC. Some episodes are downloading completely and being muxed and turning out terrific. Others are stalling when it comes to downloading the audio and then just ending.

The download process for CBC at default is, download video, extract CC subs from video, download audio
CBC has CC subs embedded in the video file, not as a separate stream the manifest can pull.
So, after the video is downloaded the cbc service instructs the usage of ccextractor, which has a bug in the most recent version.

To prove this, use the workers command and grab everything
--workers 4 --downloads 3

Roll back to version .93 and Devine will work correctly.

cextractor --version
CCExtractor 0.93, Carlos Fernandez Sanz, Volker Quetschke.
Teletext portions taken from Petr Kutalek's telxcc
--------------------------------------------------------------------------
CCExtractor detailed version info
        Version: 0.93
        Git commit: Unknown
        Compilation date: 2024-10-19
        File SHA256: Could not open file
Libraries used by CCExtractor
        libGPAC Version: 1.0.1
        zlib: 1.2.11
        utf8proc Version: 2.4.0
        protobuf-c Version: 1.3.1
        libpng Version: 1.6.37
        FreeType 
        libhash
        nuklear
        libzvbi

Thank you for your post. I had better results today and was able to get the episodes I didn't get yesterday. I think perhaps it was my Internet speed's fault. I have DSL and sometimes it goes in and out. My Nord VPN runs out tomorrow. I might go with Windscribe this time.

[SaLTy]

Does anyone know the best way to download a series on iPlayer?

The episodes all have a unique pid and if you do the series pid it doesn't follow the /episodes/pid needed for the service to recognise the correct URL.

If I run

devine dl iP https://www.bbc.co.uk/programmes/m0024qb9

it says KeyError: 'episode'. Is it best to do them one by one or create a script and add each URL to automate each episode?

[stabbedbybrick]

@SaLTy:

Update the service. This was fixed a while back.

[iamghost]

devine dl -w s01 iP m0024qb9

[SaLTy]

@SaLTy:

Update the service. This was fixed a while back.

Oops thought I was on the latest one. Thanks, working now

[SummerArk]

All,

What are some software tools that I could/should/must use that will help me in my making of a devine service?

For example, what are some network tools?

[stabbedbybrick]

All,

What are some software tools that I could/should/must use that will help me in my making of a devine service?

For example, what are some network tools?

What tools you need mostly depends on whether you're using the web or app API. If it's for web, you can honestly do just fine with web developer tools in your browser. If it's for an app, you'll need a network sniffer along with Frida and possibly some unpinning and anti-root scripts.

A few of the more commonly used network tools:

• HTTPToolkit
• Charles Proxy
• Postman
• Burp Suite
• mitmproxy
• Reqable

[SummerArk]

Thanks stabbedbybrick,

I will download, install, and use those tools.

I have had only had some experience with the following:

• Slight experience with some Google Chrome extensions mentioned in this forum
• Slight experience with Google Developer Tools
• Slight experience with Postman
• Slight experience with Wireshark
• Slight and more experience with Telerik Fiddler

At my "start" of trying to write a Service, I started to realize, that the case seems, that I need better tools than what I have had experience using.

[remmy64]

Is stv in devine down I keep getting errors

[PSXman_uk]

Is stv in devine down I keep getting errors

url and error ?

[remmy64]

IndexError: list index out of range
https://www.svtplay.se/the-pirate-bay
or this JSONDecodeError: Expecting value: line 2 column 1 (char 1)

[PunchDrunkL0ve]

I think the service is for STV UK only?

[WhiskeyJack]

IndexError: list index out of range
https://www.svtplay.se/the-pirate-bay
or this JSONDecodeError: Expecting value: line 2 column 1 (char 1)

That's not STV, that's SVT in sweden.
STV is Scottish TV

[WhiskeyJack]

[QUOTE=WhiskeyJack;2756687]

IndexError: list index out of range
https://www.svtplay.se/the-pirate-bay
or this JSONDecodeError: Expecting value: line 2 column 1 (char 1)

That's not STV, that's SVT in sweden.
STV is Scottish TV
The link for THe Pirate Bay documntary on SVT is not drm protected.

use yt-dlp
yt-dlp "https://www.svtplay.se/video/edPgaJG/the-pirate-bay/avsnitt-1?video=visa"

As for the language options in that video, the 4 audio and two sub tracks, I'll leave that for you to read up in the manpages.
https://man.archlinux.org/man/extra/yt-dlp/yt-dlp.1.en

[Tom Saurus]

My Nord VPN ran out but I am staving off renewing. I took some advice here and the Proton Free VPN is working for two of the U.S. streaming websites I use Freevine and Devine for. Of course Roku U.S. doesn't work with Nord or Proton VPNs. But I can use Windscribe Free 10 GB to get what I want over time. I have never been able to grasp the proxy way but maybe someday I will figure it out. I want to say thanks once again to stabbedbybrick for all that he has done and I really appreciate it. I read if one gets the Eset Nod 32 Anti Virus with VPN that the Company uses Windscribe VPN as their VPN. I have not went that route because it is very expensive to go to that level.

STV is a terrific service. I recommend "Taggart" and "Pie In The Sky" as I am fan of Mystery Shows. When it comes to the Scottish shows or other U.K. shows it is nice to have subtitles as sometimes the accent is quite thick and some characters talk fast.