Devine - Modular Movie, TV, and Music Archival Software

[SaLTy]

I have a Debian machine running NordVPN and it works, so your base setup will be fine. I'd chack the ccextractor you have downloaded as its suggesting it might not be compatible with your version of Deb.

What version of ccextractor are you using? I installed this one ccextractor_0.94-dmo4_amd64.deb

Glad to hear.

my cfg file looks like this:

dl:
  sub_format: srt

That's exactly how I had it as well, and have it on my windows machine like that but just doesn't like it on Debian.


Tried downloading from UKTV but goes straight to failing on the subs and says:

❌ Download Failed...
An unexpected error occurred in one of the download workers.
See the error trace above for more information.

Using this command:

devine dl --sub-format srt UKTV https://u.co.uk/shows/jamali-maddix-follow-the-leader/series-1/episode-1/6361425730112

[mkjustuk]

What version of ccextractor are you using? I installed this one ccextractor_0.94-dmo4_amd64.deb

That looks right, my package list shows:

ccextractor 1:0.94-dmo4 amd64

[mkjustuk]

CalledProcessError: Command '[PosixPath('/usr/local/bin/ccextractor'), '-trim',
'-nobom', '-noru', '-ru1', '-o',
PosixPath('/tmp/devine/Subtitle_ccextractor-d4d91e4a_en.srt'),
PosixPath('/tmp/devine/Video_d4d91e4a_repack.mp4')]' returned non-zero exit
status 4.

I checked my directories and I have ccextractor showing in /usr/bin and not in /usr/local/bin
Maybe try checking your paths (echo $PATH) for it's presence and copy it around?

[SaLTy]

CalledProcessError: Command '[PosixPath('/usr/local/bin/ccextractor'), '-trim',
'-nobom', '-noru', '-ru1', '-o',
PosixPath('/tmp/devine/Subtitle_ccextractor-d4d91e4a_en.srt'),
PosixPath('/tmp/devine/Video_d4d91e4a_repack.mp4')]' returned non-zero exit
status 4.

I checked my directories and I have ccextractor showing in /usr/bin and not in /usr/local/bin
Maybe try checking your paths (echo $PATH) for it's presence and copy it around?

I moved it to /usr/bin but it's still having the issues. It's really random as I can download from ALL4 and MY5 using the default downloader (not with aria2c even though that is located in /usr/bin as well) and without adding --sub-format srt. Other sites like STV and UKTV fail straight away.

[grimeire]

I have a Debian machine running NordVPN and it works, so your base setup will be fine. I'd chack the ccextractor you have downloaded as its suggesting it might not be compatible with your version of Deb.

What version of ccextractor are you using? I installed this one ccextractor_0.94-dmo4_amd64.deb

Glad to hear.

my cfg file looks like this:

dl:
  sub_format: srt

That's exactly how I had it as well, and have it on my windows machine like that but just doesn't like it on Debian.


Tried downloading from UKTV but goes straight to failing on the subs and says:

❌ Download Failed...
An unexpected error occurred in one of the download workers.
See the error trace above for more information.

Using this command:

devine dl --sub-format srt UKTV https://u.co.uk/shows/jamali-maddix-follow-the-leader/series-1/episode-1/6361425730112

im getting the same error on windows.


I tired different downloaders(default and aria2c) and different paths to temp/cache etc.. but no luck. tired different VPNs and different vpn endpoints but still the same. The logs files dont seem to shed any light on whats going on.

I confirmed all the prerequisites are installed and path are valid as i can auto complete and run the commands in powershell. I renamed all shaka-packager and ccextractor from the repo names to just shaka-packager.exe and ccextractor.exe




[Attachment 82322 - Click to enlarge]


my config

directories:
    services: 'C:\Users\winvictim\AppData\Local\devine\services'
    WVDs: 'C:\Users\winvictim\AppData\Local\devine\utils\wvd\generic_8159_l3.wvd'


cdm:
    default: generic_8159_l3

#downloader:
#    aria2c

#dl:
#    sub_format: srt
    
key_vaults:
  - type: SQLite
    name: Local Vault
    path: 'C:\Users\winvictim\AppData\Local\devine\key_store.db'

Error i am seein. I dont understand why i am getting error "WindowsPath('C:/Users/WINVIC~1/AppData/Local/Temp/devine')]' returned non-zero
exit status 1."

C:\Users\winvictim\AppData\Local\Programs\Python\Python311\Lib\site-packages │
│ \devine\core\drm\widevine.py:291 in decrypt                                  │
│                                                                              │
│   288 │   │   │   p.wait()                                                   │
│   289 │   │   │                                                              │
│   290 │   │   │   if p.returncode != 0 or had_error:                         │
│ > 291 │   │   │   │   raise subprocess.CalledProcessError(p.returncode, argu │
│   292 │   │   │                                                              │
│   293 │   │   │   path.unlink()                                              │
│   294 │   │   │   if not stream_skipped:                                     │
└──────────────────────────────────────────────────────────────────────────────┘
CalledProcessError: Command
'['input=C:\\Users\\WINVIC~1\\AppData\\Local\\Temp\\devine\\Audio_18c4dafb.mp4,s
tream=0,output=C:\\Users\\WINVIC~1\\AppData\\Local\\Temp\\devine\\Audio_18c4dafb
_decrypted.mp4,output_format=MP4', '--enable_raw_key_decryption', '--keys',
'label=0:key_id=fd3f12e5ce87b118e02111141c6cf48f:key=4f398a28d85a3b1f2543843bb54
5e119,label=1:key_id=84b9d4747155f216cc022107ea9cee2a:key=1c23f8d407882245e0dcb6
cd984eafb3,label=2:key_id=727f6690e69d915c22208caba3c7b089:key=d0d662127a60b09e6
b13b8f16bafa360,label=3:key_id=00000000000000000000000000000000:key=4f398a28d85a
3b1f2543843bb545e119,label=4:key_id=00000000000000000000000000000000:key=1c23f8d
407882245e0dcb6cd984eafb3,label=5:key_id=00000000000000000000000000000000:key=d0
d662127a60b09e6b13b8f16bafa360', '--temp_dir',
WindowsPath('C:/Users/WINVIC~1/AppData/Local/Temp/devine')]' returned non-zero
exit status 1.

     ❌ Download Failed...
        An unexpected error occurred in one of the download workers.
        Binary call failed, Process exit code: 1
        See the error trace above for more information.

devine running

devine.exe dl -A -q 360p ctv "https://www.ctv.ca/shows/reba/pilot-s1e1"

                       / __ \/ ____/ |  / /  _/ | / / ____/
                      / / / / __/  | | / // //  |/ / __/
                     / /_/ / /___  | |/ // // /|  / /___
                    /_____/_____/  |___/___/_/ |_/_____/
                    ⠀
                    v3.3.3 Copyright © 2019-2024 rlaphoenix
                      https://github.com/devine-dl/devine

     Service Config loaded
     Loaded Cdm Widevine CDM: 8159 (L3)
     Loaded 1 Vaults

  ─────────────────────────────── Service: CTV ───────────────────────────────

     Service is not Geoblocked in your region

  ─────────────────────────────── Series: Reba ───────────────────────────────

     1 Season, 1 Episode

  ──────────────────────────── Reba S01E01 Pilot ─────────────────────────────


     1 Audio
     └── [DD] | en | 6.0 | 384 kb/s
         ⠦ - ------------------------------------ • 00:37 • DASH 290.0 kB/s

     Widevine(AAAAU3Bzc2gAAAAA7e+LqXnWSs6jyCfc1R0h7QAAADMIARIQ/T8S5c6HsRjgI
     REUHGz0jxoJYmVsbG1lZGlhIhJmZi0wY2QxZTI5Yy0zMTIyMTA=)
     └── fd3f12e5ce87b118e02111141c6cf48f:4f398a28d85a3b1f2543843bb545e119*
         from Local Vault SQLite

[stabbedbybrick]

@grimeire:
If you read the output from calling shaka-packager, it clearly says that you've downloaded mpd_generator and not packager.

--------------------

PSA: I've updated the MY5 service to no longer make use of any hmac keys. Nothing should change for the end user, but it means I don't have to maintain anything in the background. It just seems silly to use old methods when it's not needed. Just pull the update and you should be ready to roll.

And since I know there are other projects here who're using my github for the hmac, I'll leave it up until the current keys are dead so you have a chance to update your code before it breaks. Feel free to use my code if you want.

[grimeire]

@grimeire:
If you read the output from calling shaka-packager, it clearly says that you've downloaded mpd_generator and not packager.

--------------------

PSA: I've updated the MY5 service to no longer make use of any hmac keys. Nothing should change for the end user, but it means I don't have to maintain anything in the background. It just seems silly to use old methods when it's not needed. Just pull the update and you should be ready to roll.

And since I know there are other projects here who're using my github for the hmac, I'll leave it up until the current keys are dead so you have a chance to update your code before it breaks. Feel free to use my code if you want.

im an idiot.

Thank you for your answer and all your work with the services.

Just seen the first windows release in the list and downloaded that. you need to expand it to get the actual windows release.

For anyone else having the issue make sure you download packager-win-x64.exe. I renamed it to shaka-packager.exe but not sure if thats needed. You have to add the path too.

[PunchDrunkL0ve]

im an idiot.

Thank you for your answer and all your work with the services.

Just seen the first windows release in the list and downloaded that. you need to expand it to get the actual windows release.

For anyone else having the issue make sure you download packager-win-x64.exe. I renamed it to shaka-packager.exe but not sure if thats needed. You have to add the path too.

If it makes you feel better I did exactly that not long ago

[stabbedbybrick]

I've been getting a lot requests for TVNZ so I finally went ahead and added it to the repo.

The problem is that NZ VPNs are borderline unusable, so testing is a nightmare. Feel free to report any bugs and I'll do my best to fix them.

[achilles]

Thx for TVNZ.

Is there a way to limit search results in devine? I can't find anything in the help. Getting 100 search results on some services is rather annoying.

[PunchDrunkL0ve]

Attempting to d/l https://www.bbc.co.uk/programmes/m0023ff5 leads to the following error. Any suggestions? Many thanks.

┌───────────────────── Traceback (most recent call last) ──────────────────────┐
│ in _run_module_as_main:198 │
│ in _run_code:88 │
│ │
│ in <module>:7 │
│ │
│ C:\Users\me\AppData\Roaming\Python\Python312\site-packages\click\core. │
│ py:1157 in __call__ │
│ │
│ C:\Users\me\AppData\Roaming\Python\Python312\site-packages\click\core. │
│ py:1078 in main │
│ │
│ C:\Users\me\AppData\Roaming\Python\Python312\site-packages\click\core. │
│ py:1688 in invoke │
│ │
│ C:\Users\me\AppData\Roaming\Python\Python312\site-packages\click\core. │
│ py:1688 in invoke │
│ │
│ C:\Users\me\AppData\Roaming\Python\Python312\site-packages\click\core. │
│ py:1657 in _process_result │
│ │
│ C:\Users\me\AppData\Roaming\Python\Python312\site-packages\click\core. │
│ py:783 in invoke │
│ │
│ C:\Users\me\AppData\Local\Programs\Python\Python31 2\Lib\site-packages\ │
│ devine\commands\dl.py:301 in result │
│ │
│ 298 │ │ │ │ self.log.info("Authenticated with Service") │
│ 299 │ │ │
│ 300 │ │ with console.status("Fetching Title Metadata...", spinner="dot │
│ > 301 │ │ │ titles = service.get_titles() │
│ 302 │ │ │ if not titles: │
│ 303 │ │ │ │ self.log.error("No titles returned, nothing to downloa │
│ 304 │ │ │ │ sys.exit(1) │
│ │
│ C:\Users\me\AppData\Local\devine\services\iP\__ini t__.py:113 in │
│ get_titles │
│ │
│ 110 │ │ elif data is None: │
│ 111 │ │ │ raise ValueError(f"Metadata was not found - if {pid} is an │
│ 112 │ │ │
│ > 113 │ │ if "Film" in data["labels"]["category"]: │
│ 114 │ │ │ data = self.session.get(self.config["endpoints"]["episodes │
│ 115 │ │ │ if not data.get("episodes"): │
│ 116 │ │ │ │ raise ValueError(f"Metadata was not found for {pid}") │
└───────────────────────────────────────────────── ─────────────────────────────┘
TypeError: 'NoneType' object is not subscriptable

[SaLTy]

I'm still having issues with some of the services on Linux... I'm not sure if it's due to the VPN connection.

Does this look like it is from the logs?

[stabbedbybrick]

Thx for TVNZ.

Is there a way to limit search results in devine? I can't find anything in the help. Getting 100 search results on some services is rather annoying.

I'm afraid not. Would be nice feature, though. It's not always possible to enforce a page limit on the search results for a service.

TypeError: 'NoneType' object is not subscriptable

I just pushed small fix for this. Should work now, until the next thing breaks. I feel like IP is held together by sheer force of will at this point

I'm still having issues with some of the services on Linux... I'm not sure if it's due to the VPN connection.

Does this look like it is from the logs?

Most likely not a VPN issue. When the downloader fails without a proper traceback, it's typically an issue with one of the dependencies.

[PunchDrunkL0ve]

TypeError: 'NoneType' object is not subscriptable

I just pushed small fix for this. Should work now, until the next thing breaks. I feel like IP is held together by sheer force of will at this point

Indeed it does. Thanks a million.

PS Is there any movement towards making the N_m3u8DL-RE enabled version of devine public?

[mickmars]

I seem to be getting this a lot these days, and have to keep retrying until things eventually download -Any suggestions ?

❌ Download Failed...
HTTPSConnectionPool(host='vod-hls-uk-live.akamaized.net',
port=443): Max retries exceeded with url:
/usp/auth/vod/piff_abr_full_hd/2d397d-m00233yx/vf_m00233yx_74bd039b-0f
b8-4f9c-aec2-9188eebfea28.ism/hls/vf_m00233yx_74bd039b-0fb8-4f9c-aec2-
9188eebfea28-audio_eng_1=128000.m3u8 (Caused by
SSLError(SSLEOFError(8, '[SSL: UNEXPECTED_EOF_WHILE_READING] EOF
occurred in violation of protocol (_ssl.c:1000)')))

[SaLTy]

I'm still having issues with some of the services on Linux... I'm not sure if it's due to the VPN connection.

Does this look like it is from the logs?

Most likely not a VPN issue. When the downloader fails without a proper traceback, it's typically an issue with one of the dependencies.

Ah, OK.. I don't suppose there's an easy way to tell which one it could be? It's strange as I can download fine from ALL4, MY5 and ITV but others just don't work.

When trying iP I get this error -

ValueError: Cannot set verify_mode to CERT_NONE when check_hostname is enabled.

I know iP can be a pain for VPNs so not sure if that's what it is.

[stabbedbybrick]

PS Is there any movement towards making the N_m3u8DL-RE enabled version of devine public?

The only way to do it right now would be to create a separate fork, and I'm not so sure I'm up for it. Considering so many people seem to have difficulty using one version of Devine, having another one coexist just sounds a like recipe for disaster.

Ah, OK.. I don't suppose there's an easy way to tell which one it could be? It's strange as I can download fine from ALL4, MY5 and ITV but others just don't work.

When trying iP I get this error -

ValueError: Cannot set verify_mode to CERT_NONE when check_hostname is enabled.

I know iP can be a pain for VPNs so not sure if that's what it is.

In "devine/commands/dl.py", line 548, you can add a print statement that should give you more info:

except Exception as e:  # noqa
    print(e)

As for your other error, you can try downgrading requests:

pip install requests==2.31.0

[F1sav28]

Whatever you decide Stabbedbybrick, it's another thanks from me. We all appreciate the maintenance with the services.

I must admit, personally I've never really noticed the current downloaders to be slow anyway. Typically it's about 3 mins for an episode my side. Now and then, I might have to wait much longer for something like a sport thing off DSCP ot even iP but I consider those to be outliers.

[PunchDrunkL0ve]

PS Is there any movement towards making the N_m3u8DL-RE enabled version of devine public?

The only way to do it right now would be to create a separate fork, and I'm not so sure I'm up for it. Considering so many people seem to have difficulty using one version of Devine, having another one coexist just sounds a like recipe for disaster.

Agreed. A pull request seems to be the way to go, but I understand that would be too much work on your side. 100% your call.

[SaLTy]

Ah, OK.. I don't suppose there's an easy way to tell which one it could be? It's strange as I can download fine from ALL4, MY5 and ITV but others just don't work.

When trying iP I get this error -

ValueError: Cannot set verify_mode to CERT_NONE when check_hostname is enabled.

I know iP can be a pain for VPNs so not sure if that's what it is.

In "devine/commands/dl.py", line 548, you can add a print statement that should give you more info:

except Exception as e:  # noqa
    print(e)

As for your other error, you can try downgrading requests:

pip install requests==2.31.0

Thanks you, I added that but looks like it's still the same. I am yet to downgrade requests but I have attached the log and the output devine shows just incase anyone else is able to spot something more obvious.

[stabbedbybrick]

I must admit, personally I've never really noticed the current downloaders to be slow anyway. Typically it's about 3 mins for an episode my side. Now and then, I might have to wait much longer for something like a sport thing off DSCP ot even iP but I consider those to be outliers.

Most people seem to want N_m3u8DL-RE because they think it's magically faster - which it isn't. There might be a slight increase for services that actively throttle you (I'm looking at you, iPlayer), but almost all speed-related issues are due to proxies or general user errors, which wouldn't be solved by using another downloader. For the services I personally use, like AMZN, DSNP, MAX, etc., I max out my connection on a regular basis using the default requests downloader, so I know it's fine. N_m3u8DL-RE comes with other benefits, though, like I mentioned earlier.

@SaLTy:
Downgrade requests and watch all your troubles go away

[PunchDrunkL0ve]

Most people seem to want N_m3u8DL-RE because they think it's magically faster - which it isn't. There might be a slight increase for services that actively throttle you (I'm looking at you, iPlayer), but almost all speed-related issues are due to proxies or general user errors, which wouldn't be solved by using another downloader. For the services I personally use, like AMZN, DSNP, MAX, etc., I max out my connection on a regular basis using the default requests downloader, so I know it's fine. N_m3u8DL-RE comes with other benefits, though, like I mentioned earlier.

For me personally, it's about stability. Both requests and Aria2c are far less stable than N_m3u8DL-RE on my Win10 box. What's your experience?

[stabbedbybrick]

For me personally, it's about stability. Both requests and Aria2c are far less stable than N_m3u8DL-RE on my Win10 box. What's your experience?

Absolutely. The current Devine downloaders are written to be as "safe" as possible, with the intention of trying to emulate an actual device. That, along with how they handle threading, can cause some irregularities and odd behavior like broken progress bars and difficulty to break out of active downloads. N_m3u8DL-RE on the other hand just goes balls to the wall, and is a smoother experience in that sense. And since N_m is also a parser on its own, it can be used for unsupported manifests like ISM or multi-period content, which is the main reason I wanted to add it.

[SaLTy]

@SaLTy:
Downgrade requests and watch all your troubles go away

Thanks stabbedbybrick, I downgraded and looks like it's working now. Although aria2c doesn't seem to work and iP still has issues but I'm putting that down to VPN as I know they're a pain for that.

Appreciate the help

[pauli3]

Out of curiosity, those of you that report consistently slow speeds, are you using Hola proxy per chance? Have you experimented with the '--workers' option? Because while N_m3u8DL-RE is often faster, both Aria and Requests have treated me pretty well on basically all services. At least when it comes to speed. But if you have Hola added as a dependency, it will automatically load if the service require a proxy, and things will turn to syrup.

Nope, purposely use --no proxy switch with devine. Have tinkered with --workers too, but little to no difference. The issue is likely that N_m3u8DL-RE splits the files into smaller chunks which downloads significantly faster all at once. The reason why some people have speed issues and others dont is due to geographical location and needing more than 1 thread to achieve a faster speed. (oceania on the other side to uk/eu)

[sipho]

Most people seem to want N_m3u8DL-RE because they think it's magically faster - which it isn't. There might be a slight increase for services that actively throttle you (I'm looking at you, iPlayer), but almost all speed-related issues are due to proxies or general user errors, which wouldn't be solved by using another downloader.

I'm in deepest darkest Africa and I can download five (5) episodes of the same show using freevine in the same that devine downloads just one.

no-proxy and 16 threads for devine.

[stabbedbybrick]

All right, I'll take your word for it. I'll see if I can figure something out.

[SB47x]

I've been getting a lot requests for TVNZ so I finally went ahead and added it to the repo.

The problem is that NZ VPNs are borderline unusable, so testing is a nightmare. Feel free to report any bugs and I'll do my best to fix them.

Thanks, works well. I'd been using Ozivine, but it's handy to have as many services as possible in Devine.

[PunchDrunkL0ve]

All right, I'll take your word for it. I'll see if I can figure something out.