Decryption: The Last Crusade

[A_n_g_e_l_a]

Decryption: The last Crusade

This post is a follow on to 'Decryption: The Temple of Doom'
And 'Decryption: The Dungeon of Despair' and to a lesser extent Downloading-and-decryption-on-your-Android-phone

By now, if you've have followed those threads you are hopefully using the tools therein and happily getting keys, downloading, decrypting and merging files into playable media. In otherwords you've escpaped both The Temple and The Dungeon and find yourself in pleasant green sunlit meadows.

But the tools you are using for downloading and decryption may have a shelf-life or be eclipsed by other tools. There is a whisper that yt-dlp may be a target for a take-down notice. Thus to remain online the '--allow-u' option to download encrypted media may disappear as a yt-dlp option. There is a newer, alternative downloader available. So have no fear.

Mp4decrypt is slower than its alternative - shaka-packager and ffmpeg can have some of its many long-winded duties eased too, by mkvmerge.

Here is the list of alternatives. I am, very probably, not telling you anything new, after all these tools have been around for a while, but if you haven't yet tried them yet, here is a description of the tools. I'll expand on their use after this short list:

1. Shaka-packager
   We use it to apply keys to a media file to decrypt it, however it was built to encrypt widevine media! This is a faster alternative to mp4decrypt. Download one of the files suitable for your machine labelled packager* , extract the file and save the binary as 'shaka-packager in your machine's PATH
   
2. mkvmerge
   Mkvmerge is part of a package of tools called MKVToolNix found at https://mkvtoolnix.download/downloads.html.
   Mkv files are a few percent smaller in size than the same video saved as mp4. It does exactly what its name suggests; it merges audio, video and subtitle files into one playable stream.
   
3. The Stream Detector (Firefox or Chrome plugin
   The Stream Detector is something I use a lot. Not only does it find mpds but it can also create the command to directly download the media with N_m3u8DL-RE
   Find here: Firefox version - https://addons.mozilla.org/en-US/firefox/addon/hls-stream-detector/
   Chrome version: https://github.com/rowrawer/stream-detector/releases choose hls_stream_detector-2.1X.XX.crx
   and in chrome's address bar type 'chrome://extensions/'; set then Developer mode (top-right window) and drag and drop the crx file into the window to install
   
4. N_m3u8DL-RE
   - very catchy name that! - this tool morphed from N_m3u8DL-CLI which was a straight m3u8 downloader. The 'RE' version copes with Dynamic Adaptive Streaming over Https (DASH) encrypted streams as well as Http Live Streaming (HLS) streams. DASH streams are associated with an mpd and HLS with m3u8 files that decribe the media. It is the candidate to replace yt-dlp.
   
   Download from https://github.com/nilaoda/N_m3u8DL-RE/releases , extract and place the binary somewhere in your machine's PATH
   From a command window or terminal, typing 'N_m3u8DL-RE --help', will reveal all if it is installed correctly.
   

Using the tools:

Shaka-packager
A typical shaka-packager call to decrypt a stream looks like this:

shaka-packager in=Frost.faudio\=96000.m4a,stream=audio,output=Frost.m4a --enable_raw_key_decryption --keys key_id=c2471312525641a38487ac8adc5b6f37:key=94b0ba348ce2fa88049378987c462e18
shaka-packager in='Frost.fvideo=3295464.mp4',stream=video,output=Frost.mp4 --enable_raw_key_decryption --keys key_id=c2471312525641a38487ac8adc5b6f37:key=94b0ba348ce2fa88049378987c462e18

The same code quoted for easy readability:-

shaka-packager in=Frost.faudio\=96000.m4a,stream=audio,output=Fro st.m4a --enable_raw_key_decryption --keys key_id=c2471312525641a38487ac8adc5b6f37:key=94b0ba 348ce2fa88049378987c462e18
shaka-packager in='Frost.fvideo=3295464.mp4',stream=video,output= Frost.mp4 --enable_raw_key_decryption --keys key_id=c2471312525641a38487ac8adc5b6f37:key=94b0ba 348ce2fa88049378987c462e18

Note that in the command string, the commas are not followed by spaces, in fact spaces will prevent the routine working. Both the kid and the key are specifically identified. And also note the in-filenames have an equals-sign that either needs escaping with backslash (\) or the filename can quoted with single quotes. I've used both styles in the example.
If you want to run through the command yourself the programme details are :-
https://www.itv.com/watch/a-touch-of-frost/Ya1774/Ya1774a0004
frost s02E01 key c2471312525641a38487ac8adc5b6f37:94b0ba348ce2fa880 49378987c462e18


mkvmerge
Mkvmerge is quite simple to use from the command line:

mkvmerge -o output.mkv Video.mp4 Audio.m4a subs.srt

And a fully qualified real example

 mkvmerge -o Woman_on_the_Run.mkv 'Woman On The Run _ New to Encore _ Talking Pictures TV.en.m4a' 'Woman On The Run _ New to Encore _ Talking Pictures TV.mp4' --language 0:en --track-name 0:English 'Woman On The Run _ New to Encore _ Talking Pictures TV.en-GB.srt'

And repeated as a quote so you can read it:

mkvmerge -o Woman_on_the_Run.mkv 'Woman On The Run _ New to Encore _ Talking Pictures TV.en.m4a' 'Woman On The Run _ New to Encore _ Talking Pictures TV.mp4' --language 0:en --track-name 0:English 'Woman On The Run _ New to Encore _ Talking Pictures TV.en-GB.srt'

The Stream Detector
As installed, The Stream Detector will list playable media streams such as mpd and m3u8. Look for 'master.m3u8 as that is a descriptor for all the stream parts. Its strength in use is that it can collect a history of the media you have visited. With one click a copy of a the mpd may be made. But there is even more power; it is possible to copy the media in a number of formats. I will be looking at two:_

1. N_m3u8DL-RE command string,
   
   [Attachment 69162 - Click to enlarge]
2. Table entry
   
   [Attachment 69161 - Click to enlarge]

I will deal with these specifically in use in the next sections.
My options for The Stream Detector, so that a sensible programme title appears whereever possible is:-

[Attachment 69166 - Click to enlarge]

N_m3u8DL-RE
At its simplest you can use N_m3u8DL-RE <mpd url> for example (Token has expired!)

N_m3u8DL-RE https://manifest.prod.boltdns.net/manifest/v1/dash/live-baseurl/bccenc/1242911124001/64f141c8-40a5-4bc4-a593-318ba8c0950e/6s/manifest.mpd?fastly_token=NjQwOWUxYTlfNDFlNWFlNjdkODE0YWY3NjI2YTRiYjc2YzcyZWVmODJjZmNkNTczMDQwZDVmZjgyZjBmNGJlYTVjYzU4NDM1Yw%3D%3D

Now The stream detector can save a full N_m3u8DL-RE command which mainly worked in testing (except not with ITV.com - more on that later).

For example, this is an m3u8 downloaded from https://www.tptvencore.co.uk/Video/Woman-On-The-Run?id=918aabec-7afe-4f83-9063-8816fd255f10 ; the media link (m3u8) generated by The Stream Detector is

N_m3u8DL-RE "https://manifest.prod.boltdns.net/manifest/v1/hls/v4/aes128/6272132012001/591c148f-5b79-4bf3-99ba-23f082f1c336/6s/master.m3u8?fastly_token=NjNlNTJjOWJfYjllZTQ1OGM4NjZjYWQ4ZmQwZjNjOTk5OTdiZDFjN2VjOTY0ZTc3YzE3YzU4NTI3MTEzMjA1MzQ4ZWIzZTFjOQ%3D%3D&bcov_auth=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJhY2NpZCI6IjYyNzIxMzIwMTIwMDEifQ.kfwThZX49U2WSqRYLANL5xP-1CkNYut8YFBgMMwc6fY" --header "User-Agent: Mozilla%2F5.0%20(X11%3B%20Linux%20x86_64%3B%20rv%3A109.0)%20Gecko%2F20100101%20Firefox%2F109.0|Referer: https%3A%2F%2Fwww.tptvencore.co.uk%2F" --save-name "Woman On The Run _ New to Encore _ Talking Pictures TV"

And again as a quote:-

N_m3u8DL-RE "https://manifest.prod.boltdns.net/manifest/v1/hls/v4/aes128/6272132012001/591c148f-5b79-4bf3-99ba-23f082f1c336/6s/master.m3u8?fastly_token=NjNlNTJjOWJfYjllZTQ1OGM4N jZjYWQ4ZmQwZjNjOTk5OTdiZDFjN2VjOTY0ZTc3YzE3YzU4NTI 3MTEzMjA1MzQ4ZWIzZTFjOQ%3D%3D&bcov_auth=eyJ0eXAiOi JKV1QiLCJhbGciOiJIUzI1NiJ9.eyJhY2NpZCI6IjYyNzIxMzI wMTIwMDEifQ.kfwThZX49U2WSqRYLANL5xP-1CkNYut8YFBgMMwc6fY" --header "User-Agent: Mozilla%2F5.0%20(X11%3B%20Linux%20x86_64%3B%20rv%3 A109.0)%20Gecko%2F20100101%20Firefox%2F109.0|Refer er: https%3A%2F%2Fwww.tptvencore.co.uk%2F" --save-name "Woman On The Run _ New to Encore _ Talking Pictures TV"

And that is nearly good enough.

If you use as given you will end up with three files - am mp4 an m4a and an srt which you will need to merge. However; N_m3u8DL-RE can be made to merge files to produce a playable video by adding

-M format=mkv:muxer=mkvmerge

to the end of the command string.

If you want a faster download - then add '-mt' as an option marker for "Multi-Threading" to your command, forcing N_n3u8DL-RE to download streams for video, audio and subtitle simultaneously.

N_m3u8DL-RE "https://manifest.prod.boltdns.net/manifest/v1/hls/v4/aes128/6272132012001/591c148f-5b79-4bf3-99ba-23f082f1c336/6s/master.m3u8?fastly_token=NjNlNTJjOWJfYjllZTQ1OGM4NjZjYWQ4ZmQwZjNjOTk5OTdiZDFjN2VjOTY0ZTc3YzE3YzU4NTI3MTEzMjA1MzQ4ZWIzZTFjOQ%3D%3D&bcov_auth=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJhY2NpZCI6IjYyNzIxMzIwMTIwMDEifQ.kfwThZX49U2WSqRYLANL5xP-1CkNYut8YFBgMMwc6fY" --header "User-Agent: Mozilla%2F5.0%20(X11%3B%20Linux%20x86_64%3B%20rv%3A109.0)%20Gecko%2F20100101%20Firefox%2F109.0|Referer: https%3A%2F%2Fwww.tptvencore.co.uk%2F" --save-name "Woman On The Run _ New to Encore _ Talking Pictures TV" -M format=mkv:muxer=mkvmerge

And again - more readable

N_m3u8DL-RE "https://manifest.prod.boltdns.net/manifest/v1/hls/v4/aes128/6272132012001/591c148f-5b79-4bf3-99ba-23f082f1c336/6s/master.m3u8?fastly_token=NjNlNTJjOWJfYjllZTQ1OGM4N jZjYWQ4ZmQwZjNjOTk5OTdiZDFjN2VjOTY0ZTc3YzE3YzU4NTI 3MTEzMjA1MzQ4ZWIzZTFjOQ%3D%3D&bcov_auth=eyJ0eXAiOi JKV1QiLCJhbGciOiJIUzI1NiJ9.eyJhY2NpZCI6IjYyNzIxMzI wMTIwMDEifQ.kfwThZX49U2WSqRYLANL5xP-1CkNYut8YFBgMMwc6fY" --header "User-Agent: Mozilla%2F5.0%20(X11%3B%20Linux%20x86_64%3B%20rv%3 A109.0)%20Gecko%2F20100101%20Firefox%2F109.0|Refer er: https%3A%2F%2Fwww.tptvencore.co.uk%2F" --save-name "Woman On The Run _ New to Encore _ Talking Pictures TV" -M format=mkv:muxer=mkvmerge

If you want to follow along the video is here https://www.tptvencore.co.uk/Video/Woman-On-The-Run?id=91732d44-0280-4a95-93c4-f51908245410

when the command is run in a terminal window we get this:

[Attachment 69159 - Click to enlarge]
N_m3u8DL-RE halts for the user to select which streams they want - not all available are shown.
If you wish, adding --auto-select to the command will download without N_m3u8DL-RE stopping to wait for your selection but makes a pretty good job of selecting the best streams available in most cases.

[Attachment 69160 - Click to enlarge]

So there we are almost done. The problem we have is getting the link to the media and putting a name to the media. If we use programs to download we all like to have as little human intervention as possible.
You might want to spend hours using httptoolkit to find mpd by python code and parsing html or json to arrive at programme, series and episode description. But in my mind using' copy as Table Entry' in stream detector goes pretty far along the full automation route for me.

A few ITV programmes saved as table entries:-

https://itvpnpdotcom.blue.content.itv.com/10-3617-0004-001/34/1/VAR028-HD-s/10-3617-00...0f3f26c321cc69 | Ghislaine: Partner in Crime - Series 1 - Episode 4 - ITVX | 09/02/2023 14:12:15
https://itvpnpdotcom.blue.content.itv.com/10-3617-0003-001/34/1/VAR028-HD-s/10-3617-00...7ee793dac83a56 | Ghislaine: Partner in Crime - Series 1 - Episode 3 - ITVX | 09/02/2023 14:12:11
https://itvpnpdotcom.blue.content.itv.com/10-3617-0002-001/34/1/VAR028-HD-s/10-3617-00...d97ffafb5f4617 | Ghislaine: Partner in Crime - Series 1 - Episode 2 - ITVX | 09/02/2023 14:12:09
https://itvpnpdotcom.blue.content.itv.com/10-3617-0001-001/34/1/VAR028-HD-s/10-3617-00...0aee24d1c85df1 | Ghislaine: Partner in Crime - Series 1 - Episode 1 - ITVX | 09/02/2023 14:12:04

So now we have in easily readable form a list of mpd and programme identifiers for saving.

And a fully worked up program to download that batch of results is ITVXbatch.py It is intended to be run in the WKS-KEYS folder and have access to your own Content Decryption Module.

An ITVX Batch Downloader
Using a clipboard of table entries made by The Stream Detector.

#!/usr/bin/env python3

'''
ITVXbatch.py - a batch downloader for ITVX
coded by A_n_g_e_l_a
08:02:2023

Requirements: N_m3u8DL-RE, shaka-packager and mkvmerge in $PATH and python modules
pyperclip and termcolor may be needed, install with 'pip install pyperclip termcolor'

Written for Linux systems using WKS-KEYS with a working CDM, place in the WKS-KEYS folder.
Should work on Windows with .exe sprinkled where necessary 

This program loads content from The Stream Detector to provide mpds and program name.

The program reads the clipboard for 'Table Entries' saved from 'The Stream Detector'.

Use 'The Stream Detector' browser plugin (Chrome + Firefox)
Find here: Firefox version - https://addons.mozilla.org/en-US/firefox/addon/hls-stream-detector/
Chrome version: https://github.com/rowrawer/stream-detector/releases  choose  hls_stream_detector-2.1X.XX.crx 
and in chrome://extensions/  set Developer mode (top-right window) and drag and drop the crx file into the window to install

First clear all old urls from The Stream Detector
And then set 'Copy stream URL as' to 'Table Entry' in The Stream Detector GUI

Now just visit any number of video pages at ITV.com and the mpd urls will shown in the GUI. 
Copy the Table Entry or Entries to the clipboard from  'The Stream Detector' window when finished selecting videos.
Then run this program

'''

import os
import base64
import requests
from pywidevine.L3.cdm import deviceconfig
from base64 import b64encode
from pywidevine.L3.decrypt.wvdecryptcustom import WvDecrypt
import shutil
import subprocess
import re
from termcolor import colored
import  pyperclip as PC

headers = {
    'Accept': '*/*',
    'Accept-Language': 'en-GB,en;q=0.9',
    'Connection': 'keep-alive',
    'Referer': 'itv.com',
}

def WV_Function(pssh, lic_url, cert_b64=None):
    wvdecrypt = WvDecrypt(init_data_b64=pssh, cert_data_b64=cert_b64, device=deviceconfig.device_android_generic)                   
    widevine_license = requests.post(url=lic_url, data=wvdecrypt.get_challenge(), headers=None)
    license_b64 = b64encode(widevine_license.content)
    wvdecrypt.update_license(license_b64)
    Correct, keyswvdecrypt = wvdecrypt.start_process()
    # chain any multiple keys with --key at the start of all but the first kid:key pair
    if Correct:
        mykeys = ''
        for key in keyswvdecrypt:
            mykeys += key + '--key'
    if mykeys.endswith('--key'):
        mykeys = mykeys[:-5]
    divides("key found")
    print(mykeys)
    return mykeys  

## pretty print screen divisions
def divides(text):
    text = text
    l = len(text)
    count = int(shutil.get_terminal_size().columns) - 2
    if count <= 120:
        count = int(count)
    else:
        count = int(count/2)
    count = count - (l)   
    #line = ('-' * int(count/2))
    line = (chr(9601) * int(count/2))
    print('\n', colored(line, 'green'), " ",  text, " ",   colored(line, 'green'))

def findlicense(mpd_url):
    bit = mpd_url.split('/',8)
    ContentID = bit[7].rsplit('_',2 )
    license = "https://itvpnp.live.ott.irdeto.com/Widevine/getlicense?CrmId=itvpnp&AccountId=itvpnp&ContentId=" + ContentID[0]
    divides('license')
    print(license)
    return license

def generate_pssh(kid: str):
    str1 = '000000387073736800000000edef8ba979d64acea3c827dcd51d21ed000000181210'
    str3 = '48e3dc959b06'
    return base64.b64encode(bytes.fromhex(str1+kid+str3)).decode()


def getnm3u8(mpd_url, pssh):
    # decrypt
    lic_url = findlicense(mpd_url)
    key = WV_Function(pssh, lic_url)
    # organize cookie
    cookie = mpd_url.split("&")
    cookie = cookie[1].split('exp%3D')
    cookie = 'Cookie: hdntl=exp=' + cookie[1].replace('%3D', '=').replace('%2A', '*').replace('nohubplus', 'hdntl,nohubplus')
    #print (cookie)
    divides('Downloading')                                                                                         
    command = [
        "N_m3u8DL-RE",
        mpd_url,
        '--append-url-params',
        '--header',
        "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
        '--header',
        "accept-encoding: identity",
        '--header',
        "Accept-Language: en-us,en;q=0.5",
        '--header',
        "connection: close",
        '--header',
        cookie,
        '--header',
        "Sec-Fetch-Mode: navigate",
        '--header',
        "host': itvpnpdotcom.blue.content.itv.com",
        '--header',
        "User-Agent': Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36", 
        "--auto-select",
        "--save-name",
        videoname,
        "--save-dir",
        "./output",
        "--tmp-dir",
        "./",
        "-mt",
        "--use-shaka-packager",
        "--key",
        key,
        "-M",
        "format=mkv:muxer=mkvmerge",
        ]
    #print(f"\n\ncommand for N_m3u8DL-re is: {command}\n\n")
    subprocess.run(command)


if __name__ == "__main__":
    print("Copy stream URL as 'Table Entry' from The Stream Detector")
    input("Press Enter when ready!")
    fmpd = PC.paste().split('\n')
    divides('ITVX')
    for line in fmpd:
        linetuple = line.split('|')
        mpd_url = linetuple[0].replace('\n','').replace(' ', '')
        print(f"[info] downloading {mpd_url}")
        videoname = linetuple[1].replace('\n','').replace(' ','').replace('-ITVX','')
        divides('downloading ' + videoname)
        # parse for KID to get PSSH
        mpd = requests.get(mpd_url,headers).text
        mylines = mpd.split("\n")
        for myline in mylines:
        
            m = re.search('cenc:default_KID=\"(.+?)\">', myline)
            if m:
                KID = m.group(1)
        KID = KID.replace('-', '')
        
        if KID != '':
            divides(f'Default_KID:')
            print(KID)
            pssh = generate_pssh(KID) 
            divides('pssh found')
            print(pssh)
        else:
            print('No KID was found; exiting!')
            exit(0)
  
        if not os.path.exists("./output/"):
            os.system("mkdir ./output/")
        # download    
        getnm3u8(mpd_url, pssh)
    divides("All Done.")
    print('[info] Your files are in ./output/\n')
    exit(0)

The code above should be a good starting point for adapting for your own url targets. Note specifically the python way of calling N_m3u8DL-RE. ITV proved a little difficult, in that, with N_m3u8DL-RE, it needed a cookie sent and --append-url-params which normally is not needed..

And that's all I know!!

PS. I get unwanted messages from people telling me they cannot understand all this. And I agree, in January and February of last year neither could I.
I read loads of posts, evening after evening. and not much made sense.
But I Googled to clarify terms being used and I kept reading and re-reading posts from the heroes on this site.
Eventually I formed some understanding - enough to have a go - I learn best when doing, who doesn't? So I tried examples, myself, wherever possible. I eventually found code from others on github and made adaptations to it. I learnt to look and read the error messages that python gives. (It is trying to be helpful). And I attempted to make sense of the error and correct my code.
Eventually I found I had learned enough to code some python for my own purposes. And I keep learning. The code I write today looks noting lke the stuff I was producing last Summer.
So I think my message to you is do not be put off by the seeming complexity of decryption and the need to program. Do not expect to understand it over-night. But if you keep at it, bit by bit the mists will clear.

Please ask questions in this thread and not to me via PM.

And no I don't make videos.

[A_n_g_e_l_a]

STV Downloader

Stv.tv have made changes to their website and SSAI (Server Side Advert Insertion) is currently not part of their offering (whoopee!!) But my previously posted stv script has been broken by the changes.

This update is a single or a batch downloader for encrypted or encryption-free videos.
The program reads the clipboard of urls saved from 'The Stream Detector' browser plugin using 'copy stream URL as Table Entry'.
Any urls in the list not of the form .mpd or master.m3u8 are ignored.

 
#!/usr/bin/env python3
 
import  pyperclip as PC
import requests
from pywidevine.L3.cdm import deviceconfig
from base64 import b64encode
from pywidevine.L3.decrypt.wvdecryptcustom import WvDecrypt
import shutil
from termcolor import colored
import subprocess
import re

'''
For stv.tv  - a series downloader

'pip install pyperclip termcolor' may be needed before you start

Written for Linux systems using WKS-KEYS with a working CDM 

The program reads the clipboard of urls saved from 'The Stream Detector' browser plugin using 'copy stream URL as Table Entry'.
Any urls in the list not of the form .mpd or master.m3u8 are ignored.

Use 'The Stream Detector' browser plugin (Chrome + Firefox)
Find here: Firefox version - https://addons.mozilla.org/en-US/firefox/addon/hls-stream-detector/
Chrome version: https://github.com/rowrawer/stream-detector/releases  choose  hls_stream_detector-2.1X.XX.crx 
and in chrome://extensions/  set Developer mode (top-right window) and drag and drop the crx file into the window to install

Now just visit any number of video pages at STV and the videos urls will be listed by
The Stream Detector. Copy the urls to the clipboard from 'The Stream Detector' window when finished selecting videos.
This program ignores all links except md or master.m3u8
'''

headers = {
    'Accept': '*/*',
    'Accept-Language': 'en-GB,en;q=0.7',
    'Connection': 'keep-alive',
}

def WV_Function(pssh, lic_url, cert_b64=None):
    wvdecrypt = WvDecrypt(init_data_b64=pssh, cert_data_b64=cert_b64, device=deviceconfig.device_android_generic)                   
    widevine_license = requests.post(url=lic_url, data=wvdecrypt.get_challenge(), headers=None)
    license_b64 = b64encode(widevine_license.content)
    wvdecrypt.update_license(license_b64)
    Correct, keyswvdecrypt = wvdecrypt.start_process()
    if Correct:
        return keyswvdecrypt  

def divides(text):
    text = text
    l = len(text)
    count = int(shutil.get_terminal_size().columns) - 2
    if count <= 120:
        count = int(count)
    else:
        count = int(count/2)
    count = count - (l)   
    #line = ('-' * int(count/2))
    line = (chr(9601) * int(count/2))
    print('\n', colored(line, 'green'), " ",  text, " ",   colored(line, 'green'))

def get_pssh_lic(mpd_url):
    mpd = requests.get(mpd_url, headers).text
    lines = mpd.split("\n")
    for line in lines:
        m = re.search('<cenc:pssh>(AAAA.+?)</cenc:pssh>', line)
        n = re.search('bc:licenseAcquisitionUrl=\"(http.+?)\" xmlns:bc=\"urn:brightcove:2015\"', line)
        if m:
            pssh = m.group(1)
        if n:
            lic_url = n.group(1)
    return pssh, lic_url

divides('STV Downloader from abatch of master.m3u8 or mpd url')
print("Ready to read from clipboard")
input("Press Enter when ready... ")
print('reading from the clipboard ...')

mylines = PC.paste().split('\n')
for line in mylines:
    linetuple = line.split('|')
    map_url = linetuple[0].replace(' ', '')
    videoname = (linetuple[1]+ linetuple[2])
    #clean up videoname
    videoname = re.sub(r"[\s]", "_", videoname) # replace whitespace 
    while videoname.endswith('_'):
        videoname = videoname[:-1]
    while videoname.startswith('_'):
        videoname = videoname[1:] 
             
    if 'mpd' in map_url:
        # encrypted stream
        divides('encrypted video')
        mpdmap = requests.get(map_url, headers=headers).text
        pssh, lic_url = get_pssh_lic(map_url)
        keys =  WV_Function(pssh, lic_url)
        key = keys[0]
        divides('keys found')
        print(colored(key, 'red'))
        divides('downloading video')
        command = [
        "N_m3u8DL-RE",
        map_url,
        "--auto-select",
        "--save-name",
        videoname,
        "--save-dir",
        "./output",
        "--tmp-dir",
        "./",
        "-mt",
        "--use-shaka-packager",
        "--key",
        key,
        "-M",
        "format=mkv:muxer=mkvmerge",
        ]
    elif 'master.m3u' in map_url:  
        divides('downloading an encryption free video')
        command = [
        "N_m3u8DL-RE",
        map_url,
        "--auto-select",
        "--save-name",
        videoname,
        "--save-dir",
        "./output/",
        "--tmp-dir",
        "./",
        "-mt",
        "-M",
        "format=mkv:muxer=mkvmerge",
        ]
    else:
        continue
    subprocess.run(command)
exit(0)

Should you wish to download a set of particular streams and not accept the auto selected offer then this command shows setting stream id.

        command = [
        "N_m3u8DL-RE",
        map_url,
        "-sv",
        "best",
        "-sa",
        "id='audio-2'",  # using a named id. STV often have three anonymous streams audio-0, 1, and 2
        "-ss",
        "id='subtitles-0':for=all",
        "--save-name",
        videoname,
        "--save-dir",
        "./output/",
        "--tmp-dir",
        "./",
        "-mt",
        "--use-shaka-packager",
        "-M",
        "format=mp4",
        ]

[codehound]

Excellent as usual

[jack_666]

I have to echo codehound's remarks. Again, another excellent post. Keep up the truly excellent work.

[cedric8528]

Well done Teacher!

Thanks a lot A_n_g_e_l_a

[deccavox]

As usual, excellent work Angela.

One or two observations though.
As with your previous STV downloader, by using "--auto-select" as an option for N_m3u8DL-RE when the stream is none-encypted, it selects the 64Kbps audio stream and not the "best" which is 128Kbps. By omitting that option, you get a list of the streams. Just press enter and it will download the best video (1080p) and the best audio (128Kbps).

But that is not the case for encrypted streams.

However, big problem with encrypted streams. Your last downloader worked perfectly for everything I threw at it. But this time, most end up with an unplayable video. MediaInfo typically says:

Video
ID                                       : 1
Format                                   : V_QUICKTIME
Codec ID                                 : V_QUICKTIME
Duration                                 : 51 min 17 s
Bit rate                                 : 3 554 kb/s
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 25.000 FPS

However, it will not even play in Windows Media player 12 (which does play Quicktime), even after changing extension to .mov
Neither ffmpeg nor HandBrake recognise it as a valid codec.

But, not all (just most that I've tried).
Catching Milat S01E01 is actually perfect: https://player.stv.tv/episode/461c/banijay1-milat

Some that fail:
I've tried most of The Bridge: https://player.stv.tv/summary/banijay1-thebridge
and
Six Nations Rugby: Italy v France: https://player.stv.tv/episode/4ejz/six-nations-live

I'll have to spend more time on it when I do have the time to try to suss it out.

The strange thing is, during downloading with N_m3u8DL-RE, it does actually say "[0x1]: Video, h264 (avc1), 1920x1080", which of course is correct.

Incidently, many of my final files I want as mp4, so instead of using
"-M",
"format=mkv:muxer=mkvmerge",
I use:
"-M",
"format=mp4:muxer=ffmpeg",

[A_n_g_e_l_a]

One or two observations though.

"format=mp4:muxer=ffmpeg",

One or two observations of my own.

1. One: My aged-ears with attendant high frequency hearing loss are more low-bit rate rather than high these days: I cannot hear the difference. If you think you really stil can, then you are a capable coder; change it to your preference.
2. Two: I only tried The_Bridge Pilot, saved as mkv; it downloaded and it played fine. More likely the problem is with your Windows set-up missing the needed codecs. Try using VLC as your media player. Or getting decent operating system
3. Three: "format=mp4:muxer=ffmpeg" You do not meed any muxer specified; so "format=mp4" is enough.
   
   Edit: Just tested the Rugby stream and that works too.

[codehound]

Surely no one nowadays is using anything but VLC (?) Its the ripping go-to for all testing purposes, especially if you're an uploader. You test in VLC and if its G2G in VLC - that's your testing done. Anyone using a micky mouse media player does it at their own peril.

VLC needs no sideloading of any codecs. Those days when out with the ark and the k-lite codec pack.

[Sorenb]

Surely no one nowadays is using anything but VLC (?) Its the ripping go-to for all testing purposes, especially if you're an uploader. You test in VLC and if its G2G in VLC - that's your testing done. Anyone using a micky mouse media player does it at their own peril.

VLC needs no sideloading of any codecs. Those days when out with the ark and the k-lite codec pack.

MPV with smplayer player here, and as back up for tagging purposes Haruna Media Player. (VLC is loaded somewhere, but I'm a linux head on testing KDE Plasma, need the interfaces to be smooth with QT6 just around the corner.)

[deccavox]

Of course I used VLC to test it. That's my default player. It was VLC that came up with:
Codec not supported:
VLC could not decode the format " " (No description for this codec)
and that's what prompted me to try other things.

My comments weren't meant as a criticism, merely what I had found happened at my end when first trying it. Maybe there is something in the code that's good for linux and not windows. I'll investigate when I have time. I made them mainly wondering if others were finding the same effects.

Reading through many posts on here about many things, there's an awful lot of snide comments towards members. Not nice.

[codehound]

Of course I used VLC to test it. That's my default player. It was VLC that came up with:
Codec not supported:
VLC could not decode the format " " (No description for this codec)
and that's what prompted me to try other things.

My comments weren't meant as a criticism, merely what I had found happened at my end when first trying it. Maybe there is something in the code that's good for linux and not windows. I'll investigate when I have time. I made them mainly wondering if others were finding the same effects.

Fair comment but your in the 1% minority with linux

Reading through many posts on here about many things, there's an awful lot of snide comments towards members. Not nice.

Its frustration through having to repeat ad-nauseum what's been posted multiple times before and the more times its repeated and spoonfed the more chance its got of being revoked. You've only got to monitor what's happened over vdo on here to see there are moles amongst us.

If you use the forum's own search facility 95% of asked questions have already been answered.

[Sorenb]

VLC could not decode the format " " (No description for this codec)

It's a rare error report, and usually reserved for x265/h265 codecs (hevc is a whole different level of system requirements and multi threaded cpus if the host machine is older than the codec. VLC may not compile with support if the host can't play.)

[Sorenb]

As usual, excellent work Angela.

One or two observations though.
As with your previous STV downloader, by using "--auto-select" as an option for N_m3u8DL-RE when the stream is none-encypted, it selects the 64Kbps audio stream and not the "best" which is 128Kbps. By omitting that option, you get a list of the streams. Just press enter and it will download the best video (1080p) and the best audio (128Kbps).

But that is not the case for encrypted streams.

However, big problem with encrypted streams. Your last downloader worked perfectly for everything I threw at it. But this time, most end up with an unplayable video. MediaInfo typically says:

Video
ID                                       : 1
Format                                   : V_QUICKTIME
Codec ID                                 : V_QUICKTIME
Duration                                 : 51 min 17 s
Bit rate                                 : 3 554 kb/s
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 25.000 FPS

However, it will not even play in Windows Media player 12 (which does play Quicktime), even after changing extension to .mov
Neither ffmpeg nor HandBrake recognise it as a valid codec.

But, not all (just most that I've tried).
Catching Milat S01E01 is actually perfect: https://player.stv.tv/episode/461c/banijay1-milat

Some that fail:
I've tried most of The Bridge: https://player.stv.tv/summary/banijay1-thebridge
and
Six Nations Rugby: Italy v France: https://player.stv.tv/episode/4ejz/six-nations-live

I'll have to spend more time on it when I do have the time to try to suss it out.

The strange thing is, during downloading with N_m3u8DL-RE, it does actually say "[0x1]: Video, h264 (avc1), 1920x1080", which of course is correct.

Incidently, many of my final files I want as mp4, so instead of using
"-M",
"format=mkv:muxer=mkvmerge",
I use:
"-M",
"format=mp4:muxer=ffmpeg",

V_QUICKTIME
Your video stream download is corrupted.

"V_QUICKTIME

Codec ID: V_QUICKTIME

Codec Name: Video taken from QuickTime(TM) files

Description: Several codecs as stored in QuickTime, e.g., Sorenson or Cinepak.

Initialization: The Private Data contains all additional data that is stored in the ‘stsd’ (sample description) atom in the QuickTime file after the mandatory video descriptor structure (starting with the size and FourCC fields). For an explanation of the QuickTime file format read QuickTime File Format Specification."

https://www.matroska.org/technical/codec_specs.html



https://forum.videolan.org/viewtopic.php?t=156824

[deccavox]

Its frustration through having to repeat ad-nauseum what's been posted multiple times before and the more times its repeated and spoonfed the more chance its got of being revoked. You've only got to monitor what's happened over vdo on here to see there are moles amongst us.

I'm not quite sure which bit of my above post has been repeated multiple times before (did you read it?).

Anyway, it's the "--use-shaka-packager", "--key", key,
that's killing it. Basically, it's not decrypting. I'm not sure why yet. But I will invesigate on my own.
If I replace that small part of the code with good old fashioned mp4decrypt script it all works fine.

and yes, before you ask, of course I have packager-win-x64.exe in my Windows PATH.

[Sorenb]

Its frustration through having to repeat ad-nauseum what's been posted multiple times before and the more times its repeated and spoonfed the more chance its got of being revoked. You've only got to monitor what's happened over vdo on here to see there are moles amongst us.

I'm not quite sure which bit of my above post has been repeated multiple times before (did you read it?).

Anyway, it's the "--use-shaka-packager", "--key", key,
that's killing it. Basically, it's not decrypting. I'm not sure why yet. But I will invesigate on my own.
If I replace that small part of the code with good old fashioned mp4decrypt script it all works fine.

and yes, before you ask, of course I have packager-win-x64.exe in my Windows PATH.

Rename the Shaka packager in your $PATH and recheck you have the syntax correct.

[larits]

Reading through many posts on here about many things, there's an awful lot of snide comments towards members. Not nice.

This is unfortunately a very common occurrence in this part of the internet, and not exclusive to this forum. I'm new here, but was a part of the piracy/release scene for many years, and it tends to attract lots of unhappy people who get a little buzz from being condescending towards others. Reading through many posts on here reminds me of old-school IRC channels where scene members would jump at the chance to show of their skills/knowledge, just so they could tell people to **** off when someone inevitably asked how it was done. It's like their favorite thing. This is speaking generally, of course; I'm not referring to a specific person. It just comes with the territory, and the best practice is to just let it roll of your back.

Anyway, it's the "--use-shaka-packager", "--key", key,
that's killing it. Basically, it's not decrypting. I'm not sure why yet. But I will invesigate on my own.
If I replace that small part of the code with good old fashioned mp4decrypt script it all works fine.

and yes, before you ask, of course I have packager-win-x64.exe in my Windows PATH.

Try renaming it to shaka-packager.exe.

[A_n_g_e_l_a]

.... it tends to attract lots of unhappy people who get a little buzz from being condescending towards others. . This is speaking generally, of course; I'm not referring to a specific person.

Try renaming it to shaka-packager.exe.

I started here, on this forum, this time last year and agree people, including me, can be unhelpful at times. My motivation for these Decryption threads was to provide answers to common problems I had met and was continuing to meet on my decryption journey.

I confess to feelings of frustration when people jump straight in and demand answers to stuff that I know for certain has already been answered over, and over, and over again. I feel frustration when people ask questions in the forum that Google is better placed to answer instead.

I confess to enormous frustration when people skim read and expect instant insight into the tools and methods used in decrypting. Or those that claim to have read a thread but then happily proclaim their ignorance and demand help to understand. There is a case in point right here. Renaming shaka-packager was explained in the very first post on this thread.

I confess to feeling of annoyance when people take a script, written as an example and posted in a tutorial style, and start complaining the script doesn't work for them for some reason. I clearly say my scripts are written for Linux; I only post them when they work on my machine. I really do not wish to become software support person with endless discussion about getting the scripts to work on Windows. Not my job.. The example scripts are just that, things for you to change to suit your circumstance and in the process learn a little about coding. And things to cut and paste from when you start stringing Python code together yourself. If I was offering a 'solution' to downloading from itvx or stv I would have posted these scripts each in their own new thread.

And should you make a big discovery and improve one of my example scripts, great, I am pleased the acorn I set has grown, but please start your own thread should you wish to tell the World about it.

So I understand the old-timers and indeed notice myself getting a bit short with people sometimes. So I would say in summary, I think it is a truism people generally wish to be helpful but there is a wariness to avoid fools. Here 'fools' are people who ask questions without doing their own research first; or those who wish to be baby-stepped through everything and have no concept of 'study'.

I'll happily help those whom have truly tried to help themselves first.

[larits]

@A_n_g_e_l_a

I promise, I wasn't singling you out here. I genuinely meant in more general terms. I've spent a couple of months here reading through everything I can get my eyes on, and you've been an immense help in regards to my learning of both decryption and Python. I'm stuck on a Linux machine at the moment, and studying your scripts have been a fantastic learning tool. But there are a lot of posts around here that serve no other purpose than to make the poster feel superior for knowing something the OP don't. And that is a frustrating thing for a new guy. But of course I understand the lack of patience when someone hasn't made any kind of effort before asking or even complaining. There are a lot of those posts, as well.

Either way, sorry for derailing your thread. I hope you continue to teach us new guys how it's done

[A_n_g_e_l_a]

@A_n_g_e_l_a

I promise, I wasn't singling you out here.

I know you weren't. Just as I am not picking on you. I am trying to make the point that it is not some 'malicious delight in knowing and not telling' that drives me, and I assume, probably everyone else, but a frustration that grown adults can appear as helpless as small children when faced with a little bit of complexity.

Decryption is a complex topic and it is not within everyone's grasp. People arrive here from differing knowledge backgrounds and experiences. Sadly, some need to be shown the door otherwise threads become clogged with "OK I've done what you said but it didn't work?" type questions which do no-one any good and only breed dependency and even higher expectations.

[A_n_g_e_l_a]

Rename the Shaka packager in your $PATH and recheck you have the syntax correct.

One gotcha occurs if you are used to yt-dlp, and prefacing your keys from WideVine with "--key". So if you just drop N_m3u8DL-RE in as a replacement and specify the ~RE --key option and then the key variable, it might actually look like this --key --key xxxxxxx

[Sorenb]

Rename the Shaka packager in your $PATH and recheck you have the syntax correct.

One gotcha occurs if you are used to yt-dlp, and prefacing your keys from WideVine with "--key". So if you just drop N_m3u8DL-RE in as a replacement and specify the ~RE --key option and then the key variable, it might actually look like this --key --key xxxxxxx

You are correct. But Deccavox, in amongst his rather verbose reply stated the script was failing when n-m3u8-re was calling shaka-packager.
He stated again, in that verbose response, that he had shaka-packager added to his $PATH.
The solution for windows is the same as for linux, the binary needs to be renamed. Either by cp on linux, or whatever equivalent windows uses.


As for why certain responses can seem terse, or abrupt. Well, we all have lives outside this forum. Some of us have other projects we are involved in and offer help on a voluntary basis.
Seeing the same type of query appear again and again, the "I've not read the man pages and haven't bothered to really look at the project page for this tool I'm trying to use. The search bar is beyond my capabilities, give me my answer now god damn it."
Well, it can get up certain peoples backs.
Brevity can go a long way too.

And worse still, the scammers who see a reply on the thread/forum and proceed to flood your inbox with even more demands.

[deccavox]

Let's move on. Please. Angela posted very good information and scripts for ITV and STV. I gave feedback of my initial experiences for STV. Yes, I had/ have a problem with shaka-packager on this script, despite having used same package very successfully in the past. So I was basically wondering if others were having the same issue. I may not be a pro coder but I am at the stage where I'm hopefully not a noobie asking questions that have been asked before. Indeed, I have posted some of my own solutions to various downloaders for others to share.

I tried renaming shaka-packager as some suggested. Still the same. I even tried putting shaka-packager in same folder as the STV downloader, so PATH would not be required. The same.

I've been through the script. There's nothing specific to linux or Windows. As I said, all works fine (including N_m3u8DL-RE downloading encrypted streams). It's just that final shaka-packager stage of decryption. But, it's not the downloaded streams that are faulty. Using the very same key and same downloads and using mp4decrypt instead, it's good.

I've tried a few STV encrypted downloads as I said, all the same, apart from "Catching Milat". Why the difference? It can't be because Milat has no subtitles and all the rest do, surely?

As I said, I was hoping to promote positive discussion about this specific problem. Has anybody else actually tried to download (say) The Bridge S01E01: https://player.stv.tv/episode/461s/banijay1-thebridge

I know Angela said she did when she was composing the script. Could somebody else have a go please?

And I'm sorry if this is a little verbose.

[A_n_g_e_l_a]

...[snip]....
Brevity can go a long way too.

2 ->!

[Sorenb]

"I tried renaming shaka-packager as some suggested. Still the same. I even tried putting shaka-packager in same folder as the STV downloader, so PATH would not be required. The same."
If you are flagging n-m3u8re to use shaka packager, it has to be on your systems path, you are calling an installed application.

In cmd

 mkdir C:\bin

reame the binary to shaka-packager.exe and save in the newly created folder bin at root of your C: drive

 setx PATH "C:\bin;%PATH%"

restart cmd
type shaka-packager and you get the same man page readout that packager gives


In fact, in that newly created bin folder, you can place ALL your apps that you keep in a folder (yt-dlp.exe ffmpeg.exe even GPAC if you want it. )

[A_n_g_e_l_a]

My working Linux versions
shaka-packager --version
shaka-packager version v2.6.1-634af65-release
mkvmerge --version
mkvmerge v73.0.0 ('25 or 6 to 4') 64-bit
N_m3u8DL-RE --version
0.1.5.3

[libero08]

Decryption: The last Crusade


[*]The Stream Detector (Firefox of Chrome plugin
The Stream Detector is something I use a lot. Not only does it find mpds but it can also create the command to directly download the media with N_m3u8DL-RE
Find here: Firefox version - https://addons.mozilla.org/en-US/firefox/addon/hls-stream-detector/
Chrome version: https://github.com/rowrawer/stream-detector/releases choose hls_stream_detector-2.1X.XX.crx
and in chrome's address bar type 'chrome://extensions/'; set then Developer mode (top-right window) and drag and drop the crx file into the window to install



And no I don't make videos.

i try to install hlsd for chrome drag in drop and chrome return me an eror, after a initial warn. solution for install?

[A_n_g_e_l_a]

... set then Developer mode (top-right window) and drag and drop the crx file into the window to install

i try to install hlsd for chrome drag in drop and chrome return me an eror, after a initial warn. solution for install?

Setting developer options to 'on' (small slider to right) is the only really important step. Also I found the right-hand side of the screen responsive. Make sure the file you have ends with crx and not xpi.

[libero08]

i have a lot of stuffs into dev area, and 110% , 24/7 ,365 day /year "small button" into rigth upper corner are ever "pushed" on the rigth side
well maybe i am not at the first step ad i knew "crx" is for chrome
I repat, when i drag & drop, a warn message on bottom of dev windows come, after when i allow , in the left corner, upper side come an error message . Nothing more, i thougth someone have a idea, because i think are very weird.

PACK ERROR:
error: "CRX_REQUIRED_PROOF_MISSING"

[A_n_g_e_l_a]

i have a lot of stuffs into dev area, and 110% , 24/7 ,365 day /year "small button" into rigth upper corner are ever "pushed" on the rigth side
well maybe i am not at the first step ad i knew "crx" is for chrome
I repat, when i drag & drop, a warn message on bottom of dev windows come, after when i allow , in the left corner, upper side come an error message . Nothing more, i thougth someone have a idea, because i think are very weird.

PACK ERROR:
error: "CRX_REQUIRED_PROOF_MISSING"

Look all sorts come on here asking for help. Ok you've got lots of experience - sorry I was unable to help you.
But if you had posted the information you just have I might have suggested you have a corrupted crx and you try to re-download. But since that is such an obvious thing to try, of course you will already have tried it. And of course you will have also tried an earlier release. So maestro I can't think of a thing you can do.

[deccavox]

@libero08
Stream Detector extension doesn't work in Chrome browser. Chrome moans it's not an official extension from their web store and it won't let you enable it. I've even tried it in Edge, which is essentially Chrome. Although Edge has an additional option: Allow extensions from other stores, it still won't let you enable it.
It more or less tells you that on the Github entry anyway: https://github.com/rowrawer/stream-detector
Quote "The Chrome version of this addon is not maintained or supported in any way. It's only included on the off chance that it works. Don't expect it to."