VideoHelp Forum


Try StreamFab Downloader and download from Netflix, Amazon, Youtube! Or Try DVDFab and copy Blu-rays!


Try StreamFab Downloader and download streaming video from Youtube, Netflix, Amazon! Download free trial.


+ Reply to Thread
Page 1 of 8
1 2 3 ... LastLast
Results 1 to 30 of 217
Thread
  1. Member
    Join Date
    Feb 2022
    Location
    Search the forum first!
    Search PM
    ITVX batch downloader that downloads and merges subtitles.

    Pick and Mix Mode
    Use the Stream Detectorfor pick and mix selection of programme URLs

    Image
    [Attachment 71939 - Click to enlarge]

    Configure the stream detector to store page urls.

    Image
    [Attachment 71940 - Click to enlarge]

    Copy to clipboard from the Stream Detector

    Series Mode;
    Or there is a Series Mode where you enter a start end end series number to download.
    Or when in series mode, enter 'all', where prompted, to download everything.

    Enter series mode by starting the program with 'S' as an argument, for example:-
    Code:
    python bestITVX.py  S
    Note that some ITVX programmes have missing series. To cope, this script uses absolute numbers. ITVX may have three series listed. Called series 9,10 and 15, for example The script sees them as series 1 2 and 3

    Three style of CDM choices
    Choose pywidevine, WKS-KEYS Decryption Module to call. See the script text to set your preference.

    I post scripts as examples of possibilities; and for you to adapt. Scripts works for me at the point of posting. Sort out your own issues.



    bestITVX.py
    Code:
    # Angela 26:07:2023
    
    # requirements - use pip install for each python module below
    '''
    httpx
    pyfiglet
    pyperclip
    pywidevine
    selectolax
    termcolor
    pysubs2
    '''
    # google installing python modules if you need help.
    
    # also needed will be N_m3u8DL-RE and it's dependencies; ffmpeg, mkvmerge
    # see https://forum.videohelp.com/threads/408557-Decryption-The-Last-Crusade for details
    
    # for PICK & MIX DOWNLOAD
    #------------------------------------------------------
    # PowerShell or terminal window:
    # start script with
    # python bestITVX.py  
    # and have a clipboard of video page urls - as many as you like from ITVX
    
    # For SERIES DOWNLOAD
    #-------------------------------------------------------
    # start program by 'python bestITVX.py S' (use S or s)
    # and have a single URL ready to PASTE when asked.
    # Enter:-
    # 'all' for all series or use series start and end number.
    # Note:-
    # ITVX misses out series from sequence sometimes, e.g. Monster Carp - missing series 5
    # script uses absolute numbering. If series five is missing, script sees series 6 as the missing 5 etc.
    
    
    ## This program can use The Stream Detector to capture 
    ## multiple page URLs
    ## Stream Detector select 'options'
    ## in the box adjacent to'user-defined-commands enter -->  %origin%
    ## In TSD window select copy stream url as User-Defined-Command 1
    
    ## Or for one-off programmes copy the url address in the screen window.
    
    ## Changes at ITVX provide better programme curation so now episode titles
    ## as well as the series name are included in the videoname
    
    # Permission to use this software means accepting that
    # =======there is absolutley no support.============= 
    # Target users are proficient at running python scripts; 
    # if that doesn't decribe you, do not use this software.
    
    import re
    import subprocess
    from base64 import b64encode
    from pathlib import Path
    import httpx
    from httpx import URL, Client
    from selectolax.lexbor import LexborHTMLParser
    import os
    import pyperclip as PC
    import pyfiglet as PF
    from termcolor import colored
    import json
    
    
    # GLOBALS
    OUT_PATH = Path('output')
    OUT_PATH.mkdir(exist_ok=True, parents=True)
    
    global SUBS
    SERIES = False
    global PYWIDEVINE
    
    #@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
    # There are choices of Content Decryption Module to use
    #
    # 1 Use your own CDM within WKS-KEYS subfolders
    # Options needs pywidevine/L3/decrypt/wvdecryptcustom   and
    # pywidevine/L3/cdm  sub-folder structures in WKS-KEYS (some versions differ)
    
    # 2 Local CDM using pywidevine module and a wvd file instead of WKS-KEYS
    # set PYWIDEVINE= True 
    #
    # Also set the WVD_PATH point to the location of your wvd file, if not already set, see below.
    # Ensure the script is not run within the WKS-KEYS folder for this option.
    
    # Crib to create wvd file (use full paths):
    # pywidevine create-device -k "path/to/device_private_key" -c "path/to/device_client_id_blob" -t "ANDROID" -l 3 -o "output_path"
    # 
    # Running a system with both WKS-KEYS and pywidevine installed is possible.
    # WKS-KEYS uses an older protobuf version 3.20.3 installed by pip
    # if you 'pip install pywidevine' after that, pywidevinve expects the protobuf version to be 4.x.x
    # It will complain that the protobufs need re-compiling
    # See here for instruction how'
    # https://forum.videohelp.com/threads/409040-Correcting-Protobuf-Downgrade-to-3-19-0-error#post2684955
    
    #Set false for WKS-KEYS use
    PYWIDEVINE = False
    # if using pywidevine set below to your wvd path
    WVD_PATH = "/home/angela/android_sdk.wvd"
    
    
    
    class ITV:
        def __init__(self):
            self.host = 'itvpnpdotcom.blue.content.itv.com'
            timeout = httpx.Timeout(10.0, connect=60.0)
            self.client = Client(
                headers={
                    'authority': 'www.itv.com',
                    'user-agent': 'Dalvik/2.9.8 (Linux; U; Android 9.9.2; ALE-L94 Build/NJHGGF)',
                },
            timeout=timeout,
            )
            return
    
        def download(self, url: str) -> None:
            title, data = self.get_data(url)
        
            video = data['Playlist']['Video']
            media = video['MediaFiles']
            illegals = "*'%$!(),.:;"
            replacements = {
                '  Episode ': 'E',
                '  Series ': '_S',
                'otherepisodes': '_extra',
                'ITVX': '',
                ' ': '_',
                '&': 'and',
                '?': '',
            }
    
            # replace  extraneous title data and 'illegal' characters
            videoname = ''.join(c for c in title if c.isprintable() and c not in illegals)
            # standardize and compact videoname    
            for rep in replacements:  
                videoname = videoname.replace(rep, replacements[rep])
            videoname = re.sub(r"(\d+)", pad_number, videoname).lstrip('_').rstrip('_')
            result = re.search(r"(^.*S\d+)_(E\d*.*)", videoname)
            # not always a series with episodes
            try:
                pre = result.group(1)
                post = result.group(2)
                videoname = pre+post
            except:
                pass
    
            print(videoname)
            try:
                subs_url = video['Subtitles'][0]['Href']
                subs = self.client.get(subs_url)
                if subs.status_code==200:
                    SUBS = True
                    f = open(f"{videoname}.subs.vtt", "wb") # bytes needed for N_m's subtitles
                    subtitles = subs.content
                    f.write(subtitles)
                    f.close()
                    # convert subtitles
                    os.system(f"python -m pysubs2 --to srt {videoname}.subs.vtt")
                else:
                    SUBS=False
            except:
                SUBS = False    
            myvideoname = videoname
            mpd_url = [f'{video["Base"]}{y}' for x in media if (y := URL(x['Href'])).path.endswith('.mpd')][0]
            lic_url = [x['KeyServiceUrl'] for x in media][0]
    
            pssh = self._get_pssh(mpd_url)
            key = self._get_key(pssh, lic_url)
            temp = URL(mpd_url).params['hdnea']
            temp = temp.replace('nohubplus', 'hdntl,nohubplus')
            cookie = f"cookie: {re.sub(r'^.*(?<=exp=)', 'hdntl=exp=', temp)}"
    
            if SUBS: 
                subs = f"--mux-import:path=./{videoname}.subs.srt:lang=eng::name='English'"
            else:
                subs = '--no-log'
    
            m3u8dl = 'N_m3u8DL-RE'
            
            subprocess.run([
                m3u8dl,
                mpd_url,
                '--append-url-params',
                '--header',
                cookie,
                '--header',
                f'host: {self.host}',
                '--header',
                f'user-agent: {self.client.headers["user-agent"]}',
                '--auto-select',
                '--save-name',
                myvideoname,
                '--save-dir',
                './output',
                '--tmp-dir',
                './',
                '-mt',
                '--key',
                key,
                '-M',
                'format=mkv:muxer=mkvmerge',
                subs, 
            ])
            
            print(f"[info] {myvideoname}.mkv is in {OUT_PATH}")
            if SUBS:
                os.remove(f"{videoname}.subs.vtt")
    
        def get_series_json(self, url: str) -> list:
            urls = [] # list
            r = self.client.get(url)
            if r.status_code == 200:
                tree = LexborHTMLParser(r.text)
                jsondata = tree.root.css_first('#__NEXT_DATA__').text()
                myjson = json.loads(jsondata)
                jsonseriesdata = len(myjson["props"]["pageProps"]["seriesList"])
                print("To download all series, enter 'all' below.\nOr to download a range, enter the series\nstart number")
                series_num_start = input("Which series start number? ")
                if series_num_start == 'all':
                    series_num_start = 0     
                    series_num_end = jsonseriesdata 
                else:
                    series_num_start = int(series_num_start)             
                    series_num_end = int(input("Which series end number? "))
                    series_num_start -= 1 
                # trap for non sequential and missing series numbers 
                # where last series number is greater that total series
                if series_num_end > jsonseriesdata:
                    series_num_end = jsonseriesdata
    
                parturl = url.rsplit('/',2)[0]
                #series search
                programmeId = myjson["query"]["programmeId"]  # 
                for j in range(series_num_start, series_num_end):
                    numepisodes = myjson["props"]["pageProps"]["seriesList"][j]["numberOfAvailableEpisodes"]
                    for i in range(0,numepisodes): 
                        letterA = myjson["props"]["pageProps"]["seriesList"][j]["titles"][i]["encodedEpisodeId"]["letterA"]
                        urls.append(f"{parturl}/{programmeId}/{letterA}")
    
            return urls
    
        def get_data(self, url: str) -> tuple:
            r = self.client.get(url)
            if r.status_code == 200:
                tree = LexborHTMLParser(r.text)
                jsondata = tree.root.css_first('#__NEXT_DATA__').text()
                myjson = json.loads(jsondata)
                title = myjson["props"]["pageProps"]["programme"]["title"]
                try:
                    extendtitle = myjson["props"]["pageProps"]["episode"]["contentInfo"]
                    title = f"{title}_{extendtitle}"
                except:
                    pass
                try:
                    magni_url = myjson["props"]["pageProps"]["episode"]["playlistUrl"]
                except:
                    magni_url = myjson["props"]["pageProps"]["seriesList"][0]["titles"][0]["playlistUrl"]      
    
                features = ['mpeg-dash', 'widevine', 'outband-webvtt', 'hd', 'single-track']
                payload = {
                    'client': {'id': 'browser'},
                    'variantAvailability': {
                        'featureset': {'min': features, 'max': features},
                        'platformTag': 'dotcom',
                    }
                }
                r = self.client.post(magni_url, json=payload)
                return title, r.json()
            else:
                print(f"The response code {r.status_code} did not indicate successs!")
                print(f"The web page content returned was:-\n\n {r.content}\n\nExiting!")
                print(f"Be sure to copy a page url for a playable video ")
                exit(1)
    
        # Local WKS-KEYS CDM or ywidevine module and wvd file pointing to local CDM
        # note: create a wvd using the template below
        # pywidevine create-device -k "KEY_PATH" -c "BLOB_PATH" -t "ANDROID" -l 3 -o "OUTPUT_PATH"
        def _get_key(self, pssh: str, lic_url: str , cert_b64=None) -> str:
            
            if PYWIDEVINE:  
                print("Using pywidevine WVD on this machine")   
                from pywidevine.cdm import Cdm
                from pywidevine.device import Device
                from pywidevine.pssh import PSSH
                pssh = PSSH(pssh)
                device = Device.load(WVD_PATH)
                cdm = Cdm.from_device(device)
                session_id = cdm.open()
                challenge = cdm.get_license_challenge(session_id, pssh)
                licence = httpx.post(lic_url, data=challenge, headers=None)
                #licence.raise_for_status()
                cdm.parse_license(session_id, licence.content)
                mykeys = ''
                for key in cdm.get_keys(session_id):
                    if key.type=='SIGNING':
                        pass
                    else:
                        print(f"Keys found {key.kid.hex}:{key.key.hex()}")
                        mykeys += f"{key.kid.hex}:{key.key.hex()}"
                cdm.close(session_id)
                return mykeys 
            else:
                print("Using CDM on this machine")
                from pywidevine.L3.cdm import deviceconfig 
                from pywidevine.L3.decrypt.wvdecryptcustom import WvDecrypt
                wvdecrypt = WvDecrypt(init_data_b64=pssh, cert_data_b64=cert_b64, device=deviceconfig.device_android_generic)                   
                widevine_license = httpx.post(url=lic_url, data=wvdecrypt.get_challenge(), headers=None)
                license_b64 = b64encode(widevine_license.content)
                wvdecrypt.update_license(license_b64)
                Correct, keyswvdecrypt = wvdecrypt.start_process()
                if Correct:
                    mykeys = ''
                    for key in keyswvdecrypt:
                        mykeys += key +' '
                print(f"Keys found {mykeys}\n")
                return mykeys 
            
        def _get_pssh(self, mpd_url: str) -> str:
            r = self.client.get(mpd_url)
            kid = (
                LexborHTMLParser(r.text)
                .css_first('ContentProtection')
                .attributes.get('cenc:default_kid')
                .replace('-', '')
            )
            s = f'000000387073736800000000edef8ba979d64acea3c827dcd51d21ed000000181210{kid}48e3dc959b06'
            return b64encode(bytes.fromhex(s)).decode()
    
    # add leading zero to series or episode
    def pad_number(match):
        number = int(match.group(1))
        return format(number, "02d")
    
    def main() -> int:
        myITV = ITV()
        if SERIES:
            urls = []
            input("SERIES download: Have the first video link ready in clipboard.\n" + \
                  "Enter when ready!\n")
            # ignores any extra urls, 
            # use first for series collection
            mystr = PC.paste().split('\n')[0] 
            urls = myITV.get_series_json(mystr)
        else:
            input("Press enter with PAGE urls in clipboard ")
            urls = PC.paste().split('\n')  
            
        count = 0
        count = len(urls)
        print(f"The URL list has {count} video(s)")
        for url in urls:
            url = url.encode('utf-8', 'ignore').decode().strip()
            myITV.download(url)
        return 0
    
    if __name__ == "__main__":
        from sys import argv
        if argv and len(argv) > 1:
            if argv[1]=='S' or argv[1]=='s':
                SERIES = True
        title = PF.figlet_format(' I T V X ', font='smslant')
        print(colored(title, 'green'))
        main()
        exit(0)
    Absolutely no support! If you choose to use this software you are expected to be proficient at running python scripts and finding and installing dependencies .
    Last edited by A_n_g_e_l_a; 26th Aug 2023 at 04:52. Reason: Updated with experirmental series download facility
    Quote Quote  
  2. Sweet, works like a charm, thanks!

    Image
    [Attachment 71945 - Click to enlarge]



    Edit: First time I tried this it worked fine but complained about missing mkvmerge at the end:

    Code:
    FileNotFoundError: [Errno 2] No such file or directory: 'mkvmerge'
    So I installed mkvtoolnix and when the download was finished all the files (subs/video/audio) were deleted, then the output was:

    Code:
    20:53:31.038 WARN : Reading media info...                                               
    20:53:31.129 INFO : NaN: Audio, aac (mp4a), 96 kb/s                                                 
                                                                                                        
    20:53:41.420 WARN : Force Exit...━━━━━━━━━━━━━━━━━━━━━━━━━  30% 48.32MB/164.01MB 4.92MBps 00:00:24 ⣷
                                                                                                        
    Vid 1280x720 | 1874 Kbps ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━  30% 48.32MB/164.01MB 4.92MBps 00:00:24 ⣷Aud 96 Kbps | 2CH        ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━  20%   9.93MB/49.55MB 1.04MBps 00:00:39 ⣷
                                                                                                        
    Error: The file 'Bodies_of_Evidence__The_Butcher_Surgeon.mp4' could not be opened for reading: open file error.
    mv: cannot stat 'Bodies_of_Evidence__The_Butcher_Surgeon.mkv': No such file or directory
    But it was the script that deleted them. The only way I can stop this happening is to remove mkvtoolnix, then I end up with video and audio in 1 file and subs separate. Also the files don't go in the output folder, which I presume they're supposed to.

    Still works fine for me as I don't need subs just thought I would point this out.
    Last edited by vidsrme; 23rd Jun 2023 at 15:21.
    Quote Quote  
  3. Member
    Join Date
    Feb 2022
    Location
    Search the forum first!
    Search PM
    Originally Posted by vidsrme View Post
    Sweet, works like a charm, thanks!

    Image
    [Attachment 71945 - Click to enlarge]



    Edit: First time I tried this it worked fine but complained about missing mkvmerge at the end:

    Code:
    FileNotFoundError: [Errno 2] No such file or directory: 'mkvmerge'
    So I installed mkvtoolnix and when the download was finished all the files (subs/video/audio) were deleted, then the output was:

    Code:
    20:53:31.038 WARN : Reading media info...                                               
    20:53:31.129 INFO : NaN: Audio, aac (mp4a), 96 kb/s                                                 
                                                                                                        
    20:53:41.420 WARN : Force Exit...━━━━━━━━━━━━━━━━━━━━━━━━━  30% 48.32MB/164.01MB 4.92MBps 00:00:24 ⣷
                                                                                                        
    Vid 1280x720 | 1874 Kbps ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━  30% 48.32MB/164.01MB 4.92MBps 00:00:24 ⣷Aud 96 Kbps | 2CH        ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━  20%   9.93MB/49.55MB 1.04MBps 00:00:39 ⣷
                                                                                                        
    Error: The file 'Bodies_of_Evidence__The_Butcher_Surgeon.mp4' could not be opened for reading: open file error.
    mv: cannot stat 'Bodies_of_Evidence__The_Butcher_Surgeon.mkv': No such file or directory
    But it was the script that deleted them. The only way I can stop this happening is to remove mkvtoolnix, then I end up with video and audio in 1 file and subs separate. Also the files don't go in the output folder, which I presume they're supposed to.

    Still works fine for me as I don't need subs just thought I would point this out.
    This error had me going for quite while a few months back.
    The full title to the video is
    Code:
    Bodies of Evidence: The Butcher Surgeon
    And N_m3u8DL-RE does not cope with colons. It changes them to a full-stop and that messes up the code!
    I have corrected that to trap it and other 'illegal' characters that cause hiccups. See the edited version above.

    Image
    [Attachment 71951 - Click to enlarge]

    Try again it should work this time.
    Quote Quote  
  4. Member
    Join Date
    Dec 2020
    Location
    Croatia
    Search PM
    Does it download 1080p streams?
    Quote Quote  
  5. Member
    Join Date
    Feb 2022
    Location
    Search the forum first!
    Search PM
    Originally Posted by ampersand View Post
    Does it download 1080p streams?
    1280x720 is the best they provide on the web. STV have much the same portfolio and have 1080p
    Quote Quote  
  6. Member
    Join Date
    May 2023
    Location
    England
    Search Comp PM
    Great!

    I gotta say, though, y'all should be very careful of running a script that has
    Code:
    os.system("rm *.mp4 *.vtt *.srt")
    in it.

    Some of us prefer mp4 to mkv. If you've already downloaded other mp4s etc to that folder (i.e. from another script), this script will happily delete them all.
    Quote Quote  
  7. Member
    Join Date
    Feb 2022
    Location
    Search the forum first!
    Search PM
    Originally Posted by PhilipG View Post
    Great!

    I gotta say, though, y'all should be very careful of running a script that has
    Code:
    os.system("rm *.mp4 *.vtt *.srt")
    in it.

    Some of us prefer mp4 to mkv. If you've already downloaded other mp4s etc to that folder (i.e. from another script), this script will happily delete them all.
    Yes sorry about that. I did consider the problem and thought no-one is going to mix programs and videos. Are they?

    If you do mix video and python scripts
    Make these edits

    Insert this line.
    Image
    [Attachment 71963 - Click to enlarge]


    remove this line
    Image
    [Attachment 71964 - Click to enlarge]

    The line to insert
    Code:
    os.system(f"rm {videoname}.mp4 {videoname}.subs.vtt {videoname}.subs.srt")
    Last edited by A_n_g_e_l_a; 24th Jun 2023 at 10:05.
    Quote Quote  
  8. Member
    Join Date
    Mar 2023
    Location
    Australia
    Search Comp PM
    Hi I'm trying to copy @Angela's script and keep getting the following error
    https://files.videohelp.com/u/305113/TSD.png
    https://files.videohelp.com/u/305113/Options.png
    https://files.videohelp.com/u/305113/bestITVX.png

    Hoping someone can help please
    Image Attached Thumbnails Click image for larger version

Name:	bestITVX.png
Views:	169
Size:	59.3 KB
ID:	71970  

    Click image for larger version

Name:	Options.png
Views:	55
Size:	8.3 KB
ID:	71971  

    Click image for larger version

Name:	TSD.png
Views:	108
Size:	29.3 KB
ID:	71972  

    Quote Quote  
  9. Member
    Join Date
    May 2023
    Location
    England
    Search Comp PM
    Thanks very much, just the ticket!

    I don't mix programs and videos but a couple of months ago a similar script wiped out my mp4s in my Completed folder, it was only a couple of hours' worth of downloads but since then I do pay attention to rm commands in scripts!
    Quote Quote  
  10. Member
    Join Date
    Feb 2022
    Location
    Search the forum first!
    Search PM
    Originally Posted by ringy9 View Post
    Hi I'm trying to copy @Angela's script and keep getting the following error


    Hoping someone can help please
    It all looks well set up. Reading through the error response the issue seems to be an unrecognised character in the url. I myself have download this series with no issues.
    My copy from the stream detector
    Code:
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0006
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0005
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0004
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0003
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0002
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    Is your clipboard doing anything different?

    I see Australia is home. ITVX will need a VPN - are you using one? And what is it doing to the urls?
    Quote Quote  
  11. Member
    Join Date
    May 2023
    Location
    England
    Search Comp PM
    Originally Posted by ringy9 View Post
    Hi I'm trying to copy @Angela's script and keep getting the following error
    https://files.videohelp.com/u/305113/TSD.png

    Hoping someone can help please
    Maybe you need to run it like this:

    Code:
    python bestITVX.py
    or (possibly)

    Code:
    python3 bestITVX.py
    ?

    Quote Quote  
  12. Member
    Join Date
    Feb 2022
    Location
    Search the forum first!
    Search PM
    Originally Posted by PhilipG View Post
    Thanks very much, just the ticket!

    I don't mix programs and videos but a couple of months ago a similar script wiped out my mp4s in my Completed folder, it was only a couple of hours' worth of downloads but since then I do pay attention to rm commands in scripts!
    Absolutely! And I have edited the program to avoid wildcard deletes

    Originally Posted by PhilipG View Post
    Originally Posted by ringy9 View Post
    Hi I'm trying to copy @Angela's script and keep getting the following error
    https://files.videohelp.com/u/305113/TSD.png

    Hoping someone can help please
    Maybe you need to run it like this:

    Code:
    python bestITVX.py
    or (possibly)

    Code:
    python3 bestITVX.py
    ?

    They may have made the python executable by setting the 'run as program' bit. chmod +x program name (On my system)
    Quote Quote  
  13. Debbie akduyurye's Avatar
    Join Date
    Jun 2023
    Location
    Knightsbridge, UK
    Search Comp PM
    How flattering, looks like someone has copied my code


    Those looking for a much simpler solution, see below for the original implementation:


    No browser extensions, no clipboard dependencies, just paste your URLs and go.

    Simply copy URLs from the address bar

    E.g.

    Code:
    python itv.py https://www.itv.com/watch/love-island/2a3697/2a3697a0460

    Setup

    Code:
    sudo pacman -S mkvtoolnix-gui
    yay -S n-m3u8dl-re-bin
    pip install httpx selectolax
    Get mp4decrypt
    https://github.com/axiomatic-systems/Bento4#on-linux-and-other-platforms-using-cmake


    Script

    itv.py

    Code:
    import sys
    import platform
    import subprocess
    from base64 import b64encode
    from concurrent.futures import ThreadPoolExecutor, as_completed
    from pathlib import Path
    
    from httpx import URL, Client
    from selectolax.lexbor import LexborHTMLParser
    
    out = Path('out')
    out.mkdir(exist_ok=True,parents=True)
    
    
    def download(url: str) -> None:
        client = Client(headers={'authority':'www.itv.com','user-agent':'AppleWebKit/537.36'},timeout=20)
        title,data = get_data(client,url)
        video = data['Playlist']['Video']
        media = video['MediaFiles']
        mpd_url = [f'{video["Base"]}{y}' for x in media if (y:=URL(x['Href'])).path.endswith('.mpd')][0]
        lic_url = [x['KeyServiceUrl'] for x in media][0]
        pssh = get_pssh(client,mpd_url)
        key = get_key(client,pssh,lic_url)
        m3u8dl = 'N_m3u8DL-RE' if platform.system() in {'Windows','Darwin'} else 'n-m3u8dl-re'
        subprocess.run([
            m3u8dl,
            mpd_url,
            '--append-url-params',
            '--header','cookie: hdntl=~data=hdntl~hmac=' + '.',  # append any random byte
            '--header','host: itvpnpdotcom.blue.content.itv.com',
            '--header',f'user-agent: {client.headers["user-agent"]}',
            '--auto-select',
            '--save-name',title,
            '--save-dir','./out',
            '--tmp-dir','./',
            '-mt',
            '--key',key,
            '-M',
            'format=mkv:muxer=mkvmerge',
            '--no-log',
        ])
    
    
    def get_data(client: Client, url: str) -> tuple:
        html = LexborHTMLParser(client.get(url).text)
        title = html.css_first('title').text()
        magni_url = html.css_first('[data-video-id]').attributes.get('data-video-id')
        featureset = {k:('mpeg-dash','widevine','outband-webvtt','hd','single-track') for k in ('min','max')}
        payload = {'client':{'id':'browser'},'variantAvailability':{'featureset':featureset,'platformTag':'dotcom'}}
        return title, client.post(magni_url,json=payload).json()
    
    
    def get_key(client:Client, pssh:str, lic_url:str) -> str:
        payload = {'license':lic_url,'headers':'connection: keep-alive','pssh':pssh,'buildInfo':'','proxy':'','cache':True}
        return LexborHTMLParser(client.post('https://cdrm-project.com/wv',json=payload).text).css_first('li').text()
    
    
    def get_pssh(client:Client, mpd_url:str) -> str:
        version = '3870737368'  # b'8pssh'
        system_id = 'EDEF8BA979D64ACEA3C827DCD51D21ED'
        data = '48E3DC959B06'
        key_id = (LexborHTMLParser(client.get(mpd_url).text).css_first('ContentProtection').attributes.get('cenc:default_kid').replace('-',''))
        s = f'000000{version}00000000{system_id}000000181210{key_id}{data}'
        return b64encode(bytes.fromhex(s)).decode()
    
    
    def main():
        urls = sys.argv[1:]
        with ThreadPoolExecutor() as e: [f.result() for f in as_completed(e.submit(download,u) for u in urls)]
    
    
    if __name__ == "__main__":
        main()
    Quote Quote  
  14. Member
    Join Date
    Feb 2022
    Location
    Search the forum first!
    Search PM
    Originally Posted by akduyurye View Post
    How flattering, looks like someone has copied my code

    Don't deceive yourself. You are the copyist in chief aren't you? It's no good trying to hide behind pseudonyms. You had to have had an agenda to make a 'debut' post slagging others off as you did; in the process revealing your ego as big as a planet. So who do we know that fits that bill? Who has not shame?
    And since I called you out before for copying my work, I thought it excellent fun to return the favour. But even your post wasn't your original work I spotted the remnants of my old ITVX script. So nothing changes code monkey; nothing much original in your head, is there?
    Last edited by A_n_g_e_l_a; 24th Jun 2023 at 13:39.
    Quote Quote  
  15. Member
    Join Date
    Mar 2023
    Location
    Australia
    Search Comp PM
    It all looks well set up. Reading through the error response the issue seems to be an unrecognised character in the url. I myself have download this series with no issues.
    My copy from the stream detector

    Is your clipboard doing anything different?

    I see Australia is home. ITVX will need a VPN - are you using one? And what is it doing to the urls?[/QUOTE]

    Thanks for the reply @Angela. My clipboard is doing the exact same as yours. I am using a VPN as you asked, "What is it doing to URLs?" They look exactly the same if that's what you mean? I'm a real newbie so perhaps you have a different thought?

    Also tried as suggested python bestITVX.py and python3 BestITVX.py to no avail
    Quote Quote  
  16. Member
    Join Date
    Dec 2020
    Location
    Croatia
    Search PM
    try this

    after this line at the bottom of the file:
    Code:
    urls = PC.paste().split('\n')
    add this code:
    Code:
    for url in urls:
        chars = []
        for char in url:
            chars.append(char)
        print(url)
        print(chars)
    (and make sure it's aligned properly with the rest of the code in "main")

    this should print out the characters that are in every link you pasted before crashing
    copy the printout here (screenshot like you did already or copy/paste in code tags) and let's see if there's any hidden characters there lurking
    Quote Quote  
  17. Member
    Join Date
    Mar 2023
    Location
    Australia
    Search Comp PM
    @ampersand this is the result from your extra code
    Code:
    C:\ffmpeg\bin>new_bestITVX.py
    Press enter when urls in clipboard
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't', 't']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't', 't', 'p']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't', 't', 'p', 's']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't', 't', 'p', 's', ':']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't', 't', 'p', 's', ':', '/']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't', 't', 'p', 's', ':', '/', '/']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't', 't', 'p', 's', ':', '/', '/', 'w']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-', 's']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-', 's', 'p']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-', 's', 'p', 'e']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-', 's', 'p', 'e', 'a']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-', 's', 'p', 'e', 'a', 'k']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-', 's', 'p', 'e', 'a', 'k', 'i']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-', 's', 'p', 'e', 'a', 'k', 'i', 'n']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-', 's', 'p', 'e', 'a', 'k', 'i', 'n', 'g']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-', 's', 'p', 'e', 'a', 'k', 'i', 'n', 'g', '/']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-', 's', 'p', 'e', 'a', 'k', 'i', 'n', 'g', '/', '1']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-', 's', 'p', 'e', 'a', 'k', 'i', 'n', 'g', '/', '1', '0']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-', 's', 'p', 'e', 'a', 'k', 'i', 'n', 'g', '/', '1', '0', 'a']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-', 's', 'p', 'e', 'a', 'k', 'i', 'n', 'g', '/', '1', '0', 'a', '3']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-', 's', 'p', 'e', 'a', 'k', 'i', 'n', 'g', '/', '1', '0', 'a', '3', '3']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-', 's', 'p', 'e', 'a', 'k', 'i', 'n', 'g', '/', '1', '0', 'a', '3', '3', '7']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-', 's', 'p', 'e', 'a', 'k', 'i', 'n', 'g', '/', '1', '0', 'a', '3', '3', '7', '7']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-', 's', 'p', 'e', 'a', 'k', 'i', 'n', 'g', '/', '1', '0', 'a', '3', '3', '7', '7', '/']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-', 's', 'p', 'e', 'a', 'k', 'i', 'n', 'g', '/', '1', '0', 'a', '3', '3', '7', '7', '/', '1']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-', 's', 'p', 'e', 'a', 'k', 'i', 'n', 'g', '/', '1', '0', 'a', '3', '3', '7', '7', '/', '1', '0']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-', 's', 'p', 'e', 'a', 'k', 'i', 'n', 'g', '/', '1', '0', 'a', '3', '3', '7', '7', '/', '1', '0', 'a']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-', 's', 'p', 'e', 'a', 'k', 'i', 'n', 'g', '/', '1', '0', 'a', '3', '3', '7', '7', '/', '1', '0', 'a', '3']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-', 's', 'p', 'e', 'a', 'k', 'i', 'n', 'g', '/', '1', '0', 'a', '3', '3', '7', '7', '/', '1', '0', 'a', '3', '3']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-', 's', 'p', 'e', 'a', 'k', 'i', 'n', 'g', '/', '1', '0', 'a', '3', '3', '7', '7', '/', '1', '0', 'a', '3', '3', '7']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-', 's', 'p', 'e', 'a', 'k', 'i', 'n', 'g', '/', '1', '0', 'a', '3', '3', '7', '7', '/', '1', '0', 'a', '3', '3', '7', '7']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-', 's', 'p', 'e', 'a', 'k', 'i', 'n', 'g', '/', '1', '0', 'a', '3', '3', '7', '7', '/', '1', '0', 'a', '3', '3', '7', '7', 'a']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-', 's', 'p', 'e', 'a', 'k', 'i', 'n', 'g', '/', '1', '0', 'a', '3', '3', '7', '7', '/', '1', '0', 'a', '3', '3', '7', '7', 'a', '0']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-', 's', 'p', 'e', 'a', 'k', 'i', 'n', 'g', '/', '1', '0', 'a', '3', '3', '7', '7', '/', '1', '0', 'a', '3', '3', '7', '7', 'a', '0', '0']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-', 's', 'p', 'e', 'a', 'k', 'i', 'n', 'g', '/', '1', '0', 'a', '3', '3', '7', '7', '/', '1', '0', 'a', '3', '3', '7', '7', 'a', '0', '0', '0']
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-', 's', 'p', 'e', 'a', 'k', 'i', 'n', 'g', '/', '1', '0', 'a', '3', '3', '7', '7', '/', '1', '0', 'a', '3', '3', '7', '7', 'a', '0', '0', '0', '1']
    Traceback (most recent call last):
      File "C:\ffmpeg\bin\new_bestITVX.py", line 178, in <module>
        main()
      File "C:\ffmpeg\bin\new_bestITVX.py", line 173, in main
        ITV().download(url)
      File "C:\ffmpeg\bin\new_bestITVX.py", line 48, in download
        title, data = self.get_data(url)
      File "C:\ffmpeg\bin\new_bestITVX.py", line 114, in get_data
        r = self.client.get(url)
      File "C:\Users\User\AppData\Local\Programs\Python\Python310\lib\site-packages\httpx\_client.py", line 1041, in get
        return self.request(
      File "C:\Users\User\AppData\Local\Programs\Python\Python310\lib\site-packages\httpx\_client.py", line 801, in request
        request = self.build_request(
      File "C:\Users\User\AppData\Local\Programs\Python\Python310\lib\site-packages\httpx\_client.py", line 346, in build_request
        url = self._merge_url(url)
      File "C:\Users\User\AppData\Local\Programs\Python\Python310\lib\site-packages\httpx\_client.py", line 376, in _merge_url
        merge_url = URL(url)
      File "C:\Users\User\AppData\Local\Programs\Python\Python310\lib\site-packages\httpx\_urls.py", line 113, in __init__
        self._uri_reference = urlparse(url, **kwargs)
      File "C:\Users\User\AppData\Local\Programs\Python\Python310\lib\site-packages\httpx\_urlparse.py", line 160, in urlparse
        raise InvalidURL("Invalid non-printable ASCII character in URL")
    httpx.InvalidURL: Invalid non-printable ASCII character in URL
    
    C:\ffmpeg\bin>
    Quote Quote  
  18. Member
    Join Date
    Feb 2022
    Location
    Search the forum first!
    Search PM
    Originally Posted by ringy9 View Post
    @ampersand this is the result from your extra code
    Nothing untoward there.

    let's check your paste. Open a command window (cmd -w) from the 'type here for search' window box.

    type
    Code:
     curl
    and use ctrl v to paste a single url for itvx

    You should see a whole load of data in the command window if the connection is ok . Report back

    I see you have python 3.10 any chance of updating to 3.11? But I'm clutching straws here - somehow your system is seeing a character that shouldn't be there and where it is coming from is a mystery.
    Quote Quote  
  19. Member
    Join Date
    Mar 2023
    Location
    Australia
    Search Comp PM
    @Angela please forgive my ignorance as I don't quite understand your instruction "open a command window cmd -w from the 'type here for search' window box.#" Does that mean the regular search area at the bottom of the windows screen?

    C:\Users\User>curl https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    curl: (35) schannel: next InitializeSecurityContext failed: Unknown error (0x80092012) - The revocation function was unable to check revocation for the certificate.
    Last edited by ringy9; 25th Jun 2023 at 03:51. Reason: extra information
    Quote Quote  
  20. Member
    Join Date
    Feb 2022
    Location
    Search the forum first!
    Search PM
    Originally Posted by ringy9 View Post
    @Angela please forgive my ignorance as I don't quite understand your instruction "open a command window cmd -w from the 'type here for search' window box.#" Does that mean the regular search area at the bottom of the windows screen?

    C:\Users\User>curl https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    curl: (35) schannel: next InitializeSecurityContext failed: Unknown error (0x80092012) - The revocation function was unable to check revocation for the certificate.
    Yes that box. And you've run it and it suggests things are not well with your VPN. When I run curl <url from clipboard> using an ITVX url it dumps the whole page of HTML. So your problem seems to be a connection issue and beyond the scope of my abilities. Ask for technical help from your VPN provider.
    Quote Quote  
  21. Member
    Join Date
    Mar 2023
    Location
    Australia
    Search Comp PM
    @Angela thanks for your efforts
    Quote Quote  
  22. It could be a Windows vs Linux issue. Different platforms will add different characters when copying stuff from the browser, and Win CMD is pretty grumpy about those. Curl on windows can also be a pain in the ass, so it's not necessarily a VPN issue either.

    @ringy9: Before giving up, you can try these things:

    1. At the bottom, edit the script to look like this:

    Code:
    def main() -> int:
        input("Press enter when urls in clipboard ")
        urls = PC.paste().split('\n')
        urls = urls.encode('ascii', 'ignore').decode()
        for url in urls:
            ITV().download(url)
        return 0
    2. Try PowerShell instead of CMD. It's generally more able.

    If it's still not working, then it's most likely a connection issue. But it might be worth a try.
    Quote Quote  
  23. Member
    Join Date
    Feb 2022
    Location
    Search the forum first!
    Search PM
    Originally Posted by stabbedbybrick View Post

    Code:
    def main() -> int:
        input("Press enter when urls in clipboard ")
        urls = PC.paste().split('\n')
        urls = urls.encode('ascii', 'ignore').decode()
        for url in urls:
            ITV().download(url)
        return 0
    .
    Helpful intervention, thanks. But that code produces an error AttributeError: 'list' object has no attribute 'encode'

    Try this:-
    Code:
    def main() -> int:
        input("Press enter with urls in clipboard ")
        urls = PC.paste().split('\n')
        for url in urls:
            url = url.encode('ascii', 'ignore').decode()
            ITV().download(url)
        return 0
    Quote Quote  
  24. Member
    Join Date
    Mar 2023
    Location
    Australia
    Search Comp PM
    @Angela and @stabbedbybrick tried both with cmd same error. I've never used Powershell but do you just call bestITVXest.py from it's command window?
    Quote Quote  
  25. Member
    Join Date
    Feb 2022
    Location
    Search the forum first!
    Search PM
    Assume so; not a Windows user
    Quote Quote  
  26. Yes, you use it in the same way.
    Quote Quote  
  27. Member
    Join Date
    Mar 2023
    Location
    Australia
    Search Comp PM
    @ stabbedbybrick Thanks same error
    Quote Quote  
  28. As a last ditch effort, you could activate the Windows Subsystem for Linux (WSL) and run the script and see if it works. It should be a part of Win 10+ and will allow you to run Linux in a virtual environment while in Windows.

    Other than that, I'm out of ideas.
    Quote Quote  
  29. Member
    Join Date
    Feb 2022
    Location
    Search the forum first!
    Search PM
    Here's a quick test it should output a stream of html if the connection to itvx works. It is minimal; no longer uses the clipboard, so paste to screen, This way we will know if it is your connection or not.

    qtest.py
    Code:
    from httpx import Client
    
    
    
    url = input("Enter a single url \n")
    print(url)
    client = Client(
                headers={
                    'authority': 'www.itv.com',
                    'user-agent': 'Dalvik/2.9.8 (Linux; U; Android 9.9.2; ALE-L94 Build/NJHGGF)',
                })
    resp = client.get(url)
    html = resp.text
    print(html)
    Quote Quote  
  30. Member
    Join Date
    Mar 2023
    Location
    Australia
    Search Comp PM
    Got a heap of code this time
    C:\ffmpeg\bin>qtest.py
    Enter a single url
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
    <!DOCTYPE html><html lang="en"><head><meta charSet="utf-8"/><meta name="viewport" content="width=device-width"/><script id="GTM-MXQXBVC" async="">
    (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':
    new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],
    j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:''; j.async=true;j.src=
    'https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);
    })(window,document,'script','dataLayer','GTM-MXQXBVC');

    The whole list kept failing when I tried to put into code
    Quote Quote  



Similar Threads

Visit our sponsor! Try DVDFab and backup Blu-rays!