ITVX batch downloader that downloads and merges subtitles.
Pick and Mix Mode
Use the Stream Detectorfor pick and mix selection of programme URLs
[Attachment 71939 - Click to enlarge]
Configure the stream detector to store page urls.
[Attachment 71940 - Click to enlarge]
Copy to clipboard from the Stream Detector
Series Mode;
Or there is a Series Mode where you enter a start end end series number to download.
Or when in series mode, enter 'all', where prompted, to download everything.
Enter series mode by starting the program with 'S' as an argument, for example:-
Note that some ITVX programmes have missing series. To cope, this script uses absolute numbers. ITVX may have three series listed. Called series 9,10 and 15, for example The script sees them as series 1 2 and 3Code:python bestITVX.py S
Three style of CDM choices
Choose pywidevine, WKS-KEYS Decryption Module to call. See the script text to set your preference.
I post scripts as examples of possibilities; and for you to adapt. Scripts works for me at the point of posting. Sort out your own issues.
bestITVX.py
Absolutely no support! If you choose to use this software you are expected to be proficient at running python scripts and finding and installing dependencies .Code:# Angela 26:07:2023 # requirements - use pip install for each python module below ''' httpx pyfiglet pyperclip pywidevine selectolax termcolor pysubs2 ''' # google installing python modules if you need help. # also needed will be N_m3u8DL-RE and it's dependencies; ffmpeg, mkvmerge # see https://forum.videohelp.com/threads/408557-Decryption-The-Last-Crusade for details # for PICK & MIX DOWNLOAD #------------------------------------------------------ # PowerShell or terminal window: # start script with # python bestITVX.py # and have a clipboard of video page urls - as many as you like from ITVX # For SERIES DOWNLOAD #------------------------------------------------------- # start program by 'python bestITVX.py S' (use S or s) # and have a single URL ready to PASTE when asked. # Enter:- # 'all' for all series or use series start and end number. # Note:- # ITVX misses out series from sequence sometimes, e.g. Monster Carp - missing series 5 # script uses absolute numbering. If series five is missing, script sees series 6 as the missing 5 etc. ## This program can use The Stream Detector to capture ## multiple page URLs ## Stream Detector select 'options' ## in the box adjacent to'user-defined-commands enter --> %origin% ## In TSD window select copy stream url as User-Defined-Command 1 ## Or for one-off programmes copy the url address in the screen window. ## Changes at ITVX provide better programme curation so now episode titles ## as well as the series name are included in the videoname # Permission to use this software means accepting that # =======there is absolutley no support.============= # Target users are proficient at running python scripts; # if that doesn't decribe you, do not use this software. import re import subprocess from base64 import b64encode from pathlib import Path import httpx from httpx import URL, Client from selectolax.lexbor import LexborHTMLParser import os import pyperclip as PC import pyfiglet as PF from termcolor import colored import json # GLOBALS OUT_PATH = Path('output') OUT_PATH.mkdir(exist_ok=True, parents=True) global SUBS SERIES = False global PYWIDEVINE #@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ # There are choices of Content Decryption Module to use # # 1 Use your own CDM within WKS-KEYS subfolders # Options needs pywidevine/L3/decrypt/wvdecryptcustom and # pywidevine/L3/cdm sub-folder structures in WKS-KEYS (some versions differ) # 2 Local CDM using pywidevine module and a wvd file instead of WKS-KEYS # set PYWIDEVINE= True # # Also set the WVD_PATH point to the location of your wvd file, if not already set, see below. # Ensure the script is not run within the WKS-KEYS folder for this option. # Crib to create wvd file (use full paths): # pywidevine create-device -k "path/to/device_private_key" -c "path/to/device_client_id_blob" -t "ANDROID" -l 3 -o "output_path" # # Running a system with both WKS-KEYS and pywidevine installed is possible. # WKS-KEYS uses an older protobuf version 3.20.3 installed by pip # if you 'pip install pywidevine' after that, pywidevinve expects the protobuf version to be 4.x.x # It will complain that the protobufs need re-compiling # See here for instruction how' # https://forum.videohelp.com/threads/409040-Correcting-Protobuf-Downgrade-to-3-19-0-error#post2684955 #Set false for WKS-KEYS use PYWIDEVINE = False # if using pywidevine set below to your wvd path WVD_PATH = "/home/angela/android_sdk.wvd" class ITV: def __init__(self): self.host = 'itvpnpdotcom.blue.content.itv.com' timeout = httpx.Timeout(10.0, connect=60.0) self.client = Client( headers={ 'authority': 'www.itv.com', 'user-agent': 'Dalvik/2.9.8 (Linux; U; Android 9.9.2; ALE-L94 Build/NJHGGF)', }, timeout=timeout, ) return def download(self, url: str) -> None: title, data = self.get_data(url) video = data['Playlist']['Video'] media = video['MediaFiles'] illegals = "*'%$!(),.:;" replacements = { ' Episode ': 'E', ' Series ': '_S', 'otherepisodes': '_extra', 'ITVX': '', ' ': '_', '&': 'and', '?': '', } # replace extraneous title data and 'illegal' characters videoname = ''.join(c for c in title if c.isprintable() and c not in illegals) # standardize and compact videoname for rep in replacements: videoname = videoname.replace(rep, replacements[rep]) videoname = re.sub(r"(\d+)", pad_number, videoname).lstrip('_').rstrip('_') result = re.search(r"(^.*S\d+)_(E\d*.*)", videoname) # not always a series with episodes try: pre = result.group(1) post = result.group(2) videoname = pre+post except: pass print(videoname) try: subs_url = video['Subtitles'][0]['Href'] subs = self.client.get(subs_url) if subs.status_code==200: SUBS = True f = open(f"{videoname}.subs.vtt", "wb") # bytes needed for N_m's subtitles subtitles = subs.content f.write(subtitles) f.close() # convert subtitles os.system(f"python -m pysubs2 --to srt {videoname}.subs.vtt") else: SUBS=False except: SUBS = False myvideoname = videoname mpd_url = [f'{video["Base"]}{y}' for x in media if (y := URL(x['Href'])).path.endswith('.mpd')][0] lic_url = [x['KeyServiceUrl'] for x in media][0] pssh = self._get_pssh(mpd_url) key = self._get_key(pssh, lic_url) temp = URL(mpd_url).params['hdnea'] temp = temp.replace('nohubplus', 'hdntl,nohubplus') cookie = f"cookie: {re.sub(r'^.*(?<=exp=)', 'hdntl=exp=', temp)}" if SUBS: subs = f"--mux-import:path=./{videoname}.subs.srt:lang=eng::name='English'" else: subs = '--no-log' m3u8dl = 'N_m3u8DL-RE' subprocess.run([ m3u8dl, mpd_url, '--append-url-params', '--header', cookie, '--header', f'host: {self.host}', '--header', f'user-agent: {self.client.headers["user-agent"]}', '--auto-select', '--save-name', myvideoname, '--save-dir', './output', '--tmp-dir', './', '-mt', '--key', key, '-M', 'format=mkv:muxer=mkvmerge', subs, ]) print(f"[info] {myvideoname}.mkv is in {OUT_PATH}") if SUBS: os.remove(f"{videoname}.subs.vtt") def get_series_json(self, url: str) -> list: urls = [] # list r = self.client.get(url) if r.status_code == 200: tree = LexborHTMLParser(r.text) jsondata = tree.root.css_first('#__NEXT_DATA__').text() myjson = json.loads(jsondata) jsonseriesdata = len(myjson["props"]["pageProps"]["seriesList"]) print("To download all series, enter 'all' below.\nOr to download a range, enter the series\nstart number") series_num_start = input("Which series start number? ") if series_num_start == 'all': series_num_start = 0 series_num_end = jsonseriesdata else: series_num_start = int(series_num_start) series_num_end = int(input("Which series end number? ")) series_num_start -= 1 # trap for non sequential and missing series numbers # where last series number is greater that total series if series_num_end > jsonseriesdata: series_num_end = jsonseriesdata parturl = url.rsplit('/',2)[0] #series search programmeId = myjson["query"]["programmeId"] # for j in range(series_num_start, series_num_end): numepisodes = myjson["props"]["pageProps"]["seriesList"][j]["numberOfAvailableEpisodes"] for i in range(0,numepisodes): letterA = myjson["props"]["pageProps"]["seriesList"][j]["titles"][i]["encodedEpisodeId"]["letterA"] urls.append(f"{parturl}/{programmeId}/{letterA}") return urls def get_data(self, url: str) -> tuple: r = self.client.get(url) if r.status_code == 200: tree = LexborHTMLParser(r.text) jsondata = tree.root.css_first('#__NEXT_DATA__').text() myjson = json.loads(jsondata) title = myjson["props"]["pageProps"]["programme"]["title"] try: extendtitle = myjson["props"]["pageProps"]["episode"]["contentInfo"] title = f"{title}_{extendtitle}" except: pass try: magni_url = myjson["props"]["pageProps"]["episode"]["playlistUrl"] except: magni_url = myjson["props"]["pageProps"]["seriesList"][0]["titles"][0]["playlistUrl"] features = ['mpeg-dash', 'widevine', 'outband-webvtt', 'hd', 'single-track'] payload = { 'client': {'id': 'browser'}, 'variantAvailability': { 'featureset': {'min': features, 'max': features}, 'platformTag': 'dotcom', } } r = self.client.post(magni_url, json=payload) return title, r.json() else: print(f"The response code {r.status_code} did not indicate successs!") print(f"The web page content returned was:-\n\n {r.content}\n\nExiting!") print(f"Be sure to copy a page url for a playable video ") exit(1) # Local WKS-KEYS CDM or ywidevine module and wvd file pointing to local CDM # note: create a wvd using the template below # pywidevine create-device -k "KEY_PATH" -c "BLOB_PATH" -t "ANDROID" -l 3 -o "OUTPUT_PATH" def _get_key(self, pssh: str, lic_url: str , cert_b64=None) -> str: if PYWIDEVINE: print("Using pywidevine WVD on this machine") from pywidevine.cdm import Cdm from pywidevine.device import Device from pywidevine.pssh import PSSH pssh = PSSH(pssh) device = Device.load(WVD_PATH) cdm = Cdm.from_device(device) session_id = cdm.open() challenge = cdm.get_license_challenge(session_id, pssh) licence = httpx.post(lic_url, data=challenge, headers=None) #licence.raise_for_status() cdm.parse_license(session_id, licence.content) mykeys = '' for key in cdm.get_keys(session_id): if key.type=='SIGNING': pass else: print(f"Keys found {key.kid.hex}:{key.key.hex()}") mykeys += f"{key.kid.hex}:{key.key.hex()}" cdm.close(session_id) return mykeys else: print("Using CDM on this machine") from pywidevine.L3.cdm import deviceconfig from pywidevine.L3.decrypt.wvdecryptcustom import WvDecrypt wvdecrypt = WvDecrypt(init_data_b64=pssh, cert_data_b64=cert_b64, device=deviceconfig.device_android_generic) widevine_license = httpx.post(url=lic_url, data=wvdecrypt.get_challenge(), headers=None) license_b64 = b64encode(widevine_license.content) wvdecrypt.update_license(license_b64) Correct, keyswvdecrypt = wvdecrypt.start_process() if Correct: mykeys = '' for key in keyswvdecrypt: mykeys += key +' ' print(f"Keys found {mykeys}\n") return mykeys def _get_pssh(self, mpd_url: str) -> str: r = self.client.get(mpd_url) kid = ( LexborHTMLParser(r.text) .css_first('ContentProtection') .attributes.get('cenc:default_kid') .replace('-', '') ) s = f'000000387073736800000000edef8ba979d64acea3c827dcd51d21ed000000181210{kid}48e3dc959b06' return b64encode(bytes.fromhex(s)).decode() # add leading zero to series or episode def pad_number(match): number = int(match.group(1)) return format(number, "02d") def main() -> int: myITV = ITV() if SERIES: urls = [] input("SERIES download: Have the first video link ready in clipboard.\n" + \ "Enter when ready!\n") # ignores any extra urls, # use first for series collection mystr = PC.paste().split('\n')[0] urls = myITV.get_series_json(mystr) else: input("Press enter with PAGE urls in clipboard ") urls = PC.paste().split('\n') count = 0 count = len(urls) print(f"The URL list has {count} video(s)") for url in urls: url = url.encode('utf-8', 'ignore').decode().strip() myITV.download(url) return 0 if __name__ == "__main__": from sys import argv if argv and len(argv) > 1: if argv[1]=='S' or argv[1]=='s': SERIES = True title = PF.figlet_format(' I T V X ', font='smslant') print(colored(title, 'green')) main() exit(0)
+ Reply to Thread
Results 1 to 30 of 217
-
Last edited by A_n_g_e_l_a; 26th Aug 2023 at 04:52. Reason: Updated with experirmental series download facility
-
Sweet, works like a charm, thanks!
[Attachment 71945 - Click to enlarge]
Edit: First time I tried this it worked fine but complained about missing mkvmerge at the end:
Code:FileNotFoundError: [Errno 2] No such file or directory: 'mkvmerge'
Code:20:53:31.038 WARN : Reading media info... 20:53:31.129 INFO : NaN: Audio, aac (mp4a), 96 kb/s 20:53:41.420 WARN : Force Exit...━━━━━━━━━━━━━━━━━━━━━━━━━ 30% 48.32MB/164.01MB 4.92MBps 00:00:24 ⣷ Vid 1280x720 | 1874 Kbps ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 30% 48.32MB/164.01MB 4.92MBps 00:00:24 ⣷Aud 96 Kbps | 2CH ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 20% 9.93MB/49.55MB 1.04MBps 00:00:39 ⣷ Error: The file 'Bodies_of_Evidence__The_Butcher_Surgeon.mp4' could not be opened for reading: open file error. mv: cannot stat 'Bodies_of_Evidence__The_Butcher_Surgeon.mkv': No such file or directory
Still works fine for me as I don't need subs just thought I would point this out.Last edited by vidsrme; 23rd Jun 2023 at 15:21.
-
This error had me going for quite while a few months back.
The full title to the video isCode:Bodies of Evidence: The Butcher Surgeon
I have corrected that to trap it and other 'illegal' characters that cause hiccups. See the edited version above.
[Attachment 71951 - Click to enlarge]
Try again it should work this time. -
-
Great!
I gotta say, though, y'all should be very careful of running a script that hasCode:os.system("rm *.mp4 *.vtt *.srt")
Some of us prefer mp4 to mkv. If you've already downloaded other mp4s etc to that folder (i.e. from another script), this script will happily delete them all. -
Yes sorry about that. I did consider the problem and thought no-one is going to mix programs and videos.
Are they?
If you do mix video and python scripts
Make these edits
Insert this line.
[Attachment 71963 - Click to enlarge]
remove this line
[Attachment 71964 - Click to enlarge]
The line to insert
Code:os.system(f"rm {videoname}.mp4 {videoname}.subs.vtt {videoname}.subs.srt")
Last edited by A_n_g_e_l_a; 24th Jun 2023 at 10:05.
-
Hi I'm trying to copy @Angela's script and keep getting the following error
https://files.videohelp.com/u/305113/TSD.png
https://files.videohelp.com/u/305113/Options.png
https://files.videohelp.com/u/305113/bestITVX.png
Hoping someone can help please -
Thanks very much, just the ticket!
I don't mix programs and videos but a couple of months ago a similar script wiped out my mp4s in my Completed folder, it was only a couple of hours' worth of downloads but since then I do pay attention to rm commands in scripts! -
It all looks well set up. Reading through the error response the issue seems to be an unrecognised character in the url. I myself have download this series with no issues.
My copy from the stream detector
Code:https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0006 https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0005 https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0004 https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0003 https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0002 https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
I see Australia is home. ITVX will need a VPN - are you using one? And what is it doing to the urls? -
-
-
How flattering, looks like someone has copied my code
Those looking for a much simpler solution, see below for the original implementation:
No browser extensions, no clipboard dependencies, just paste your URLs and go.
Simply copy URLs from the address bar
E.g.
Code:python itv.py https://www.itv.com/watch/love-island/2a3697/2a3697a0460
Setup
Code:sudo pacman -S mkvtoolnix-gui yay -S n-m3u8dl-re-bin pip install httpx selectolax
https://github.com/axiomatic-systems/Bento4#on-linux-and-other-platforms-using-cmake
Script
itv.py
Code:import sys import platform import subprocess from base64 import b64encode from concurrent.futures import ThreadPoolExecutor, as_completed from pathlib import Path from httpx import URL, Client from selectolax.lexbor import LexborHTMLParser out = Path('out') out.mkdir(exist_ok=True,parents=True) def download(url: str) -> None: client = Client(headers={'authority':'www.itv.com','user-agent':'AppleWebKit/537.36'},timeout=20) title,data = get_data(client,url) video = data['Playlist']['Video'] media = video['MediaFiles'] mpd_url = [f'{video["Base"]}{y}' for x in media if (y:=URL(x['Href'])).path.endswith('.mpd')][0] lic_url = [x['KeyServiceUrl'] for x in media][0] pssh = get_pssh(client,mpd_url) key = get_key(client,pssh,lic_url) m3u8dl = 'N_m3u8DL-RE' if platform.system() in {'Windows','Darwin'} else 'n-m3u8dl-re' subprocess.run([ m3u8dl, mpd_url, '--append-url-params', '--header','cookie: hdntl=~data=hdntl~hmac=' + '.', # append any random byte '--header','host: itvpnpdotcom.blue.content.itv.com', '--header',f'user-agent: {client.headers["user-agent"]}', '--auto-select', '--save-name',title, '--save-dir','./out', '--tmp-dir','./', '-mt', '--key',key, '-M', 'format=mkv:muxer=mkvmerge', '--no-log', ]) def get_data(client: Client, url: str) -> tuple: html = LexborHTMLParser(client.get(url).text) title = html.css_first('title').text() magni_url = html.css_first('[data-video-id]').attributes.get('data-video-id') featureset = {k:('mpeg-dash','widevine','outband-webvtt','hd','single-track') for k in ('min','max')} payload = {'client':{'id':'browser'},'variantAvailability':{'featureset':featureset,'platformTag':'dotcom'}} return title, client.post(magni_url,json=payload).json() def get_key(client:Client, pssh:str, lic_url:str) -> str: payload = {'license':lic_url,'headers':'connection: keep-alive','pssh':pssh,'buildInfo':'','proxy':'','cache':True} return LexborHTMLParser(client.post('https://cdrm-project.com/wv',json=payload).text).css_first('li').text() def get_pssh(client:Client, mpd_url:str) -> str: version = '3870737368' # b'8pssh' system_id = 'EDEF8BA979D64ACEA3C827DCD51D21ED' data = '48E3DC959B06' key_id = (LexborHTMLParser(client.get(mpd_url).text).css_first('ContentProtection').attributes.get('cenc:default_kid').replace('-','')) s = f'000000{version}00000000{system_id}000000181210{key_id}{data}' return b64encode(bytes.fromhex(s)).decode() def main(): urls = sys.argv[1:] with ThreadPoolExecutor() as e: [f.result() for f in as_completed(e.submit(download,u) for u in urls)] if __name__ == "__main__": main()
-
Don't deceive yourself. You are the copyist in chief aren't you? It's no good trying to hide behind pseudonyms. You had to have had an agenda to make a 'debut' post slagging others off as you did; in the process revealing your ego as big as a planet. So who do we know that fits that bill? Who has not shame?
And since I called you out before for copying my work, I thought it excellent fun to return the favour. But even your post wasn't your original work I spotted the remnants of my old ITVX script. So nothing changes code monkey; nothing much original in your head, is there?Last edited by A_n_g_e_l_a; 24th Jun 2023 at 13:39.
-
It all looks well set up. Reading through the error response the issue seems to be an unrecognised character in the url. I myself have download this series with no issues.
My copy from the stream detector
Is your clipboard doing anything different?
I see Australia is home. ITVX will need a VPN - are you using one? And what is it doing to the urls?[/QUOTE]
Thanks for the reply @Angela. My clipboard is doing the exact same as yours. I am using a VPN as you asked, "What is it doing to URLs?" They look exactly the same if that's what you mean? I'm a real newbie so perhaps you have a different thought?
Also tried as suggested python bestITVX.py and python3 BestITVX.py to no avail -
try this
after this line at the bottom of the file:
Code:urls = PC.paste().split('\n')
Code:for url in urls: chars = [] for char in url: chars.append(char) print(url) print(chars)
this should print out the characters that are in every link you pasted before crashing
copy the printout here (screenshot like you did already or copy/paste in code tags) and let's see if there's any hidden characters there lurking -
@ampersand this is the result from your extra code
Code:C:\ffmpeg\bin>new_bestITVX.py Press enter when urls in clipboard https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't', 't'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't', 't', 'p'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't', 't', 'p', 's'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't', 't', 'p', 's', ':'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't', 't', 'p', 's', ':', '/'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't', 't', 'p', 's', ':', '/', '/'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't', 't', 'p', 's', ':', '/', '/', 'w'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-', 's'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-', 's', 'p'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-', 's', 'p', 'e'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-', 's', 'p', 'e', 'a'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-', 's', 'p', 'e', 'a', 'k'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-', 's', 'p', 'e', 'a', 'k', 'i'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-', 's', 'p', 'e', 'a', 'k', 'i', 'n'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-', 's', 'p', 'e', 'a', 'k', 'i', 'n', 'g'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-', 's', 'p', 'e', 'a', 'k', 'i', 'n', 'g', '/'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-', 's', 'p', 'e', 'a', 'k', 'i', 'n', 'g', '/', '1'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-', 's', 'p', 'e', 'a', 'k', 'i', 'n', 'g', '/', '1', '0'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-', 's', 'p', 'e', 'a', 'k', 'i', 'n', 'g', '/', '1', '0', 'a'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-', 's', 'p', 'e', 'a', 'k', 'i', 'n', 'g', '/', '1', '0', 'a', '3'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-', 's', 'p', 'e', 'a', 'k', 'i', 'n', 'g', '/', '1', '0', 'a', '3', '3'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-', 's', 'p', 'e', 'a', 'k', 'i', 'n', 'g', '/', '1', '0', 'a', '3', '3', '7'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-', 's', 'p', 'e', 'a', 'k', 'i', 'n', 'g', '/', '1', '0', 'a', '3', '3', '7', '7'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-', 's', 'p', 'e', 'a', 'k', 'i', 'n', 'g', '/', '1', '0', 'a', '3', '3', '7', '7', '/'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-', 's', 'p', 'e', 'a', 'k', 'i', 'n', 'g', '/', '1', '0', 'a', '3', '3', '7', '7', '/', '1'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-', 's', 'p', 'e', 'a', 'k', 'i', 'n', 'g', '/', '1', '0', 'a', '3', '3', '7', '7', '/', '1', '0'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-', 's', 'p', 'e', 'a', 'k', 'i', 'n', 'g', '/', '1', '0', 'a', '3', '3', '7', '7', '/', '1', '0', 'a'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-', 's', 'p', 'e', 'a', 'k', 'i', 'n', 'g', '/', '1', '0', 'a', '3', '3', '7', '7', '/', '1', '0', 'a', '3'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-', 's', 'p', 'e', 'a', 'k', 'i', 'n', 'g', '/', '1', '0', 'a', '3', '3', '7', '7', '/', '1', '0', 'a', '3', '3'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-', 's', 'p', 'e', 'a', 'k', 'i', 'n', 'g', '/', '1', '0', 'a', '3', '3', '7', '7', '/', '1', '0', 'a', '3', '3', '7'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-', 's', 'p', 'e', 'a', 'k', 'i', 'n', 'g', '/', '1', '0', 'a', '3', '3', '7', '7', '/', '1', '0', 'a', '3', '3', '7', '7'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-', 's', 'p', 'e', 'a', 'k', 'i', 'n', 'g', '/', '1', '0', 'a', '3', '3', '7', '7', '/', '1', '0', 'a', '3', '3', '7', '7', 'a'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-', 's', 'p', 'e', 'a', 'k', 'i', 'n', 'g', '/', '1', '0', 'a', '3', '3', '7', '7', '/', '1', '0', 'a', '3', '3', '7', '7', 'a', '0'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-', 's', 'p', 'e', 'a', 'k', 'i', 'n', 'g', '/', '1', '0', 'a', '3', '3', '7', '7', '/', '1', '0', 'a', '3', '3', '7', '7', 'a', '0', '0'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-', 's', 'p', 'e', 'a', 'k', 'i', 'n', 'g', '/', '1', '0', 'a', '3', '3', '7', '7', '/', '1', '0', 'a', '3', '3', '7', '7', 'a', '0', '0', '0'] https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001 ['h', 't', 't', 'p', 's', ':', '/', '/', 'w', 'w', 'w', '.', 'i', 't', 'v', '.', 'c', 'o', 'm', '/', 'w', 'a', 't', 'c', 'h', '/', 'r', 'u', 'b', 'y', '-', 's', 'p', 'e', 'a', 'k', 'i', 'n', 'g', '/', '1', '0', 'a', '3', '3', '7', '7', '/', '1', '0', 'a', '3', '3', '7', '7', 'a', '0', '0', '0', '1'] Traceback (most recent call last): File "C:\ffmpeg\bin\new_bestITVX.py", line 178, in <module> main() File "C:\ffmpeg\bin\new_bestITVX.py", line 173, in main ITV().download(url) File "C:\ffmpeg\bin\new_bestITVX.py", line 48, in download title, data = self.get_data(url) File "C:\ffmpeg\bin\new_bestITVX.py", line 114, in get_data r = self.client.get(url) File "C:\Users\User\AppData\Local\Programs\Python\Python310\lib\site-packages\httpx\_client.py", line 1041, in get return self.request( File "C:\Users\User\AppData\Local\Programs\Python\Python310\lib\site-packages\httpx\_client.py", line 801, in request request = self.build_request( File "C:\Users\User\AppData\Local\Programs\Python\Python310\lib\site-packages\httpx\_client.py", line 346, in build_request url = self._merge_url(url) File "C:\Users\User\AppData\Local\Programs\Python\Python310\lib\site-packages\httpx\_client.py", line 376, in _merge_url merge_url = URL(url) File "C:\Users\User\AppData\Local\Programs\Python\Python310\lib\site-packages\httpx\_urls.py", line 113, in __init__ self._uri_reference = urlparse(url, **kwargs) File "C:\Users\User\AppData\Local\Programs\Python\Python310\lib\site-packages\httpx\_urlparse.py", line 160, in urlparse raise InvalidURL("Invalid non-printable ASCII character in URL") httpx.InvalidURL: Invalid non-printable ASCII character in URL C:\ffmpeg\bin>
-
Nothing untoward there.
let's check your paste. Open a command window (cmd -w) from the 'type here for search' window box.
type
Code:curl
You should see a whole load of data in the command window if the connection is ok . Report back
I see you have python 3.10 any chance of updating to 3.11? But I'm clutching straws here - somehow your system is seeing a character that shouldn't be there and where it is coming from is a mystery. -
@Angela please forgive my ignorance as I don't quite understand your instruction "open a command window cmd -w from the 'type here for search' window box.#" Does that mean the regular search area at the bottom of the windows screen?
C:\Users\User>curl https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
curl: (35) schannel: next InitializeSecurityContext failed: Unknown error (0x80092012) - The revocation function was unable to check revocation for the certificate.Last edited by ringy9; 25th Jun 2023 at 03:51. Reason: extra information
-
Yes that box. And you've run it and it suggests things are not well with your VPN. When I run curl <url from clipboard> using an ITVX url it dumps the whole page of HTML. So your problem seems to be a connection issue and beyond the scope of my abilities. Ask for technical help from your VPN provider.
-
It could be a Windows vs Linux issue. Different platforms will add different characters when copying stuff from the browser, and Win CMD is pretty grumpy about those. Curl on windows can also be a pain in the ass, so it's not necessarily a VPN issue either.
@ringy9: Before giving up, you can try these things:
1. At the bottom, edit the script to look like this:
Code:def main() -> int: input("Press enter when urls in clipboard ") urls = PC.paste().split('\n') urls = urls.encode('ascii', 'ignore').decode() for url in urls: ITV().download(url) return 0
If it's still not working, then it's most likely a connection issue. But it might be worth a try. -
Helpful intervention, thanks. But that code produces an error AttributeError: 'list' object has no attribute 'encode'
Try this:-
Code:def main() -> int: input("Press enter with urls in clipboard ") urls = PC.paste().split('\n') for url in urls: url = url.encode('ascii', 'ignore').decode() ITV().download(url) return 0
-
@Angela and @stabbedbybrick tried both with cmd same error. I've never used Powershell but do you just call bestITVXest.py from it's command window?
-
As a last ditch effort, you could activate the Windows Subsystem for Linux (WSL) and run the script and see if it works. It should be a part of Win 10+ and will allow you to run Linux in a virtual environment while in Windows.
Other than that, I'm out of ideas. -
Here's a quick test it should output a stream of html if the connection to itvx works. It is minimal; no longer uses the clipboard, so paste to screen, This way we will know if it is your connection or not.
qtest.py
Code:from httpx import Client url = input("Enter a single url \n") print(url) client = Client( headers={ 'authority': 'www.itv.com', 'user-agent': 'Dalvik/2.9.8 (Linux; U; Android 9.9.2; ALE-L94 Build/NJHGGF)', }) resp = client.get(url) html = resp.text print(html)
-
Got a heap of code this time
C:\ffmpeg\bin>qtest.py
Enter a single url
https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
https://www.itv.com/watch/ruby-speaking/10a3377/10a3377a0001
<!DOCTYPE html><html lang="en"><head><meta charSet="utf-8"/><meta name="viewport" content="width=device-width"/><script id="GTM-MXQXBVC" async="">
(function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':
new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],
j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:''; j.async=true;j.src=
'https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);
})(window,document,'script','dataLayer','GTM-MXQXBVC');
The whole list kept failing when I tried to put into code
Similar Threads
-
Crash course on downloading ITVx hub please?
By adrian44 in forum Video Streaming DownloadingReplies: 301Last Post: 30th Jan 2025, 08:16 -
ITVX & Error 403
By PhilipG in forum Video Streaming DownloadingReplies: 14Last Post: 20th May 2023, 02:39 -
Portable Windows Video DownLoader & SubTitles?
By TEH in forum Video Streaming DownloadingReplies: 4Last Post: 21st Jul 2022, 11:41 -
4K-Video-Downloader and playlist subtitles
By ChasVideo in forum Video Streaming DownloadingReplies: 0Last Post: 11th Sep 2020, 05:49 -
Using TV downloader
By frankopstaele in forum Newbie / General discussionsReplies: 0Last Post: 2nd Feb 2019, 18:44