UK Free to Air Downloader

[jake2k8]

Thank you again for the good work Angela!

[tenxob]

Now we just need a queueing system to process in order so it can work overnight!!!

[A_n_g_e_l_a]

Now we just need a queueing system to process in order so it can work overnight!!!

Well it is doable so long as there is no token on the mpd that expires too shortly. Just add each compiled N_m3u8DL-RE command to a saved file. Then the system's cron could run a download job, at a set time, reading in each line of the file and downloading. You could probably do that yourself?

Not found cron on windows though.

[peterb999]

You just need to use Task Scheduler in windows instead of CRON.

[A_n_g_e_l_a]

You just need to use Task Scheduler in windows instead of CRON.

Thanks peter, thought there would have to be.

I've roughed out a working solution to create a batch file instead of downloading for all channels. All works fine as does reading the batch file to run the download. Only issue is ITVX; their subs have to be downloaded and muxed into N_m3u8DL_RE's output. That's OK in the main script but not when run as a batch as the subs files do not get saved. I'll accept that. If you want ITVX subs muxed in the video download normally.

Not sure about scheduling - could leave you to set your own cron job or task scheduler...??

[A_n_g_e_l_a]

[Attachment 80437 - Click to enlarge]

Ready to go. No scheduler - you can use your OS' facilities to run getbatch.py whenever you need. Or I think the likely use will see people populating a batch file and processing it straight away, freeing them from waiting for each download to finish. Although it is already possible to have multiple tabs open and downloading.

Testing, tidying and prettifying, expect a release later.

[A_n_g_e_l_a]

https://files.videohelp.com/u/301890/uk-fta_15_0.zip

Batch downloader added. Switch is in Configs. When set all downloads are deferred and a batch file is created. Use the designated menu item to run the batch-file when ready. Just remember some mpd have a timestamp - typically 6 hours - but varies. Allhell3 is included for normal and batch-download and is a slightly tweaked version to remove the stops that would prevent an unattended download.

[tenxob]

"..knock, and it shall be opened unto you"

[A_n_g_e_l_a]

"..knock, and it shall be opened unto you"

Only slightly ajar, the door was, but all the same a couple of bugs managed to creep in!


https://files.videohelp.com/u/301890/uk-fta_15_1.zip remedies

[pssh]

Will ITV 1080p work with L1?

[A_n_g_e_l_a]

Will ITV 1080p work with L1?

You can can have the extra-special secret-developmental super-restricted humongously-secure L007, but it will not make any difference if the CDN does not produce 1080p for web distribution.

[pssh]

Will ITV 1080p work with L1?

You can can have the extra-special secret-developmental super-restricted humongously-secure L007,
but it will not make any difference if the CDN does not produce 1080p for web distribution.

Your reply is in your typically "Angela" style
(I like it especially after the last few days that have been so unlucky for some here in the UK)


The reason I am asking as Devine supports ITV 1080p with L1,
so supplying L1 to UK-FTA will also lead to access to 1080p content?

[A_n_g_e_l_a]

Will ITV 1080p work with L1?

You can can have the extra-special secret-developmental super-restricted humongously-secure L007,
but it will not make any difference if the CDN does not produce 1080p for web distribution.

Your reply is in your typically "Angela" style
(I like it especially after the last few days that have been so unlucky for some here in the UK)


The reason I am asking as Devine supports ITV 1080p with L1,
so supplying L1 to UK-FTA will also lead to access to 1080p content?

No. Simply. IF Devine supports 1080p then it will most likely be using an endpoint meant IPTV. UK-FTA uses web endpoints for ITV. If you want 1080p then STV carries much of ITV output but usually at the expense of bandwidth.

[A_n_g_e_l_a]

Double posted!

[tenxob]

One small observation noted - batch will not download to location with space in name ( e.g., SAVEPATH = r"//nasstation/Data 3/FTADownloads" ).

Pesky Gremlins!

[tenxob]

@Angela Have you deprecated get_iplayer now?

[A_n_g_e_l_a]

One small observation noted - batch will not download to location with space in name ( e.g., SAVEPATH = r"//nasstation/Data 3/FTADownloads" ).
Pesky Gremlins!

Linux does not to use spaces in file paths unless the path is in quote or an escape(\) preceeds the space. e.g
SAVEPATH = r"//nasstation/Data\ 3/FTADownloads"

Windows should accommodate such things automagically. But your save path doesn't have a drive letter ( C: ) ??

@Angela Have you deprecated get_iplayer now?

No.

[tenxob]

@Angela

UNC network path = worked until batch!

[PSXman_uk]

One small observation noted - batch will not download to location with space in name ( e.g., SAVEPATH = r"//nasstation/Data 3/FTADownloads" ).

Pesky Gremlins!

Would say it's the // at the beginning it doesn't like or it wouldn't like that path in windows.

if it's on the C drive for example you could put /Users/psxma/Videos

Tried

SAVEPATH = r"/temp/Ukfta Download"

Downloaded but gave error

Traceback (most recent call last):
File "C:\Scripts\uk-fta\UK-FTA\ukfta\bbc_dl\getBBCLinks.py", line 450, in <module>
main(pid)
File "C:\Scripts\uk-fta\UK-FTA\ukfta\bbc_dl\getBBCLinks.py", line 272, in main
BBC.dodownload(videoname)
File "C:\Scripts\uk-fta\UK-FTA\ukfta\bbc_dl\BBC.py", line 65, in dodownload
print(f"File saved to {OUT_PATH}/{title}")
^^^^^
NameError: name 'title' is not defined. Did you mean: 'time'?

Correction works with my 5 not bbc comes up with an error

[A_n_g_e_l_a]

One small observation noted - batch will not download to location with space in name ( e.g., SAVEPATH = r"//nasstation/Data 3/FTADownloads" ).

Pesky Gremlins!

Would say it's the // at the beginning it doesn't like or it wouldn't like that path in windows.

if it's on the C drive for example you could put /Users/psxma/Videos

Tried

SAVEPATH = r"/temp/Ukfta Download"

Downloaded but gave error

Traceback (most recent call last):
File "C:\Scripts\uk-fta\UK-FTA\ukfta\bbc_dl\getBBCLinks.py", line 450, in <module>
main(pid)
File "C:\Scripts\uk-fta\UK-FTA\ukfta\bbc_dl\getBBCLinks.py", line 272, in main
BBC.dodownload(videoname)
File "C:\Scripts\uk-fta\UK-FTA\ukfta\bbc_dl\BBC.py", line 65, in dodownload
print(f"File saved to {OUT_PATH}/{title}")
^^^^^
NameError: name 'title' is not defined. Did you mean: 'time'?

Correction works with my 5 not bbc comes up with an error

That gremlin in 15.0 is corrected in 15.1. Isn't it?

[A_n_g_e_l_a]

@Angela

UNC network path = worked until batch!

Because batch isn't using the same libraries.

One for you. The script is there ready to adapt. Your starter task is to learn some Python. Explore 'from pathlib import Path' and wrap your offending savepath with Path(). Report back; success or otherwise.

[tenxob]

Useful pointer - investigation commences!

[PSXman_uk]

Got an error message when trying to download series spent from BBC complains about a file existing.

 Enter key word(s) for search : spen
⢿ Loading...
⡿ Loading...Episodes:
Spent: Series 1 - Episode 1, BBC Two, m001zpr7
INFO: 1 total programmes

⣻ Loading...INFO: Processing tv: 'Spent: Series 1 - 01. Episode 1 (m001zpr7)'
INFO: Downloading subtitles [editorial]
⣻ Loading...
⡿ Loading...Episodes:
Spent: Series 1 - Episode 2, BBC Two, m001zprb
INFO: 1 total programmes

⢿ Loading...INFO: Processing tv: 'Spent: Series 1 - 02. Episode 2 (m001zprb)'
INFO: Downloading subtitles [editorial]
Traceback (most recent call last):
  File "C:\Scripts\uk-fta\UK-FTA\ukfta\bbc_dl\getBBCLinks.py", line 450, in <module>
    main(pid)
  File "C:\Scripts\uk-fta\UK-FTA\ukfta\bbc_dl\getBBCLinks.py", line 270, in main
    os.rename( f, "subs.srt")
FileExistsError: [WinError 183] Cannot create a file when that file already exists:
'Spent_Series_1_-_02._Episode_2_m001zprb_editorial.srt' -> 'subs.srt'
⣯ Loading...

[A_n_g_e_l_a]

Got an error message when trying to download series spent from BBC complains about a file existing.

 Enter key word(s) for search : spen
⢿ Loading...
⡿ Loading...Episodes:
Spent: Series 1 - Episode 1, BBC Two, m001zpr7
INFO: 1 total programmes

⣻ Loading...INFO: Processing tv: 'Spent: Series 1 - 01. Episode 1 (m001zpr7)'
INFO: Downloading subtitles [editorial]
⣻ Loading...
⡿ Loading...Episodes:
Spent: Series 1 - Episode 2, BBC Two, m001zprb
INFO: 1 total programmes

⢿ Loading...INFO: Processing tv: 'Spent: Series 1 - 02. Episode 2 (m001zprb)'
INFO: Downloading subtitles [editorial]
Traceback (most recent call last):
  File "C:\Scripts\uk-fta\UK-FTA\ukfta\bbc_dl\getBBCLinks.py", line 450, in <module>
    main(pid)
  File "C:\Scripts\uk-fta\UK-FTA\ukfta\bbc_dl\getBBCLinks.py", line 270, in main
    os.rename( f, "subs.srt")
FileExistsError: [WinError 183] Cannot create a file when that file already exists:
'Spent_Series_1_-_02._Episode_2_m001zprb_editorial.srt' -> 'subs.srt'
⣯ Loading...

You've had some testing sessions probably and prevented things working through. Some subs have been downloaded and not removed by the script as you bailed out too early. If you let scripts finish it all should take care of itself. Just search for subs.srt and remove it (them) and try again.

[tenxob]

Whilst on my way to Great Snake Enlightenment, I have also been testing the Bach feature and found now I can download from BBC ( except for the prog " EPISODES" but now not from MY5.
Following is produced in Batch File:

N_m3u8DL-RE https://akadash0.akamaized.net/hydra/C5439660001/C5439660001C/20240704171229/C5439660001.mpd --header 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36' --auto-select --save-name The_Night_Caller_S01E01 --save-dir \\diskstation\Data4\FTADownloads/MY5/The_Night_Caller --tmp-dir ./tmp --check-segments-count False -mt --use-shaka-packager --key 155c73db4a5f4edcaa2bc682eb166415:dcb728f54f21e7dbd 6c45cec66ae59cc -M format=mkv:muxer=mkvmerge --no-log
N_m3u8DL-RE https://akadash0.akamaized.net/hydra/C5439660002/C5439660002A/20240706223828/C5439660002.mpd --header 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36' --auto-select --save-name The_Night_Caller_S01E02 --save-dir \\diskstation\Data4\FTADownloads/MY5/The_Night_Caller --tmp-dir ./tmp --check-segments-count False -mt --use-shaka-packager --key b6bace1a4f224c5790a62cc70a6b148d:a17d598c202d161dc 0fccf8ef9101adc -M format=mkv:muxer=mkvmerge --no-log
N_m3u8DL-RE https://akadash0.akamaized.net/hydra/C5439660003/C5439660003A/20240708165235/C5439660003.mpd --header 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36' --auto-select --save-name The_Night_Caller_S01E03 --save-dir \\diskstation\Data4\FTADownloads/MY5/The_Night_Caller --tmp-dir ./tmp --check-segments-count False -mt --use-shaka-packager --key d34080bd748d461182b1fd4b798f071b:b6984f085f7ad904a fb0a215dbf0be3d -M format=mkv:muxer=mkvmerge --no-log
N_m3u8DL-RE https://akadash0.akamaized.net/hydra/C5439660004/C5439660004A/20240708205303/C5439660004.mpd --header 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36' --auto-select --save-name The_Night_Caller_S01E04 --save-dir \\diskstation\Data4\FTADownloads/MY5/The_Night_Caller --tmp-dir ./tmp --check-segments-count False -mt --use-shaka-packager --key

[tenxob]

Also, whilst working very well, for some reason, BBC lumps all progs in one BBC folder - not named subfolders?

[A_n_g_e_l_a]

Whilst on my way to Great Snake Enlightenment, I have also been testing the Bach feature and found now I can download from BBC ( except for the prog " EPISODES" but now not from MY5.
Following is produced in Batch File:

N_m3u8DL-RE https://akadash0.akamaized.net/hydra/C5439660001/C5439660001C/20240704171229/C5439660001.mpd --header "'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36'" --auto-select --save-name The_Night_Caller_S01E01 --save-dir \\diskstation\Data4\FTADownloads/MY5/The_Night_Caller --tmp-dir ./tmp --check-segments-count False -mt --use-shaka-packager --key 155c73db4a5f4edcaa2bc682eb166415:dcb728f54f21e7dbd 6c45cec66ae59cc -M format=mkv:muxer=mkvmerge --no-log

It is appears to need quotes round the whole header string. I've added them in red to your quote. Where the issue came from I have no idea as my5 was tested and working.
But never mind. The header string appears not to be needed anyway now... if it ever was.

Edit my5getter.py and comment out the two lines in green below. Put a # at the front as in the image

[Attachment 80653 - Click to enlarge]
In this case less is more!

[A_n_g_e_l_a]

Also, whilst working very well, for some reason, BBC lumps all progs in one BBC folder - not named subfolders?

Yes it is working correctly. I told you getbatch was a quick and dirty solution. It doesn't do subtitles on batch download for BBC or ITV either. These are the trade-offs

[tenxob]

My5getter edited and all working as expected now. Thanks for the pointer.

You didn't mention just how dirty!

[A_n_g_e_l_a]

My5getter edited and all working as expected now. Thanks for the pointer.

You didn't mention just how dirty!

What you are asking for is what any respectable sys admin does daily - filing files. A small batch routine to group-move files into a folder of the file-name stem.

When you ask for add-ons that were not part of the original spec it presents various levels of difficulty integrating what is to become with what is. If starting from scratch, one understands the needs to tag and remember parts for later use. If coming back to a script with and add-ons, sometimes full integration isn't straight-forward. Sometimes in coding a little rain falls.

How did you get on with your task? Did I spot a cheat with \\diskstation\Data4\FTADownloads - have you left out the space this time?

As an aside your Path is incorrectly written. It might get through the interpreter this time but not always. Don't use windows backslashes in Python Paths
And no Path has a double slash. If 'diskstation' signals a Synology NAS then the underlying operating system is Linux anyway. So in Config set your SAVE_PATH=r"/diskstation/Data4/FTADownloads" That is, if, of course, diskstation is really at the root of the NAS - which would be unusual. Unless that is a windows alias for a remote drive. If you have to specify a windows drive letter, just write the letter colon single forward-slash eg 'Z:/diskstation/Data4..... etc