UK Free to Air Downloader

[Akirainblack]

Much better, thank you!
It gets to the download, pulls down the srt which appears in the iplayer recordings folder on the desktop.
Then:

⣽ Loading...Episodes:
Red Dwarf - I, The End, BBC Two, b008nddx
INFO: 1 total programmes

⣾ Loading...INFO: Processing tv: 'Red Dwarf: I - 01. The End (b008nddx)'
INFO: Downloading subtitles [technical]
path empty or file not exists!

Description:
  N_m3u8DL-RE (Beta version) 20230628

Usage:
  N_m3u8DL-RE <input> [options]

Arguments:
  <input>  Input Url or File

Still there with 13.2

[A_n_g_e_l_a]

https://files.videohelp.com/u/301890/uk-fta_13.2.zip with reworked choice of get_iplayer call depending on system.

[white_snake]

def populatetable(pid):
command = [
'get_iplayer.cmd',[/CODE]

Yet a bare get_iplayer works from the command line? Another reason to choose Linux. Thanks for the wrinkle.

Ah, wait, yes, it does work from cmd (sorry Windows), but not when it's called from subprocess.

https://files.videohelp.com/u/301890/uk-fta_13.2.zip with reworked choice of get_iplayer call depending on system.

I was about to say that this is one of the cases where shell=True in subprocess makes things work (not sure if it breaks anything on Linux though). Either way, I just tested your new version and can confirm it works fine on Windows.

[Akirainblack]

def populatetable(pid):
command = [
'get_iplayer.cmd',[/CODE]

Yet a bare get_iplayer works from the command line? Another reason to choose Linux. Thanks for the wrinkle.

Ah, wait, yes, it does work from cmd (sorry Windows), but not when it's called from subprocess.

https://files.videohelp.com/u/301890/uk-fta_13.2.zip with reworked choice of get_iplayer call depending on system.

I was about to say that this is one of the cases where shell=True in subprocess makes things work (not sure if it breaks anything on Linux though). Either way, I just tested your new version and can confirm it works fine on Windows.

Odd, that it isn't fully working for me.
I was going to ask if it could be down to the default location for get_iplayer's output having a space:
C:\Users\<username>\Desktop\iPlayer Recordings

@white_snake
Have you changed anything from default? Are you using the x64 or the x86 version?

[A_n_g_e_l_a]

Odd, that it isn't fully working for me.
I was going to ask if it could be down to the default location for get_iplayer's output having a space:
C:\Users\<username>\Desktop\iPlayer Recordings

Try this version https://files.videohelp.com/u/301890/uk-fta_13.3.zip with White_Snake's suggestion 'shell=True' added.

[white_snake]

Odd, that it isn't fully working for me.
I was going to ask if it could be down to the default location for get_iplayer's output having a space:
C:\Users\<username>\Desktop\iPlayer Recordings

@white_snake
Have you changed anything from default? Are you using the x64 or the x86 version?

I actually can't reproduce your problem because I have another: I can't seem to find an UK proxy that works with BBC (windscr1be worked up until a few days ago), in fact I get:

⣻ Loading...WARNING: No media streams found for requested programme versions and recording quality.
WARNING: The programme may not be available - check the iPlayer or Sounds site.
WARNING: The programme may only be available in an unsupported format (e.g., Flash) - check the iPlayer or Sounds site.
WARNING: If you use a VPN/VPS/Smart DNS/web proxy, it may have been blocked.

[Akirainblack]

Sadly exactly the same.

[BenDWire]

Angela. Firstly a huge thank you for all your work on these scripts. I really appreciate the time you've spent on educating people like me.

You'll be happy to hear that it's all running perfectly on my Debian system, but there is a small bug in vadapav.py. I noticed from the standalone version that if the DOWNLOAD_FOLDER and SAVE_PATH are set to the same place then the script can't resume. Your latest version implicitly does this, so expect a barrage of users complaining in 3 ... 2 ... 1 ...

The fix is trivial for you, I'm sure. However, for those who have already noticed this then edit vadapav.py and add something like the following:

Line 47      DOWNLOAD_FOLDER = Path(f"{config.SAVEPATH}/VADA/Download")
Line 58      DOWNLOAD_FOLDER.mkdir(exist_ok=True, parents=True)

Thanks again, and best wishes.

[A_n_g_e_l_a]

Sadly exactly the same.

well you are on your own on this one. If you can install VS Code you could run getBBCLinks using a debugger. Following each command through until it fails will point you at the error. get_iplayer works though - just a bit more effort. Or freevine - that may work as it doesn't use get_iplayer.

[A_n_g_e_l_a]

Angela. Firstly a huge thank you for all your work on these scripts. I really appreciate the time you've spent on educating people like me.

You'll be happy to hear that it's all running perfectly on my Debian system, but there is a small bug in vadapav.py. I noticed from the standalone version that if the DOWNLOAD_FOLDER and SAVE_PATH are set to the same place then the script can't resume. Your latest version implicitly does this, so expect a barrage of users complaining in 3 ... 2 ... 1 ...

The fix is trivial for you, I'm sure. However, for those who have already noticed this then edit vadapav.py and add something like the following:

Line 47      DOWNLOAD_FOLDER = Path(f"{config.SAVEPATH}/VADA/Download")
Line 58      DOWNLOAD_FOLDER.mkdir(exist_ok=True, parents=True)

Thanks again, and best wishes.

Golly! You're right I remember making the change while cleaning up code and thinking it will make no difference.

I'll sort it out tomorrow. A clean uninterrupted download works fine - and that is as far as I tested.

[A_n_g_e_l_a]

https://files.videohelp.com/u/301890/uk-fta_13.4.zip fixes vadapav download resume error.

[Akirainblack]

Sadly exactly the same.

well you are on your own on this one. If you can install VS Code you could run getBBCLinks using a debugger. Following each command through until it fails will point you at the error. get_iplayer works though - just a bit more effort. Or freevine - that may work as it doesn't use get_iplayer.

It's absolutely not a problem. I'll just use get_iplayer seperately.
Everything else works absolutely amazingly.
I was thinking of trying to put this into a docker container which may well make things easier in the end.
I'm slowly making progress with it.

[A_n_g_e_l_a]

Sadly exactly the same.

well you are on your own on this one. If you can install VS Code you could run getBBCLinks using a debugger. Following each command through until it fails will point you at the error. get_iplayer works though - just a bit more effort. Or freevine - that may work as it doesn't use get_iplayer.

It's absolutely not a problem. I'll just use get_iplayer seperately.
Everything else works absolutely amazingly.
I was thinking of trying to put this into a docker container which may well make things easier in the end.
I'm slowly making progress with it.

Some Windows problems are caused by the command window not having enough privileges. Windows Home usually is the culprit. You might try the 'Terminal app', from windows store, and compare the difference.

[A_n_g_e_l_a]

I was about to say that this is one of the cases where shell=True in subprocess makes things work (not sure if it breaks anything on Linux though). Either way, I just tested your new version and can confirm it works fine on Windows.

shell=True is not needed. I took a look at the code more carefully this morning and the subprocess call doesn't make it to shell display; it is captured.

The 'command' variable contains the get_iplayer string,

moreresults = subprocess.run(command, capture_output=True, \
                               text=True, stderr=None)

I can vaguely remember having problems coding this as we are calling get_iplayer and it wants to take over the download. But instead we just want its list. So capture_output=True absolves the iplayer shell of any responsibility.

Windows cmd.exe is being asked to do a lot here . It is recommended to use Windows Terminal which more faithfully follows Linux.

[white_snake]

I was about to say that this is one of the cases where shell=True in subprocess makes things work (not sure if it breaks anything on Linux though). Either way, I just tested your new version and can confirm it works fine on Windows.

shell=True is not needed. I took a look at the code more carefully this morning and the subprocess call doesn't make it to shell display; it is captured.

The 'command' variable contains the get_iplayer string,

moreresults = subprocess.run(command, capture_output=True, \
                             text=True, stderr=None)

I can vaguely remember having problems coding this as we are calling get_iplayer and it wants to take over the download. But instead we just want its list. So capture_output=True absolves the iplayer shell of any responsibility.

Windows cmd.exe is being asked to do a lot here . It is recommended to use Windows Terminal which more faithfully follows Linux.

With

iplayer = 'get_iplayer'

it does make a difference for me:

moreresults = subprocess.run(command, capture_output=True,
                             text=True, stderr=None, shell=True)

so shell=True is needed for the code to correctly run on Windows + cmd.exe. Also, the fact that the output is captured doesn't mean that the get_iplayer command isn't called from the cmd shell: when you're in the program description screen and it tells you "(Confirm with enter)", after I press enter I see another instance of cmd.exe being run for a brief second and then closing again (which doesn't happen without shell=True), confirming cmd.exe is being used.

This said, I fail to see why you say it's not needed (at least for Windows users and assuming it doesn't break anything on Linux).

Just to be perfectly clear: of course it's not needed if you already have the system check you wrote:

    if os.name == 'posix':
        iplayer = 'get_iplayer'
    else:
        iplayer = 'get_iplayer.cmd'

but without it, with the original code, shell=True is needed to make things work on Windows.

[A_n_g_e_l_a]

This said, I fail to see why you say it's not needed (at least for Windows users and assuming it doesn't break anything on Linux).

This current release is 13 and in all the earlier releases no one has ever said BBC was broken. And that was with running with get_iplayer and without shell=True. But I note what you say.

[Akirainblack]

Sadly exactly the same.

well you are on your own on this one. If you can install VS Code you could run getBBCLinks using a debugger. Following each command through until it fails will point you at the error. get_iplayer works though - just a bit more effort. Or freevine - that may work as it doesn't use get_iplayer.

It's absolutely not a problem. I'll just use get_iplayer seperately.
Everything else works absolutely amazingly.
I was thinking of trying to put this into a docker container which may well make things easier in the end.
I'm slowly making progress with it.

Some Windows problems are caused by the command window not having enough privileges. Windows Home usually is the culprit. You might try the 'Terminal app', from windows store, and compare the difference.

I'm already running Terminal. I've tried both standard and elevated. It's interesting that it pulls the SRT but then falls over.

[white_snake]

This said, I fail to see why you say it's not needed (at least for Windows users and assuming it doesn't break anything on Linux).

This current release is 13 and in all the earlier releases no one has ever said BBC was broken. And that was with running with get_iplayer and without shell=True. But I note what you say.

One would think it would have happened before, I feel you. I'm on 10.0.19045.4046, not sure it this was fixed in Win11 or if it worked in past versions. Either way, if it doesn't break anything, I would leave shell=True there for maximum compatibility.

Now that we're at it, wanna know another funny little bug for Windows users? This time cmd is the culprit. When I run menu.py I don't see colors, but the color codes instead:

←[32m     __  ____ __    _____________
    / / / / //_/___/ __/_  __/ _ |
   / /_/ / ,< /___/ _/  / / / __ |
   \____/_/|_|   /_/   /_/ /_/ |_|

←[0m
←[31mChoose Your Search, Selector and Downloader.
←[0m
←[32mWhich Channel?←[0m

On Windows this is fixed by adding

os.system('color')

in the first lines of menu.py. "But where's the fun part?" you might say. Well, the submenus all work fine even without that line. Although, if I run them directly:

python getBBCLinks.py

bug presents itself again.

←[32m     ___    ___    _____
    / _ )  / _ )  / ___/
   / _  | / _  | / /__
  /____/ /____/  \___/

←[0m
←[31mA BBC Video Selector and Downloader.

←[0m
Enter the Series url:

Just Windows things. ¯\_(ツ)_/¯

[A_n_g_e_l_a]

I'm already running Terminal. I've tried both standard and elevated. It's interesting that it pulls the SRT but then falls over.

I've pulled out an old windows machine and have the same error. I hate windows!! Everything about it seems designed to stop ypu doing what you need to do.

But I'm closer; now something is upsetting N_m3u8DL-RE.

[A_n_g_e_l_a]

Now that we're at it, wanna know another funny little bug for Windows users? This time cmd is the culprit. When I run menu.py I don't see colors, but the color codes instead:

←[32m     __  ____ __    _____________
    / / / / //_/___/ __/_  __/ _ |
   / /_/ / ,< /___/ _/  / / / __ |
   \____/_/|_|   /_/   /_/ /_/ |_|

←[0m
←[31mChoose Your Search, Selector and Downloader.
←[0m
←[32mWhich Channel?←[0m

[/CODE]

Just Windows things. ¯\_(ツ)_/¯

The use of Terminal corrects colour and odd characters. You'll get a pretty close rendition to what I see on Linux.

[white_snake]

The use of Terminal corrects colour and odd characters. You'll get a pretty close rendition to what I see on Linux.

Yup Terminal works fine, same thing goes for the VS Code terminal. Strangely enough it breaks in Powershell (5.1).

[A_n_g_e_l_a]

I'm already running Terminal. I've tried both standard and elevated. It's interesting that it pulls the SRT but then falls over.

Narrowed it down to the N_M3u8DL-RE command. It doesn't like mkvmerge as a bare name. Help says give a binary path but it complains when I do that too.

Had it running when formatted as mp4 but it took an age; ffmpeg complained about timestamps endlessly.

If you want to test with mp4 find '--format=mkv:muxer=mkvmerge',

and replace with

'--format=mp4',

And comment out the lines
'--mux-import',
'path=subs.srt:lang=eng:name="English"',

[A_n_g_e_l_a]

OK. Red herrings apart I seem to have arrived at a fix. If your getBBCLinks.py fails ON A WINDOWS MACHINE ONLY then replace the uk-fta/bbc_dl/getBBCLinks.py with this new file - https://files.videohelp.com/u/301890/getBBCLinks.bbc_version. Rename the new file back to getBBCLinks.py.

I think the issue devolves down to get_iplayer: it was saving the subtitle file to a location in C:\Uses\Uname\Desktop\iPlayer Recordings. You need to find where get_iplayer is storing yours and edit a line declaring the location at the top of the new script.

[Akirainblack]

@A_n_g_e_l_a you're a GENIUS. It works PERFECTLY now! Thank you so much!

[stevepen1974]

Upgraded from fta2 to fta13.4.

fta13.4 runs ok with done displayed and no errors. The files are now muxed to video format quicktime and will not play.

Running both fta2 and fta13.4 in env.

fta2 works perfect with no mux issues, just wondered if anyone can confirm what the issue is with fta13.4?

Cheers

Steve

[A_n_g_e_l_a]

The files are now muxed to video format quicktime Upgraded from fta2 to fta13.4

fta2 works perfect with no mux issues, just wondered if anyone can confirm what the issue is with fta13.4?

On a Mac I assume. I really have little idea especially as fta2 is no longer available to check back with.

I did make a format change from mp4 to mkv in the N_m3u8DL-RE mux process and perhaps is that which is giving quicktime errors.

I suggest you edit the mux command in one of the files below and see what happens.

./c4_dl/C4.py
./itv_dl/ITVX.py
./my5_dl/my5getter.py
./stv_dl/stvgetter.py
./uktvp/slimuktvp.py
./tptvencore/TPTV.py
./bbc_dl/BBC.py
./bbc_dl/getBBCLinks.py

Search and find

'format=mkv:muxer=mkvmerge',

in one of the files for the service of your choice and change the line
to

'format=mp4',

If doing it on linux I would use something like this to edit in place for all files (after making a backup). Not tested for Mac!!

grep -rl  format=mkv . | xargs sed -i 's/format=mkv:muxer=mkvmerge/format=mp4)/g'

run this code from the ukfta folder

[stevepen1974]

The files are now muxed to video format quicktime Upgraded from fta2 to fta13.4

fta2 works perfect with no mux issues, just wondered if anyone can confirm what the issue is with fta13.4?

On a Mac I assume. I really have little idea especially as fta2 is no longer available to check back with.

I did make a format change from mp4 to mkv in the N_m3u8DL-RE mux process and perhaps is that which is giving quicktime errors.

I suggest you edit the mux command in one of the files below and see what happens.

./c4_dl/C4.py
./itv_dl/ITVX.py
./my5_dl/my5getter.py
./stv_dl/stvgetter.py
./uktvp/slimuktvp.py
./tptvencore/TPTV.py
./bbc_dl/BBC.py
./bbc_dl/getBBCLinks.py

Search and find

'format=mkv:muxer=mkvmerge',

in one of the files for the service of your choice and change the line
to

'format=mp4',

If doing it on linux I would use something like this to edit in place for all files (after making a backup). Not tested for Mac!!

grep -rl  format=mkv . | xargs sed -i 's/format=mkv:muxer=mkvmerge/format=mp4)/g'

run this code from the ukfta folder

Thanks Angela,

File did change to mp4, but did not decrypt. No errors flagged.

I have replaced c4.py from fta2 in fta 13.4, issues remains, muxs to quicktime.


Forgot to mention I am running this on Windows 11.

Cheers

[A_n_g_e_l_a]

Thanks Angela,

File did change to mp4, but did not decrypt. No errors flagged.

I have replaced c4.py from fta2 in fta 13.4, issues remains, muxs to quicktime.


Forgot to mention I am running this on Windows 11.

Cheers

Windows 11 and Quicktime?????

Have you seen this

[Attachment 77815 - Click to enlarge]

[stevepen1974]

Thanks Angela,

File did change to mp4, but did not decrypt. No errors flagged.

I have replaced c4.py from fta2 in fta 13.4, issues remains, muxs to quicktime.


Forgot to mention I am running this on Windows 11.

Cheers

Windows 11 and Quicktime?????

Have you seen this

[Attachment 77815 - Click to enlarge]

Amended c4.py as per:

TMP_DIR,
'--use-shaka-packager',
'--decryption-binary-path', # windows users: uncomment this and next. Edit next path
'./shaka-packager.exe',
'--key',

Running this :

https://github.com/shaka-project/shaka-packager/releases/download/v3.0.3/mpd_generator-win-x64.exe

Result the same, muxed file is quicktime/un-playable?

Cheers

[A_n_g_e_l_a]

https://github.com/shaka-project/shaka-packager/releases/download/v3.0.3/mpd_generator-win-x64.exe

Result the same, muxed file is quicktime/un-playable?

Cheers

Well if you are saying it worked on v2 nothing around decryption has changed very much. You'll of course seen that the script is fetching keys. What I suggest is to let the script fetch keys. I think C4 shows the mpd too. So with both those you could use N_m3u8DL-RE to download the encrypted video and audio stream and then decrypt them by hand using mp4decrypt and the keys provided by the script.

I'm confused when you say Quicktime as that is an Apple Mac proprietary codec AFAIK. How can you tell it is Quicktime? Have you tried to use VLC as your player? That comes with all the codecs for playback you need.

If the script completes without an error being shown, it really suggests something elsewhere is wrong.