Freevine - A downloader for free streaming services

[stabbedbybrick]

The Richard Osman House of Games is a show where the episode numbering is all over the place.

https://www.bbc.co.uk/iplayer/episodes/b094mjv0

S07E02 is named Wednesday. It is Wednesday's episode so should really be S07E03.

Richard.Osmans.House.of.Games.S07E00.Week.1.Friday
Richard.Osmans.House.of.Games.S07E01.Week.1.Thursday
Richard.Osmans.House.of.Games.S07E02.Week.1.Wednesday
Richard.Osmans.House.of.Games.S07E03.Week.1.Tuesday
Richard.Osmans.House.of.Games.S07E04.Week.1.Monday

Ah, crap. It starts at 00 when it shouldn't, resulting in everything being shifted one place. Everything is still the correct episode according to its name, though. So S07E02 is the Wednesday episode, S07E04 is Monday etc. It's just that they're ordered incorrectly for these particular shows. So at the very least, you'll get the episode you're after in the end if you go by name. I guess that's a step forward.

I'll take a look at it at some later time. Dealing with this nightmare of a service is making me tired.

Simply searching on iPlayer for Antiques Roadshow doesn't produce the correct URL at all. Still needs to be fudged.

This is the first search result and it works perfectly fine: https://www.bbc.co.uk/iplayer/episodes/b006mj2y/antiques-roadshow

[stabbedbybrick]

Thank you for fantastic program! It works really well.

Just a question I had, hope its not rude to ask here. I use the ITVX script by Angela but its now broke. Apparantly ITV has made changes and requires some login cookies and tokens. But your ITV still work perfectly without any login. Is ther a reason for this?

Not rude at all. But I'm not aware of any changes being made on ITV. I don't perform any login sessions or collect any cookies at all, and everything works as usual, so whatever is not working for you is because of something else. Angela is probably on top of it.

[Tom Saurus]

peacocktv have 21 jump street tv series so
here key for S04E05

mpd: https://g003-vod-us-cmaf-prd-cl.cdn.peacocktv.com/pub/global/eX7/Eoo/PCK_1617124354077.721_01/cmaf/mpeg_cenc/master_cmaf.mpd?c3.ri=3777202375631605628&audio=all&subtitle=all&forcedNarrative=true
--key 0016eb99393944cfefc41bedf4ec9bca:8a176e793c6bdf80fdfc89df3e5297ee

iamghost: Thank you for your help and also thank you on behalf of many other people who you have helped here at VideoHelp.

[deccavox]

Simply searching on iPlayer for Antiques Roadshow doesn't produce the correct URL at all. Still needs to be fudged.

It worked for me, I did a search for antiques roadshow and I got https://www.bbc.co.uk/iplayer/episodes/b006mj2y/antiques-roadshow
I've tried it on both today's v0.5.4-beta and on the previous v0.5.3-beta

And I can confirm subs are now good to go when I choose option to have them separate.
And
py freevine.py -t https://www.bbc.co.uk/iplayer/episodes/b006nb9z/bargain-hunt
is no longer throwing up a shit load of E00's
Very good stabbedbybrick.

[sipho]

This is the first search result and it works perfectly fine: https://www.bbc.co.uk/iplayer/episodes/b006mj2y/antiques-roadshow

It's a browser issue. I use Opera for iPlayer stuff and that doesn't render the show URL correctly. Chrome works perfectly.

It was late when I tried but should have thought about another browser. Sorry.

@deccavox Thanks.

[Tom Saurus]

I ran across another what I am terming a "dud" file on CTV with "21 Jump Street S04E11". I went to the Browser and looked it up you hover your mouse cursour over the thumbnail for the video and it doesn't change to a hand to select it like the other files do. Seasons 1, 2, 3 and 5 had no such dud files when it comes to 21 Jump Street.

[deccavox]

Your scripts are superb stabbedbybrick but I have a little niggle.

I hate dots in filenames separating individual words.

Now I thought the config.yaml settings could provide the solution.
I had already changed to this:

filename: 
  series: "{title}.{number}.{name}.{resolution}"
  movies: "{title}.{year}.{resolution}"

It worked exactly as it should do. But, as you'd expect, with dots.

So, I tried

filename: 
  series: "{title} {number} {name} [{resolution}]"
  movies: "{title} ({year}) [{resolution}]"

Still effing dots. It ignored the spaces instead of dots. It did sort out the [{resolution}]
here's an example:
Home.and.Away.S25E2497.Tuesday.3.October.[1080p].mp4

It even puts dots within the {name} lol

[stabbedbybrick]

@deccavox:

Yes, that's by design. You can re-arrange and remove sections from the filename, but any empty spaces are replaced by dots. Even within the the {name}. Removing spaces from filenames has been the standard since forever, and dots are used as separator because it's been the scene/P2P convention for 40 years and is the format most peoples' library will look like if they've ever downloaded anything of quality from FTP/torrent/DDL/etc. Using misplaced spaces and odd characters are considered "dirty" and might even cause issues when using any 3rd party tools. Using dots guarantees that whatever you do, the filename won't cause any errors since every system or piece of software dealing with media files in any way will be built on that conventional filename.

I can put it on the list for possible cosmetic changes, but in all honesty, I don't think I'll ever change it.

---------------------------------

In other news, it's come to my attention that Channel4 will serve up the descriptive audio track in random ways, even when specifying otherwise. If anyone here uses it and have gotten the AD track even when specifying "id=audio_eng=128000" in the config file, I'd love to see some examples. Because I can't seem to reproduce it. Everything I download comes with the correct standard audio track.

Thank you.

[sipho]

In other news, it's come to my attention that Channel4 will serve up the descriptive audio track in random ways, even when specifying otherwise. If anyone here uses it and have gotten the AD track even when specifying "id=audio_eng=128000" in the config file, I'd love to see some examples. Because I can't seem to reproduce it. Everything I download comes with the correct standard audio track.

Changing that config line will definitely cause the correct track to be downloaded.

But for certain episodes of certain shows, the manifest has the AD track flagged as default.

When you select -a for those episodes the AD track is the first ID and has the default track tag.

[A_n_g_e_l_a]

Couldn't resist this, sorry!

Universal video-file naming convention in the real World.


[Attachment 74156 - Click to enlarge]

I was looking for the 1990's film Chocolat

[A_n_g_e_l_a]

In other news python module httpx==0.25.0 is to be avoided.

[sipho]

In other news python module httpx==0.25.0 is to be avoided.

And yet oddly enough it works fine with Stabby's freevine script.

[A_n_g_e_l_a]

In other news python module httpx==0.25.0 is to be avoided.

And yet oddly enough it works fine with Stabby's freevine script.

No httpx doesn't raise an issue because stabby, in this instance, calls the older python module 'requests'. Requests is OK but slower than httpx as it doesn't do connection pooling. No big issue; just developer choice.

[stabbedbybrick]

In other news, it's come to my attention that Channel4 will serve up the descriptive audio track in random ways, even when specifying otherwise. If anyone here uses it and have gotten the AD track even when specifying "id=audio_eng=128000" in the config file, I'd love to see some examples. Because I can't seem to reproduce it. Everything I download comes with the correct standard audio track.

Changing that config line will definitely cause the correct track to be downloaded.

But for certain episodes of certain shows, the manifest has the AD track flagged as default.

When you select -a for those episodes the AD track is the first ID and has the default track tag.

Okay, gotcha. So there isn't actually an issue with requesting the specific track then, just sometimes the order if you don't specify. Which was kind of my point to begin with when giving the tip on how to download the correct tracks.

[stabbedbybrick]

Couldn't resist this, sorry!

Universal video-file naming convention in the real World.

74156

I was looking for the 1990's film Chocolat

All I can say is that the scene and internal groups are behind over 90% of everything that gets released, have been for decades, and they all follow the same conventions. I chose to do the same because, similar to how you should program with as much accessibility as possible in mind, using the same file naming as the vast, vast majority of pirated content out there simply makes the most sense in that regard. You might not like it for whatever reason, but with the staggering amount of these releases that exist it in this format it is objectively the most common way of naming video files.

That being said, people can do whatever they want. I like it this way.

And regarding httpx, I currently use it for everything except some singular instances on a couple of services where reading the manifests caused an issue. Which is probably why I haven't noticed any problems with the version. And Requests is neither old or incapable of connection pooling. You use the Session class just like the Client class in httpx. Pretty much the only reason I’m using httpx to begin with is because it comes with async, which I use for a couple of services. The overall difference in speed between the two libraries are imperceptible at best on such a small scale, and Requests is a lot more stable. It's going to be a while before it can be considered replaced.

[scryclwn]

lol stabbedbybrick has the patience of a saint

[Fallito]

@stabbedbybrick,

You should stick to the universal naming convention. For those wanting spaces, there is this utility which runs in every OS, which rename your episodes. It is called : Rename My TV Series. I have been using it for years now in Plex so it can download correctly all the metadata.


https://www.tweaking4all.com/home-theatre/rename-my-tv-series-v2/

[stabbedbybrick]

@stabbedbybrick,

You should stick to the universal naming convention. For those wanting spaces, there is this utility which runs in every OS, which rename your episodes. It is called : Rename My TV Series. I have been using it for years now in Plex so it can download correctly all the metadata.


https://www.tweaking4all.com/home-theatre/rename-my-tv-series-v2/

I actually didn't consider Plex's way of gathering metadata; that's a genuinely good reason to at least have the option to change the filenames.

I'll see if I can add some kind of Plex-friendly options for storing files.

[deccavox]

BBC are so maddening with their numbering. There's already been some examples highlighted above. But how about this one. You'd expect a single series would be series 1 (S01). Just like this one:

https://www.bbc.co.uk/iplayer/episodes/p0gd25kn/union-with-david-olusoga

12:13:36.041 INFO : Union.with.David.Olusoga.S01E01.The.Making.of.Britain
12:13:36.042 INFO : Union.with.David.Olusoga.S01E02.Creating.Britishness
12:13:36.042 INFO : Union.with.David.Olusoga.S01E03.The.Two.Nations
12:13:36.043 INFO : Union.with.David.Olusoga.S01E04.Union.and.Disunion

Until we come to this one:

https://www.bbc.co.uk/iplayer/episodes/b00yyc7b/story-of-ireland

12:21:59.902 INFO : Story.of.Ireland.S00E01.Age.of.Invasions
12:21:59.903 INFO : Story.of.Ireland.S00E02.The.Age.of.Conquest
12:21:59.903 INFO : Story.of.Ireland.S00E03.The.Age.of.Revolution
12:21:59.904 INFO : Story.of.Ireland.S00E04.The.Age.of.Union
12:21:59.905 INFO : Story.of.Ireland.S00E05.Age.of.Nations

S00???? Eh????
Conclusion: make sure you perform a 'py freevine.py -t URL' before downloading stuff.
Performing a download on the latter using s01 instead of s00 will get you nowhere.

@stabbedbybrick. This is absolutely no reflection on you. It's the madness of the BBC.

[optimalberry]

I've been lurking for a while but just registered to say a big thanks to stabbedbybrick for putting a lot of work into this.

I genuinely don't understand what the issue is with the filenames, though?
Unless something changed in the latest versions (I haven't updated), it's what everyone else is using anyways and Plex/Kodi are able to understand that just fine! So color me confused...

[stabbedbybrick]

@stabbedbybrick. This is absolutely no reflection on you. It's the madness of the BBC.

It's on me a little bit, though, since this is how the program works. The problem is that when you're collecting seasons and episodes in an object oriented way like this where you iterate through all the data, you need to find the correlating numbers somehow. If it's a well-designed API, there will be clear and consistent entries for all numbers, titles, data etc for all shows. But for sites like BBC, there is no consistency. The entries change from title to title, and some of them don't have anything to "grab on to" at all. Normally, "00" is meant for specials and extras that are outside of the main show. But with BBC shows that don't have those clear distinctions, the program interprets that as a special and assigns it 00. Same thing with episodes. Which means you have to rely on a bunch of regex and "if else" type of programming, which is really annoying.

If this was a single script meant for only BBC it wouldn't be as difficult to manage. But since I've tried to make Freevine as modular as possible where many functions are serving all streaming services, it makes it a lot more difficult to incorporate changes that don't follow the regular pattern.

But I have it on the list of fixes to be made. At least the actual downloading works correctly.

[stabbedbybrick]

I've been lurking for a while but just registered to say a big thanks to stabbedbybrick for putting a lot of work into this.

I genuinely don't understand what the issue is with the filenames, though?
Unless something changed in the latest versions (I haven't updated), it's what everyone else is using anyways and Plex/Kodi are able to understand that just fine! So color me confused...

You're very welcome.

As for the filenames, I think it's mostly a classic case of ignorance. But I do know that tools like Sonarr have built-in naming schemes for Plex/Jellyfin, so it might be worth looking into eventually anyway.

[sipho]

Running this command to download a BBC episode.

freevine.py -i -e S01E01 https://www.bbc.co.uk/iplayer/episodes/p0g5h206/boiling-point

Throws a hissy fit with these errors.

File "D:\GitHub\freevine\services\bbc.py", line 284, in clean_subtitles
    f.write(srt)
  File "C:\Program Files\Python310\lib\encodings\cp1252.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode character '\u2015' in position 30987: character maps to <undefined>

I snipped a bunch in the middle.

Other episodes worked fine, just the first episode.

I was able to fix the problem by changing line 283 in bbc.py to this.

with open(self.save_path / f"{filename}.srt", "w", encoding='utf-8') as f:

However setting a system variable in Windows also fixed the problem. So not sure if this is something you need to add to your scripts as I'm sure it won't affect those running on Linux or perhaps even MacOS?

[stabbedbybrick]

Utf-8 is the default encoding and shouldn't even be needed to specify. But it if does fix the problem, I guess it does matter in some cases. I'll keep that in mind for the future, thank you.

I've been busy with life stuff and haven't had any time to work on it, but next update should hopefully fix some of these growing pains.

[sipho]

No probs, just wanted to let you know.

[stabbedbybrick]

Small update! (20231008)

You should now be able to customize the file names pretty much however you want. Dots, dashes, underscores, spaces, parentheses, and square brackets are allowed. Each keyword in curly brackets represent a section of the file name, use them however you see fit. I'm sure some of you will be able to break this somehow, just let me know.
NOTE: {year} can be unreliable for series since it's completely up the service API, so if you need to have it included it's recommended to just add it manually with whatever year fits.

BBC iPlayer season and episode number are slightly improved. Shows without any clear numbering should now at least be ordered according to how they're posted on the site. If they appear out of order, it's not my choice. First episodes of seasons for these particular shows might display as E00, but it's how it has to be for now. For any normal shows, the "00" will indicate that it's a special season or episode.

I also added an additional download path to separate movies and series.

[sipho]

Thank you squire. This gets better, brick by brick.

[deccavox]

Thank you squire. This gets better, brick by brick.

Indeed it does. We must be at roof level by now. Well done stabby
I'll check out the update later.

[stabbedbybrick]

New update! (20231010)

You can now download episodes directly by URL (Pluto TV not yet supported):

python freevine.py EPISODE_URL
python freevine.py --quality EPISODE_URL
python freevine.py --info EPISODE_URL

MY5:
Fresh hmac and aes keys are fetched remotely each session, so there's no need to hardcode them in or update them yourself.
They are automatically updated on a regular basis, so they never expire.

[deccavox]

Very good stabby and thanks.
One I meant to mention to you. With ITVX and Emmerdale.
Try this: py freevine.py -t https://www.itv.com/watch/emmerdale/Ya0524

You get:

20:08:51.920 INFO : ITV
20:08:52.088 INFO : Emmerdale: 1 Season(s), 32 Episode(s)

20:08:52.089 INFO : Emmerdale S52E9779
20:08:52.089 INFO : Emmerdale S52E9780
20:08:52.089 INFO : Emmerdale S52E9781
20:08:52.090 INFO : Emmerdale S52E9782
20:08:52.090 INFO : Emmerdale S52E9784
20:08:52.090 INFO : Emmerdale S52E9785
20:08:52.091 INFO : Emmerdale S52E9786
20:08:52.091 INFO : Emmerdale S52E9787
20:08:52.091 INFO : Emmerdale S52E9788
20:08:52.092 INFO : Emmerdale S52E9790
20:08:52.092 INFO : Emmerdale S52E9791
20:08:52.092 INFO : Emmerdale S52E9792
20:08:52.093 INFO : Emmerdale S52E9793
20:08:52.093 INFO : Emmerdale S52E9793
20:08:52.093 INFO : Emmerdale S52E9794
20:08:52.093 INFO : Emmerdale S52E9794
20:08:52.094 INFO : Emmerdale S52E9796
20:08:52.094 INFO : Emmerdale S52E9796
20:08:52.094 INFO : Emmerdale S52E9797
20:08:52.095 INFO : Emmerdale S52E9797
20:08:52.096 INFO : Emmerdale S52E9798
20:08:52.096 INFO : Emmerdale S52E9798
20:08:52.099 INFO : Emmerdale S52E9799
20:08:52.099 INFO : Emmerdale S52E9799
20:08:52.100 INFO : Emmerdale S52E9800
20:08:52.101 INFO : Emmerdale S52E9800
20:08:52.101 INFO : Emmerdale S52E9802
20:08:52.102 INFO : Emmerdale S52E9802
20:08:52.103 INFO : Emmerdale S52E9803
20:08:52.103 INFO : Emmerdale S52E9803
20:08:52.104 INFO : Emmerdale S52E9804
20:08:52.105 INFO : Emmerdale S52E9804

Note how it gives duplicates of episodes and misses some (eg. S52E9801).
You get similar with Corronation Street.

Using STV player instead, you don't get the duplicate episodes. They're all S00 though