I'm new here so don't know if this is the best place to ask but I am specifically looking for a way to do this from a Linux command prompt on a headless server, without using a browser or anything that requires a GUI or desktop environment, or software that only runs in Windows. And I'm pretty certain this will only work in the USA, and probably only if you have an account with a cable or satellite TV provider (some streaming accounts may also work but I don't know for sure).
If I were using a desktop, in a web browser I would open the "Developer Tools" and under the Network tab put m3u8 (or possibly master.m3u8 to be more specific) in the search field. I would then go to https://www.nbc.com/live?brand=nbc&callsign=nbc and if necessary log in (it seems like it doesn't always ask you to log in if you have a previously stored cookie) and then in the Developer Tools pane select the very first m3u8 link that appears. It has a format similar to this:
where the x's replace hexadecimal digits (which may be tied to a specific account, which is why I replaced them) and "unixtimeseconds" replaces a Unix timestamp (in seconds). I am try to find out is if there is a way to obtain this same URL from a Linux command prompt. The reason is that if you can get this URL it will play in VLC and possibly other media players, without the necessity of playing it in a browser window.Code:https://tvenbcaffgroup4-x.akamaized.net/v1/master/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/PROD_NBC_AFFILIATE_G4_PASSTHROUGH/master.m3u8?aws.sessionId=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx&hdnts=exp=unixtimeseconds~acl=/v1/master/*!/Content/*~hmac=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
This link does change every so often so you can't just grab it once and keep using it - by the next day it will have changed. It may change every few minutes for all I know. So I would like to find a way to get this link from the command prompt using one or more Linux commands, or even a shell script. I did try yt-dlp and it does not work, it falls back to the generic extractor and then fails, but I am not trying to save the stream anyway, I just need a way to get that m3u8 link. I'd be grateful for any suggestions as long as they don't require using a desktop environment or running Windows.
+ Reply to Thread
Results 1 to 8 of 8
Last edited by Prairie Dog; 1st Nov 2022 at 14:37.
"I am specifically looking for a way to do this from a Linux command prompt on a headless server, without using a browser or anything that requires a GUI or desktop environment, or software that only runs in Windows."
If I can get this to work it's going to be part of a bash script. If I have to have a browser open it won't work on the machine I want to run it on (which has no desktop and therefore no ability to run Firefox) but also if I have to go into a browser then I can just get the stream link using Developer Tools, therefopre I would not need any Firefox addons. But that would defeat the whole purpose.
I'll give you some pointers.
Study the site code. Start from the top url and work your way down. The m3u8 link is within the code, your challenge is to find it. Once you have found the page that contains the link, work your way back up. Use curl together with the appropriate referer/header info to get the pages. Use a combination of awk, sed and grep to capture the appropriate urls from the pages.
That's all there is to it.
So, as I understand it, any automated process would need to go to 'https://www.nbc.com/live?brand=nbc&callsign=nbc', look to see if it is asking for a login and if so supply the login information, and click anything else necessary to start the stream playing - that much might be doable in a headless browser. But concurrently with that, it would need to go into the browser's menu structure and select "Developer Tools" (would that even exist in a "headless" browser?), click on the Network tab, put 'master.m3u8' in the search box, find the first result once it appears, right click on it and find the item to copy the link and click that, and then save that link to someplace it can be accessed in the calling script. It's that last part that I suspect may not work, because that information is not part of a web page, it is something that is in the browser's memory. It may have arrived as part of the setup for playing the stream, but it is never displayed in the browser's main window.
And all that assumes that you can figure out how to even use a headless browser (and again I would ask if any of those even include Developer Tools) and Selenium. I ran into this once before when trying to figure out how to do something completely different and no one was ever able to point me to any simple setup instructions for use on a Linux server with no desktop. I guess I will ask, has anyone here ever installed and used a headless browser and Selenium under the server (no desktop) version of Ubuntu/Debian/Raspberry Pi OS, or any of their derivatives, for a task even remotely similar to this? If so, how did you get it working? Please keep in mind that not everyone that uses Linux is a Linux expert, and I'm definitely not (nor am I a programmer, unless you call having cobbled together a few short bash scripts programming) so at this point the workings of a headless browser are a complete mystery to me.
Now if there were a web page where one could put in a link like 'https://www.nbc.com/live?brand=nbc&callsign=nbc' and it would cut through all the crap and give you the m3u8 link (kind of like what https://wheregoes.com/ does with shortened URLs) then I could see where a headless browser and Selenium might be useful. But unfortunately in this case I don't even think a service like that would work due to the need for a login (or perhaps a stored cookie).
Beyond that, in that Reddit link all they talked about was what to do with the .m3u8 link once you already have it (for example using it with ffmpeg). I already know what to do with it so that's not the issue, the issue is that I want to be able to get it in a bash script that might be run as part of an unattended or remote process. But I see that guy never got a useful answer to his question either (unless I am completely missing it).
To emulate a real browser from command line I recommend you to use "curl" instead "wget".
You must get the same response from curl than from a real browser if you send the same http headers and treat the cookies in the same way, curl allows to do that easily.
Use curl with the same http headers (like User Agent) than the browser you get the m3u8.
If you need to do a login you can do in curl too, and send data through POST.
You can use mitmproxy to know exactly every byte that your browser sends to server.