The site I download from stopped working - how to fix?

[BosseB]

I have been downloading MSNBC news shows from https://livenewschat.eu/politics/ for later viewing and it has worked fine for a long time. But since some time it no longer works.
If I visit that URL I am getting redirected to https://livenewschat.eu/msnbc-live-stream/ instead.
I can play the video fine in my browser (Firefox) using this URL, but my download script has stopped working over the weekend...

When I try to run my normal download command I get this response:

Data from /home/user/www/MSNBC/url_strm.txt
URL used: https://livenewschat.eu/politics/ https://cdn-ca2-na.lncnetworks.host/hls//index.m3u8
[https @ 0x55cf84f14840] HTTP error 404 Not Found
[in#0 @ 0x55cf84f13ec0] Error opening input: Server returned 404 Not Found
Error opening input file https://cdn-ca2-na.lncnetworks.host/hls//index.m3u8.
Error opening input files: Server returned 404 Not Found

Can someone please help?

Details:
I use ffmpeg version N-121199-g5cb6d2221a-20250924 (a more recent version than is provided by apt on Ubuntu) so I can use the moov handling.

My script (a normal bash shellscript) needs the m3u8 url of the video stream in order to retrieve the video and also a working referer URL.

The ffmpeg command looks like this:

CMD="ffmpeg -hide_banner -loglevel warning ${MODE} -i \"${M3U8URL}\" -r 30 -vf scale=w=-4:h=480 -c:v libx264 -preset fast -crf 26 -x264-params keyint=30:scenecut=0:open-gop=0 -c:a aac ${MOVFLCMD} -t ${CAPTURETIME} ${TARGETFILE}"
#

The variables are set as follows:

read -r VIDEOURL M3U8URL < "$URLFILE" #Read the current URL and M3U8 from file
MODE="-referer \"${VIDEOURL}\""
MOVFLCMD="-movflags +frag_keyframe+empty_moov+default_base_moof -frag_duration 2000000"

CAPTURETIME and TARGETFILE are script arguments for the duration in seconds and the output mp4 file name

As can be seen the two crucial items are VIDEOURL and M3U8URL and these are extracted separately just minutes before the download starts using a separate script. I believe the main problem is the m3u8 extraction script command which writes the file...

This is the current content of the URLFILE as created by my getm3u8 script, which worked fine until Friday:

https://livenewschat.eu/msnbc-live-stream/ https://cdn-qc1-na.lncnetworks.host/hls//index.m3u8

I can switch download sites using the m3u8 URL creator script with a site selector for argument and it writes the two items to the file read by the download script.

What I need now is an extraction script command to retrieve the correct VIDEOURL and M3U8URL for the changed site.

Can someone here please provide an extraction command which will work to get the correct referer and m3u8 for the site?

[Gromyko]

change your M3U8URL to https://cdn-tx1-na.lncnetworks.host/hls/msnbc_live/index.m3u8

and add


-referer "https://livenewschat.eu/"


to your


CMD="ffmpeg -hide_banner -loglevel warning ${MODE} -i \"${M3U8URL}\" -r 30 -vf scale=w=-4:h=480 -c:v libx264 -preset fast -crf 26 -x264-params keyint=30:scenecut=0:open-gop=0 -c:a aac ${MOVFLCMD} -t ${CAPTURETIME} ${TARGETFILE}"

[LZAA]

C:\Documents and Settings\Administrator\Desktop>ffmpeg -headers "Referer: https://livenewschat.eu
/msnbc-live-stream/" -hide_banner -i "https://cdn-qc1-na.lncnetworks.host/hls/msnbc_live/index.m3
u8" -c copy v.ts
[https @ 07a9b940] No trailing CRLF found in HTTP header. Adding it.
[hls @ 07a9af80] Skip ('#EXT-X-VERSION:3')
[hls @ 07a9af80] Opening 'https://cdn-qc1-na.lncnetworks.host/hls/msnbc_live/1762791173500.key' f
or reading
[hls @ 07a9af80] Opening 'crypto+https://cdn-qc1-na.lncnetworks.host/hls/msnbc_live/1762791191500
.ts' for reading
Input #0, hls, from 'https://cdn-qc1-na.lncnetworks.host/hls/msnbc_live/index.m3u8':
Duration: N/A, start: 12913.767000, bitrate: N/A
Program 0
Metadata:
variant_bitrate : 0
Stream #0:0: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p(tv, bt709), 720x404 [SAR 1:1
DAR 180:101], 30 fps, 29.97 tbr, 90k tbn
Metadata:
variant_bitrate : 0
Stream #0:1: Audio: aac (HE-AAC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp
Metadata:
variant_bitrate : 0
Output #0, mpegts, to 'v.ts':
Metadata:
encoder : Lavf60.11.100
Stream #0:0: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p(tv, bt709), 720x404 [SAR 1:1
DAR 180:101], q=2-31, 30 fps, 29.97 tbr, 90k tbn
Metadata:
variant_bitrate : 0
Stream #0:1: Audio: aac (HE-AAC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp
Metadata:
variant_bitrate : 0
Stream mapping:
Stream #0:0 -> #0:0 (copy)
Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
frame= 60 fps=0.0 q=-1.0 size= 205kB time=00:00:02.01 bitrate= 831.4kbits/s speed=3.23x
frame= 103 fps= 87 q=-1.0 size= 256kB time=00:00:03.48 bitrate= 601.8kbits/s speed=2.93x
frame= 133 fps= 77 q=-1.0 size= 512kB time=00:00:04.43 bitrate= 945.2kbits/s speed=2.58x
[hls @ 07a9af80] Opening 'https://cdn-qc1-na.lncnetworks.host/hls/msnbc_live/1762791197500.key' f
or reading
[hls @ 07a9af80] Opening 'crypto+https://cdn-qc1-na.lncnetworks.host/hls/msnbc_live/1762791197500
.ts' for reading
frame= 179 fps= 42 q=-1.0 size= 512kB time=00:00:06.01 bitrate= 697.2kbits/s speed=1.42x
frame= 233 fps= 47 q=-1.0 size= 768kB time=00:00:07.78 bitrate= 807.7kbits/s speed=1.59x
frame= 263 fps= 48 q=-1.0 size= 768kB time=00:00:08.79 bitrate= 715.8kbits/s speed=1.59x
...

[BosseB]

change your M3U8URL to https://cdn-tx1-na.lncnetworks.host/hls/msnbc_live/index.m3u8

and add

-referer "https://livenewschat.eu/"

to your

CMD="ffmpeg -hide_banner -loglevel warning ${MODE} -i \"${M3U8URL}\" -r 30 -vf scale=w=-4:h=480 -c:v libx264 -preset fast -crf 26 -x264-params keyint=30cenecut=0pen-gop=0 -c:a aac ${MOVFLCMD} -t ${CAPTURETIME} ${TARGETFILE}"

I ended up in getting a script that gave me:

VIDEOURL="https://livenewschat.eu/msnbc-live-stream/"
MODE="-referer \"${VIDEOURL}\""
and
M3U8URL="https://cdn-qc1-na.lncnetworks.host/hls/msnbc_live/index.m3u8"

The problem earlier was how to extract the full M3U8URL, so it became non-working.
Also it seems like the start of the M3U8URL following // is changing from call to call among about a dozen candidates.

Now running.