Trying to record a m3u8 stream from uplynk but having problems with it

23rd Jun 2016 12:06 #1

Member

Hi Everyone,

I'm trying to get a recording of this stream here: www.abc7news.com/live

I dug around the page and found this url:

Code:

http://content.uplynk.com/channel/ext/4413701bf5a1488db55b767f8ae9d4fa/KGOwatch.m3u8?tc=1&exp=1466699965226&rn=3070.9982744148&ct=c&oid=4413701bf5a1488db55b767f8ae9d4fa&eid=KGOwatch&ad=watchkgolive&ad.kv=station%2Ckgo&ad.adUnit=midroll&ad.flex=0&ad._v=2&sig=c78c896a2bfd6e87246863a847d1b041d8117f951267d1609b4522192157c7c0

I've tried several stream recorders to capture that stream with various levels of success. The best one so far is mencoder. I'm using this command line:

Code:

mencoder -nocache -rtsp-stream-over-http "http://content.uplynk.com/channel/ext/4413701bf5a1488db55b767f8ae9d4fa/KGOwatch.m3u8?tc=1&exp=1466699965226&rn=3070.9982744148&ct=c&oid=4413701bf5a1488db55b767f8ae9d4fa&eid=KGOwatch&ad=watchkgolive&ad.kv=station%2Ckgo&ad.adUnit=midroll&ad.flex=0&ad._v=2&sig=c78c896a2bfd6e87246863a847d1b041d8117f951267d1609b4522192157c7c0" -endpos 01:10:00 -oac mp3lame -ovc copy -o "c:\temp\abc_news_%date:~4,2%-%date:~7,2%-%date:~10,4%.avi"

That basically records the show for 1h 10mins to an avi file. Everything works fine EXCEPT: it looks like mencoder captures the lowest quality stream in that broadcast. I know that the KGOwatch.m3u8 has several streams available inside it but mencoder goes for a low quality one. Can I force it to use another stream? How? I've attempted using an url from inside the m3u8 file but I get a forbidden message

m3u8 contents:

Code:

#EXTM3U
#EXT-X-VERSION:5
#EXT-UPLYNK-LIVE
#EXT-X-START:TIME-OFFSET=0.00
#EXT-X-INDEPENDENT-SEGMENTS
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="aac",NAME="unspecified",LANGUAGE="en",AUTOSELECT=YES,DEFAULT=YES
#UPLYNK-MEDIA0:416x234x30,baseline-13,2x48000
#EXT-X-STREAM-INF:PROGRAM-ID=1,RESOLUTION=416x234,BANDWIDTH=471244,CODECS="mp4a.40.5,avc1.42000d",FRAME-RATE=30.000,AUDIO="aac",AVERAGE-BANDWIDTH=413250
http://content-ausc2.uplynk.com/channel/0194acb14e714610b2598d206811d951/d.m3u8
#UPLYNK-MEDIA0:704x396x30,main-30,2x48000
#EXT-X-STREAM-INF:PROGRAM-ID=1,RESOLUTION=704x396,BANDWIDTH=873267,CODECS="mp4a.40.5,avc1.4d001e",FRAME-RATE=30.000,AUDIO="aac",AVERAGE-BANDWIDTH=710215
http://content-ausc2.uplynk.com/channel/0194acb14e714610b2598d206811d951/e.m3u8
#UPLYNK-MEDIA0:896x504x30,main-31,2x48000
#EXT-X-STREAM-INF:PROGRAM-ID=1,RESOLUTION=896x504,BANDWIDTH=1554841,CODECS="mp4a.40.5,avc1.4d001f",FRAME-RATE=30.000,AUDIO="aac",AVERAGE-BANDWIDTH=1249600
http://content-ausc2.uplynk.com/channel/0194acb14e714610b2598d206811d951/f.m3u8
#UPLYNK-MEDIA0:1280x720x30,main-31,2x48000
#EXT-X-STREAM-INF:PROGRAM-ID=1,RESOLUTION=1280x720,BANDWIDTH=3328000,CODECS="mp4a.40.5,avc1.4d001f",FRAME-RATE=30.000,AUDIO="aac",AVERAGE-BANDWIDTH=2607154
http://content-ausc2.uplynk.com/channel/0194acb14e714610b2598d206811d951/g.m3u8
#UPLYNK-MEDIA0:192x108x15,baseline-11,2x48000
#EXT-X-STREAM-INF:PROGRAM-ID=1,RESOLUTION=192x108,BANDWIDTH=117145,CODECS="mp4a.40.5,avc1.42000b",FRAME-RATE=15.000,AUDIO="aac",AVERAGE-BANDWIDTH=110225
http://content-ausc2.uplynk.com/channel/0194acb14e714610b2598d206811d951/b.m3u8
#UPLYNK-MEDIA0:256x144x30,baseline-12,2x48000
#EXT-X-STREAM-INF:PROGRAM-ID=1,RESOLUTION=256x144,BANDWIDTH=236953,CODECS="mp4a.40.5,avc1.42000c",FRAME-RATE=30.000,AUDIO="aac",AVERAGE-BANDWIDTH=225375
http://content-ausc2.uplynk.com/channel/0194acb14e714610b2598d206811d951/c.m3u8

results of using a link from the m3u8 file:

Code:

MEncoder Redxii-SVN-r37872-4.9.3 (x86_64) (C) 2000-2016 MPlayer Team
FFmpeg version: N-80353-g8b069b1
Build date: 2016-06-17 01:39:07 EDT
Resolving content-ausc2.uplynk.com for AF_INET...
Connecting to server content-ausc2.uplynk.com[54.241.162.9]: 80...

Server returned 403: Forbidden
STREAM_ASF, URL: http://content-ausc2.uplynk.com/channel/0194acb14e714610b2598d206811d951/g.m3u8
Resolving content-ausc2.uplynk.com for AF_INET...
Connecting to server content-ausc2.uplynk.com[54.241.162.9]: 80...

Server returned 403:Forbidden
Failed to parse header.
Failed, exiting.
Resolving content-ausc2.uplynk.com for AF_INET...
Connecting to server content-ausc2.uplynk.com[54.241.162.9]: 80...

Server returned 403: Forbidden
No stream found to handle url http://content-ausc2.uplynk.com/channel/0194acb14e714610b2598d206811d951/g.m3u8
Cannot open file/device.

Exiting...

I also tried ffmpeg to capture that stream with this command line:

Code:

ffmpeg -t 00:01:00 -i "http://content.uplynk.com/channel/ext/4413701bf5a1488db55b767f8ae9d4fa/KGOwatch.m3u8?tc=1&exp=1466699377474&rn=1308.5294741229313&ct=c&oid=4413701bf5a1488db55b767f8ae9d4fa&eid=KGOwatch&ad=watchkgolive&ad.kv=station%2Ckgo&ad.adUnit=midroll&ad.flex=0&ad._v=2&sig=ccffcc93c2d2e835e77002b8643aa19f48e52f5bf039c72ed798d35179afa5a6"  -c:v libx264 -c:a copy output.avi

The process starts off fine for about 3 seconds (good quality file output) but then it dies throwing this error:
Non-monotonous DTS in output stream 0:1; previous: 102, current: 9; changing to 103. This may result in incorrect timestamps in the output file.

Here's the actual code that I see:

Code:

ffmpeg version N-80522-ge0faad8 Copyright (c) 2000-2016 the FFmpeg developers
  built with gcc 5.4.0 (GCC)
  configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-dxva2 --enable-libmfx --enable-nvenc --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinger --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzimg --enable-lzma --enable-decklink --enable-zlib
  libavutil      55. 24.100 / 55. 24.100
  libavcodec     57. 46.100 / 57. 46.100
  libavformat    57. 38.102 / 57. 38.102
  libavdevice    57.  0.101 / 57.  0.101
  libavfilter     6. 46.102 /  6. 46.102
  libswscale      4.  1.100 /  4.  1.100
  libswresample   2.  1.100 /  2.  1.100
  libpostproc    54.  0.100 / 54.  0.100
[aac @ 0000000000666320] element type mismatch 1 != 0
[mpegts @ 00000000006355c0] DTS 0 < 354352 out of order
[aac @ 00000000006814c0] element type mismatch 1 != 0
[mpegts @ 000000000064a040] DTS 0 < 354352 out of order
[aac @ 0000000002f7f9a0] element type mismatch 1 != 0
[mpegts @ 00000000006a4420] DTS 0 < 354352 out of order
[aac @ 0000000002f81b60] element type mismatch 1 != 0
[mpegts @ 0000000003f50680] DTS 0 < 354352 out of order
[aac @ 00000000006d7780] element type mismatch 1 != 0
[mpegts @ 000000000444e900] DTS 0 < 354352 out of order
[aac @ 000000000334ff40] element type mismatch 1 != 0
[mpegts @ 000000000452ee60] DTS 0 < 354352 out of order
[aac @ 000000000062ea00] element type mismatch 1 != 0
[aac @ 0000000002f80ce0] element type mismatch 1 != 0
[aac @ 0000000002f81680] element type mismatch 1 != 0
[aac @ 0000000002f7f000] element type mismatch 1 != 0
[aac @ 00000000006d4c40] element type mismatch 1 != 0
[aac @ 000000000334d400] element type mismatch 1 != 0
Input #0, hls,applehttp, from 'http://content.uplynk.com/channel/ext/4413701bf5a1488db55b767f8ae9d4fa/KGOwatch.m3u8?tc=1&exp=1466699377474&rn=1308.5294741229313&ct=c&oid=4413701bf5a1488db55b767f8ae9d4fa&eid=KGOwatch&ad=watchkgolive&ad.kv=station%2Ckgo&ad.adUnit=midroll&ad.flex=0&ad._v=2&sig=ccffcc93c2d2e835e77002b8643aa19f48e52f5bf039c72ed798d35179afa5a6':
  Duration: N/A, start: 0.000000, bitrate: N/A
  Program 0
    Metadata:
      variant_bitrate : 471244
    Stream #0:0(en): Audio: aac (HE-AAC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 78 kb/s (default)
    Metadata:
      comment         : unspecified
      variant_bitrate : 471244
    Stream #0:1: Video: h264 (Constrained Baseline) ([27][0][0][0] / 0x001B), yuv420p, 416x234, 30 fps, 29.97 tbr, 90k tbn, 60 tbc
    Metadata:
      variant_bitrate : 471244
    Stream #0:2: Data: timed_id3 (ID3  / 0x20334449)
    Metadata:
      variant_bitrate : 471244
  Program 1
    Metadata:
      variant_bitrate : 873267
    Stream #0:3(en): Audio: aac (HE-AAC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 96 kb/s (default)
    Metadata:
      comment         : unspecified
      variant_bitrate : 873267
    Stream #0:4: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p, 704x396, 30 fps, 29.97 tbr, 90k tbn, 60 tbc
    Metadata:
      variant_bitrate : 873267
    Stream #0:5: Data: timed_id3 (ID3  / 0x20334449)
    Metadata:
      variant_bitrate : 873267
  Program 2
    Metadata:
      variant_bitrate : 1554841
    Stream #0:6(en): Audio: aac (HE-AAC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      comment         : unspecified
      variant_bitrate : 1554841
    Stream #0:7: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p, 896x504, 30 fps, 29.97 tbr, 90k tbn, 60 tbc
    Metadata:
      variant_bitrate : 1554841
    Stream #0:8: Data: timed_id3 (ID3  / 0x20334449)
    Metadata:
      variant_bitrate : 1554841
  Program 3
    Metadata:
      variant_bitrate : 3328000
    Stream #0:9(en): Audio: aac (HE-AAC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      comment         : unspecified
      variant_bitrate : 3328000
    Stream #0:10: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p, 1280x720, 30 fps, 29.97 tbr, 90k tbn, 60 tbc
    Metadata:
      variant_bitrate : 3328000
    Stream #0:11: Data: timed_id3 (ID3  / 0x20334449)
    Metadata:
      variant_bitrate : 3328000
  Program 4
    Metadata:
      variant_bitrate : 117145
    Stream #0:12(en): Audio: aac (HE-AAC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 30 kb/s (default)
    Metadata:
      comment         : unspecified
      variant_bitrate : 117145
    Stream #0:13: Video: h264 (Constrained Baseline) ([27][0][0][0] / 0x001B), yuv420p, 192x108, 15 fps, 14.99 tbr, 90k tbn, 30 tbc
    Metadata:
      variant_bitrate : 117145
    Stream #0:14: Data: timed_id3 (ID3  / 0x20334449)
    Metadata:
      variant_bitrate : 117145
  Program 5
    Metadata:
      variant_bitrate : 236953
    Stream #0:15(en): Audio: aac (HE-AAC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 48 kb/s (default)
    Metadata:
      comment         : unspecified
      variant_bitrate : 236953
    Stream #0:16: Video: h264 (Constrained Baseline) ([27][0][0][0] / 0x001B), yuv420p, 256x144, 30 fps, 29.97 tbr, 90k tbn, 60 tbc
    Metadata:
      variant_bitrate : 236953
    Stream #0:17: Data: timed_id3 (ID3  / 0x20334449)
    Metadata:
      variant_bitrate : 236953
File 'output.avi' already exists. Overwrite ? [y/N] y
[libx264 @ 000000000567b0e0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
[libx264 @ 000000000567b0e0] profile High, level 3.1
[avi @ 00000000032947a0] Using AVStream.codec to pass codec parameters to muxers is deprecated, use AVStream.codecpar instead.
    Last message repeated 1 times
Output #0, avi, to 'output.avi':
  Metadata:
    ISFT            : Lavf57.38.102
    Stream #0:0: Video: h264 (libx264) (H264 / 0x34363248), yuv420p, 1280x720, q=-1--1, 29.97 fps, 29.97 tbn, 29.97 tbc
    Metadata:
      variant_bitrate : 3328000
      encoder         : Lavc57.46.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
    Stream #0:1(en): Audio: aac (HE-AAC) ([255][0][0][0] / 0x00FF), 48000 Hz, stereo, 78 kb/s (default)
    Metadata:
      comment         : unspecified
      variant_bitrate : 471244
Stream mapping:
  Stream #0:10 -> #0:0 (h264 (native) -> h264 (libx264))
  Stream #0:0 -> #0:1 (copy)
Press [q] to stop, [?] for help
[avi @ 00000000032947a0] Non-monotonous DTS in output stream 0:1; previous: 93, current: 0; changing to 94. This may result in incorrect timestamps in the output file.
[avi @ 00000000032947a0] Non-monotonous DTS in output stream 0:1; previous: 94, current: 1; changing to 95. This may result in incorrect timestamps in the output file.
[hls,applehttp @ 00000000006124a0] DTS 0 < 354352 out of order
[avi @ 00000000032947a0] Non-monotonous DTS in output stream 0:1; previous: 95, current: 2; changing to 96. This may result in incorrect timestamps in the output file.
[avi @ 00000000032947a0] Non-monotonous DTS in output stream 0:1; previous: 96, current: 3; changing to 97. This may result in incorrect timestamps in the output file.
[avi @ 00000000032947a0] Non-monotonous DTS in output stream 0:1; previous: 97, current: 4; changing to 98. This may result in incorrect timestamps in the output file.
[avi @ 00000000032947a0] Non-monotonous DTS in output stream 0:1; previous: 98, current: 5; changing to 99. This may result in incorrect timestamps in the output file.
[avi @ 00000000032947a0] Non-monotonous DTS in output stream 0:1; previous: 99, current: 6; changing to 100. This may result in incorrect timestamps in the output file.
[avi @ 00000000032947a0] Non-monotonous DTS in output stream 0:1; previous: 100, current: 7; changing to 101. This may result in incorrect timestamps in the output file.
[avi @ 00000000032947a0] Non-monotonous DTS in output stream 0:1; previous: 101, current: 8; changing to 102. This may result in incorrect timestamps in the output file.
[avi @ 00000000032947a0] Non-monotonous DTS in output stream 0:1; previous: 102, current: 9; changing to 103. This may result in incorrect timestamps in the output file.

Any thoughts?

Quote

23rd Jun 2016 12:26 #2

Anonymous84347344

Guest

Originally Posted by ser_zone

www.abc7news.com/live

Livestreamer:

Code:

livestreamer --http-header "User-Agent=Mozilla/5.0 (Windows NT 10.0; WOW64; rv:46.0) Gecko/20100101 Firefox/46.0" "hlsvariant://content.uplynk.com/channel/ext/4413701bf5a1488db55b767f8ae9d4fa/KGOwatch.m3u8?tc=1&exp=1466702501183&rn=9638.535741590636&ct=c&oid=4413701bf5a1488db55b767f8ae9d4fa&eid=KGOwatch&ad=watchkgolive&ad.adUnit=midroll&ad.flex=0&ad._v=2&ad.kv=station%2Ckgo%2C_fw_vcid2%2Cc041_6299437393561595251%2Cdatg_yx_bk%2C2HIEq5x499eyjthU&euid=e9cbd587-4291-458c-8a5b-cfa10dd2d4cb_000_0_001_LF&sig=78d6ad45ba8dc8b6dd5b569ecbba4e1e35273cabcf3380238e6af6405e742ab9&ad.cping=1&referer=abc7news.com" best -o ABC7.ts
[cli][info] Found matching plugin stream for URL hlsvariant://content.uplynk.com/channel/ext/4413701bf5a1488db55b767f8ae9d4fa/KGOwatch.m3u8?tc=1&exp=1466702501183&rn=9638.535741590636&ct=c&oid=4413701bf5a1488db55b767f8ae9d4fa&eid=KGOwatch&ad=watchkgolive&ad.adUnit=midroll&ad.flex=0&ad._v=2&ad.kv=station%2Ckgo%2C_fw_vcid2%2Cc041_6299437393561595251%2Cdatg_yx_bk%2C2HIEq5x499eyjthU&euid=e9cbd587-4291-458c-8a5b-cfa10dd2d4cb_000_0_001_LF&sig=78d6ad45ba8dc8b6dd5b569ecbba4e1e35273cabcf3380238e6af6405e742ab9&ad.cping=1&referer=abc7news.com
[cli][info] Available streams: 108p (worst), 144p, 234p, 396p, 504p, 720p (best)
[cli][info] Opening stream: 720p (hls)
[download][ABC7.ts] Written 19.8 MB (1m50s @ 184.5 KB/s)

Code:

Video: MPEG4 Video (H264) 1280x720 30fps [V: h264 main L3.1, yuv420p, 1280x720]
Audio: AAC 48000Hz stereo 128kbps [A: aac he-aac, 48000 Hz, stereo, 128 kb/s]

Quote

23rd Jun 2016 12:46 #3

Anonymous84347344

Guest

Currently there is no live stream on this channel, so I can't test it properly. Additionally I can see EXT-X-DISCONTINUITY tags in their HLS playlist. Not a good sign, if you ask me.

Quote

23rd Jun 2016 13:47 #4

ser_zone

Member

Yes, it looks like I might be out of luck with this stream. I tried the livestreamer and it creates a .ts file as expected but it's not playable by vlc. Also, once you get a recording going can you stop it somehow? Other than hitting ctrl+c to interrupt it?

I've tried mencoder too during the actual broadcast but the stream seems to be "jumping" around, at least during commercials, and when it "returns" the audio is way off.

What is EXT-X-DISCONTINUITY and is there a way to deal with it?

Thanks again for your help.

Quote

23rd Jun 2016 15:09 #5

Anonymous84347344

Guest

Originally Posted by ser_zone

I tried the livestreamer and it creates a .ts file as expected but it's not playable by vlc.

I can open it with MPC-HC, but the file is not seekable and video playback suddenly stops for some seconds during the commercials. I have no time right now to investigate it further, but it seems to be an encoding issue:

EXT-X-DISCONTINUITY tag indicates an encoding discontinuity between the media segment that follows it and the one that preceded it.

Quote

23rd Jun 2016 20:25 #6

ponens

Member Emeritus

It works with the higher quality if you specify the appropriate url to mencoder, but the pbs parameter must be included.

Last edited by ponens; 24th Jun 2016 at 17:18.

Quote

24th Jun 2016 00:27 #7

Anonymous84347344

Guest

Some further investigation: I simply changed the container to MP4. Video playback works fine now (without any interruptions) and the file is seekable.
Command line I've used:
Code:
ffmpeg -i ABC7.ts -c copy -bsf:a aac_adtstoasc ABC7.mp4
Seems that Livestreamer downloads it correctly and I only have to correct the header / metadata with FFMPEG.

Quote

Trying to record a m3u8 stream from uplynk but having problems with it

Thread Tools

Search Thread

Similar Threads

Help with Uplynk HLS/m3u8 from ABC

record Mtv adria m3u8 playlist

How to record a M3U8 streaming on windows 7?

How to record part of live stream m3u8(30min)& save it as mp4 using ffmpeg?

Record M3U8 HLS stream or cache it: possible ?