View Profile
View Forum Posts
Private Message
Hi!
There's certain streaming I'd like to save but it won't be available till the moment it starts. I have the link though (it says unavailable if you open it now), and it's in this format: http://vmf.edge-apps.net/embed/live.php?streamname=<name of the stream>&autoplay=true
I've installed livestreamer and wanted to try recording something else from "vmf.edge-apps.net" to try it out, since there's no time for trying things when the stream I want starts.
So, there's this TV channel, http://www.tvpublica.com.ar/vivo/, that if you look in the page's source you can find a link in the same format as above. Here's the link: http://vmf.edge-apps.net/embed/live.php?streamname=c7live01-20034&autoplay=true
I looked around and found this thread: https://forum.videohelp.com/threads/379186-Tricky-stream-to-download-%28
A kind user there shows the OP the command line for livestreamer, but I'm curious to know how did he extract the "hlsvariant://" link, given that the OP only posted a link from vmf.edge-apps.net
Thanks for any help
Try StreamFab Downloader and download from Netflix, Amazon, Youtube! Or Try DVDFab and copy Blu-rays! or rip iTunes movies!
+ Reply to Thread
Results 1 to 11 of 11
Thread
-
-
When the stream is active look for the playlist url in dev tools network.
Last edited by ponens; 20th Jun 2018 at 01:41.
-
I didn't find anything like that, and neither with 'playback, 'session' or 'json' (there's something that contains the latter but doesn't seem like anything relevant). I even tried looking manually for a long string of characters, but nothing similar shows up.
Edit: I made some "progress". I installed the VideoDownloadHelper addon and it does show me an m3u8 playlist on the TV channel. The playlist however doesn't seem to contain anything if I download it.
A 403 error. If I check the details I get the playlist's link. So I tried to loading it with livestreamer, replacing http:// with hlsvariant://<!DOCTYPE html>
<html>
<head>
<title>403 Access denied.</title>
</head>
<body>
<h1>Error 403 Access denied.</h1>
<p>Access denied.</p>
<h3>Guru Meditation:</h3>
<p>XID: 93265376</p>
<hr>
<p>Varnish cache server</p>
</body>
</html>
livestreamer --http-header "Referer=http://vmf.edge-apps.net/embed/live.php?streamname=c7live01-20034" "hlsvariant://g3.vxral-slo.transport.edge-access.net/b17/ngrp:c7cam2-20034_all/c7cam2-20034_720p.m3u8" best -o Livestream.ts
and livestreamer's response:
Dead end?[cli][info] Found matching plugin stream for URL hlsvariant://g3.vxral-slo.transport.edge-access.net/b17/ngrp:c7cam2-20034_all/c7cam2-20034_720p.m3u8
error: No streams found on this URL: hlsvariant://g3.vxral-slo.transport.edge-access.net/b17/ngrp:c7cam2-20034_all/c7cam2-20034_720p.m3u8
Edit 2: Even MORE progress. Actually that add-on is not really needed. By having the TV channel's page open with video playing you can toggle the developer tools and go to the network tab. Now *this* is interesting. Every few seconds 3 files show up. An .m3u8, .key an .ts files. Right clicking the m3u8 and "copy as curl" gets me the original link and the playlist link
The playlist is different from what the add-on had, but whatever. Livestreamer still doesn't work. If I try downloading that playlist I get the 403 error. BUT if I choose "see response" in the network logger I get *something*curl "https://g2.mc-hor.transport.edge-access.net/b17/ngrp:c7cam2-20034_all/c7cam2-20034_720p.m3u8" -H "Host: g2.mc-hor.transport.edge-access.net" -H "User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:56.0) Gecko/20100101 Firefox/56.0" -H "Accept: */*" -H "Accept-Language: en-GB,en-US;q=0.7,en;q=0.3" --compressed -H "Referer: http://vmf.edge-apps.net/embed/live.php?streamname=c7live01-20034&autoplay=true" -H "Origin: http://vmf.edge-apps.net" -H "Connection: keep-alive" -H "If-Modified-Since: Wed, 20 Jun 2018 07:28:28 GMT"
It was much longer, I trimmed it here. That does look like a playlist now. "AES-128" pops up, along with the .key file it seems there's some kind of encryption here, and I have no idea what to do with livestreamer now.Code:#EXTM3U #EXT-X-VERSION:3 #EXT-X-MEDIA-SEQUENCE:32871 #EXT-X-TARGETDURATION:4 #EXT-X-KEY:METHOD=AES-128,URI="https://vmf.edge-apps.net/keys/00D2A735511A71B0D8449A57CF2520AA/c7cam2-20034_720p-1529475921665.key",IV=0x0000000000000000000001641BDE7701 #EXTINF:4.000, c7cam2-20034_720p-1529475924580.ts #EXTINF:4.000, c7cam2-20034_720p-1529475928078.ts #EXTINF:4.000, c7cam2-20034_720p-1529475931744.ts #EXTINF:4.000, c7cam2-20034_720p-1529475936245.ts #EXTINF:4.000, c7cam2-20034_720p-1529475939701.ts #EXTINF:4.000, c7cam2-20034_720p-1529475943736.ts #EXTINF:4.000, c7cam2-20034_720p-1529475947698.ts #EXTINF:4.000, c7cam2-20034_720p-1529475952382.ts #EXTINF:4.000, c7cam2-20034_720p-1529475955940.ts #EXT-X-KEY:METHOD=AES-128,URI="https://vmf.edge-apps.net/keys/00D2A735511A71B0D8449A57CF2520AA/c7cam2-20034_720p-1529475959689.key",IV=0x0000000000000000000001641BDF0B89 #EXTINF:4.000, c7cam2-20034_720p-1529475959689.ts #EXTINF:4.000, c7cam2-20034_720p-1529475963695.ts #EXTINF:4.000, c7cam2-20034_720p-1529475967708.ts #EXTINF:4.000, c7cam2-20034_720p-1529475971696.ts #EXTINF:4.000, c7cam2-20034_720p-1529475975705.ts #EXTINF:4.000, c7cam2-20034_720p-1529475979691.ts #EXTINF:4.000, c7cam2-20034_720p-1529475983700.ts #EXTINF:4.000, c7cam2-20034_720p-1529475987740.ts #EXTINF:4.000, c7cam2-20034_720p-1529475991692.ts #EXTINF:4.000, c7cam2-20034_720p-1529475995693.ts #EXT-X-KEY:METHOD=AES-128,URI="https://vmf.edge-apps.net/keys/00D2A735511A71B0D8449A57CF2520AA/c7cam2-20034_720p-1529475999748.key",IV=0x0000000000000000000001641BDFA804
Last edited by Sapito; 20th Jun 2018 at 02:35.
-
It can't be downloaded unless it is some way that sets the Referer. Any direct download that doesn't set the Referer will result in 403.
720p.m3u8 is not a variant playlist. So, if you're going to use livestreamer with the 720p.m3u8 playlist, use "hls://" instead of "hlsvariant://"
If you used the variant playlist with livestreamer directly (instead of trying to download it), you wouldn't get a 403 error as long as the Referer is properly set.Last edited by ponens; 20th Jun 2018 at 10:32.
-
I was able (with a VPN in Argentina) to download with youtube-dl with:
I got the url and referrer from Internet Explorer's Developer Tools.Code:youtube-dl --referer "http://vmf.edge-apps.net/embed/live.php?streamname=c7live01-20034&autoplay=true" https://g1.proy-slo.transport.edge-access.net/b17/ngrp:c7cam2-20034_all/playlist.m3u8
Last edited by jagabo; 20th Jun 2018 at 10:16.
-
Jagabo,Code:
youtube-dl --referer "http://vmf.edge-apps.net/embed/live.php?streamname=c7live01-20034&autoplay=true" https://g1.proy-slo.transport.edge-access.net/b17/ngrp:c7cam2-20034_all/playlist.m3u8
What exactly does the "referer" do? Why is it necessary? You say you got it from browser developer tools. How did you know the above (ending in "true") was the "referer"? -
I may be wrong but I think true means yes as in "autoplay is yes" - it will play automatically. https://github.com/rg3/youtube-dl/issues/1854 may explain the refeer bitOriginally Posted by jimdagys
-
The --referer field of an http request is site from which the link was obtained.
https://en.wikipedia.org/wiki/List_of_HTTP_header_fields
https://en.wikipedia.org/wiki/HTTP_referer
So in that command line you are saying "I want to open this file: https://g1.proy-slo.transport.edge-access.net/b17/ngrp:c7cam2-20034_all/playlist.m3u8, and "I got that link from http://vmf.edge-apps.net/embed/live.php?streamname=c7live01-20034&autoplay=true.
In cases like this the --referer is to verify that you got the link from a legit web page -- other sites that link to it will not be able to play the video. With this particular request only the "http://vmf.edge-apps.net" part of the referer field is really needed.
I got the -referer field from Developer Tools. Look at the Details of the request:Last edited by jagabo; 20th Jun 2018 at 17:26.
-
WE MADE IT
So, livestreamer refuses to work. See:
But youtube-dl as jagabo says works wonderfully. It needed a ffmpeg to do anything, I downloaded the "static" Windows build. It works so I guess that's the one.Code:C:\Program Files (x86)\Livestreamer>livestreamer --http-header "Referer=http://vmf.edge-apps.net/embed/live.php?streamname=c7live01-20034&autoplay=true" "hls://g1.proy-slo.transport.edge-access.net/b17/ngrp:c7cam2-20034_all/playlist.m3u8" best -o Livestream.ts [cli][info] Found matching plugin stream for URL hls://g1.proy-slo.transport.edge-access.net/b17/ngrp:c7cam2-20034_all/playlist.m3u8 [cli][info] Available streams: live (worst, best) [cli][info] Opening stream: live (hls) [cli][error] Could not open stream: Unable to open URL: http://g1.proy-slo.transport.edge-access.net/b17/ngrp:c7cam2-20034_all/playlist.m3u8 (HTTPConnectionPool(host='g1.proy-slo.transport.edge-access.net', port=80): Max retries exceeded with url: /b17/ngrp:c7cam2-20034_all/playlist.m3u8 (Caused by <class 'socket.gaierror'>: [Errno 11001] getaddrinfo failed))
Now there's a slight problem. There's no way to stop recording. If I close the cmd window or press ctrl+c I end up with a .mp4.part file that's unplayable (since live TV is infinite I have to trigger the "stop" myself). The only way to have youtube-dl produce a playable file at the end is to disconnect the network adapter, so I get this:Code:youtube-dl --referer "http://vmf.edge-apps.net/embed/live.php?streamname=c7live01-20034&autoplay=true" https://g1.mc-hor.transport.edge-access.net/b17/ngrp:c7cam2-20034_all/c7cam2-20034_1080p.m3u8
There the file gets renamed (finalized?) to .mp4 and it plays perfect. Any ideas? I tried outputting a TS file with "-o testvideo.ts" but it's just the same mp4 as before with a .ts extension (the player identifies it as an mp4 container).Code:[hls,applehttp @ 061f4ac0] Opening 'crypto+https://g1.mc-hor.transport.edge-access.net/b17/ngrp:c7cam2-20034_all/c7cam2-20034_1080p-1529537100290.ts' for reading [https @ 062db240] Opening 'https://g1.mc-hor.transport.edge-access.net/b17/ngrp:c7cam2-20034_all/c7cam2-20034_1080p.m3u8' for reading [tls @ 06299fc0] Error in the pull function. [hls,applehttp @ 061f4ac0] keepalive request failed for 'https://g1.mc-hor.transport.edge-access.net/b17/ngrp:c7cam2-20034_all/c7cam2-20034_1080p.m3u8', retrying with new connection: I/O error [tcp @ 06b07d00] Failed to resolve hostname g1.mc-hor.transport.edge-access.net: The name does not resolve for the supplied parameters [hls,applehttp @ 061f4ac0] Failed to reload playlist 0 [tcp @ 06b08180] Failed to resolve hostname g1.mc-hor.transport.edge-access.net: The name does not resolve for the supplied parameters [hls,applehttp @ 061f4ac0] Failed to reload playlist 0 frame= 3800 fps= 25 q=-1.0 Lsize= 60191kB time=00:02:31.97 bitrate=3244.5kbits/s speed= 1x video:57669kB audio:2386kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.226303% [ffmpeg] Downloaded 61635404 bytes [download] 100% of 58.78MiB in 02:34
On the other side the -F switch to list video formats doesn't do much either. Of course, since I'm using the 1080 playlist there's bound to be an only option. I just see that the container is mp4 (why? aren't the individual streamed packets in .ts?). I expected that it could throw the video/audio codecs at least.
Thanks for for helping me to get this runningCode:youtube-dl -F --referer "http://vmf.edge-apps.net/embed/live.php?streamname=c7live01-20034&autoplay=true" https://g1.mc-hor.transport.edge-access.net/b17/ngrp:c7cam2-20034_all/c7cam2-20034_1080p.m3u8 [generic] c7cam2-20034_1080p: Requesting header [generic] c7cam2-20034_1080p: Downloading m3u8 information [info] Available formats for c7cam2-20034_1080p: format code extension resolution note 0 mp4 unknown
-
Yes, live streams never end so you have to manually stop the download. Pressing 'q' while downloading used to stop youtube-dl gracefully (?) but the option was removed at some point. I find that very short downloads tend not to play after CTRL+C. Longer ones work more often. Of course, risking a one-time-only live recording with CTRL+C isn't great. Maybe you can switch to using ffmpeg for downloading instead ('q' is supported there).
Code:ffmpeg.exe -headers "Referer: http://vmf.edge-apps.net" -i "https://g1.proy-slo.transport.edge-access.net/b17/ngrp:c7cam2-20034_all/playlist.m3u8" -c copy output.mp4
-
I see...Originally Posted by jagabo
I found out that by using the "--no-part" command youtube-dl writes directly into the .mp4, avoiding the nasty .part file. In this case I can CTRL+C safely and I always end with a playable file. Are they any drawbacks of using "--no-part"? I can't see any so far. Or what's the actual purpose of making the .part file by default.
Anyway, now I have another doubt. The TV channel above allows you to go back 1 hour in the live stream. I tried going back in the player at the page and checked the playlist but it's the same as if I were watching the live stream. How do I tell youtube-dl to start capturing 1 hour before the current transmission?
If my net connection goes down and the streaming is interrupted, it can be useful to go back to the point it stopped and resume capturing from there. Then I can cut and append the mp4 files to get the whole thing together.
QuoteSimilar Threads
-
help downloading this m3u8 with livestreamer (have the link)
By mulliganman in forum Video Streaming DownloadingReplies: 1Last Post: 29th May 2018, 09:32 -
Wireshark + Livestreamer: Have URL and auth token but cannot play it
By VideoBob132 in forum Video Streaming DownloadingReplies: 1Last Post: 10th Mar 2017, 23:18 -
Explain youtube URL direct download link sections
By Waffa in forum Video Streaming DownloadingReplies: 4Last Post: 27th Jun 2015, 05:59 -
Getting rtmpdump URL for USTREAM feed
By i336_ in forum Video Streaming DownloadingReplies: 1Last Post: 15th Apr 2015, 09:18 -
Extract Steraming Link from given URL
By bablu15 in forum Video Streaming DownloadingReplies: 1Last Post: 2nd May 2014, 08:06