Would appreciate help from m3u8 expert, on a site that perhaps guards dl.

[rchase]

Hello, here is the link to a video that I want to save for offline viewing with the VLC player.

http://www.learner.org/vod/vod_window.html?pid=682

I have successfully downloaded videos from this site as mkv before by using ffmpeg and the terminal on the Mac. But the site seemed to have updated the stream so that it blocks my older way of download.

Here is the m3u8 link from the page source:
http://telvue_abr-f.akamaihd.net/i/secure_vod/T01719/french_in_action/french_in_action...ad2412e4b3bf0e

What I have noticed is, this link expires after some time, and a refresh of the original page generates a new m3u8 stream for the same video.

Thanks, please forgive me for gaps in my understanding, I can barely use the Terminal on my Mac...

[aazerty]

Hi rchase ,

From your link : http://.../i/.../master.m3u8hdnea=...3bf0e
Do changes like : http://.../z/.../manifest.f4m
Create a batch file using livestreamer : livestreamer "hds://http://.../z/.../manifest.f4m" best
See below for watching with VLC in the bestest quality :

livestreamer "hds://http://telvue_abr-f.akamaihd.net/z/secure_vod/T01719/french_in_action/french_in_action_01-,0350,0750,1250,.mp4.csmil/manifest.f4m" best

[cli][info] Found matching plugin stream for URL hds://http://telvue_abr-f.akamaihd.net/z/secure_vod/T01719/french_in_action/french_in_action_01-,0350,0750,1250,.mp4.csmil/manifest.f4m
[cli][info] Available streams: 1310k (best), 410k (worst), 810k
[cli][info] Opening stream: 1310k (hds)
[cli][info] Starting player: 'C:\Program Files\VideoLAN\VLC\vlc.exe'

If you want to DL it in medium quality :
livestreamer "hds://http://.../z/.../manifest.f4m" 810k -o FR_in_action_01.flv

====
I think it's possible to DL 52 episodes by changing the number :
/french_in_action_01 => /french_in_action_02 and so on
FR_in_action_01.flv => FR_in_action_02.flv and so on

Cheers .

[pub]

You can download some of them directly from their server. For example:

http://telvue_abr-f.akamaihd.net/i/secure_vod/T01719/connect_with_english/connect_with_english_split_01-,0350,0750,1250,.mp4.csmil/master.m3u8?hdnea=st=1399259853~exp=1399260153~acl=%2F*~hmac=2ed28a5ff1d288eaee2863cde7cc5bafc19c43c2b84b07a9285635d7147ed8a2

results in

http://cdn-origin.peg.tv/vod/annenberg_android/connect_with_english_split_01.mp4

Somehow doesn't work for your video.

[rchase]

Hi aazerty and pub, thank you for your help.

aazerty, do you mind showing me how the m3u8 might be downloaded with ffmpeg? I am still learning to use livestreamer and it's a little tough. I am more familiar with ffmpeg. Thanks again.

pub, I noticed that Connect with English video too, but it's not part of the French In Action series. It's an interesting website. Thanks!

[aazerty]

Hi rchase ,

livestreamer is easy to use for your try . Follow those instructions :
_ Go to the link in red => select a portable version . Depending your OS !!!
_ DL it => extract it in a folder without administrator's right .
_ Create a batch file ( under windows )

@echo on
livestreamer "hds://http://telvue_abr-f.akamaihd.net/z/secure_vod/T01719/french_in_action/french_in_action_01-,0350,0750,1250,.mp4.csmil/manifest.f4m" 810k -o FR_in_action_01.flv
pause
:fin

!!! Save it in the "livestreamer.exe's directory" !!!
Launch it => FR_in_action_01.flv

NOTE : livestreamer is very powerfull for downloading videos & watching (directly in VLC) .

Cheers .

[rchase]

Hi aazerty, thank you again. I use a Mac and was learning how to use the Terminal to execute commands, but I have no clue how to "create a batch" with the code that you have provided. But I want you to know that I appreciate your help very much.

I was trying to see if there are tutorials on the web that does a step-by-step walk-through of how to use livestreamer on the Mac and I didn't find anything...

Sorry again. I wish I understood the steps you've provided immediately. (The reason I was asking about ffmpeg was I spent hours last year learning how to use it to get the video from the site for off-line viewing and did it successfully; but the site administrator must have updated the stream settings.)

[ponens]

(The reason I was asking about ffmpeg was I spent hours last year learning how to use it to get the video from the site for off-line viewing and did it successfully; but the site administrator must have updated the stream settings.)

ffmpeg still works with the m3u8 link from this site (just tested it). Are you retrieving the m3u8 by viewing the source of the page (Ctril-U)? What does your command line for terminal look like?

Yes, the code at the end of the m3u8 link expires, but there should be enough time to use it.

[rchase]

Hi ponens, thank you.

I haven't been able to download the m3u8 feed with ffmpeg. Here is how I have been doing it with Terminal.

/Applications/ffmpeg -i "[the m3u8 feed]" -c copy Name1.mkv

Do you mind telling me how you were able to get it to work? I appreciate your help very much!

[aazerty]

Hi ,

Read :
http://livestreamer.readthedocs.org/en/latest/install.html
===
Mac OS X

# easy_install -U livestreamer
...
===
Sorry but I don't use Mac or linux and I can't help for command's line .

Thanks for the previous link :
=> for the moment I have DL 10 episodes with the same process .
( no issue with the time 'expire' ! )

Cheers .

[rchase]

Hi aazerty!

I'm not sure if you're interested in learning French, but this seems to be a nice program that Yale developed. Or maybe you know French already!

You know, I just didn't quite understand what "# easy_install -U livestreamer" means. That means nothing to me...

[ponens]

1. Are you retrieving the m3u8 by viewing the source of the page (Ctril-U)? If not, then how do you get the m3u8 link?

2. What does your command line for terminal look like? Not a template of the command line, but an actual command line.

For your template: "[the m3u8 feed]", do you include the all of the code in the feed link, including the entire suffix?

What error does ffmpeg report?

[rchase]

Hi ponens, thanks! Here are my links and codes that I'm using.

1. Without the quotation mark.

"http://telvue_abr-f.akamaihd.net/i/secure_vod/T01719/french_in_action/french_in_action_01-,0350,0750,1250,.mp4.csmil/master.m3u8?hdnea=st=1427933623~exp=1427933923~acl =%2F*~hmac=a89746a79d40457f22626fb06d684cea5a65bb0 af80bf29eb1b7231405b31110"

2. What I enter into Terminal.

/Applications/ffmpeg -i "http://telvue_abr-f.akamaihd.net/i/secure_vod/T01719/french_in_action/french_in_action_01-,0350,0750,1250,.mp4.csmil/master.m3u8?hdnea=st=1427933623~exp=1427933923~acl =%2F*~hmac=a89746a79d40457f22626fb06d684cea5a65bb0 af80bf29eb1b7231405b31110" -c copy French1.mkv

3. The error is long, but I'll paste it in another post.

[rchase]

ffmpeg version 2.6.1 Copyright (c) 2000-2015 the FFmpeg developers
built with llvm-gcc 4.2.1 (LLVM build 2336.11.00)
configuration: --prefix=/Volumes/Ramdisk/sw --enable-gpl --enable-pthreads --enable-version3 --enable-libspeex --enable-libvpx --disable-decoder=libvpx --enable-libmp3lame --enable-libtheora --enable-libvorbis --enable-libx264 --enable-avfilter --enable-libopencore_amrwb --enable-libopencore_amrnb --enable-filters --enable-libgsm --enable-libvidstab --enable-libx265 --disable-doc --arch=x86_64 --enable-runtime-cpudetect
libavutil 54. 20.100 / 54. 20.100
libavcodec 56. 26.100 / 56. 26.100
libavformat 56. 25.101 / 56. 25.101
libavdevice 56. 4.100 / 56. 4.100
libavfilter 5. 11.102 / 5. 11.102
libswscale 3. 1.101 / 3. 1.101
libswresample 1. 1.100 / 1. 1.100
libpostproc 53. 3.100 / 53. 3.100
[h264 @ 0x7fbc5c800000] non-existing SPS 0 referenced in buffering period
Last message repeated 1 times
[h264 @ 0x7fbc5b821c00] non-existing SPS 0 referenced in buffering period
Last message repeated 1 times
[h264 @ 0x7fbc5c816800] non-existing SPS 0 referenced in buffering period
Last message repeated 1 times
[h264 @ 0x7fbc5b892e00] non-existing SPS 0 referenced in buffering period
[h264 @ 0x7fbc5b828a00] non-existing SPS 0 referenced in buffering period
[h264 @ 0x7fbc5b919200] non-existing SPS 0 referenced in buffering period
Input #0, hls,applehttp, from 'http://telvue_abr-f.akamaihd.net/i/secure_vod/T01719/french_in_action/french_in_action_01-,0350,0750,1250,.mp4.csmil/master.m3u8?hdnea=st=1427933623~exp=1427933923~acl =%2F*~hmac=a89746a79d40457f22626fb06d684cea5a65bb0 af80bf29eb1b7231405b31110':
Duration: 00:28:22.53, start: 0.100667, bitrate: 0 kb/s
Program 0
Metadata:
variant_bitrate : 410000
Stream #0:0: Video: h264 (Constrained Baseline) ([27][0][0][0] / 0x001B), yuv420p, 320x240 [SAR 1:1 DAR 4:3], 29.97 fps, 29.97 tbr, 90k tbn, 59.94 tbc
Metadata:
variant_bitrate : 410000
Stream #0:1: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 61 kb/s
Metadata:
variant_bitrate : 410000
Stream #0:2: Data: timed_id3 (ID3 / 0x20334449)
Metadata:
variant_bitrate : 410000
Program 1
Metadata:
variant_bitrate : 810000
Stream #0:3: Video: h264 (Constrained Baseline) ([27][0][0][0] / 0x001B), yuv420p, 400x300 [SAR 1:1 DAR 4:3], 29.97 fps, 29.97 tbr, 90k tbn, 59.94 tbc
Metadata:
variant_bitrate : 810000
Stream #0:4: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 61 kb/s
Metadata:
variant_bitrate : 810000
Stream #0:5: Data: timed_id3 (ID3 / 0x20334449)
Metadata:
variant_bitrate : 810000
Program 2
Metadata:
variant_bitrate : 1310000
Stream #0:6: Video: h264 (Constrained Baseline) ([27][0][0][0] / 0x001B), yuv420p, 640x480 [SAR 1:1 DAR 4:3], 29.97 fps, 29.97 tbr, 90k tbn, 59.94 tbc
Metadata:
variant_bitrate : 1310000
Stream #0:7: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 61 kb/s
Metadata:
variant_bitrate : 1310000
Stream #0:8: Data: timed_id3 (ID3 / 0x20334449)
Metadata:
variant_bitrate : 1310000
[matroska @ 0x7fbc5c03d200] Error parsing AAC extradata, unable to determine samplerate.
Output #0, matroska, to 'French1.mkv':
Metadata:
encoder : Lavf56.25.101
Stream #0:0: Video: h264 (H264 / 0x34363248), yuv420p, 640x480 [SAR 1:1 DAR 4:3], q=2-31, 29.97 fps, 29.97 tbr, 1k tbn, 90k tbc
Metadata:
variant_bitrate : 1310000
Stream #0:1: Audio: aac ([255][0][0][0] / 0x00FF), 48000 Hz, stereo, 61 kb/s
Metadata:
variant_bitrate : 410000
Stream mapping:
Stream #0:6 -> #0:0 (copy)
Stream #0:1 -> #0:1 (copy)
Could not write header for output file #0 (incorrect codec parameters ?): Invalid argument

[ponens]

Try changing this part of your command line:

-c copy French1.mkv

to either:

-c:v copy -c:a copy -f mpegts French1.mkv

or

-c:v copy -c:a copy -f mpegts French1.flv

The m3u8 link is good for 5 minutes, you'll get a 403 error if it has expired.

[rchase]

Oh my, it's working.

ponens, you have no idea how magical this seems to me... thank you so much

What is the difference between -c copy and -c:v copy -c:a copy -f mpegts?

[ponens]

The relevant part of your error is:

"Could not write header for output file #0 (incorrect codec parameters ?): Invalid argument"

For whatever reason, your version of ffmpeg (and/or your OS) couldn't write the raw data to the file container (mkv) in the raw codec format.

"-c:v copy -c:a copy -f mpegts" specifies the codec format and treats the audio and video tracks independently.

However, "-c copy" works with my current version of ffmpeg on windows.

[rchase]

How would the resultant mkv files be different between these two ways of downloading? BTW, how did you learn all these?

Very interesting stuff. I'll try downloading with a different ffmpeg and on a different version of OS X and report back later. Thanks again!

[aazerty]

Hi rchase ,

I am french .

The streams given with .m3u8 are fragmented => outfile.ts
you can specify -f mpegts .

=======
In your case there are 8 streams inclued in your m3u8's link :
Stream #0:0 to Stream #0:7
(https://forum.videohelp.com/threads/371095-Would-appreciate-help-from-m3u8-expert-on-a-...=1#post2383284)

By this you can select what you want !
Stream #0:3: Video: h264 (Constrained Baseline) ([27][0][0][0] / 0x001B), yuv420p, 400x300 [SAR 1:1 DAR 4:3], 29.97 fps, 29.97 tbr, 90k tbn, 59.94 tbc
Stream #0:4: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 61 kb/s

Your writes in the commands line will be
-map 0:3 -map 0:4 -c:v copy -c:a copy "FR_in_action_01.ts"
( 1st map for the video , 2nd one for audio )
====

ffmpeg -i "..." -map 0:3 -map 0:4 -c:v copy -c:a copy "FR_in_action_01.ts"

While the running process you'll see
Stream mapping:
Stream #0:3 -> #0:0 (copy)
Stream #0:4 -> #0:1 (copy)
=======

Cheers .

[rchase]

Hi aazerty! You are French! What are the chances?

I just downloaded an episode with the command that you wrote,

-map 0:3 -map 0:4 -c:v copy -c:a copy

and it worked! And the size of the file is about 195MB. And it played well in VLC.

When I downloaded with the command that ponens wrote,

-c:v copy -c:a copy -f mpegts

it downloaded a bigger video that is about 310MB.

How interesting! BTW, what do you think of the video series; I assume that you've looked at a few of them.

[aazerty]

Hi rchase ,

You can try :

ffmpeg -i 1stfile
and after
ffmpeg -i 2ndfile

To know what streams are inclued .
in one case it's 810k ( 195Mb ) and for the other it's 1310k ( 310Mb ) .

I don't need the hightest quality ( less time fro DL each video ) .

I haven't yet watches the videos .
I'll do it later , to improve my english knowledge .

Cheers .

[ponens]

I just downloaded an episode with the command that you wrote,

-map 0:3 -map 0:4 -c:v copy -c:a copy

and it worked! And the size of the file is about 195MB. And it played well in VLC.

When I downloaded with the command that ponens wrote,

-c:v copy -c:a copy -f mpegts

it downloaded a bigger video that is about 310MB.

You're just selecting a lower quality stream by doing that. In the m3u8 url:

french_in_action_01-,0350,0750,1250,.mp4.csmil/master.m3u8

0350 is 240p (320x240)
0750 is 300p (400x300)
1250 is 480p (640x480)

The -map parameter you used (-map 0:3 -map 0:4) selected the 0750 quality.

When ffmpeg is used without any -map arguments, it defaults to the highest quality.

[rchase]

Hi aazerty, that's cool that I can check the stream specs that way (thanks again for the tips). I just did it to the bigger file and found this.

Duration: 00:28:22.50, start: 1.400000, bitrate: 1445 kb/s
Program 1
Metadata:
service_name : Service01
service_provider: FFmpeg
Stream #0:0[0x100]: Video: h264 (Constrained Baseline) ([27][0][0][0] / 0x001B), yuv420p, 640x480 [SAR 1:1 DAR 4:3], 29.97 fps, 29.97 tbr, 90k tbn, 59.94 tbc
Stream #0:1[0x101]: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 69 kb/s

Hi ponens, I see what you mean. It's amazing that the video info is already in the stream address. I think 480p is pretty good! Thanks again for your help!

[aazerty]

Hi ,

The solution given by 'pub' ( thanks to him ) is faster and works very well !!!
( http://.../connect_with_english_split_01.mp4 )

Cheers .

[aazerty]

Hi ,

Attention !!!

Example :

====
With 
aaaa://telvue_abr-f.akamaihd.net/z/secure_vod/T01719/french_in_action/french_in_action_22-,0350,0750,1250,.mp4.csmil/manifest.f4m"
the video has a duration = 28mn13s

with
aaaa://cdn-origin.peg.tv/vod/annenberg_android/connect_with_english_split_22.mp4
the video has a duration = 14mn7s
====
replace aaaa by htpp

For each video downloaded , you'll have to use "flvextractcl.exe" and "MP4Box.exe" :
( because some fragments are missing !!!
[stream.hds][error] Failed to parse fragment 1-1: Failed to read data: timed out
[download][FR_in_action_22.flv] Written 1.2 MB (27s @ 45.6 KB/s) )

rem ====
"flvextractcl.exe" -v -a -o "FR_in_action_22.flv"
---
FLV Extract CL v1.6.1
Copyright 2006-2010 J.D. Purcell
http://www.moitah.net/

True Frame Rate: 29,9700801309031 (10147000/338571)
Average Frame Rate: 29,9724430030924 (3382600/112857)

Finished.
---
rem "FR_in_action_22.264"
rem "FR_in_action_22.aac"
---
"MP4Box.exe" -add FR_in_action_22.264:fps=25.000#video -add FR_in_action_22.aac#audio -new "FR_in_action_22_final.mp4"
---
AVC-H264 import - frame size 400 x 300 at 29.970 FPS
AVC Import results: 50739 samples - Slices: 1272 I 49467 P 0 B - 52590 SEI - 564 IDR
AAC import  - sample rate 48000 - MPEG-4 audio - 2 channels
Saving FR_in_action_22_final.mp4: 0.500 secs Interleaving
rem ====

Cheers .