Need help downloading videos from extremely tricky website

[vikeyev]

Could somebody here please help me to download videos from this website http://www.maoritelevision.com/tv/shows/road-nats

I've tried many different programs, RMC, Orbit, StreamTransport, URL Snooper, Wireshark etc etc. But none of them are finding the actual download link. Any help would be greatly appreciated.

[Alex_ander]

It's HLS stream (not RTMP). You can get the link to m3u8 manifest file (master .m3u8 -> link to .m3u8 for the wanted bitrate version) e. g. using AddBlock Plus Firefox plugin, then try youtube-dl or livestreamer. Not sure which one will handle (if will) keyfiles from brightcove.com used for encryption here.

[vikeyev]

It's HLS stream (not RTMP). You can get the link to m3u8 manifest file (master .m3u8 -> link to .m3u8 for the wanted bitrate version) e. g. using AddBlock Plus Firefox plugin

Sorry, I'm quite a noob at this. How do I go about getting the correct link?

[Alex_ander]

With AddBlock Plus plugin installed in Firefox, select 'Open blockable items' from its dropdown menu. That will open a window with links below the video page. After starting playback, you'll see links to .m3u8 and .ts files. The .ts fragments are downloadable directly, but they can't be played due to encryption, so the only hope to get the proper video file is to use the .m3u8 link in a program handling the encryption.

[jimdagys]

I used Adblocker Plus with Firefox and got this for episode 10:

http://c.brightcove.com/services/mobile/streaming/index/master.m3u8?videoId=5042285719001&pubId=1614493167001

However, I was not successful to download with either youtube-dl or livestreamer.
No doubt there are some people on this board (should they choose to answer) who would know how to download by free methods your above videos.
For the time being, I tested Tubedigger ($15-$25 for lifetime license) and it downloads your video no problem. The trial version (free) will download 50% of the video.
I downloaded the 50% and posted it here for your convenience:
https://files.videohelp.com/u/61125/tubedigger.ts
You can also use the freeware called Debut Free video capture:
Link to software:
https://files.videohelp.com/u/61125/debutsetupfree.zip
The result won't be as good as the downloaded copy, but you can screen capture the whole video by this method. I captured a one minute segment here for your convenience:
https://files.videohelp.com/u/61125/debut.avi

[nullacht]

I used Adblocker Plus with Firefox and got this for episode 10:

http://c.brightcove.com/services/mobile/streaming/index/master.m3u8?videoId=5042285719001&pubId=1614493167001

However, I was not successful to download with either youtube-dl or livestreamer.

Here is an example command line:

youtube-dl "http://c.brightcove.com/services/mobile/streaming/index/master.m3u8?videoId=5042285719001&pubId=1614493167001" --force-generic-extractor -o HLS.mp4
[generic] master: Requesting header
[generic] master: Downloading m3u8 information
[download] Destination: HLS.mp4

[jimdagys]

Ok, I see what you did. Adding the magic words:
--force-generic-extractor
allows you to download the videos. I don't understand the reason, but it just works.
Using the above link from Adblock Plus:

http://c.brightcove.com/services/mobile/streaming/index/master.m3u8?videoId=5042285719001&pubId=1614493167001

Then add to the beginning to get available formats:
youtube-dl -F
Then add the magic words at the end (if you don't add the magic words you get an error message and no download):
--force-generic-extractor
So the whole code is this:

youtube-dl -F "http://c.brightcove.com/services/mobile/streaming/index/master.m3u8?videoId=5042285719001&pubId=1614493167001" --force-generic-extractor

This yields a list of available formats; I choose 697 from the list (see below red line).

Microsoft Windows XP [Version 5.1.2600]
(C) Copyright 1985-2001 Microsoft Corp.

C:\Documents and Settings\default>cd..

C:\Documents and Settings>cd..

C:\>cd youtube-dl

C:\youtube-dl>youtube-dl -F "http://c.brightcove.com/services/mobile/streaming/i
ndex/master.m3u8?videoId=5042285719001&pubId=1614493167 001" --force-generic-extr
actor
[generic] master: Requesting header
[generic] master: Downloading m3u8 information
[info] Available formats for master:
format code extension resolution note
meta mp4 multiple Quality selection URL
399 mp4 320x180 399k
697 mp4 640x360 697k
1321 mp4 1024x576 1321k
1919 mp4 1280x720 1919k (best)

Putting 697 into the code gives the final code to download the video:

youtube-dl -f 697 "http://c.brightcove.com/services/mobile/streaming/index/master.m3u8?videoId=5042285719001&pubId=1614493167001" --force-generic-extractor

Microsoft Windows XP [Version 5.1.2600]
(C) Copyright 1985-2001 Microsoft Corp.

C:\Documents and Settings\default>cd..

C:\Documents and Settings>cd..

C:\>cd youtube-dl

C:\youtube-dl>youtube-dl -f 697 "http://c.brightcove.com/services/mobile/streami
ng/index/master.m3u8?videoId=5042285719001&pubId=1614493167 001" --force-generic-
extractor
[generic] master: Requesting header
[generic] master: Downloading m3u8 information
[download] Destination: master-master.mp4
ffmpeg version N-67673-g933eca9 Copyright (c) 2000-2014 the FFmpeg developers
built on Nov 14 2014 22:01:58 with gcc 4.9.2 (GCC)
configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-av
isynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enab
le-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --
enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-lib
modplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrw
b --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinge
r --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --en
able-libvidstab --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libvorbis
--enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-
libx265 --enable-libxavs --enable-libxvid --enable-zlib
libavutil 54. 11.100 / 54. 11.100
libavcodec 56. 12.100 / 56. 12.100
libavformat 56. 12.103 / 56. 12.103
libavdevice 56. 3.100 / 56. 3.100
libavfilter 5. 2.103 / 5. 2.103
libswscale 3. 1.101 / 3. 1.101
libswresample 1. 1.100 / 1. 1.100
libpostproc 53. 3.100 / 53. 3.100
Input #0, hls,applehttp, from 'https://secure.brightcove.com/services/mobile/str
eaming/index/rendition.m3u8?assetId=5042329842001&expiration=14 92987440000&token
=6e4e9e4ede43dd58f3fafe406438884074bd3063&pubId=16 14493167001&videoId=5042285719
001':
Duration: 00:26:14.00, start: 9.940500, bitrate: 0 kb/s
Program 0
Metadata:
variant_bitrate : 0
Stream #0:0: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p(tv, bt709)
, 640x360 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc
Stream #0:1: Audio: aac (LC) ([15][0][0][0] / 0x000F), 44100 Hz, stereo, flt
p, 110 kb/s
Output #0, mp4, to 'file:master-master.mp4.part':
Metadata:
encoder : Lavf56.12.103
Stream #0:0: Video: h264 ([33][0][0][0] / 0x0021), yuv420p, 640x360 [SAR 1:1
DAR 16:9], q=2-31, 25 fps, 90k tbn, 90k tbc
Stream #0:1: Audio: aac ([64][0][0][0] / 0x0040), 44100 Hz, stereo, 110 kb/s

Stream mapping:
Stream #0:0 -> #0:0 (copy)
Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
frame= 60 fps=0.0 q=-1.0 size= 252kB time=00:00:02.41 bitrate= 851.9kbits/
frame= 93 fps= 90 q=-1.0 size= 392kB time=00:00:03.73 bitrate= 859.2kbits/
frame= 109 fps= 70 q=-1.0 size= 493kB time=00:00:04.37 bitrate= 921.3kbits/
frame= 131 fps= 64 q=-1.0 size= 639kB time=00:00:05.25 bitrate= 995.4kbits/
frame= 151 fps= 59 q=-1.0 size= 754kB time=00:00:06.05 bitrate=1019.2kbits/
frame= 156 fps= 58 q=-1.0 Lsize= 851kB time=00:00:06.25 bitrate=1114.0kbits
/s
video:773kB audio:72kB subtitle:0kB other streams:0kB global headers:0kB muxing
overhead: 0.638967%
Received signal 2: terminating.

[sysanin]

I used Adblocker Plus with Firefox
However, I was not successful to download with either youtube-dl or livestreamer

Change User Agent on Android - Chrome and disable Flash

[nullacht]

There are many different ways to download videos from websites. A slightly different approach would be to use Brightcove IDs.

http://www.maoritelevision.com/tv/shows/road-nats/S01E010/road-nats-series-1-episode-10

id="video-player"
data-account="1614493167001"
data-player="r1IDOA4s"
data-embed="default"
data-video-id="5042285719001"

youtube-dl "http://players.brightcove.net/1614493167001/r1IDOA4s_default/index.html?videoId=5042285719001" -F
[brightcove:new] 5042285719001: Downloading webpage
[brightcove:new] 5042285719001: Downloading JSON metadata
[brightcove:new] 5042285719001: Downloading m3u8 information
[brightcove:new] 5042285719001: Downloading m3u8 information
[brightcove:new] 5042285719001: Downloading MPD manifest
[brightcove:new] 5042285719001: Downloading MPD manifest
[info] Available formats for 5042285719001:
format code                     extension  resolution note
hls-meta-0                      mp4        multiple   Quality selection URL
hls-meta-1                      mp4        multiple   Quality selection URL
dash-5042301349001_audio_und-0  m4a        audio only DASH audio   67k , mp4a.40.2 (44100Hz)
dash-5042301349001_audio_und-1  m4a        audio only DASH audio   67k , mp4a.40.2 (44100Hz)
dash-5042301349001_video_1-0    mp4        426x240    DASH video  435k , avc1.4D4015, 25fps, video only
dash-5042301349001_video_1-1    mp4        426x240    DASH video  435k , avc1.4D4015, 25fps, video only
dash-5042303913001_video_1-0    mp4        640x360    DASH video  871k , avc1.4D401E, 25fps, video only
dash-5042303913001_video_1-1    mp4        640x360    DASH video  871k , avc1.4D401E, 25fps, video only
dash-5042303151001_video_1-0    mp4        1024x576   DASH video 1743k , avc1.4D401F, 25fps, video only
dash-5042303151001_video_1-1    mp4        1024x576   DASH video 1743k , avc1.4D401F, 25fps, video only
dash-5042303916001_video_1-0    mp4        1280x720   DASH video 2606k , avc1.4D401F, 25fps, video only
dash-5042303916001_video_1-1    mp4        1280x720   DASH video 2606k , avc1.4D401F, 25fps, video only
dash-5042303544001_video_1-0    mp4        1920x1080  DASH video 3672k , avc1.4D4028, 25fps, video only
dash-5042303544001_video_1-1    mp4        1920x1080  DASH video 3672k , avc1.4D4028, 25fps, video only
rtmp-361k-240p                  mp4        426x240     361k , MP4 container, H264, 68.53MiB
http-361k-240p-0                mp4        426x240     361k , MP4 container, H264, 68.53MiB
http-361k-240p-1                mp4        426x240     361k , MP4 container, H264, 68.53MiB
hls-399-0                       mp4        320x180     399k
hls-399-1                       mp4        320x180     399k
rtmp-694k-360p                  mp4        640x360     694k , MP4 container, H264, 130.72MiB
http-694k-360p-0                mp4        640x360     694k , MP4 container, H264, 130.72MiB
http-694k-360p-1                mp4        640x360     694k , MP4 container, H264, 130.72MiB
hls-697-0                       mp4        640x360     697k
hls-697-1                       mp4        640x360     697k
hls-1321-0                      mp4        1024x576   1321k
hls-1321-1                      mp4        1024x576   1321k
rtmp-1321k-576p                 mp4        1024x576   1321k , MP4 container, H264, 248.57MiB
http-1321k-576p-0               mp4        1024x576   1321k , MP4 container, H264, 248.57MiB
http-1321k-576p-1               mp4        1024x576   1321k , MP4 container, H264, 248.57MiB
hls-1919-0                      mp4        1280x720   1919k
hls-1919-1                      mp4        1280x720   1919k
rtmp-1919k-720p                 mp4        1280x720   1919k , MP4 container, H264, 360.69MiB
http-1919k-720p-0               mp4        1280x720   1919k , MP4 container, H264, 360.69MiB
http-1919k-720p-1               mp4        1280x720   1919k , MP4 container, H264, 360.69MiB (best)

youtube-dl "http://players.brightcove.net/1614493167001/r1IDOA4s_default/index.html?videoId=5042285719001" -f http-1919k-720p-1 -g

https://udso-a.akamaihd.net/1614493167001/1614493167001_5042331909001_5042285719001.mp4?pubId=1614493167001&videoId=5042285719001

Video: MPEG4 Video (H264) 1280x720 25fps 1792kbps [V: h264 main L3.1, yuv420p, 1280x720, 1792 kb/s]
Audio: AAC 44100Hz stereo 127kbps [A: aac lc, 44100 Hz, stereo, 127 kb/s]

1920x1080 is also available.

youtube-dl "http://players.brightcove.net/1614493167001/r1IDOA4s_default/index.html?videoId=5042285719001"
[brightcove:new] 5042285719001: Downloading webpage
[brightcove:new] 5042285719001: Downloading JSON metadata
[brightcove:new] 5042285719001: Downloading m3u8 information
[brightcove:new] 5042285719001: Downloading m3u8 information
[brightcove:new] 5042285719001: Downloading MPD manifest
[brightcove:new] 5042285719001: Downloading MPD manifest
[dashsegments] Total fragments: 788

[vikeyev]

Thank you guys so much it worked. Took me a little while to figure it out but I got it working.