TVNZ anyone?

[KiwiDemander]

Anyone figured out how to record TVNZ OnDemand videos? I use rtmpdump for TV3 (rtmpsrv doesn't work so I use gdb to dump browser RAM then extract strings to find the RTMPE URL, but that's OK by me, I only need it once in a while). TVNZ have become a bit more clever lately; the browser RAM is full of RTMP strings but which one (if any!) is the right one?

[garciahigh]

I've found RTMPExloreX seems to work for a lot of the videos on TVNZ.

Just choose 'rtmpsrv', click montor then go to the TVNZ url.

https://github.com/andr3jx/RTMPExploreX/downloads

Below is an example of rtmpdump command RTMPExploreX generates.

However, I am having an issue with some of the videos which timeout & need resuming.

rtmpdump -r "rtmpe://cp128344.edgefcs.net:1935/ondemand?videoId=1054618655001&lineUpId=&pubId=963482467001&playerId=1029272630001&affiliateId=" -a "ondemand?videoId=1054618655001&lineUpId=&pubId=963482467001&playerId=1029272630001&affiliateId=" -f "WIN 11,5,502,110" -W "http://admin.brightcove.com/viewer/us20121113.1511/connection/ExternalConnection_2.swf" -p "http://tvnz.co.nz/the-food-truck/s1-e4-video-4270493" -y "mp4:videos/963482467001/963482467001_1054674049001_4270493.mp4?videoId=1054618655001&lineUpId=&pubId=963482467001&playerId=1029272630001&affiliateId=" -o 963482467001_1054674049001_4270493.flv

[KiwiDemander]

@garciahigh: Thanks for that. I tried RTMPExploreX and got it working after compiling it and some fiddling around getting all the executables in the right places. As you noted, it stopped half way through, but I just edited the generated rtmpdump command to add the "-e" flag then ran it again, and it resumed the download correctly and finished it. So, that's all good.

I would prefer a technique that works for TVNZ on Linux, if anyone has had success there.

[Chug a Bug]

I can't get TVNZ to play here, Geo-blocked probably. But this is what I did to make RTMPdump/RTMPsrv/RTMPsuck work generally on Linux (Ubuntu 10.04 LTS)

Go to System -> Administration -> User Groups and create a new user. I have used the "rtmp" username for the new user, and didn't let the user to have any additional user privileges. The user will be used to monitor RTMP/RTMPE streams.


Create folder "home/USER/Download/dumps"
Open "/home/USER/Downloads/", and right-click on the "dumps" folder.
Select the Share tab.
Check "Share this folder", "Allow others to create and delete files in this folder", "Guest access (for people without a user account)".
Hit the "Close" button.
Note that sharing this folder is needed if you want to save videos with rtmpsuck.

I created a Bash script that looks like this:

#!/bin/bash

sudo iptables -t nat -A OUTPUT -p tcp --dport 1935 -m owner \! --uid-owner rtmp -j REDIRECT
cd /home/USER/Downloads/dumps
sudo su rtmp -c rtmpsuck
sudo iptables -t nat -D OUTPUT -p tcp --dport 1935 -m owner \! --uid-owner rtmp -j REDIRECT


and saved it in /bin folder and made it executable. Then I created a desktop launcher to launch it.

Now visit the site that has the stream or start playing the stream, rtmpsuck will begin saving. Press Ctrl+C to exit script.

[KiwiDemander]

@Chug a Bug: Yes, geo-locked to .nz I think. Thanks for the script; I had already tried something like that, just entering the commands manually, but get no response from rtmpsuck when I play the desired video on the TVNZ site. It just displays

RTMP Proxy Server v2.3
(c) 2010 Andrej Stepanchuk, Howard Chu; license: GPL

Streaming on rtmp://0.0.0.0:1935

when started up, and nothing more.

[Chug a Bug]

You have to configure your firewall to redirect RTMP traffic through a local port i.e.


sudo iptables -t nat -A OUTPUT -p tcp --dport 1935 -j REDIRECT

then start RtmpSrv/Suck

RtmpSrv will be analyzing data from your local port. It will output a command to RtmpDump that will download video.

When it's finished, press CTRL+C. This will stop rtmpsrv.

You can then delete the firewall redirect created above:


sudo iptables -t nat -D OUTPUT -p tcp --dport 1935 -j REDIRECT

All of which is time consuming, hence the script

[KiwiDemander]

You have to configure your firewall to redirect RTMP traffic through a local port i.e.
sudo iptables -t nat -A OUTPUT -p tcp --dport 1935 -j REDIRECT
then start RtmpSrv/Suck
...
All of which is time consuming, hence the script

Oops - I 'improved' your script to the point where it didn't work!

Now, entering the 'iptables' command correctly, then running rtmpsuck, I get this:

RTMP Proxy Server v2.3
(c) 2010 Andrej Stepanchuk, Howard Chu; license: GPL

Streaming on rtmp://0.0.0.0:1935
Processing connect
app: ondemand?videoId=1118635852001&lineUpId=&pubId=963 482467001&playerId=1029272630001&affiliateId=
flashVer: LNX 11,0,1,152
swfUrl: http://admin.brightcove.com/viewer/us20121120.1354/connection/ExternalConnection_2.swf
tcUrl: rtmpe://cp128344.edgefcs.net:1935/ondemand?videoId=1118635852001&lineUpId=&pubId=963 482467001&playerId=1029272630001&affiliateId=
pageUrl: http://tvnz.co.nz/cheers-to-50-years/goodnight-kiwi-video-1097027
ERROR: RTMP_Connect1, handshake failed.
ERROR: RTMP_ReadPacket, failed to read RTMP packet header
Closing connection... done!

ERROR: Handshake failed
Closing connection... done!

I believe that displays all the info needed to build an rtmpdump command, except for the -y (playpath) value. Is it perhaps missing that because of the errors it mentions?

[Chug a Bug]

Well thats an improvement at least!

Yes ondemand Akamai server will likely need a playpath, certainly the BBC's do.

Do you know if theres an media selector XML page available? I can't tell here, I'm blocked... BBC ondemand has one with all the information but many sites hide the data.

Have you tried rtmpsrv instead? Sometimes one works, but not the other. Also make sure you're using the latest versions, official repo is outdated last time I looked, compile your own if possible, it's version. 2.4.x now, your version 2.3 is well out of date.

[KiwiDemander]

Getting closer!

Don't know about 'media selector XML page' - what is that? Google found this:
http://www.google.co.nz/search?hl=en-NZ&source=hp&biw=&bih=&q=tvnz+media+selector+XML+....1.gBxLkPSh08Y
which mentions TVNZOnDemand, the site I'm interested in.

Tried with rtmpsrv. It printed a complete rtmpdump command, which I pasted into the shell and executed; here are the results:

$ sudo iptables -t nat -A OUTPUT -p tcp --dport 1935 -j REDIRECT
$
$ /usr/sbin/rtmpsrv
RTMP Server v2.3
(c) 2010 Andrej Stepanchuk, Howard Chu; license: GPL

Streaming on rtmp://0.0.0.0:1935
ERROR: RTMP_ReadPacket, failed to read RTMP packet header
Closing connection... done!

rtmpdump -r "rtmpe://cp128344.edgefcs.net:1935/ondemand?videoId=1118635852001&lineUpId=&pubId=963 482467001&playerId=1029272630001&affiliateId=" -a "ondemand?videoId=1118635852001&lineUpId=&pubId=96 3482467001&playerId=1029272630001&affiliateId=" -f "LNX 11,0,1,152" -W "http://admin.brightcove.com/viewer/us20121120.1354/connection/ExternalConnection_2.swf" -p "http://tvnz.co.nz/cheers-to-50-years/goodnight-kiwi-video-1097027" -y "mp4:videos/963482467001/963482467001_1118703466001_1097027.mp4?videoId=111 8635852001&lineUpId=&pubId=963482467001&playerId=1 029272630001&affiliateId=" -o 963482467001_1118703466001_1097027.flv

Closing connection... done!

RTMPDump v2.3
(c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL
Connecting ...
WARNING: HandShake: Type mismatch: client sent 6, server answered 8
INFO: Connected...
Duplicate request, skipping.
Caught signal: 13, cleaning up, just a second...
ERROR: WriteN, RTMP send error 32 (42 bytes)
Closing connection... done!

ERROR: RTMP_ReadPacket, failed to read RTMP packet header
^CCaught signal: 2, cleaning up, just a second...
$
$ sudo iptables -t nat -D OUTPUT -p tcp --dport 1935 -j REDIRECT
$
$ rtmpdump -r "rtmpe://cp128344.edgefcs.net:1935/ondemand?videoId=1118635852001&lineUpId=&pubId=963 482467001&playerId=1029272630001&affiliateId=" -a "ondemand?videoId=1118635852001&lineUpId=&pubId=96 3482467001&playerId=1029272630001&affiliateId=" -f "LNX 11,0,1,152" -W "http://admin.brightcove.com/viewer/us20121120.1354/connection/ExternalConnection_2.swf" -p "http://tvnz.co.nz/cheers-to-50-years/goodnight-kiwi-video-1097027" -y "mp4:videos/963482467001/963482467001_1118703466001_1097027.mp4?videoId=111 8635852001&lineUpId=&pubId=963482467001&playerId=1 029272630001&affiliateId=" -o 963482467001_1118703466001_1097027.flv
RTMPDump v2.3
(c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL
Connecting ...
WARNING: HandShake: Type mismatch: client sent 6, server answered 9
WARNING: HandShake: Server not genuine Adobe!
ERROR: RTMP_Connect1, handshake failed.
$

Have also downloaded latest rtmpdump source (2.4) and compiled it, then tried rtmpsuck and rtmpsrv. Both started up normally, but gave a segmentation fault as soon as I started to play the video.

[Chug a Bug]

Hmm, I just tried your RTMPdump string here and it downloaded just fine, in both Windows and Linux! Both using ver. 2.4

rtmpdump -r "rtmpe://cp128344.edgefcs.net:1935/ondemand?videoId=1118635852001&lineUpId=&pubId=963 482467001&playerId=1029272630001&affiliateId=" -a "ondemand?videoId=1118635852001&lineUpId=&pubId=96 3482467001&playerId=1029272630001&affiliateId=" -f "LNX 11,0,1,152" -W "http://admin.brightcove.com/viewer/us20121120.1354/connection/ExternalConnection_2.swf" -p "http://tvnz.co.nz/cheers-to-50-years/goodnight-kiwi-video-1097027" -y "mp4:videos/963482467001/963482467001_1118703466001_1097027.mp4?videoId=111 8635852001&lineUpId=&pubId=963482467001&playerId=1 029272630001&affiliateId=" -o 963482467001_1118703466001_1097027.flv

Guess you have a problem elsewhere....





Can't say I've heard of a segmentation fault before

[KiwiDemander]

Success at last!

After Chug a Bug said that my rtmpdump command ran OK, I decided to have a look at what was causing the seg fault in the rtmpdump 2.4 package. Had no success making it run under kdbg, until I compiled it with -g3 flag rather than just plain -g. After that, it not only ran under the debugger, but worked properly - no seg fault - bonus! Some problem with my local compiler perhaps, I dunno - and don't care as long as it all works!

So for the benefit of anyone else having similar trouble, here's what worked for me (running Debian Squeeze, 64-bit Intel core i7 CPU)

1. Get latest rtmpdump source with git clone git://git.ffmpeg.org/rtmpdump
2. Build with make SHARED= OPT=g3 SYS=posix

@Chug a Bug: Many thanks for all your help.

[A2000]

Success at last!

After Chug a Bug said that my rtmpdump command ran OK, I decided to have a look at what was causing the seg fault in the rtmpdump 2.4 package. Had no success making it run under kdbg, until I compiled it with -g3 flag rather than just plain -g. After that, it not only ran under the debugger, but worked properly - no seg fault - bonus! Some problem with my local compiler perhaps, I dunno - and don't care as long as it all works!

So for the benefit of anyone else having similar trouble, here's what worked for me (running Debian Squeeze, 64-bit Intel core i7 CPU)

1. Get latest rtmpdump source with git clone git://git.ffmpeg.org/rtmpdump
2. Build with make SHARED= OPT=g3 SYS=posix

@Chug a Bug: Many thanks for all your help.

This is my first post in this Forum. I was going to post about something very similar to this, but as I see you already got it, I think it is better if I ask you a favor, bc seen the difficult that this videos are to "grab" and adding the GeoLocation Restriction (I'm from Argentina) I think it will be almost impossible for me to download videos from TVNZ

I managed to watch them, and I think a screen recorder perhaps works, but I know that's not the best solution.

I want to ask for some help with the same problem, plus the geo restriction problem or ask, if would you capture some videos for me, and share

I want the Season 1 of The Food Truck (For example, episode 1 is here)

http://tvnz.co.nz/the-food-truck/s1-e1-video-4216981

If you could help me, that would be great.

PS: I managed to "mask" my IP using the Ad-on "Modify Headers" in Firefox and setting "X-Forwarded-For" with a NZ IP number.