VideoHelp Forum

+ Reply to Thread
Results 1 to 19 of 19
Thread
  1. Hi,

    I'm trying to download this video
    https://www.ipam.ucla.edu/wowzavideo.aspx?vfn=10660.mp4&vfd=gss2012

    Using rtmpsrv gave an rtmpdump command, but the download jumps back to lower completion percentages, and the resulting file repeats the content.
    I'm on Linux Mint 13, using the latest git repository version of rtmpdump (compiled statically).
    Sample from terminal:

    Code:
    ~/rtmpdump $ ./rtmpdump -r "rtmp://128.97.46.13:1935/gss2012/" -a "gss2012/" -f "LNX 11,0,1,152" -W "https://www.ipam.ucla.edu/videoplayer/jwplayer.flash.swf" -p "https://www.ipam.ucla.edu/wowzavideo.aspx?vfn=10660.mp4&vfd=gss2012" -y "mp4:10660.mp4" -o mp4_10660.flv
    RTMPDump v2.4
    (c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL
    Connecting ...
    INFO: Connected...
    Starting download at: 0.000 kB
    INFO: Metadata:
    INFO: trackinfo:
    INFO:   timescale             30000.00
    INFO:   length                98732634.00
    INFO:   language              eng
    INFO: sampledescription:
    INFO:   sampletype            avc1
    INFO:   timescale             48000.00
    INFO:   length                157974528.00
    INFO:   language              eng
    INFO: sampledescription:
    INFO:   sampletype            mp4a
    INFO:   audiochannels         2.00
    INFO:   audiosamplerate       48000.00
    INFO:   videoframerate        29.97
    INFO:   aacaot                2.00
    INFO:   avclevel              40.00
    INFO:   avcprofile            100.00
    INFO:   audiocodecid          mp4a
    INFO:   videocodecid          avc1
    INFO:   width                 1920.00
    INFO:   height                1080.00
    INFO:   frameWidth            1920.00
    INFO:   frameHeight           1080.00
    INFO:   displayWidth          1920.00
    INFO:   displayHeight         1080.00
    INFO:   framerate             29.97
    INFO:   moovposition          32.00
    INFO:   duration              3291.14
    11077.478 kB / 25.96 sec (0.7%)
    INFO: Metadata:
    INFO: trackinfo:
    INFO:   timescale             30000.00
    INFO:   length                98732634.00
    INFO:   language              eng
    INFO: sampledescription:
    INFO:   sampletype            avc1
    INFO:   timescale             48000.00
    INFO:   length                157974528.00
    INFO:   language              eng
    INFO: sampledescription:
    INFO:   sampletype            mp4a
    INFO:   audiochannels         2.00
    INFO:   audiosamplerate       48000.00
    INFO:   videoframerate        29.97
    INFO:   aacaot                2.00
    INFO:   avclevel              40.00
    INFO:   avcprofile            100.00
    INFO:   audiocodecid          mp4a
    INFO:   videocodecid          avc1
    INFO:   width                 1920.00
    INFO:   height                1080.00
    INFO:   frameWidth            1920.00
    INFO:   frameHeight           1080.00
    INFO:   displayWidth          1920.00
    INFO:   displayHeight         1080.00
    INFO:   framerate             29.97
    INFO:   moovposition          32.00
    INFO:   duration              3291.14
    19733.950 kB / 21.86 sec (0.6%)
    [aborted by user]
    Then it might go up to 0.9% before going down to 0.8% again and so on, stuttering every other promille or so.
    And here is the file that resulted from that run:
    https://mega.co.nz/#!Vd1TRZoI!XEzGC11hOU1tu98k_x988UmptUAOwYBHX4-l4LOFXVE

    The video stops after the first few seconds in the flash player, so it's not even feasible to watch it that way and I would really like to have a local copy anyway as I expect to rewatch this. rtmpdump does seem to be able to access all the material though. I left it overnight and it got up to ~25% before I stopped it (that percentage is plausible looking at the content at the end, but the sound is badly out of sync by then and all the repeats give an everpresent and unpleasant feeling of déjà vu, so that file is pretty much unusable)

    Could you tell me what the problem is and point me to towards a way to fix it?
    Quote Quote  
  2. Member bat999's Avatar
    Join Date
    Feb 2008
    Location
    United Kingdom
    Search Comp PM
    Originally Posted by etiam View Post
    Could you tell me what the problem is and point me to towards a way to fix it?
    Hi
    Some streams will only download in 'real-time'.
    Try adding --live to your command.
    Quote Quote  
  3. Originally Posted by bat999 View Post
    Try adding --live to your command.
    Thank you bat999! That's considerably better.
    No more starting over with --live (or --realtime, for that matter) added to the command.


    That unmasked another problem though: The first minute or so works fine, but then the audio stays fine and the picture starts skipping frames. It looks a bit like a quick slide show of a photo sequence.
    Here's a video sample:
    https://mega.co.nz/#!FJli3TbJ!GsUe9gPYrSawNWC7SsiGT3b8TGM5XPW2t0rTQkv9q7s
    (The slideshow-esque part starts around 1:10.)
    I'm not sure about the reason here either, but it seems that dropping frames is a natural response if the bandwidth can't keep up with the transfer, and I seem to be having a surprisingly slow connection to this, so there's a strong candidate for why if you ask me.
    Same question again then, in this inner layer of the onion of live stream downloading issues. Do you know what might be the cause, and is there something I can do to compensate for it?

    (And should I have started a new thread for this followup?)
    Quote Quote  
  4. Member
    Join Date
    Jan 2013
    Location
    Heaven
    Search Comp PM
    Use -v dude!
    Quote Quote  
  5. Thanks, -v would be a more compact way to write --live , then

    That works, but now that bat999 had pointed me in the right direction some of rtmpdump's manpage entries make a bit more sense:

    --realtime -R
    Download approximately in realtime, without attempting to speed up via Pause/Unpause commands
    ("the BUFX hack"). Useful for servers that jump backwards in time at the Unpause command.
    Resuming and seeking in realtime streams is still possible.


    That last line probably means -R is an even better choice in many cases. Whether I use -v or -R though, it solves the 'starting over-problem' and unmasks the 'slideshow problem'. I've been struggling with that new issue for a couple of days, but I'm not getting anywhere with it, so I'm starting a new thread here.

    EDIT: Appearantly, no I'm not. According to the wishes of the MEGA Super Moderator I'm continuing right here in this thread.

    So, to recap:
    The issue I had with the download restarting could be resolved by adding either the --live or --realtime flag to the command, and I can now get a file that includes the full length of the video. The resulting file seems to work fine during approximately the first 70 seconds (consistent over many retries). After that, the audio keeps working well, but the picture basically turns into a sequence of still images, updated about once per second. It looks a bit like a slide show of a photo sequence.

    So, again, I'm trying to download this video:
    https://www.ipam.ucla.edu/wowzavideo...p4&vfd=gss2012

    ...and here's a short video sample of the result:
    https://mega.co.nz/#!FJli3TbJ!GsUe9g...PW2t0rTQkv9q7s
    (The problem starts around 1:10.)

    I'm on Linux Mint 13, using the git repository version of rtmpdump from the 10th of April 2013 (compiled statically).
    The first minute of a run would look something like this:

    Code:
     ~/rtmpdump/rtmpdump --realtime -r "rtmp://128.97.46.13:1935/gss2012/" -a "gss2012/" -f "LNX 11,0,1,152" -W "https://www.ipam.ucla.edu/videoplayer/jwplayer.flash.swf" -p "https://www.ipam.ucla.edu/wowzavideo.aspx?vfn=10660.mp4&vfd=gss2012" -y "mp4:10660.mp4" -o mp4_10660.flv 
    RTMPDump v2.4
    (c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL
    Connecting ...
    INFO: Connected...
    Starting download at: 0.000 kB
      in approximately realtime (disabled BUFX speedup hack)
    INFO: Metadata:
    INFO: trackinfo:
    INFO:   timescale             30000.00
    INFO:   length                98732634.00
    INFO:   language              eng
    INFO: sampledescription:
    INFO:   sampletype            avc1
    INFO:   timescale             48000.00
    INFO:   length                157974528.00
    INFO:   language              eng
    INFO: sampledescription:
    INFO:   sampletype            mp4a
    INFO:   audiochannels         2.00
    INFO:   audiosamplerate       48000.00
    INFO:   videoframerate        29.97
    INFO:   aacaot                2.00
    INFO:   avclevel              40.00
    INFO:   avcprofile            100.00
    INFO:   audiocodecid          mp4a
    INFO:   videocodecid          avc1
    INFO:   width                 1920.00
    INFO:   height                1080.00
    INFO:   frameWidth            1920.00
    INFO:   frameHeight           1080.00
    INFO:   displayWidth          1920.00
    INFO:   displayHeight         1080.00
    INFO:   framerate             29.97
    INFO:   moovposition          32.00
    INFO:   duration              3291.14
    3138.303 kB / 12.31 sec (0.3%)
    A typical data transfer rate I get seems to be in the range of 30-60 kB/s. My ISP is providing a symmetric 10 Mbps connection and I would assume that UCLA has plenty of capacity at their end. I'm located in Europe.
    Some search engine work seems to suggest that the server could have a more or less fixed rate for outputting the video and this low transfer rate isn't coping with that, so frames get dropped to stay in sync. That's just my best guess so far though, and I don't have much experience with these issues.

    Could you tell me the what's causing this problem and if there's something I can do to fix it?
    Last edited by etiam; 16th Apr 2013 at 04:28.
    Quote Quote  
  6. Originally Posted by etiam View Post
    Thanks, -v would be a more compact way to write --live , then

    That works, but now that bat999 had pointed me in the right direction some of rtmpdump's manpage entries make a bit more sense:

    --realtime -R
    Download approximately in realtime, without attempting to speed up via Pause/Unpause commands
    ("the BUFX hack"). Useful for servers that jump backwards in time at the Unpause command.
    Resuming and seeking in realtime streams is still possible.


    That last line probably means -R is an even better choice in many cases. Whether I use -v or -R though, it solves the 'starting over-problem' and unmasks the 'slideshow problem'. I've been struggling with that new issue for a couple of days, but I'm not getting anywhere with it, so I'm starting a new thread here.

    EDIT: Appearantly, no I'm not. According to the wishes of the MEGA Super Moderator I'm continuing right here in this thread.

    So, to recap:
    The issue I had with the download restarting could be resolved by adding either the --live or --realtime flag to the command, and I can now get a file that includes the full length of the video. The resulting file seems to work fine during approximately the first 70 seconds (consistent over many retries). After that, the audio keeps working well, but the picture basically turns into a sequence of still images, updated about once per second. It looks a bit like a slide show of a photo sequence.

    So, again, I'm trying to download this video:
    https://www.ipam.ucla.edu/wowzavideo...p4&vfd=gss2012

    ...and here's a short video sample of the result:
    https://mega.co.nz/#!FJli3TbJ!GsUe9g...PW2t0rTQkv9q7s
    (The problem starts around 1:10.)

    I'm on Linux Mint 13, using the git repository version of rtmpdump from the 10th of April 2013 (compiled statically).
    The first minute of a run would look something like this:

    Code:
     ~/rtmpdump/rtmpdump --realtime -r "rtmp://128.97.46.13:1935/gss2012/" -a "gss2012/" -f "LNX 11,0,1,152" -W "https://www.ipam.ucla.edu/videoplayer/jwplayer.flash.swf" -p "https://www.ipam.ucla.edu/wowzavideo.aspx?vfn=10660.mp4&vfd=gss2012" -y "mp4:10660.mp4" -o mp4_10660.flv 
    RTMPDump v2.4
    (c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL
    Connecting ...
    INFO: Connected...
    Starting download at: 0.000 kB
      in approximately realtime (disabled BUFX speedup hack)
    INFO: Metadata:
    INFO: trackinfo:
    INFO:   timescale             30000.00
    INFO:   length                98732634.00
    INFO:   language              eng
    INFO: sampledescription:
    INFO:   sampletype            avc1
    INFO:   timescale             48000.00
    INFO:   length                157974528.00
    INFO:   language              eng
    INFO: sampledescription:
    INFO:   sampletype            mp4a
    INFO:   audiochannels         2.00
    INFO:   audiosamplerate       48000.00
    INFO:   videoframerate        29.97
    INFO:   aacaot                2.00
    INFO:   avclevel              40.00
    INFO:   avcprofile            100.00
    INFO:   audiocodecid          mp4a
    INFO:   videocodecid          avc1
    INFO:   width                 1920.00
    INFO:   height                1080.00
    INFO:   frameWidth            1920.00
    INFO:   frameHeight           1080.00
    INFO:   displayWidth          1920.00
    INFO:   displayHeight         1080.00
    INFO:   framerate             29.97
    INFO:   moovposition          32.00
    INFO:   duration              3291.14
    3138.303 kB / 12.31 sec (0.3%)
    A typical data transfer rate I get seems to be in the range of 30-60 kB/s. My ISP is providing a symmetric 10 Mbps connection and I would assume that UCLA has plenty of capacity at their end. I'm located in Europe.
    Some search engine work seems to suggest that the server could have a more or less fixed rate for outputting the video and this low transfer rate isn't coping with that, so frames get dropped to stay in sync. That's just my best guess so far though, and I don't have much experience with these issues.

    Could you tell me the what's causing this problem and if there's something I can do to fix it?
    i can download it using tubedigger download speed 400-500 KB/s
    Quote Quote  
  7. Originally Posted by CristianoA View Post
    i can download it using tubedigger download speed 400-500 KB/s
    So using rtmpdump is not required for this case. That's good to know.
    I dusted off an old Windows VM and tried tubedigger as well. I get lower transfer rates ( less than 160 kB/s, and with lots of variation ), but it does seem to do the trick.
    With that, my practical problem is solved. I can get the video downloaded.
    If someone knows what's going on with rtmpdump or a workaround for the problem, I would encourage them to leave a note about it though. I'll report back myself if I get it sorted out.
    Thank you for your help!
    Quote Quote  
  8. is there any news?
    I have the exact same issue.
    for now I use rtmpdump 2.1, but not without problems.
    Quote Quote  
  9. Hi filippo.

    Unfortunately, as far as I know, this is still an unresolved issue.
    It turned out that on closer inspection the Tubedigger downloads suffered from the slideshow issue as well.
    I have tried with just about every common open source, freeware, shareware and trialware tool I could, and at least under my use they all fall short, most in similar ways. And I've had the problem verified by a friend in another location, so this seems to go beyond both rtmpdump and the specific qualities of my connection.

    The least damaged download results so far I got by using cURL (which I believe handles rtmp streams by using librtmp).
    That may be due to cURL handling the stream a bit less deficiently than the other tools, or it may be due to chance.
    In either case, those results aren't good enough either.
    If you do find a solution I would be very grateful to hear it.
    Quote Quote  
  10. Originally Posted by CristianoA View Post
    Thanks for the suggestion. Unfortunately recordings of these videos using Flash Stream Hunter "finish" after only a few seconds worth of video for me.
    Quote Quote  
  11. try this:
    Code:
    /rtmpdump-2.1 $ ./rtmpdump -m 20 -r "rtmp://128.97.46.13:1935/gss2012/" -y "mp4:10660.mp4" -q -o - | avconv -i - -qscale 0 -copyts -async 1 /home/user/Videos/graduate2012.mp4
    it isn't perfect but is the best I have found out.
    works only with rtmpdump v 2.0/2.1.
    pay attention to the file size!

    let me know if you find something better!
    Quote Quote  
  12. Is there a way to fix an old .mp4 file downloaded without the --live flag? I tried with ffmpeg -i old_file.mp4 new_file.mp4 and the video doesn't jump back but the audio DOES jump back.
    Quote Quote  
  13. Have you tried with the avconv command:
    avconv -i old_file.mp4 -qscale 0 -copyts -async 1 new_file.mp4 ?


    I don't know if this will be helpful, but this version works for me (doesn't store the same part several times), but sometimes it crashes so I recommend run by:
    Code:
    #!/bin/bash
    
    until ./rtmpdump -r "..." -y "..." -o filename -e
    do
    sleep 10
    done
    Quote Quote  
  14. Hi megaics ,

    Generally for knowning which streams are inclued in the video : ffmpeg -i video.!!!
    Code:
    ffmpeg -i ZZZ_3rd_graduate2012.ts
    
    ffmpeg version N-58869-gae33007 Copyright (c) 2000-2013 the FFmpeg developers
      built on Dec  7 2013 22:01:45 with gcc 4.8.2 (GCC)
    
      configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnu
    tls --enable-iconv --enable-libass --enable-libbluray --enable-libcaca --enable-libfreetype --enable-libgsm --enable-libilbc --enable-libmodplug --ena
    ble-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinger -
    -enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-li
    bvorbis --enable-libvpx --enable-libwavpack --enable-libx264 --enable-libxavs --enable-libxvid --enable-zlib
      libavutil      52. 56.100 / 52. 56.100
      libavcodec     55. 45.100 / 55. 45.100
      libavformat    55. 22.100 / 55. 22.100
      libavdevice    55.  5.102 / 55.  5.102
      libavfilter     3. 92.100 /  3. 92.100
      libswscale      2.  5.101 /  2.  5.101
      libswresample   0. 17.104 /  0. 17.104
      libpostproc    52.  3.100 / 52.  3.100
    Input #0, mpegts, from 'ZZZ_3rd_graduate2012.ts':
      Duration: 00:01:03.66, start: 1.456978, bitrate: 597 kb/s
      Program 1
        Metadata:
          service_name    : Service01
          service_provider: FFmpeg
        Stream #0:0[0x100]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p, 960x556 [SAR 139:135 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc
        Stream #0:1[0x101]: Audio: mp3 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, s16p, 128 kb/s
    You can watch :
    _ Stream #0:0
    _ Stream #0:1

    in other cases :
    _Stream #0:'any number'

    EXAMPLE You'll have to select :
    first the video and write -map 0:0 ( 0:9 )
    second the audio and write -map 0:1 ( 1:10 )

    in the commands-line add to indicate your need : -map 0:0 -map 0:1
    =====================

    Now with your specific link : I have done
    _ vcodec libx264
    _ size 960x556 ( the original is 1920x1080 )
    _ r 25 ( unstead 29,97 fps )
    _ audio mp3
    Code:
    3rd try
    =======
    
    rtmpdump -m 20 -r "rtmp://128.97.46.13:1935/gss2012/" -y "mp4:10660.mp4" -q -o -   | ffmpeg -i - -vcodec libx264 -acodec libmp3lame
     -r 25 -async 1 -vf scale=960:556 -aspect 16:9 -f mpegts "ZZZ_3rd_graduate2012.ts"
    
    ffmpeg version N-58869-gae33007 Copyright (c) 2000-2013 the FFmpeg developers
      built on Dec  7 2013 22:01:45 with gcc 4.8.2 (GCC)
    
      configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnu
    tls --enable-iconv --enable-libass --enable-libbluray --enable-libcaca --enable-libfreetype --enable-libgsm --enable-libilbc --enable-libmodplug --ena
    ble-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinger -
    -enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-li
    bvorbis --enable-libvpx --enable-libwavpack --enable-libx264 --enable-libxavs --enable-libxvid --enable-zlib
      libavutil      52. 56.100 / 52. 56.100
      libavcodec     55. 45.100 / 55. 45.100
      libavformat    55. 22.100 / 55. 22.100
      libavdevice    55.  5.102 / 55.  5.102
      libavfilter     3. 92.100 /  3. 92.100
      libswscale      2.  5.101 /  2.  5.101
      libswresample   0. 17.104 /  0. 17.104
      libpostproc    52.  3.100 / 52.  3.100
    Input #0, flv, from 'pipe:':
      Metadata:
        audiochannels   : 2
        videoframerate  : 30
        aacaot          : 2
        avclevel        : 40
        avcprofile      : 100
        frameWidth      : 1920
        frameHeight     : 1080
        displayWidth    : 1920
        displayHeight   : 1080
        moovposition    : 32
      Duration: 00:54:51.14, start: 0.000000, bitrate: N/A
        Stream #0:0: Video: h264 (High), yuv420p(tv, bt709), 1920x1080 [SAR 1:1 DAR 16:9], 29.97 tbr, 1k tbn, 59.94 tbc
        Stream #0:1: Audio: aac, 48000 Hz, stereo, fltp
    -async is forwarded to lavfi similarly to -af aresample=async=1:min_hard_comp=0.100000:first_pts=0.
    [libx264 @ 031d3220] using SAR=139/135
    [libx264 @ 031d3220] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 LZCNT BMI1
    [libx264 @ 031d3220] profile High, level 3.1
    Output #0, mpegts, to 'ZZZ_3rd_graduate2012.ts':
      Metadata:
        audiochannels   : 2
        videoframerate  : 30
        aacaot          : 2
        avclevel        : 40
        avcprofile      : 100
        frameWidth      : 1920
        frameHeight     : 1080
        displayWidth    : 1920
        displayHeight   : 1080
        moovposition    : 32
        encoder         : Lavf55.22.100
        Stream #0:0: Video: h264 (libx264), yuv420p, 960x556 [SAR 139:135 DAR 16:9], q=-1--1, 90k tbn, 25 tbc
        Stream #0:1: Audio: mp3 (libmp3lame), 48000 Hz, stereo, fltp
    Stream mapping:
      Stream #0:0 -> #0:0 (h264 -> libx264)
      Stream #0:1 -> #0:1 (aac -> libmp3lame)
    frame= 1545 fps=2.3 q=28.0 size=    4297kB time=00:01:01.63 bitrate= 571.2kbits/s dup=1 drop=2279
    ...
    Not finished !!!
    Cheers .
    JE SUIS CHARLIE !!!
    Quote Quote  
  15. Originally Posted by filippo.rosa View Post
    let me know if you find something better!
    Now I have. Completely undamaged, as far as I can tell.
    The crucial step was arranging for a better connection. I paid for some running time on a server close to the source and used a plain curl command from there.

    Not exactly an elegant software solution, but it has the considerable merit of being the only thing that has really worked so far.
    Quote Quote  
  16. P.S. Despite my workaround, if you find a solution to the original problem, especially with rtmpdump, please consider sharing it for the benefit of the community.
    Quote Quote  
  17. Member Emeritus
    Join Date
    May 2014
    Search PM
    Originally Posted by etiam View Post
    P.S. Despite my workaround, if you find a solution to the original problem, especially with rtmpdump, please consider sharing it for the benefit of the community.
    efiam, as alternative to rtmp you may use hls or hds.
    Examples
    Code:
    livestreamer "hlsvariant://http://128.97.46.13:1935/CP2015/12507.mp4/playlist.m3u8" best -o 12507.ts
    Code:
    adobehds.php --manifest "http://128.97.46.13:1935/CP2015/12507.mp4/manifest.f4m"
    Quote Quote  
  18. Originally Posted by etiam View Post
    P.S. Despite my workaround, if you find a solution to the original problem, especially with rtmpdump, please consider sharing it for the benefit of the community.
    it will be done.
    at the moment, I'm using the PauseBreak version of rtmpdump (see my previous post) and it's all good.
    Quote Quote  



Similar Threads