VideoHelp Forum

Try DVDFab and download streaming video, copy, convert or make Blu-rays,DVDs! Download free trial !
+ Reply to Thread
Results 1 to 21 of 21
Thread
  1. Member
    Join Date
    Sep 2012
    Location
    Dominican Republic
    Search PM
    Hello

    I am trying to capture streams from Neulion (new service provider for Univision, used to be edgefcs before via rtmp)
    They service the file as downloadable 2-second MP4 (H264) chunks.

    I am able to download all chunks, they number 0000 through 0058, then 0100 through 0158, etc.

    Example:
    http://nldsvod.cdnak.neulion.com/nlds_vod/univision/vod/2013/01/24/130123_2779233_atra...p4/00/0000.mp4
    http://nldsvod.cdnak.neulion.com/nlds_vod/univision/vod/2013/01/24/130123_2779233_atra...p4/00/0002.mp4
    http://nldsvod.cdnak.neulion.com/nlds_vod/univision/vod/2013/01/24/130123_2779233_atra...p4/00/0004.mp4
    http://nldsvod.cdnak.neulion.com/nlds_vod/univision/vod/2013/01/24/130123_2779233_atra...p4/00/0006.mp4

    I AM able to download those chunks, and even join them fine with MP4Box.
    The video plays somewhat fine (there are some bad, fragmented or pixelated frames) but more or less it's fine. What concerns me, however, is that there are many "strange" chunks, which download as mp4 but I cannot open them in any audio/video program, even if I add them to Premiere Pro they "show" just as one frame no audio/video ... however, they have "full" size the same as other regular mp4 chunks do:

    e.g. "strange" chunk

    http://nldsvod.cdnak.neulion.com/nlds_vod/univision/vod/2013/01/24/130123_2779233_atra...p4/00/0034.mp4
    http://nldsvod.cdnak.neulion.com/nlds_vod/univision/vod/2013/01/24/130123_2779233_atra...p4/00/0042.mp4
    http://nldsvod.cdnak.neulion.com/nlds_vod/univision/vod/2013/01/24/130123_2779233_atra...p4/00/0046.mp4

    The regular chunks have these properties (using Winnydows XVid4PSP):

    File: 0000.mp4
    Size: 1.1mb (1168540 bytes)
    FormatName: MOV,MP4,M4A,3GP,3G2,MJ2
    FormatLongName: QuickTime/MPEG-4/Motion JPEG 2000 format
    Duration: 00:00:02.019
    StreamCount: 4
    major_brand: isom
    minor_version: 1
    compatible_brands: isomavc1
    creation_time: 2013-01-23 19:58:45

    VideoStream 0:
    CodecName: h264
    CodecLongName: H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10
    CodecTag: 828601953 (avc1)
    Profile: Main @ L3.2
    Duration: 00:00:02.001
    Decoder Timebase: 50 / 2997
    Stream Timebase: 1 / 1000
    Frames: 60
    Bitrate: 4557kbps
    Bits: 24
    Resolution: 1280x720p
    Interlace: Progressive
    Aspect: 1.778
    PixelFormat: YUV420P
    Framerate: 29.970
    AverageFramerate: 30.000
    IsVariableFramerate (VFR)
    AverageSecondsRate: 0.033s
    Tick: 1
    B-frames mode: 1
    Max refs frames: 3
    ID: 1
    creation_time: 2013-01-23 19:58:45

    AudioStream 1:
    CodecName: aac
    CodecLongName: Advanced Audio Coding
    CodecTag: 1630826605 (mp4a)
    Duration: 00:00:02.019
    Decoder Timebase: 0 / 1
    Stream Timebase: 1 / 1000
    Frames: 86
    Bitrate: 101kbps
    Bits: 16
    Channels: 2
    Channels Layout: Stereo (3)
    Samplerate: 44100hz
    SampleFormat: AV_SAMPLE_FMT_S16
    FrameSize: 1024
    AverageSecondsRate: 0.023s

    VideoStream 2:
    CodecName: h264
    CodecLongName: H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10
    CodecTag: 828601953 (avc1)
    Duration: 00:00:02.003
    Decoder Timebase: 1 / 2000
    Stream Timebase: 1 / 1000
    Frames: 60
    Bitrate: 1741kbps
    Bits: 24
    Resolution: 0x0p
    Interlace: Progressive
    Aspect: NaN
    PixelFormat: NONE
    Framerate: 29.970
    AverageFramerate: 29.979
    IsVariableFramerate (VFR)
    AverageSecondsRate: 0.033s
    Tick: 1
    Max refs frames: 1
    ID: 1
    creation_time: 2013-01-23 19:58:53

    AudioStream 3:
    CodecName: aac
    CodecLongName: Advanced Audio Coding
    CodecTag: 1630826605 (mp4a)
    Duration: 00:00:01.998
    Start Position: 0.02s
    Decoder Timebase: 0 / 1
    Stream Timebase: 1 / 1000
    Frames: 86
    Bitrate: 108kbps
    Bits: 16
    Channels: 2
    Channels Layout: Stereo (3)
    Samplerate: 44100hz
    SampleFormat: AV_SAMPLE_FMT_S16
    FrameSize: 1024
    AverageSecondsRate: 0.023s


    The strange chunks have these properties:

    File: 0034.mp4
    Size: 454.9kb (465792 bytes)
    FormatName: MOV,MP4,M4A,3GP,3G2,MJ2
    FormatLongName: QuickTime/MPEG-4/Motion JPEG 2000 format
    Duration: 00:00:02.003
    StreamCount: 2
    major_brand: isom
    minor_version: 1
    compatible_brands: isomavc1
    creation_time: 2013-01-23 19:58:53

    VideoStream 0:
    CodecName: h264
    CodecLongName: H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10
    CodecTag: 828601953 (avc1)
    Duration: 00:00:02.003
    Decoder Timebase: 1 / 2000
    Stream Timebase: 1 / 1000
    Frames: 60
    Bitrate: 1741kbps
    Bits: 24
    Resolution: 0x0p
    Interlace: Progressive
    Aspect: NaN
    PixelFormat: NONE
    Framerate: 29.970
    AverageFramerate: 29.979
    IsVariableFramerate (VFR)
    AverageSecondsRate: 0.033s
    Tick: 1
    Max refs frames: 1
    ID: 1
    creation_time: 2013-01-23 19:58:53

    AudioStream 1:
    CodecName: aac
    CodecLongName: Advanced Audio Coding
    CodecTag: 1630826605 (mp4a)
    Duration: 00:00:01.998
    Start Position: 0.02s
    Delay: 15ms
    Decoder Timebase: 0 / 1
    Stream Timebase: 1 / 1000
    Frames: 86
    Bitrate: 108kbps
    Bits: 16
    Channels: 2
    Channels Layout: Stereo (3)
    Samplerate: 44100hz
    SampleFormat: AV_SAMPLE_FMT_S16
    FrameSize: 1024
    AverageSecondsRate: 0.023s



    Also very strange thing is that when I join all the "good" chunks, the resulting video is shorter by 6-7 minutes compared to the original video on the website, even though I cannot "perceive" *** the difference between the videos played. That leads me to believe that the "strange" chunks have to do something with it.

    *** I say perceive because I just randomly checked spots in the resulting video and everything seems to be OK (besides some bad frames like pixelation, etc.), but I have not yet had time to see the whole video back to back


    QUESTION: Does anybody know how to get the good, full video from neulion, or what to do with these "strange" chunks?
    Quote Quote  
  2. I suspect the "strange" chunks simply don't have a keyframe (and requisite header information). So they can't be decoded without the chunk before them. Joining many separate chunks together should make a file smaller than the sum of all the individual file sizes because all the redundant headers are removed.
    Quote Quote  
  3. Member
    Join Date
    Sep 2012
    Location
    Dominican Republic
    Search PM
    Well, MP4 box allows me to join the "strange" chunks along with the regular chunks, but the audio becomes delayed.

    The video is shorter by 6-7 minutes, e.g. 47 minute original video (47 minutes showing on the streaming website) becomes 40 minutes after joining only the "regular" chunks, ... so there's something wrong there
    Quote Quote  
  4. Can up upload a sequence of three consecutive chunks that include two normal chunks around a "strange" chunk?
    Quote Quote  
  5. Member
    Join Date
    Sep 2012
    Location
    Dominican Republic
    Search PM
    OK, new info, I was able to extract audio from these strange chunks, and the audio is continuous to adjacent chunks, meaning there is a noticeable audio cut between two non-consecutive chunks when leaving out the strange chunk. When a converted stranage chunk is added, the audio becomes continuous.

    This leads me to believe that this strange chunk is encoded/encrypted MP4.... any way to work with this, to decrypt it?
    Quote Quote  
  6. Your "bad" files play fine in various media players e.g. vlc, kmplayer

    Anyway seems to join ok with ffmpeg e.g. here is 500-510

    1) Create a text file "list.txt" , the contents should list the path and filenames

    eg.
    Code:
    file 'G:\PATH\0500.mp4'
    file 'G:\PATH\0502.mp4'
    file 'G:\PATH\0504.mp4'
    file 'G:\PATH\0506.mp4'
    file 'G:\PATH\0508.mp4'
    file 'G:\PATH\0510.mp4'

    2)
    I placed a ffmpeg.exe binary in the folder of files to append, along with list.txt, ran the following command
    (I got the binary from http://ffmpeg.zeranoe.com/ )

    Code:
    ffmpeg -f concat -i list.txt -c copy output.mp4
    Image Attached Files
    Quote Quote  
  7. Member
    Join Date
    Sep 2012
    Location
    Dominican Republic
    Search PM
    @poisondeathray - I actually feel stupid now, because I posted links to a wrong file (I had downloaded two chapters and thought I was ina folder for a different chapter). I have corrected the links now.
    Quote Quote  
  8. Which links did you change? Can you just zip a sequence up that has a few "bad" and "good" ones and upload them here to this site ?
    Quote Quote  
  9. Member
    Join Date
    Sep 2012
    Location
    Dominican Republic
    Search PM
    I put the same "chunk numbers" (e.g. 502, 506) but it's Episode 1 now instead of Episode 13 (posted originally)
    If you download the chunks from the link above you will see the physical difference between chunks 0500.mp4 and 0502.mp4, for example.
    Quote Quote  
  10. You're right, those "bad" files don't play by themselves, but they seem to play once joined

    Here is 500-510 using the same method , updated files

    There is a slight glitch at start of 502
    Image Attached Files
    Quote Quote  
  11. Member
    Join Date
    Sep 2012
    Location
    Dominican Republic
    Search PM
    Great thank you! I will try to play around with Ffmpeg... I was trying to join with mp4box, but it caused audio delay
    Quote Quote  
  12. Yes, the "bad" clips simply don't have any I frames. They need to be joined with the other clips to play properly. I tried joining them with My MP4Box GUI but that gave a file that didn't play properly. Demuxing them all to separate h264 and aac files (Mp4Muxer), appending those, then remuxing to MP4 (MP4Muxer again) gave a file that played properly.

    Weird that they're 1280x720 but interlaced.
    Quote Quote  
  13. ffmpeg reports many errors, but it seems to play ok. If you want to see the log file, use this instead

    Code:
    ffmpeg -f concat -i mylist.txt -c copy output.mp4 2>log.txt

    Code:
    [h264 @ 0003d620] Missing reference picture, default is 0
    [h264 @ 0003d620] decode_slice_header error
    [h264 @ 0003d620] concealing 3600 DC, 3600 AC, 3600 MV errors in P frame
    [concat @ 00032320] Estimating duration from bitrate, this may be inaccurate
    Input #0, concat, from 'mylist.txt':
      Duration: 00:00:00.01, start: 0.000000, bitrate: 218 kb/s
        Stream #0:0: Video: h264 (Main) (avc1 / 0x31637661), yuv420p, 1280x720, 2505 kb/s, 29.96 fps, 29.97 tbr, 1k tbn, 59.94 tbc
        Stream #0:1: Audio: aac (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 108 kb/s
    Output #0, mp4, to 'output.mp4':
      Metadata:
        encoder         : Lavf54.59.107
        Stream #0:0: Video: h264 ([33][0][0][0] / 0x0021), yuv420p, 1280x720, q=2-31, 2505 kb/s, 29.96 fps, 16k tbn, 1k tbc
        Stream #0:1: Audio: aac ([64][0][0][0] / 0x0040), 44100 Hz, stereo, 108 kb/s
    Stream mapping:
      Stream #0:0 -> #0:0 (copy)
      Stream #0:1 -> #0:1 (copy)
    Press [q] to stop, [?] for help
    [h264 @ 0003d620] no frame!
        Last message repeated 59 times
    [mov,mp4,m4a,3gp,3g2,mj2 @ 0003da20] decoding for stream 0 failed
    [mov,mp4,m4a,3gp,3g2,mj2 @ 0003da20] Could not find codec parameters for stream 0 (Video: h264 (avc1 / 0x31637661), 2892 kb/s): unspecified size
    Consider increasing the value for the 'analyzeduration' and 'probesize' options
    [h264 @ 0003d620] Missing reference picture, default is 0
    [h264 @ 0003d620] decode_slice_header error
    [h264 @ 0003d620] concealing 3600 DC, 3600 AC, 3600 MV errors in B frame
    [h264 @ 0003d620] no frame!
        Last message repeated 59 times
    [mov,mp4,m4a,3gp,3g2,mj2 @ 02b74020] decoding for stream 0 failed
    [mov,mp4,m4a,3gp,3g2,mj2 @ 02b74020] Could not find codec parameters for stream 0 (Video: h264 (avc1 / 0x31637661), 2676 kb/s): unspecified size
    Consider increasing the value for the 'analyzeduration' and 'probesize' options
    [h264 @ 0003d620] Missing reference picture, default is 0
    [h264 @ 0003d620] decode_slice_header error
    [h264 @ 0003d620] concealing 3600 DC, 3600 AC, 3600 MV errors in B frame
    [h264 @ 0003d620] Missing reference picture, default is 0
    [h264 @ 0003d620] decode_slice_header error
    [h264 @ 0003d620] concealing 3600 DC, 3600 AC, 3600 MV errors in P frame
    frame=  345 fps=0.0 q=-1.0 Lsize=    4122kB time=00:00:12.06 bitrate=2799.1kbits/s    
    
    video:3945kB audio:164kB subtitle:0 global headers:0kB muxing overhead 0.323091%
    There might be some way to preprocess the files so you dont get the audio glitch, not sure

    Or maybe if you were successful joining the audio with another method you can swap the audio

    It's in sync, so the timecodes are ok (these were VFR streams)
    Quote Quote  
  14. Originally Posted by jagabo View Post
    Yes, the "bad" clips simply don't have any I frames. They need to be joined with the other clips to play properly. I tried joining them with My MP4Box GUI but that gave a file that didn't play properly. Demuxing them all to separate h264 and aac files (Mp4Muxer), appending those, then remuxing to MP4 (MP4Muxer again) gave a file that played properly.

    Weird that they're 1280x720 but interlaced.

    Yes, the interlace is weird for that resolution...

    Did the demuxing / append / remux method give a file without that audio glitch ? Maybe that would be a better method?
    Quote Quote  
  15. Member
    Join Date
    Sep 2012
    Location
    Dominican Republic
    Search PM
    Thank you very much, you saved my day! I was just able to fully reconstruct the whole mp4 file with ffmpeg. Yes it did give lots of errors, but clicking-through-the-video I did not notice any problems and audio & video were well synced throughout the whole resulting mp4 file.

    Again, Thank you VERY much.
    Quote Quote  
  16. Originally Posted by poisondeathray View Post
    Did the demuxing / append / remux method give a file without that audio glitch ? Maybe that would be a better method?
    I didn't notice any audio or video glitches. File attached.
    Image Attached Files
    Quote Quote  
  17. Member
    Join Date
    Oct 2013
    Location
    New York
    Search PM
    In regards to downloading flv segmented streaming video from neulion, I am trying to use IDM. The problem is that there seem to be multiple variables in the source file name (requiring multiple wildcards with a coordinated logic). The variables sometimes increase by units other than 1 and sometimes one variable will decrease. It seems that in order to join the files back together, I would need to log the order that they streamed. I'm new to downloading video with these complexities, so I may be missing something basic. Below is an example of a source file:

    example of the end of the file source name - unedited:
    http://..._nyj_ne_2013_h_whole_1_400.mp4/00/1518.mp4



    example of file source where I've placed the 3 necessary wildcards:
    http://..._nyj_ne_2013_h_whole_1_*.mp4/*/*.mp4


    thank you and let me know if I can provide any other information.
    Last edited by emomoe; 11th Oct 2013 at 20:47.
    Quote Quote  
  18. Member
    Join Date
    Sep 2012
    Location
    Dominican Republic
    Search PM
    http://..._nyj_ne_2013_h_whole_1_*.mp4/*/*.mp4

    first * is a birtrate e.g. 400.mp4 is 400 kbps bitrate. I think it depends upon content provider, univision goes max 3000.
    The second /*/ is almost always , but sometimes 01. Try test.
    The third *.mp4 is a chunk. Try test it. Usually does not go over 58, and numbers are always even numbers.
    0000.mp4 -> 0058.mp4, then hops onto 0100.mp4 -> 0158.mp4, hops onto 0200.mp4 -> 0258.mp4, etc.
    You can try it in your browser.
    You may need to do an excel spreadsheet to easily fill in all chunks and download them by download manajer.

    Any way, first of all, try installing Coojah (for me Cookah does not work well with IE and Mozilla but fine with Chrome), and see what chunks are being received, also do it from the fastest connection you can be on, so the fastest bitrate kicks in, you will know the max bitrate then, then you download chunks with the fastest bitrate.

    Easy merge with ffmpeg works well as suggested in above posts, but if you encounter problems, try using ffmpeg filter h264 mp4toannexb investigate here: http://superuser.com/questions/133413/joining-h264-without-re-encoding
    I used this way multiple times to join various h264 files, but make sure to reencode audio when transferring back from mpegts into h264/aac, because I found out that audio issues are the most frequent causes to break output file.

    So I do:

    ffmpeg -i first_file.mp4 -c copy -bsf h264_mp4toannexb first_file.ts
    ffmpeg -i second_file.mp4 -c copy -bsf h264_mp4toannexb second_file.ts
    ffmpeg -i "concat:first_file.ts|second_file.ts" -vcodec copy -aprofile aac_low -acodec aac -strict experimental -cutoff 15000 -ab 192k (use here your actual original bitrate) [optional to adjust audio level: -af volume=XdB] -bsf aac_adtstoasc fileoutput.mp4

    you can also use
    ffmpeg -f concat -i listchunks.txt -vcodec copy -aprofile aac_low -acodec aac -strict experimental -cutoff 15000 -ab 192k (use here your actual original bitrate) [optional to adjust audio level: -af volume=XdB] -bsf aac_adtstoasc fileoutput.mp4


    again, try the simple concat method listed above for chunks, if it doe snot work you will need to convert chunks into ts and then form ts into h264/aac.
    Quote Quote  
  19. Member
    Join Date
    Oct 2013
    Location
    New York
    Search PM
    Thanks so much! I'll try this and Coojah was great advice. I'll update on how it works out.
    Quote Quote  
  20. Member
    Join Date
    Oct 2013
    Location
    New York
    Search PM
    Hi rubiohiguey - your advice was great! I used Coojah to log all the urls of the segmented video stream. After that I used wget + a simple script to download and then mp4box + a simple script to join. I'm having issue with the final output - it's a single mp4 that plays, but a complete mess. I posted the issue here:
    https://forum.videohelp.com/threads/359651-downloading-segmented-streaming-video-proble...56#post2275456
    Quote Quote  



Similar Threads