Saving streamed video that are segmented?

[ShaneJensen]

Hi. I'm trying to save streamed videos from katch.me using Flash Video Downloader (FVD) and am coming up with this issue of the video being streamed by very segmented .ts files under a half a megabyte. If the video is really long, it would take a very long time to save them all and compile manually. This happens when I try to save streaming Flash videos from various other sites that is streaming video as well. I'm very frustrated. Makes this VERY difficult for me. I've Googled around a bit for a solution and browsed around VideoHelp as well, but I'm unable to come up with anything helpful to my specific issue. I'm also overwhelmed by the number of results as well, so it's possible that if there is a solution already addressed here, I missed it due to not finding it. Can anyone help with this issue? Thanks.

[ShaneJensen]

The following image below is exactly what I'm talking about. See how many files it's broken down into? It would take me FOREVER to save all those and compile them into one. There MUST be software that does all that automatically. Can anyone please help me? Katch.me is shutting down in a week and a half and I want to save the videos on this person's account.

[ShaneJensen]

Anyone anyone? Time is of the essence here. I need a solution to this ASAP. PLEASE HELP!

[blud7]

Use TSMuxer and join them together.
MKVMerge will do the same, but the GUI is a bit more complicated.

[ShaneJensen]

Use TSMuxer and join them together.
MKVMerge will do the same, but the GUI is a bit more complicated.

Thank you, but joining them is not the issue. I should have been more clear. Is there a quicker way to DOWNLOAD them all in one click? Katch.me has these videos split into over 300 segments. With several dozen video uploads, I do not have enough time in the day to go through and download all of these splits for each video. There MUST be a quicker way to download all of these.

[ponens]

Use livestreamer or ffmpeg.

[ShaneJensen]

The "katch.me' site appears to transport video only using hls.

Start with the playlist rather than using the individual .ts segments.

For this video
https://katch.me/paulrobinett/v/78525695-0172-3c4a-8c45-737cc1f2361e

The hls playlist details can be found here
https://katch.me/videos/listVideo/78525695-0172-3c4a-8c45-737cc1f2361e

"videoUrlForPlayer":"https://d152nid216lr13.cloudfront.net/78525695-0172-3c4a-8c45-737cc1f2361e/playlist.m3u8

Use livestreamer or ffmpeg on the command line to download.

livestreamer "hls://https://d152nid216lr13.cloudfront.net/78525695-0172-3c4a-8c45-737cc1f2361e/playlist.m3u8" -o output.ts

Whether or not some program, extension, or plugin can be used instead of livestreamer or ffmpeg on the command line, I don't know.

Thank you very much for your response and the information. I don't understand how to use this Livestreamer or ffmpeg one bit. It's too complex and complicated for me. I need something simple and straight forward that will run and do what I need with a few clicks of the mouse. Those two things are just too complicated for me. But thanks anyway.

[ponens]

Thank you very much for your response and the information. I don't understand how to use this Livestreamer or ffmpeg one bit. It's too complex and complicated for me. I need something simple and straight forward that will run and do what I need with a few clicks of the mouse. Those two things are just too complicated for me. But thanks anyway.

Are you running windows OS? IDM can be made to work in one click, with a few simple extra steps beforehand.

If you're not running windows, I don't know.

note: IDM is a commercial program but has a 30 day trial period.

[ShaneJensen]

Thank you very much for your response and the information. I don't understand how to use this Livestreamer or ffmpeg one bit. It's too complex and complicated for me. I need something simple and straight forward that will run and do what I need with a few clicks of the mouse. Those two things are just too complicated for me. But thanks anyway.

Are you running windows OS? IDM can be made to work in one click, with a few simple extra steps beforehand.

If you're not running windows, I don't know.

note: IDM is a commercial program but has a 30 day trial period.

Yes, I'm running Windows. It's Windows 7 to be exact.

[ponens]

Yes, I'm running Windows. It's Windows 7 to be exact.

In that case, to save katch.me videos, follow these steps:

1. Install IDM

2. Make a note of this template:

http://d152nid216lr13.cloudfront.net/78525695-0172-3c4a-8c45-737cc1f2361e/playlist.m3u8

3. Go to the JWPlayer test page:

http://demo.jwplayer.com/stream-tester/

4. Paste the template url into the "Stream to Load" box and click "LOAD STREAM"

5. IDM button should popup over video, click it to save full video in one step.

Notes:

For other katch.me videos, replace (only) "78525695-0172-3c4a-8c45-737cc1f2361e" in the template with the number of the desired video (available in every katch.me video page). Then use the edited template on the JWplayer test page.

The JWPlayer test page requires flash.

Note that the template url starts with http and not https. It will not work with https.

[ShaneJensen]

In that case, to save katch.me videos, follow these steps:

1. Install IDM

2. Make a note of this template:

http://d152nid216lr13.cloudfront.net/78525695-0172-3c4a-8c45-737cc1f2361e/playlist.m3u8

3. Go to the JWPlayer test page:

http://demo.jwplayer.com/stream-tester/

4. Paste the template url into the "Stream to Load" box and click "LOAD STREAM"

5. IDM button should popup over video, click it to save full video in one step.

Notes:

For other katch.me videos, replace (only) "78525695-0172-3c4a-8c45-737cc1f2361e" in the template with the number of the desired video (available in every katch.me video page). Then use the edited template on the JWplayer test page.

The JWPlayer test page requires flash.

Note that the template url starts with http and not https. It will not work with https.

Thank you so much. That seems to be working. I really appreciate this. Thanks again.

[ShaneJensen]

Thank you again to those of you who helped me so far. I have saved all the files I needed from the fellow's Katch.me account. Now I have a new issue. They all need to be rotated and converted to a more friendly format. Problem is, they are unable to open in Sony Vegas, or any of the other converters and editors that I have. I tried demuxing with TsMuxer and Avidemux without any success. Does anyone have a better solution to work with these difficult files? I have one of the files uploaded as an example for any of you to work with if interested. Any help would be greatly appreciated. Thanks.

https://forum.videohelp.com/attachment.php?attachmentid=36784&stc=1&d=1461688384

[ponens]

The source videos from this guy on katch.me are pretty messed up. Either the guy, or katch.me, doesn't have the framerate stored in the mpegts, so you'll have to determine it from the elementary video stream.

Plus, the guy shoots on various devices. The phone video in your attachment seems to have a framerate of 7.5, but the first video you posted about in this thread (the one where this poor old duffer is complaining about being 49) has a framerate of 29.97, but again it is not stored in the mpegts, so you'll have to check the elementary streams. I believe the guy said that video was shot on a GoPro.

ffmpeg is the obvious tool to use here, but you mentioned before that you have an aversion to using it - at least on the command line.

You'll have to re-encode to do the rotations. If you need tools with a GUI, consider tsmuxer, winff and virtualdub. If you use winff, you'll still need to setup ffmpeg type presets though.

Here is your attachment rotated and re-encoded using ffmpeg. The file size is high because the "-crf 20" present was used. But the results are still pretty terrible because the source video is so darn awful. RENETTO20160417_01 Just catching up_transcode.mp4

[ShaneJensen]

The source videos from this guy on katch.me are pretty messed up. Either the guy, or katch.me, doesn't have the framerate stored in the mpegts, so you'll have to determine it from the elementary video stream.

Plus, the guy shoots on various devices. The phone video in your attachment seems to have a framerate of 7.5, but the first video you posted about in this thread (the one where this poor old duffer is complaining about being 49) has a framerate of 29.97, but again it is not stored in the mpegts, so you'll have to check the elementary streams. I believe the guy said that video was shot on a GoPro.

ffmpeg is the obvious tool to use here, but you mentioned before that you have an aversion to using it - at least on the command line.

You'll have to re-encode to do the rotations. If you need tools with a GUI, consider tsmuxer, winff and virtualdub. If you use winff, you'll still need to setup ffmpeg type presets though.

Here is your attachment rotated and re-encoded using ffmpeg. The file size is high because the "-crf 20" present was used. But the results are still pretty terrible because the source video is so darn awful. RENETTO20160417_01 Just catching up_transcode.mp4

Hi thank you for the response. Yeah, the videos from Katch.me are pretty messed up for a few reasons. The fact that these are captures from Pariscope livestreams mean that the quality varies due to connection speeds. So the compression and frame rate seem to fluctuate. That's how it seems to be with livestreaming. All his katch.me videos are all from his iPhone. None of them are from the GoPro. The GoPro videos are for his YouTube channel. By the way, he's Renetto on YouTube. About ten years ago he was one of the most subscribed. I still find him interesting as a vlogger. But I digress.

Also, I just realized that the video I linked here as an example is probably the worst of all his videos in terms of quality. There are other videos on his katch.me account that are much cleaner and smoother quality.

I've managed to get ffmpeg to work through simple instructions given to me in a thread on another board here on this forum using a .bat file to run with the script in it that was provided for me as a template. I managed to use that to convert all of those .ts files. However, it was suggested to me to put in a command for the 7.5 frame rate. But the rest of the videos are not 7.5. Should I force a frame rate of the standard 29.970 when I re-render them in Vegas? I do want a clean consistency up to the max. The file sizes do come out much larger, but that's okay.

I don't think the rotation script suggestion will work in my case because there are videos where he turned the iPhone around several times resulting in some of the scenes to be upsidedown. So in Vegas, I need do a split and rotate those scenes to be right side up. I'm okay with that. My biggest concern at this point is which template would be best to run in ffmpeg? The one that was provided for me on conversion forum seems good enough. I'm just curious about the frame rate issue. I'm thinking of just forcing 29.970, but want to make sure that is okay.

[poisondeathray]

Vegas should be able to do all the framerate conversions, conforming all the various clips to the project FPS. If you have smart resample on, it will add frame blends; if it's off it will add frame duplicates. That would be easiest

But using a lot of these types clips might cause vegas to crash. YMMV and save often. It would be more stable if you used an I-frame editing codec, or lossless I-frame intermediate - but it's more steps, more HDD space. I would try it as is (the stream copied mp4 option), and if it doesn't work then try re-encoding to an I-frame option

You can't just "force" a framerate to 29.97 in ffmpeg unless you re-encode. Because if you go from 7.5 (or there abouts) to 29.97, that means you have to introduce new frames (either blends, or duplicates, much like vegas would) if you want it to keep in sync and the same duration.

[JVRaines]

You can't just "force" a framerate to 29.97 in ffmpeg unless you re-encode. Because if you go from 7.5 (or there abouts) to 29.97, that means you have to introduce new frames (either blends, or duplicates, much like vegas would) if you want it to keep in sync and the same duration.

When used as an input option, -r reads the stream at the specified rate and does not drop or duplicate frames. Audio will stay in sync if you specify the correct rate. In this case, it was 7.5 fps for one file but apparently varies among the other files.

[poisondeathray]

You can't just "force" a framerate to 29.97 in ffmpeg unless you re-encode. Because if you go from 7.5 (or there abouts) to 29.97, that means you have to introduce new frames (either blends, or duplicates, much like vegas would) if you want it to keep in sync and the same duration.

When used as an input option, -r reads the stream at the specified rate and does not drop or duplicate frames. Audio will stay in sync if you specify the correct rate. In this case, it was 7.5 fps for one file but apparently varies among the other files.

That's what the statement is saying. You can't force 29.97 from 7.5, because it's not the correct rate. It will go out of sync as an input option. If you use it as an output option it will be in sync because frames will be added to make up the peroper framerate

And that's the other the problem - what is the "correct rate" ? Very likely none of these clips were 7.5 exactly for the originals. If they are phone videos, likely they are all VFR to begin with. Very unlikely that any of them have a nice round framerate. The katch.me or whatever server process probably aren't serving the original video, but you end up having an average framerate that fluctuates around 7.5 for some of the videos. Some of them might be 7.337 average like this one. That is the actual average framerate for this one, not 7.5

When you use -r as an input option that's only valid for raw input formats, or for override if you re-encode. It's not valid for stream copying. The number of frames in the original is 1195. When you "force" 7.5 with re-encoding as an output option you get 1122 frames with duplicates inserted, because the actual average framerate is actually lower than 7.5. If you "force" 7.5 with re-encoding as an input option, you get the same 1195 frames, but it becomes slightly out of sync, as the duration becomes shorter because 7.5 is faster than the actual average framerate (audio is 2min 42sec, but video is 2min 39sec , so AV mismatch and progressive desync)

[ShaneJensen]

Thank you for the response, guys. I'm understanding this a bit more with each message here. Like I said, that example I uploaded was a terrible one I realized. I have uploaded another short one that is much better quality. Same screen resolution, but the clarity and frame rate looks a lot better. I'll post that link at the end of my message. I just have a last set of questions here regarding the -r 7.5 command for the ffmpeg script that JVRaines suggested to me on the thread I posted in the Video Conversion board. Since there are other videos with better frame rates and since I have 40 of these .ts files to convert, and will be rendering the final renders with 29.97 frames, what exactly should I do with the -r command? Should I use that or leave it out? If I should use it, should I put in a higher frame rate number since the majority of them are better than that first example video I showed you? I ran the ffmpeg script that Poisondeathray gave me and added the -r 7.5, that JVRaines gave me, to Poison's template and ran that for all 40 of the .ts files. Should I tweak that -r 7.5 command, or take it out, and run those .ts files through that script again?

I'm actually going to be sharing the final rendered files with Paul (the guy in the videos) when I'm done, so I want to make sure they are right.

Here's the new example.

http://files.videohelp.com/u/81934/RENETTO20160305_03%20Tow%20truck%20is%20here.ts

[poisondeathray]

Thank you for the response, guys. I'm understanding this a bit more with each message here. Like I said, that example I uploaded was a terrible one I realized. I have uploaded another short one that is much better quality. Same screen resolution, but the clarity and frame rate looks a lot better. I'll post that link at the end of my message. I just have a last set of questions here regarding the -r 7.5 command for the ffmpeg script that JVRaines suggested to me on the thread I posted in the Video Conversion board. Since there are other videos with better frame rates and since I have 40 of these .ts files to convert, and will be rendering the final renders with 29.97 frames, what exactly should I do with the -r command? Should I use that or leave it out? If I should use it, should I put in a higher frame rate number since the majority of them are better than that first example video I showed you? I ran the ffmpeg script that Poisondeathray gave me and added the -r 7.5, that JVRaines gave me, to Poison's template and ran that for all 40 of the .ts files. Should I tweak that -r 7.5 command, or take it out, and run those .ts files through that script again?

I'm actually going to be sharing the final rendered files with Paul (the guy in the videos) when I'm done, so I want to make sure they are right.

Here's the new example.

http://files.videohelp.com/u/81934/RENETTO20160305_03%20Tow%20truck%20is%20here.ts

I thought I just explained it... The translation is don't use it as an input option, because that's not the correct frame rate. The frame rates will all be slightly different, as they usually are for phone videos. For example, this last one has an average of 29.something, but less than 29.97

If you ran the script I gave you and added -r 7.5 as an input option (before the -i) nothing extra will happen, because that was a stream copy. The framerate will remain the average 7.3 something or whatever it is for the other clips

And you don't want to change the framerate, unless you do it in a way that keeps everything in sync. That means adding frames if the desired FPS is higher than the source FPS. Using -r as an input option doesn't add or delete frames, so that will cause your file to become out of sync, unless you choose the correct average framerate (and again, it might be 12.3436 for one, 29.746 for another, etc....)

So if you want to change the framerate before importing in to vegas, this means re-encoding to an intermediate, and using -r as an output option (after the -i). As mentioned in the other thread , this is essentially the same thing vegas is doing on the fly when you drop the clips onto a 29.97 timeline. If you have smart resample off, they will be duplicate frames, if you have resmaple on, blended frames. Most people prefer duplicates, because the ghosty look of blends, but go ahead and experiment to see what you like better. The benefit of the intermediate method if you use lossless I-frame, is it will be more stable in vegas. The negatives are lots of HDD space required, takes longer than stream copying, vegas doesn't "treat" lossless YUV intermediates with studio RGB, but rather computer RGB, so you might get superbright/dark level clipping. (The exception is magicyuv with the full range option, but magicyuv is VFW only, unfortunately there is no ffmpeg component)

In my experience , 40 clips of this type should be ok. But vegas can get "squirrelly" and prone to crashes with these types of clips if you have a larger or more complex project. So I would try the copy method first. If you have problems, only then would I use the other method using -r as an output option and adding frames with a re-encoded intermediate to 29.97 before importing

[ShaneJensen]

I thought I just explained it... The translation is don't use it as an input option, because that's not the correct frame rate. The frame rates will all be slightly different, as they usually are for phone videos. For example, this last one has an average of 29.something, but less than 29.97

If you ran the script I gave you and added -r 7.5 as an input option (before the -i) nothing extra will happen, because that was a stream copy. The framerate will remain the average 7.3 something or whatever it is for the other clips

And you don't want to change the framerate, unless you do it in a way that keeps everything in sync. That means adding frames if the desired FPS is higher than the source FPS. Using -r as an input option doesn't add or delete frames, so that will cause your file to become out of sync, unless you choose the correct average framerate (and again, it might be 12.3436 for one, 29.746 for another, etc....)

So if you want to change the framerate before importing in to vegas, this means re-encoding to an intermediate, and using -r as an output option (after the -i). As mentioned in the other thread , this is essentially the same thing vegas is doing on the fly when you drop the clips onto a 29.97 timeline. If you have smart resample off, they will be duplicate frames, if you have resmaple on, blended frames. Most people prefer duplicates, because the ghosty look of blends, but go ahead and experiment to see what you like better. The benefit of the intermediate method if you use lossless I-frame, is it will be more stable in vegas. The negatives are lots of HDD space required, takes longer than stream copying, vegas doesn't "treat" lossless YUV intermediates with studio RGB, but rather computer RGB, so you might get superbright/dark level clipping. (The exception is magicyuv with the full range option, but magicyuv is VFW only, unfortunately there is no ffmpeg component)

In my experience , 40 clips of this type should be ok. But vegas can get "squirrelly" and prone to crashes with these types of clips if you have a larger or more complex project. So I would try the copy method first. If you have problems, only then would I use the other method using -r as an output option and adding frames with a re-encoded intermediate to 29.97 before importing

No no, you did good explaining it. Just needed a bit more clarification and I got it now. Still a lot of complexities I do not understand, but I know what I need to do with these clips.

In Vegas I always ALWAYS turn off resample. I always have issues with glitches when the default smart resample is enabled. I always notice a more crisp picture in the preview window when I turn resample off.

I know what I'm going to do now. Thank you very much for all your help.