Removing repeated frames from 25fps files on Vevo (originally 23.976)?

[digitalfreaknyc]

The situation:

Vevo has the highest video encoding quality currently on the net.
If you download ANY video from their site (as opposed to their complimentary youtube channel), it's 25fps. That results in each video playing back with a stutter every few seconds. The original files are almost always 23.976 so it's duplicating a frame to offset that change.
I have absolutely no idea WHY they do this but...they do. If you download it from their youtube channel, it plays back properly.

I've emailed them and they said it would be forwarded to their tech team. I don't see them doing anything about it.

Is there a program that will scan through each file and remove the duplicated frames? Obviously I know that the resulting file would have to be re-encoded but even re-encoding at a higher bitrate will be better than the original bitrate from youtube. Or...any other ideas?

[poisondeathray]

The situation:

Vevo has the highest video encoding quality currently on the net.
If you download ANY video from their site (as opposed to their complimentary youtube channel), it's 25fps. That results in each video playing back with a stutter every few seconds. The original files are almost always 23.976 so it's duplicating a frame to offset that change.
I have absolutely no idea WHY they do this but...they do. If you download it from their youtube channel, it plays back properly.

I've emailed them and they said it would be forwarded to their tech team. I don't see them doing anything about it.

Is there a program that will scan through each file and remove the duplicated frames? Obviously I know that the resulting file would have to be re-encoded but even re-encoding at a higher bitrate will be better than the original bitrate from youtube. Or...any other ideas?

Yes that's stupid of Vevo , but they've been doing that for a few years, they probably won't change. Think of all the ruined videos that they'd have to "fix".

Anyways, the answer is avisynth

TDecimate(cycler=1,cycle=25)

[digitalfreaknyc]

The situation:

Vevo has the highest video encoding quality currently on the net.
If you download ANY video from their site (as opposed to their complimentary youtube channel), it's 25fps. That results in each video playing back with a stutter every few seconds. The original files are almost always 23.976 so it's duplicating a frame to offset that change.
I have absolutely no idea WHY they do this but...they do. If you download it from their youtube channel, it plays back properly.

I've emailed them and they said it would be forwarded to their tech team. I don't see them doing anything about it.

Is there a program that will scan through each file and remove the duplicated frames? Obviously I know that the resulting file would have to be re-encoded but even re-encoding at a higher bitrate will be better than the original bitrate from youtube. Or...any other ideas?

Yes that's stupid of Vevo , but they've been doing that for a few years, they probably won't change. Think of all the ruined videos that they'd have to "fix".

Anyways, the answer is avisynth

TDecimate(cycler=1,cycle=25)

The problem is that I know NOTHING about avisynth. I'm really not very good without a GUI. Any other suggestions?

[poisondeathray]

You can copy & paste that into some GUI's that allow you to edit scripts. e.g. MeGUI, xvid4psp, ripbot, many others. Basically you copy & paste it after the 1st line . About the only popular one that you can't use it with easily is handbrake/vidcoder

I think the majority of Vevo videos are still 25p, but you might get an odd one that isn't. So double check before you blindly apply to all videos. But I agree, definitely the per-frame picture quality is higher than other sites like YT, if you pull the highest quality one from Vevo. Be careful, because sometimes they have 2 1080p versions, one slightly better, slightly higher bitrate

[digitalfreaknyc]

You can copy & paste that into some GUI's that allow you to edit scripts. e.g. MeGUI, xvid4psp, ripbot, many others. Basically you copy & paste it after the 1st line . About the only popular one that you can't use it with easily is handbrake/vidcoder

I think the majority of Vevo videos are still 25p, but you might get an odd one that isn't. So double check before you blindly apply to all videos. But I agree, definitely the per-frame picture quality is higher than other sites like YT, if you pull the highest quality one from Vevo. Be careful, because sometimes they have 2 1080p versions, one slightly better, slightly higher bitrate

I tried it on Ripbot and was told "Script error - there is no function called tdecimate."

[poisondeathray]

Hmm, I haven't used ripbot in a few years, but it used to come with all the basic avisynth plugins like tivtc.dll . Check the plugins folder. I can't recall if it uses a "portable" avisynth install or if you need a system avisynth install

If it doesn't come with it, you need to load the plugin manually. You would add a line like

LoadPlugin("PATH\tivtc.dll")

Where PATH, is the actual path. It can be any folder, where you put the .dll.

LoadPlugin("C:\Program Files (x86)\AviSynth\plugins\tivtc.dll")

If you put it in the installed version of avisynth in the avisynth\plugins folder it will autolaod without that line

[digitalfreaknyc]

Hmm, I haven't used ripbot in a few years, but it used to come with all the basic avisynth plugins like tivtc.dll . Check the plugins folder. I can't recall if it uses a "portable" avisynth install or if you need a system avisynth install

If it doesn't come with it, you need to load the plugin manually. You would add a line like

LoadPlugin("PATH\tivtc.dll")

Where PATH, is the actual path. It can be any folder, where you put the .dll.

LoadPlugin("C:\Program Files (x86)\AviSynth\plugins\tivtc.dll")

If you put it in the installed version of avisynth in the avisynth\plugins folder it will autolaod without that line

ok I added that line and then added the TDECIMATE after it.

Now it says "Script error: Invalid arguments to function TDECIMATE"

[poisondeathray]

ok I added that line and then added the TDECIMATE after it.

Now it says "Script error: Invalid arguments to function TDECIMATE"

Post your full script.

The video has to be loaded before you use TDecimate (There will be a line like FFVideoSource, or DirectShowSource(some path and your video name) , so put the TDecimate line after that line)

ie. just copy & paste it at the end. You generally have to save the script, then reload it for the changes to take effect

[digitalfreaknyc]

Here it is:

#MT


#VideoSource

video=DirectShowSource("A:\Lifehouse\Vevo\You And Me - Lifehouse - Vevo.TS",audio=false).ConvertToYV12()

#Deinterlace



#Decimate


#Crop



#Resize



#Colors



#Denoise




#Subtitles



#AudioSource
Import("C:\Temp\RipBot264temp\job1\job1_a1.avs")


#Triming



#AVSameLength



#ColorSpace

Just an FYI, there is an option for "decimate" in the program but it's greyed out. I can't change anything.

[digitalfreaknyc]

OK so it looks like I got it to work. I put the Tdecimate at the end and it seems to have worked. However, here are a few other questions:

1) The resulting file size is smaller than the original file. Is there a way to "up" the bitrate? If I have to re-encode, I prefer to go higher than lower.

2) The resulting frame rate is 23.020??? What's that about?

[poisondeathray]

Here it is:

#MT


#VideoSource

video=DirectShowSource("A:\Lifehouse\Vevo\You And Me - Lifehouse - Vevo.TS",audio=false).ConvertToYV12()

#Deinterlace



#Decimate


#Crop



#Resize



#Colors



#Denoise




#Subtitles



#AudioSource
Import("C:\Temp\RipBot264temp\job1\job1_a1.avs")


#Triming



#AVSameLength



#ColorSpace

Just an FYI, there is an option for "decimate" in the program but it's greyed out. I can't change anything.

Add a period to the directshowsource line, then TDecimate. It will look like this (the changes are in red)

video=DirectShowSource("A:\Lifehouse\Vevo\You And Me - Lifehouse - Vevo.TS",audio=false).ConvertToYV12().TDecimate(cycler=1,cycle=25)

Then save and reopen

I wouldn't use avisynth to process the audio, because you would have to re-encode the audio. I would just stream copy the audio from the original, that way you don't use audio quality. I can't recall if ripbot had a "copy" option for audio. If not, you can just use some GUI like mymp4boxgui for mp4, or mkvtoolnix for mkv, or tsmuxer for TS to add the original audio

DirectShowSource() actually might give you slightly "off" results, because it's not necessarily frame accurate. Give it a try first, but I would use FFVideoSource with seekmode=0, threads=1.

video=FFVideoSource("A:\Lifehouse\Vevo\You And Me - Lifehouse - Vevo.TS",threads=1, seekmode=0).TDecimate(cycler=1,cycle=25)

I know scripts can be difficult at first, I'm trying to think of an easier way. For example avidemux has a decimate filter that might work, you would set it to 25 (so 1 in 25). I think vdub has one too. For certain the FFVideosource/Tdecimate method works, I use it all the time for Vevo videos.

There is a way to do it with ffmpeg as well, and that will streamcopy the audio automatically ,but it's not always as accurate as the FFMS2 with threads=1, seekmode=0 method. When using threads=1, seekmode=0, it becomes very slow but that's the cost of accuracy

Actually the most accurate, most consistent way is to use DGNVtools, but it's not free and requires a compatible Nvidia card. It's the most accurate source fiilter for avisynth

[poisondeathray]

OK so it looks like I got it to work. I put the Tdecimate at the end and it seems to have worked. However, here are a few other questions:

1) The resulting file size is smaller than the original file. Is there a way to "up" the bitrate? If I have to re-encode, I prefer to go higher than lower.

2) The resulting frame rate is 23.020??? What's that about?

Ooops I missed this.

1) You can use a lower CRF value, or higher bitrate. For example if it's set to CRF "20", set it to somthing like 16 and the filesize will be larger than using "20". Or if using 2pass rate control enter a value that you want

2) Probably your DirectShowSource source filter is messing things up. Sometimes it can be buggy and it depends on the system installed codecs and splitters. Thus it can be inconsistent. If it's done an accurate job you should have 24.0 exactly, so I suspect you have errors. See my earlier post about FFVideoSource() and threads=1, seekmode=0

[poisondeathray]

Actually the ffmpeg method seems to work ok. Give me a few minutes and I'll put together some simple instructions where you double click a .bat file and a folder of TS files will be converted in batch

[poisondeathray]

It's easier to set an environment variable for your ffmpeg path, but for now, just follow these instructions.

1) Download ffmpeg.exe and place a copy in the folder of Vevo .TS files to be converted
2) Unzip and put this batch file in the same directory and double click (I've attached it in a zip, so unzip it, "batch.bat")

The batch file is actually this code below, you can modify it with notepad. I've set the output to re-wrapped into MP4 container and in the same directory as the input file, steam copying the audio, but you can modify those parameters. It's actually faster/better idea to use different physical drives. MP4 container has about 5-7% less overhead than TS, and is more compatible with devices etc.. but if you really wanted to you can use TS (larger filesize for no benefit). Each output file will take on the same name as the input file. I've set the CRF to 16, using the default "medium" x264 preset. If you have any questions about changing parameters etc.. just ask

To modify batch files, you can use notepad, then save

for %%a in ("*.ts") do ffmpeg -i "%%a" -c:v libx264 -crf 16 -vf decimate=cycle=25 -c:a copy -bsf:a aac_adtstoasc "%%~na.mp4"
pause

[flashandpan007]

Does that also work for 59.940 fps files which are converted to 25 fps by VEVO? I guess not.

In the past I contacted VEVO, to not do a frame rate conversion to 25p. A guy told me it is caused by "technical" reasons. I asked further but didn't got an answer. I personally download the youtube version which is is fine for my needs. I am happy with the quality, just look at the filesize. And I don't have any stuttery issues. I started to download Videos 8 years ago which I sorted in a good way. I won't do that much work to determine if the file should be 23.976, 24.000, true 25p, 29.970, 30.000p, 50p or 59.940p.

[digitalfreaknyc]

Actually the ffmpeg method seems to work ok. Give me a few minutes and I'll put together some simple instructions where you double click a .bat file and a folder of TS files will be converted in batch

It looks like it worked fine with Ripbot. The resulting file is just slightly over 24fps. Is that correct?

Also, how does ripbot know which frames to get rid of?

Thanks again for all of your help!

[manono]

Also, how does ripbot know which frames to get rid of?

It doesn't. TDecimate does. It removes the 1 frame in every 25 that looks most like a duplicate.

The resulting file is just slightly over 24fps. Is that correct?

If the source was 25fps then, no, it's not accurate. However, depending on how long the video is there may not have been enough frames to zero in on the real 24fps. I wouldn't worry about it.

[digitalfreaknyc]

It's amazing how much this board has helped me in the past...and how I constantly have to go back to threads that I have posted in to re-trace my steps.

I'm still having problems with Vevo. was trying to capture some Billy Joel videos and they're all over the place and look like crap. I know that the earlier method worked great on the last set of videos. For some reason, though, these aren't working. Any advice?

[poisondeathray]

I'm still having problems with Vevo. was trying to capture some Billy Joel videos and they're all over the place and look like crap. I know that the earlier method worked great on the last set of videos. For some reason, though, these aren't working. Any advice?

Post a vevo link

Could be a source issue (maybe it wasn't "filmed", or shot at a different rate) , or maybe some other processing error by vevo or production company. Or could be a decoding or matching issue

[digitalfreaknyc]

Post a vevo link

Could be a source issue (maybe it wasn't "filmed", or shot at a different rate) , or maybe some other processing error by vevo or production company. Or could be a decoding or matching issue

http://www.vevo.com/watch/billy-joel/All-About-Soul/USSM20402900

That's the first. I tried some other videos and i'm having the same issues. So it could be "user error" at this point. I wouldn't be surprised if it were me. But, then again, I had no problems with the other videos that I did.

Now, I will say that Billy Joel is Columbia so these would have been encoded more recently than the other stuff I was working with, since Columbia did their deal much more recently.

[poisondeathray]

Post a vevo link

Could be a source issue (maybe it wasn't "filmed", or shot at a different rate) , or maybe some other processing error by vevo or production company. Or could be a decoding or matching issue

http://www.vevo.com/watch/billy-joel/All-About-Soul/USSM20402900

That's the first. I tried some other videos and i'm having the same issues. So it could be "user error" at this point. I wouldn't be surprised if it were me. But, then again, I had no problems with the other videos that I did.

Now, I will say that Billy Joel is Columbia so these would have been encoded more recently than the other stuff I was working with, since Columbia did their deal much more recently.

This one has dropped frames in the source. I'm assuming all versions are the same, I check the 1080p one (which is a poorly upscaled one)

[jagabo]

The billy joel video is 30p.

[digitalfreaknyc]

Post a vevo link

Could be a source issue (maybe it wasn't "filmed", or shot at a different rate) , or maybe some other processing error by vevo or production company. Or could be a decoding or matching issue

http://www.vevo.com/watch/billy-joel/All-About-Soul/USSM20402900

That's the first. I tried some other videos and i'm having the same issues. So it could be "user error" at this point. I wouldn't be surprised if it were me. But, then again, I had no problems with the other videos that I did.

Now, I will say that Billy Joel is Columbia so these would have been encoded more recently than the other stuff I was working with, since Columbia did their deal much more recently.

This one has dropped frames in the source. I'm assuming all versions are the same, I check the 1080p one (which is a poorly upscaled one)

I just noticed something: it's not on the "official videos" page for Billy. Does this mean that any average user could upload whatever they want?

Also, what's the difference between "official videos" and "extras" and "all videos?"

The billy joel video is 30p.

Not if you look at the "source" from Vevo. It's 25fps

[poisondeathray]

What he means is the source uploaded before vevo was "30p" . ie. it wasn't "filmed" at 24p. It's not a filmed music video, it's a live performance shot on video. Vevo butchered it when converting to 25p by dropping frames

[digitalfreaknyc]

What he means is the source uploaded before vevo was "30p" . ie. it wasn't "filmed" at 24p. It's not a filmed music video, it's a live performance shot on video. Vevo butchered it when converting to 25p by dropping frames

Well, like I said, it might not be Vevo. It's not listed under the "official videos" page. What's the difference?

[poisondeathray]

What he means is the source uploaded before vevo was "30p" . ie. it wasn't "filmed" at 24p. It's not a filmed music video, it's a live performance shot on video. Vevo butchered it when converting to 25p by dropping frames

Well, like I said, it might not be Vevo. It's not listed under the "official videos" page. What's the difference?

Yeah it is vevo's fault. The other youtube version is "30p" ie. 29.97

6093 frames vs. 7302 frames . Both 4min, same cut

[digitalfreaknyc]

What he means is the source uploaded before vevo was "30p" . ie. it wasn't "filmed" at 24p. It's not a filmed music video, it's a live performance shot on video. Vevo butchered it when converting to 25p by dropping frames

Well, like I said, it might not be Vevo. It's not listed under the "official videos" page. What's the difference?

Yeah it is vevo's fault. The other youtube version is "30p" ie. 29.97

6093 frames vs. 7302 frames . Both 4min, same cut

Yes but aren't most music videos released at that frame rate? Unless they're newer?

[jagabo]

The file I downloaded was 30p.

[digitalfreaknyc]

The file I downloaded was 30p.

From youtube or from vevo?

[jagabo]

The file I downloaded was 30p.

From youtube or from vevo?

From Vevo:

http://h264-aws.vevo.com/v3/h264/2016/06/USSM20402900/634f4e79-1352-44d7-aadc-1539aa8a...00_aac_128.mp4