Plugin to interpolate dropouts within a frame?

[brassplyer]

Looking for something that will do interpolation of dropout areas within a specific frame when there aren't clean adjacent frames to use ReplaceFramesMC()

Thanks.

[poisondeathray]

There are no good approaches or shortcuts in that case. If there are no good clean directly adjacent (temporal) areas , you can only look at spatial interpolation , textures within the same frame

Some techniques like inpainting (there are a few in avisynth using masks) might reduce the amount of work you need to manually do , but it's probably better / easier to do a good spatial repair in photoshop

So the answer is painting or photoshop (clone brush/stamp, healing tool etc...) if you want it looking half decent

If there are distant adjacent "good" textures (not directly adjacent), then you can still mix & match areas with masks in a compositing tool . One way to speed that approach is use motion tracking techniques to "cover" over the bad areas borrowing textures from good areas many frames away . Obviously these techniques have problems when the content is unique (eg. faces) or motions are complex

[brassplyer]

There are no good approaches or shortcuts in that case. If there are no good clean directly adjacent (temporal) areas , you can only look at spatial interpolation , textures within the same frame

Some techniques like inpainting (there are a few in avisynth using masks) might reduce the amount of work you need to manually do , but it's probably better / easier to do a good spatial repair in photoshop

So the answer is painting or photoshop (clone brush/stamp, healing tool etc...) if you want it looking half decent

How do you export an avi frame into Photoshop - and then back into the video? I have Paintshop Pro, hopefully the same thing can be done there.

[poisondeathray]

How do you export an avi frame into Photoshop - and then back into the video? I have Paintshop Pro, hopefully the same thing can be done there.

In certain versions of photoshop (photoshop extended), you can work directly on video. Same with after effects (it's like photoshop for video, with compositing tools, masks, motion tracking)

Otherwise, you can use avspmod to export frames for use in other programs (imo the easiest, it's like an editor for avisynth), or vdub, or ImageWriter() in avisynth

You can use ImageSource() with Trim() to add it back in

[jagabo]

For that particular frame some thing like this would work pretty well:

https://forum.videohelp.com/threads/360352-Video-editing-doubling-part-of-the-video?p=2...=1#post2287054

The more detail and motion you have the worse it will look.

[brassplyer]

Otherwise, you can use avspmod to export frames for use in other programs (imo the easiest, it's like an editor for avisynth), or vdub, or ImageWriter() in avisynth

You can use ImageSource() with Trim() to add it back in

I have avspmod installed and can get video to load but it's not clear how you extract a particular frame from it to fix in photo software and then put it back in.

[poisondeathray]

Otherwise, you can use avspmod to export frames for use in other programs (imo the easiest, it's like an editor for avisynth), or vdub, or ImageWriter() in avisynth

You can use ImageSource() with Trim() to add it back in

I have avspmod installed and can get video to load but it's not clear how you extract a particular frame from it to fix in photo software and then put it back in.

Move the slider to navigate to the frame you want

Video=> save image as
(use a lossless format like bmp or png)

I would specify the correct matrix (709 vs 601) and interlace for the conversion to RGB for the screenshot at the end of the script
e.g for SD interlaced , Rec601 is default
ConvertToRGB(interlaced=true)


You can set up a keyboard shortcut in the => configure shortcuts if you want

[brassplyer]

I have avspmod installed and can get video to load but it's not clear how you extract a particular frame from it to fix in photo software and then put it back in.

Move the slider to navigate to the frame you want

Video=> save image as
(use a lossless format like bmp or png)

And to insert the treated frame back into a specific place within the video?

[poisondeathray]

And to insert the treated frame back into a specific place within the video?

e.g. Let's say you want to replace frame 666 (oooohhhh nice number )

I'm assuming you're using NTSC 29.97 (30000/1001 FPS)
Trim(0,-1) will return 1 frame from the image sequence . The AssumeFPS is to ensure the joined segments have the same frame rate (otherwise you cannot join)

orig=AVISource()
replaced=ImageSource("fixed.png").trim(0,-1).AssumeFPS(30000,1001).ConvertToYV12()

orig.trim(0,665) ++ replaced ++ orig.trim(667,0)

You probably have to add a ConvertToYV12() or ConvertToYUY2() to the replaced= line , depending on what the original AVI color space was, because most image editors will work in RGB/A (and a PNG, BMP or TIFF will be in RGB/A)

There are other plugins and methods to replace frame that might be simpler if you have many to replace

Another approach is to use a video editor or NLE . Even a free one like aviutl will work since it has layers

[brassplyer]

And to insert the treated frame back into a specific place within the video?

e.g. Let's say you want to replace frame 666 (oooohhhh nice number )

I'm assuming you're using NTSC 29.97 (30000/1001 FPS)
Trim(0,-1) will return 1 frame from the image sequence . The AssumeFPS is to ensure the joined segments have the same frame rate (otherwise you cannot join)

orig=AVISource()
replaced=ImageSource("fixed.png").trim(0,-1).AssumeFPS(30000,1001).ConvertToYV12()

orig.trim(0,665) ++ replaced ++ orig.trim(667,0)

I'm using a 47 frame sequence, trying to replace frame 40. It's not cooperating, here's what I'm getting. The file is 59.94 progressive, I get the same thing when I use AssumeFPS(60000,1001) - what's the 1001 indicate?

I gather the first orig.trim() command includes from 0 up to the frame preceding the fixed frame and the second one is the frame following the fixed frame to the end of the file or previous to the next fixed frame, correct? You can specify replacement of more than one frame, yes?

[poisondeathray]

59.94 is a 2-decimal approximation. The real framerate is 60000/1001 . Add AssumeFPS(60000,1001) after AVISource and for replaced

What format is your AVI, what colorspace ?

I gather the first orig.trim() command includes from 0 up to the frame preceding the fixed frame and the second one is the frame following the fixed frame to the end of the file or previous to the next fixed frame, correct? You can specify replacement of more than one frame, yes?

Yes, but then you have to change the ImageSource() arguments for an image sequence of the correct number of frames

[brassplyer]

59.94 is a 2-decimal approximation. The real framerate is 60000/1001 . Add AssumeFPS(60000,1001) after AVISource and for replaced

What format is your AVI, what colorspace ?

Huffyuv RGB.

I gather the first orig.trim() command includes from 0 up to the frame preceding the fixed frame and the second one is the frame following the fixed frame to the end of the file or previous to the next fixed frame, correct? You can specify replacement of more than one frame, yes?

Yes, but then you have to change the ImageSource() arguments for an image sequence of the correct number of frames

Can you clarify this? In this case, I'm just replacing one frame to see if I can make it work. It's not really a sequence per se.

[poisondeathray]

Huffyuv RGB.

If it's RGB then remove the ConvertToYV12() from replaced . You might have to use ConvertToRGB32 or ConvertToRGB24

AVISource()
Info()

That will tell you what the source is (RGB32 or RGB24) , if it has alpha channel (probably a "dummy") or not . They just have to match

Can you clarify this? In this case, I'm just replacing one frame to see if I can make it work. It's not really a sequence per se.

Yes , you just insert a sequence using the ImageSource syntax

e.g

lets say frames 40-45 are replaced with these in photoshop or image editor

img040.png
img041.png
.
.

sprintf syntax is used for wildcards

ImageSource("img%03d.png", start=40, end=45)

The "%03d" is the number of place holder digits . "%04d" would be 4 digits e.g. 0001.png

So it would look like
orig.trim(0,39) ++ replaced ++ orig.trim(46,0)

[brassplyer]

59.94 is a 2-decimal approximation. The real framerate is 60000/1001 . Add AssumeFPS(60000,1001) after AVISource and for replaced

That info script shows it being 60002/1001, changing it to that made it work.

[poisondeathray]

59.94 is a 2-decimal approximation. The real framerate is 60000/1001 . Add AssumeFPS(60000,1001) after AVISource and for replaced

That info script shows it being 60002/1001, changing it to that made it work.

Then something got messed up earlier in your workflow. I would change it to the real NTSC rate (or bobbed frame rate), especially if you're going to reinterlace and for DVD. Standards exist for a reason. The "off" framerates can cause problems

orig=AVISource().AssumeFPS(60000,1001) 
replaced=ImageSource("fixed.png").trim(0,-1).AssumeFPS(60000,1001)

orig.trim(0,39) ++ replaced ++ orig.trim(41,0)

[brassplyer]

Yes , you just insert a sequence using the ImageSource syntax

e.g

lets say frames 40-45 are replaced with these in photoshop or image editor

img040.png
img041.png
.
.

sprintf syntax is used for wildcards

ImageSource("img%03d.png", start=40, end=45)

The "%03d" is the number of place holder digits . "%04d" would be 4 digits e.g. 0001.png

So it would look like
orig.trim(0,39) ++ replaced ++ orig.trim(46,0)

Looking at the ImageSource command, I'm not clear how that wildcard syntax works. I took 4 frames 35 - 38 and marked them with a frame number in Paintshop to determine whether they were being loaded in the right place and named them 35.png, 36.png, 37.png, 38.png.

Doing it this way obviously isn't the answer:



If I do it like this I only get frame 35 and not the other 3.



How do I specify to load files 35.png, 36.png, 37.png, 38.png ?

[poisondeathray]

The 1st one didn't work because your files don't have img000.png as shown in the example . The "img" prefix is missing. Obviously you figured that out

Remove the Trim(0,-1) from "replaced" . Remember that only returns the 1st frame used in the single image case

[brassplyer]

The 1st one didn't work because your files don't have img000.png as shown in the example . The "img" prefix is missing. Obviously you figured that out

Remove the Trim(0,-1) from "replaced" . Remember that only returns the 1st frame used in the single image case

Thanks I'll give it a try.

Now what if I want to do different sequences of frames in different parts of the video? This doesn't work - it only executes whatever the last sequence referenced is.


orig=AVISource("F:\fix frame 40 test.avi")
replaced=ImageSource("%02d.png", start=20, end=22).AssumeFPS(60002,1001).ConvertToRGB32
orig.trim(0,19) ++ replaced ++ orig.trim(23,0)
replaced=ImageSource("%02d.png", start=35, end=38).AssumeFPS(60002,1001).ConvertToRGB32
orig.trim(0,34) ++ replaced ++ orig.trim(39,0)

What's the avisynth plugin that reads my mind and autowrites the script to do what I want it to do?

[poisondeathray]

The 1st one didn't work because your files don't have img000.png as shown in the example . The "img" prefix is missing. Obviously you figured that out

Remove the Trim(0,-1) from "replaced" . Remember that only returns the 1st frame used in the single image case

Thanks I'll give it a try.

Now what if I want to do different sequences of frames in different parts of the video? This doesn't work - it only executes whatever the last sequence referenced is.


orig=AVISource("F:\fix frame 40 test.avi")
replaced=ImageSource("%02d.png", start=20, end=22).AssumeFPS(60002,1001).ConvertToRGB32
orig.trim(0,19) ++ replaced ++ orig.trim(23,0)
replaced=ImageSource("%02d.png", start=35, end=38).AssumeFPS(60002,1001).ConvertToRGB32
orig.trim(0,34) ++ replaced ++ orig.trim(39,0)

What's the avisynth plugin that reads my mind and autowrites the script to do what I want it to do?

Give them different names

orig=AVISource("F:\fix frame 40 test.avi").AssumeFPS(60000,1001)
replaced1=ImageSource("%02d.png", start=20, end=22).AssumeFPS(60000,1001).ConvertToRGB32()
replaced2=ImageSource("%02d.png", start=35, end=38).AssumeFPS(60000,1001).ConvertToRGB32()

orig.trim(0,19) ++ replaced1 ++ orig.trim(23,34) ++ replaced2 ++ orig.trim(39,0)

When you have more complex or multiple sequences and layers, it might be easier to use a video editor, where you can graphically see what is going on in layers

[jagabo]

Make it a function called with parameters:

function ReplaceFramesWithImages(clip vid, int first, int last, string basename)
{
  replacements=ImageSource("basename"+"%02d.png", start=first, end=last, pixel_type="YV12)
  Trim(vid,0,first)+replacements+Trim(vid,last+1,0)
}

Then call it with

WhateverSource()
ReplaceFramesWithImages(last, 20, 22, "")
ReplaceFramesWithImages(last, 35, 38, "")

That's off the top of my head so you'll need to debug.

[poisondeathray]

You can actually use RemapFrames, ReplaceFramesSimple , or ClipClop or other frame replacement functions, because ImageSource() will insert blank frames where the frames are missing in the sequence from start= to end= . But you might need a 'real' 1st frame to get the image sequence to start reading

Trim ++ gets tedious after several entries

[brassplyer]

When you have more complex or multiple sequences and layers, it might be easier to use a video editor, where you can graphically see what is going on in layers

I have Vegas Pro 8 - you mean just trim out the replaced frames and stick in the image files?

Not sure what you mean by layers. You mean something like images from clip laid over another for a composite effect? In this case I'm just dealing with a single video file, linearly fixing frames.

I *really* appreciate all the input from those who have responded to this and in the related threads.

[poisondeathray]

When you have more complex or multiple sequences and layers, it might be easier to use a video editor, where you can graphically see what is going on in layers

I have Vegas Pro 8 - you mean just trim out the replaced frames and stick in the image files?

Not sure what you mean by layers. You mean something like images from clip laid over another for a composite effect? In this case I'm just dealing with a single video file, linearly fixing frames.

I *really* appreciate all the input from those who have responded to this and in the related threads.

Yes, you just "cover up" the video with replacement frames . Also benefit of doing it in an editor, is you have masks that you can more easily use to "cover up" parts of a frame . But if there are only a few edits, or if they are complete frames, it's simple enough in avisynth

This is what replaceframessimple syntax might look like . You need a "real" frame 0 to get ImageSource to start reading the sequence

orig=AVISource().AssumeFPS(60000,1001)
replace=ImageSource("%02d.png", start=0, end=whatever).AssumeFPS(60000,1001).ConvertToRGB32()


rfs(orig, replace, mappings="[20 22] [35 38]")

If you have many mappings, you can use a text file to list the replacement frames

[jagabo]

I debugged it. This works with names like "pic00022.png":

# assuming YV12 video
function ReplaceFramesWithImages(clip vid, int first, int last, string basename)
{
  replacements = ImageSource(basename+"%05d.png", start=first, end=last).ConvertToYV12().AssumeFPS(vid.FrameRateNumerator, vid.FrameRateDenominator)
  Trim(vid,0,first-1)+replacements+Trim(vid,last+1,0)
}

AviSource("video.avi")
ReplaceFramesWithImages(last, 22, 24, "pic")