Avisynth - Open DPX files

[procedure]

hi.
i am restoring a 16mm 2k scan DPX sequence.
as usual, avisynth seems like its the best thing out there. i tried it out for cleaning the white dirt that covers the negative and its a miracle worker.

to test it, i converted a short shot from a dpx image sequence to uncompressed AVI, then ran it through avisynth.

the problem is that i can't get the dpx file sequence to load into avisynth. i don't want to convert the whole film to AVI if i can help it. its a 17min A B roll negative so its a bit too long to convert to AVI (1080p is about 8-10TB).

supposedly this is the plugin to load dpx image sequences into avisynth "ImageMagick Reader" (immaavs) :
http://forum.doom9.net/showthread.php?t=135928

but i just can't get the plugin to load (i get an "avisynth open failure" that pops up in VirtualDub.).
Cannot load file "plugins/immaavs/immaavs.dll"

i copied the "immaavs" files into my plugin directory.
i copied the "ImageMagick" dlls and xmls into my windows/system32 folder.
i restarted the computer.

i also tried avisynth+ 32bit and 64bit and also tried AvsPMod instead of VirtualDub. no luck. similar errors.

has anyone gotten this working?

thanks.

[poisondeathray]

I've never had much luck with the avisynth version of imagemagick

ffms2 in avisynth can open single dpx images - so not very useful . I supposed you can splice them together, but that's not going to be very fun unless the shot is very short

vapoursynth can use either ffms2 with python to load dpx image sequences, or the imagemagick plugin (the vpy version works), both in the proper 10 bit depth as well . But it has fewer native plugins compared to avisynth , although it can load some native avisynth plugins too.

Another way is to use avfs to mount a "virtual" avi from the vpy script, which then can be loaded into avisynth if the filter is not supported natively in vapoursynth (basically frameserve from vapoursynth to avisynth)

Your decision no what to use might also depend on which filters you are using (some are are 8bit only, or some only work in YUV or subsampled YV12, but DPX will usually be 10bit RGB).

[jagabo]

VirtualDub Filtermod can load a sequentially numbered sequence of DPX images. You could then use its frame server to serve to an AviSynth script.

[procedure]

just checked out vapoursynth. looks good, but it doesn't have "removeDirt" which is the plugin i really need.

great idea about going from vapoursynth to avisynth. i actually tried avfs but it would not work. i get a command line error with "pfm register c:avfs.dll" (i can't remember the error now but i got mad and uninstalled Pismo Mount already). any ideas why it would not register?

i was wondering if there would be a problem with 10bit RGB. i didn't get a great 16mm transfer, and i have tried converting a dpx file to 8bit tif in photoshop and cannot detect any difference (even with extreme color correction applied). so i have no problem with converting all my dpx files to tif's. if only i could get imagemagick to work!

great comments. thank you! any more ideas?!

[poisondeathray]

just checked out vapoursynth. looks good, but it doesn't have "removeDirt" which is the plugin i really need.

It does... avisynth has about 40 different versions and variants of "removedirt", but vapoursynth has a few variants too

One here
https://forum.doom9.org/showthread.php?t=169771

great idea about going from vapoursynth to avisynth. i actually tried avfs but it would not work. i get a command line error with "pfm register c:avfs.dll" (i can't remember the error now but i got mad and uninstalled Pismo Mount already). any ideas why it would not register?

You actually don't need pismo mount, if you're using vpy only. It's included in avfs.exe with the vapoursynth install

But if you are using pismo mount, you'd need update to the newest one (there was a recent release a few days ago)

i was wondering if there would be a problem with 10bit RGB. i didn't get a great 16mm transfer, and i have tried converting a dpx file to 8bit tif in photoshop and cannot detect any difference (even with extreme color correction applied). so i have no problem with converting all my dpx files to tif's. if only i could get imagemagick to work!

jagabo has a great idea - vdfm frameserve. It's 8bit RGB but since you said that was ok, you're ready to go

[procedure]

just tried the VirtualDub Filtermod idea from jagabo. another great idea.
i'm able to start the frame server from Filtermod but i can't open the avi frame server file it creates. neither VLC or virtualDub will open it. according to VLC, it can't read its format. so it is creating an invalid avi file. any ideas on how to make it work?
the largest avi file is resolution is 1920x1080 so i tried converting a few 2k DPX files to 1920x1080 8bit tifs. but it sill serves an invalid avi.
thank you.

[poisondeathray]

You need to install the vdub frameserver client, and there are special steps if you're using x64 version of vdub/filtermod and avisynth x64

https://github.com/stax76/staxrip/issues/159#issuecomment-299569068
https://forum.doom9.org/showthread.php?p=1819655#post1819655

After you follow those steps , and the frameserver is active, the vdr file should be "openable" by AVISource("video.vdr") (or whatever the path/name is)

[procedure]

hi. yes i just noticed it said there was no client for the frameserver. this is working now after using the "auxsetup.exe" that came with VirtualDub FilterMod! thanks for the tips.

ok. i'll play with vdfm frameservering to avisynth or using vapoursynth instead (haven't tried it yet).

i also tried vdfm frameserving into Adobe Premiere CC but it wouldn't open. could that be possible somehow? do i have to switch over to all x64 programs (vdfm & avisynth) since premiere is x64?

[poisondeathray]

vdub/vdfm frameserver doesn't work with Adobe CC, even x64 doesn't work . It' s not a true virtual file emulator

AVFS does work with everything that accepts "AVI", regardless of 32bit or 64bit because it's a "virtual" AVI. (Both avisynth avfs, and vapoursynth avfs work in Adobe and other programs) , but some programs might require a certain colorspace or pixel format to work (e.g. some might need RGB32)

[procedure]

sorry i'm now seeing further instructions about using "proxyon.reg" from vdfm to help make the frameserver behave like a real avi file.

this is not working currently (don't have permission or the system is currently using certain reg keys). but i'll keep plugging away.
thanks!

[poisondeathray]

Did you get the basic vdr signpost opened in avisynth yet ? either 32bit or 64bit ?

[procedure]

yes, i can open my signpost file (for some reason i named it avi instead of vdr) in avisynth. super awesome (using 32bit versions)

[poisondeathray]

Either avi or vdr should work with AVISource() in avisynth . But neither are true virtual emulated files. For example if you use mediainfo, it won't identify it as uncompressed video. But avfs generated AVI looks like a real "AVI". Every program that supports AVI will open it. Mediainfo will "see" it as a uncompressed video file.

Another benefit of vapoursynth avfs is it supports 10bit422 (v210) and 10bit RGB (r210) , so you can frameserve into Davinci Resolve or Adobe and keep 10bit

I have pismo mount installed, and to load avs with avfs , you can just right click the avs, and in the context menu and "PFM Open" and it will mount the avs. Make sure you're using the newest version. The AVI will appear in c:\volumes and a subfolder . You can call avfs from the commandline too.

avfs script.avs

vpy avfs is loaded with a commandline only , and doesn't rely on externally installed pismo mount (but you can have it installed like I do if you want) if you use the one bundled with vapoursynth. The executable is different than the default avisynth avfs one

avfs script.vpy

[procedure]

I look forward to trying that with avfs again. thank you for all your help.

in the meantime it looks like I basically have destroyed my windows 10 installation on this project. like a total noob I tried to get the "proxyon.reg" from vdfm to work. it could not make changes to my registry since it did not have permission (i'm guessing this is a windows 10 anti-malware (or anti-idiot!) defense or something). in any case, I followed some advice on-line and changed the permission on HKEY_CLASSES_ROOT using regedit and now my computer is basically useless. I put the permission back like it was after I realized it didn't make proxyon.reg work, but now I don't have permission to do anything on my computer. Every directory is locked. I can't use the windows menu or go into settings and can't restore to a restore point either (using my emergency disk probably since I don't have permission to use any directories or programs on the computer now).

[shekh]

Forget proxyon.reg, it will not help. The avi will still be not a real file and incompatible with most programs.

[procedure]

thanks, shekh.
i wish i had never tried proxon.reg! it was my fault but i ended up having to restore an older windows backup after messing up my registry. i'm finally back to a semi-normal state again.

however not completely! after having to re-install avisynth, my original avs (which was working perfectly) is now throwing an error. My DepanEstimate.dll plugin is now saying it can't load fftw3.dll. jeez.

i found someone on this forum with a link to the fftw3.dll (ftp://ftp.fftw.org/pub/fftw/fftw3win32mingw.zip). but when i add it to windows/system32 it doesn't matter. i still get the avisynth error.

any idea why things might have changed and how i can fix it? i suppose i can replace the plugin with another one that works but i'm would prefer not to go testing other plugins when the current avs was working so nicely.

thanks.

[shekh]

If your system is 64bit you should put 32bit ddls to windows\syswow64 folder.

[procedure]

hi. yes, i put it in both system32 and syswow64. it still didn't help. i did a system restart in case as well.

[procedure]

Since it appears i cannot get an avisynth script frameserved to adobe premiere i'm looking into vapoursynth.

i tried not to ask this question but i just can't find the answer anywhere. where do i find the imagemajick plugin for download? there are instructions on how to use it but no place i can find to download it.

this seems to be the main thread:
https://forum.doom9.org/showthread.php?t=170981
which refers to a download link but i can't find it unless its the imwri test8 link (https://www.dropbox.com/s/h7pn4o5ysjhk0vc/imwri_test8.7z?dl=1) is the final link.

thanks.

[sneaker]

It is the correct link. Nothing newer (unless you compile yourself).

[poisondeathray]

why can't you get avisynth frameserved into premiere ? Can you get avisynth working ? or is that still the fftw3.dll load error ?

Or can't avfs working ?

Post the error messages verbatim - because they have precise clues as to what the problem is

[procedure]

thanks for asking. maybe i am still missing something.

the problem is that my source files are dpx. i can't get the imageMajick plugin working in avisynth (or vapoursynth darn it!). so far the only way i can load dpx files is using vdfm.

this is the part that i'm unsure about. it appears i can only use avisynth filters if i then frameserve it from vdfm to avisynth. so to get it in adobe premiere i'd have to frameserve it again from avisynth using avfs but i couldn't get that to work. so i have gotten avfs to work with your help. however i cannot get it to frameserve a file that is already been frameserved.

am i off on this?

thank you!

[poisondeathray]

Yes, that should work. It might be a bit "slow" because of the overhead, but it works here

vdfm to "read" dpx sequence => vdfm frameserver => avisynth (AVISource to read signpost) + whatever scripts/filters like remove dirt etc... => avfs => Adobe

Do you have the independent parts working ? vdfm frameserver works by itself? avisynth works by itself ? avfs works by itself ?

[raffriff42]

Using AvisynthPlus & FFmpegSource

FFImageSource("BBB_01_04_dpx_linear.dpx")

## next 2 lines only for this test image
BicubicResize(Width/4, Height/4) 
Info

## next line only for display on vdub
ConvertBits(8, dither=0).ConvertToRGB32

[Attachment 43316 - Click to enlarge]

[poisondeathray]

@rr42 - he needs an image "sequence" . Do you have a workaround in avisynth+ besides splicing ? In vpy you can use ffms2 + python to load a sequence, but not in avisynth

[raffriff42]

Okay sorry, not reading the whole thread, again. Your suggestion sounds like it will work.

[procedure]

ahhrrr! i setup everything again on a second computer (laptop) and and now it works!

vdfm to "read" dpx sequence => vdfm frameserver => avisynth (AVISource to read signpost) + whatever scripts/filters like remove dirt etc... => avfs => Adobe Premiere

this is not working on my desktop. not sure where the problem lies. i thought i was a halfway intelligent person but the persistence necessary to figure this stuff out is daunting. i don't have my desktop at the moment (which must be the computer working since this is a large video project that needs a lot of memory and cpu power). i had every individual step working except i could not open the final avi created by the last step. when i tried to open it in VLC it just throws an error saying it is not a proper video file. i was able use avfs to create a working avi from an avs. but not if it was frameserved by vdfm. tomorrow i'll try it again on my desktop and see if it was a fluke. i thought, oh well, frame serving twice must not work.

thanks very much for all your help.

[procedure]

Well I thought we had a breakthrough but...
Avisynth is reading the frame served vdr file from vdfm when the avs is very simple such as a one line script:

AviSource("VD-FilterMod-Frame-Server.vdr")

but as soon as i use my complicated film restoration package of plugins in a longer avs, both VirtualDub and AvsPmod crash when i preview the script.

[poisondeathray]

what kind of "crash" ? e.g. lockup , blue screen, etc...

Is there an error message ?

How "complicated" is the script - if there are many filters/plugins I would run x64 avisynth+/vdfm/everything to rule out memory issues . Most plugins have x64 versions now

If you run avsmeter it may reveal more info

[procedure]

for the crash, i get the standard windows 10 pop-up saying the program is not responding. no details. it takes a about 5 seconds to happen.

the script i'm using is the "film restoring" script found here: https://forum.doom9.org/showthread.php?t=144271 (thanks VideoFred!)

i tried the avsmeter tool and found that it gets stuck (i can't remember exactly how it behaved but see below):

---------

i did a little more digging. i was testing my dpx sequence with a tiny 4 frame sequence. i realized that vdmf was only showing 2 frames in the previewed sequence. so i created a new sequence that was 24 frames long. now vdmf is only showing a sequence of 14 frames. the first 9 or 10 frames are correct, and frames 11-14 are frames toward the end of the original 24 frame sequence.

so i tried a 24 frame jpg sequence and also a 24 frame tif sequence. the jpg and tif sequences are previewing correctly in vdmf. also, they work fine with my "film restoring" script applied in avisynth and are working fine frameserved a second time by avfs.

et voila, the problem seems to stem with using dpx in vdmf. i think i can live with converting my dpx files to tif if it comes to that. unless you have another suggestion maybe i should try vapourSynth again. as stated though, i so far unable to get imageMajick installed properly so no dpx files there either!

----------

i tried to autoload imageMajick (using this directory: C:\Program Files (x86)\VapourSynth\plugins64\libimwri.dll)
with this code:

import vapoursynth as vs
core = vs.get_core()
video = core.imwri.Read('D:/test.%06d.dpx')
video.set_output()

i get this error line 3

Failed to evaluate the script: Python exception: No attribute with the name imwri exists. Did you mistype a plugin namespace?

if i load imageMajick manually:

import vapoursynth as vs
core = vs.get_core()
core.std.LoadPlugin(path='C:/Program Files (x86)/VapourSynth/myPlugins64/libimwri.dll')
video = core.imwri.Read('D:/test.%06d.dpx')
video.set_output()

i get the same error line 4

programming python looks difficult, so i'm not sure what i'm getting into here with seting up plugins to restore my film even if i get dpx files to load.
thank you very much!