Frame issues with Vegas 10 pro

[Jon.G]

Ok, so I'm not a video guy, but I am a technical guy, mostly with audio. So dealing with silent film has me a bit out of my element.

So I had these reels of super 8 film, and took them to a place that provided me with 2 sets of files, prores, and uncompressed, both 720p. The prores files were made to play at the correct speed at 24fps from the 18fps source, so it doubled every 3rd frame to make a second have 24 frames instead of 18 and still run at the same speed. The uncompressed is frame for frame, but 24fps, so plays too fast, that's more for archival purposes, the prores files are what I'm dealing with to try and make dvd and bluray.

So I went through these files, and it seems like most of the files have one error in them. The error will be a point where I see a flash. When I go back to the flash and check it frame by frame, I find for some reason something has been inserted for 3 or 4 frames from earlier in the film, sometimes a few frames of leader, sometimes a few frames of an earlier part of the film. I went and checked the uncompressed files, the errors exist on there as well. This was quite disappointing, and I plan on calling the place I had the conversion done on Monday to try and find out just what in the heck happened and how this was able to pass by quality control on most of the files, this to me makes me question their abilities.

But, it seemed to me that if I edited out these 3 to 4 frames, the film ran smoothly without them. Sometimes the frame on either end of the blip were doubled, so it seems that it wasn't something missing, just somehow that got inserted by error due to some glitch in their system. So I edited out all the bad frames, and all was looking well. Great.

Well, I then rendered 2 different versions of this, one for dvd, and one for bluray. The one I rendered for bluray came out just fine it seems, although I have to check each point. But, the one for dvd, not so much. I guess dvd doesn't do a 24fps, while bluray does so I was able to render it to 24fps for bluray. But the dvd rendering had to do some frame stuff to get it to yet another frame rate. And in the process of doing this, the program decided to use these cut frames in the process. So now the bad frames I've edited out, have been blended into my good frames when doing it's frame rate conversion.

I have to ask, what in the heck is wrong with Vegas? To me, this just seems like bad programming to have the program use data that has been removed from the project. Could I have been any more clear to the program that I did not want that data used in any way, shape, or form?

What do I need to do to render this without those frames being used?

[Jon.G]

Update: I've found that at least one of these errors has also carried over to the bluray rendered file. Although it seems most haven't. But still, I find this disturbing that vegas pro 10 will use data I told it not to!

[aedipuss]

when you removed the frames in the trimmer did you make subclips of the good parts and use the subclips in the timeline? if not the removed frames are just hidden and sometime reappear at inopportune times.

on the 24fps to dvd thing. dvd can use 23.976fps progressive as input. you'll probably need to change the render from 29.97 interlaced 23.976p with 3:2 pulldown. (ntsc film) it may work better than a 29.97i render.

[Jon.G]

I just used the S key to split at either end of the bad parts, and the delete key to get rid of it, and slid everything over to fill in the gap. I'll have to look into how to make subclips. Ok, I right click on the clip, there's an option to create subclip, is this all I need to do?

I'll have to try that for dvd. Thanks for the tip.

[Cornucopia]

Well, first, you should deal with a BAD transfer by either getting your money back or by having them do it right this time. There's no excuse for that. Also, Pro-res is of little use to you unless you had a Mac. They should be offering you a framerate and a fileformat that you can work with right away.

Next, it's better to understand what to expect:
Super8mm film is usually 18fps (though there can be some variation). Theatrical 35mm film runs a 24fps. Video runs at either 25fps (in PAL areas of the world) or at 29.97fps (in NTSC areas of the world). Your avatar doesn't say, so I don't correctly know where you are, but for expediency's sake, I'm going to assume you are in the US (NTSC)...

DVD only stores MPEG/VOB material as 23.976 (aka 24fps slightly slowed down) with Pulldown flags to make it seem 29.97, or at 29.97 (with NO pulldown flags).

If your material had been Theatrical film, you could have just taken the 24fps Uncompressed files and encoded them to 23.976fps MPEG2 for DVD (or h.264 for BD) with pulldown flags (aka "soft" Telecine). Then an Author and Burn to disc would have gotten you what you needed.

Since you've got 18fps, one of the best ways to get from Here (18) to There (29.97) is to use DGPulldown. You can then encode MPEG2 at 18fps directly (but fps notwithstanding, otherwise all according to DVD spec) and then Apply an 18 -> 29.97 adjustment through DGPulldown and it'll modify the MPEG2 with additional flags such that while it is stored at 18fps (and not normally DVD compliant) the presence of the flags will make it acceptable to DVD authoring apps and it will play correctly and at the right speed.
Unfortunately, there is no similar app for h.264, so you may need to use a different app (I suggest AVISynth frameserved to x264) to hardcode those pulled-down frames.

You could use Vegas to do that part, but it's frame blending/interpolation is just mediocre. That's why I mentioned AVISynth+x264. However, the problem you are experiencing with Vegas is only partly the fault of Vegas; the rest being the fault of the bad original pulldown, and the bad editing. Any time you intend to work with something with duplicated/interpolated frames you are asking for trouble. One should always get back to the original unduplicated frames, which in this case would be 18fps, before doing most editing. Only a VERY FEW apps can correctly untangle and work with the mess that Telecine & Duplicated frames causes.

BTW, yes, the 29.97 is slightly slowed down from 30fps, but not by enough to worry about motion changes, and since as you say it is SILENT, you don't even need to worry about doing a TimeCompress/PitchShift on the audio.

However, are you sure it's silent? Many Super8mm films have sound.

HTH,
Scott

[aedipuss]

should work, then it can't reference missing parts.

[Jon.G]

Wow....lots of information that is over my head there. Yes, I need to contact them and find out what went wrong, but it didn't go wrong with the frame conversions, since the errors are on the uncompressed files that are frame by frame. And yes, I'm quite sure they're silent, we never had a sound super 8 camera.

I thank you for that detailed write up. But, I wouldn't know where to begin with those instructions. I guess I'd have to ask for step by step instructions, or a link to a specific guide. Are you suggesting I should start with the uncompressed and not bother using the prores? If so, then I'd need to know how to take that file, which plays back the 18fps movie fast at 24fps since it is frame by frame, and step by step turn it into files that play correctly that I would use to author bluray and dvd.

But before I do that, I need to address things with the transfer place, and see if they need to redo the job, or if it's just inserted data that editing out will leave me with a perfect transfer. I suppose I could get the viewer out, and compare the uncompressed frame by frame to the film frame by frame, and see if any frames are missing at those points.

[Cornucopia]

Part of the problem with the uncompressed footage is that the 18fps is playing at 24fps, so of course it's going to play faster. You would need an app that can read the header (the part that tells at what fps to run) and change it and it alone to 18fps (ReStream could do this with MPEG2, AVIFrate with AVI, etc.). Then you could play it at 18fps, and it would have unduplicated frames, giving you a decent master with which to convert to how you need.

However, that's still THEIR problem, as they should have given you that to begin with...

And yes, check their output frame by frame if you need to.

If there are other steps needed down the road, we can guide.

Scott

[budwzr]

I have to ask, what in the heck is wrong with Vegas? To me, this just seems like bad programming to have the program use data that has been removed from the project. Could I have been any more clear to the program that I did not want that data used in any way, shape, or form?

Geez, what are the chances that your video service did a lousy job, AND you also discovered Vegas has bad programming? Boy, yer a genius!

[Jon.G]

Well, I did the suggestion of rendering to 23.976p with 3:2 pulldown for dvd, and now I have a file that dvd-lab pro 2 can't open, so that doesn't work. There's a waste of rendering time.

[Jon.G]

Also, as an update to those keeping track of my video woes, I have found out that things have not been accurately described to me. I was told that the prores were speed corrected (which they are), and the uncompressed are not, for archival purposes. But, when I cued up the uncompressed to do a frame by frame comparison to determine if there are missing frames where these errors are, I found that the uncompressed are also speed corrected, with each 3rd frame doubled to bring it up to 24. Another point I will be bringing up with the film transfer place on Monday. I"m hoping this issue came into play when the frame rate stuff was done, and that they have an unfooled around with version of these still (they told me they keep my files for 10 days in case there are problems), so that I don't have to deal with dropping off my films with them again. Hopefully they will be good at clearing up these issues, I'll keep you all posted.

[Cornucopia]

If that's true, then you'll HAVE to drop your films back off with them, as those files won't do much good. It would be a way lot more effort trying to FIX them than to just re-scan the film correctly. And if they don't know how (or don't have the righ equipment) to do Super8mm and 18fps directly to 18fps digital files, they should have told you that up front (so you could go to a different conversion house). Hell, I know people on this site who can do a better job. I would myself, but I don't have the right equipment.

Just so that we are all on the same page, what OS are you running? What kind of playback equipment? What is your intended output?...

Scott

[aedipuss]

not sure what's wrong at your end but dvdlab pro 2.51 opens and uses vegas pro10 23.97w/3:2 pulldown renders just fine here.

[Jon.G]

Well, they did tell me they can do 18fps files, although that they would be 24fps not speed corrected....what they told me and what they gave me are 2 different things, so I'm thinking there's a chance they do have frame by frame files from the job on their system.

I'm running windows 7, in the US, so I'm going to need to do this to NTSC, and am intending to make dvd and bluray discs of these materials. They are just old home movies taken on a less than professional camera, so the source isn't exactly top of the line, but I was hoping to get these preserved to have a digital master that was done right, and would leave me with a digital source that would make me not feel a need to go back to the original film again, unless I wanted to view them on a projector for the heck of it. Most places I find talk about going to dvd, sometimes bluray, not so much raw digital master. This place offered 720p HD, and uncompressed files, it sounded like this place knew what they are doing and what they're talking about. I hope this was just some isolated mishap and that it will be corrected quickly. Anyways, until I talk to them Monday, there's not much more to do here, I really shouldn't bother with these files and fixing them anymore and should just await a proper resolution to these errors.

[Jon.G]

Ok, so first, I figured out the uncompressed files are what they told me they are. I'm new to using this software, so I didn't realize not having the framerate set to match the file I'm pulling in will cause the preview to add frames and such. I had done so much to the session by this point, I didn't know what I was looking at anymore, so I started a new session, with correct frame rate, pulled in an uncompressed file, and it's frame by frame. So, I suppose I could revisit the header modification stuff and all that jazz when I get back to it.

Meanwhile, I've contacted the transfer place, and it sounded to me like they see this problem at times, occasionally, perhaps a few times a year. Lucky me I guess. I got the feeling from the guy that their raw files won't have the errors on them and they'll need to export a new set of files for me, but he's going to check.

Also, earlier I mentioned problems with the 23.976 render not working in dvd lab pro. Well, it did work with dvd architect. But, at this point, I don't even know if I started that session right, through discussion here and playing around, I've been learning about all these things. So, I suppose it's good that these files weren't ready, this gave me a trial run so that now I know what I'm doing once I get the real files.

Thanks for all the help here. I think I now know enough of what I'm doing for the purposes of this project, and should be on my way to having error free files to make a great dvd and bluray.

[Jon.G]

Ok, here's the update:

They determined the error doesn't exist on their raw scans, only on the 2 sets of files, and they will have a new set of files ready for me this afternoon. I will be taking a trip in today to get the files loaded onto my external hard drive.

So now that some of you have brought up interesting tips, and have suggested doing things starting with the uncompressed files rather than the speed corrected prores files, I want to see what the recommended procedure is for best results. At this point, I have figured out enough to be able to use the 24fps prores files to lay them out into vegas pro 10, render to 2 different .mpgs at 23.976p for dvd and bluray, and author 2 discs with dvd architect, a bluray and a dvd.

I'm not sure why I had problems with my experimenting with the rendering to 23.976p 3:2 pulldown and using that in dvd lab pro 2, but since others have gotten that to work I probably could, but instead of playing around with that I decided, may as well use dvd architect since I'm using that for the bluray, and that worked.

So basically, I know how I can make these discs using the prores files.

However, I'm curious if one thinks I should use the uncompressed instead, if this would make a difference in quality of the final product.

The codec I am provided for playback of the uncompressed files is Blackmagic_Desktop_Video_Windows_8.2. As I've stated before, the files I am provided with play at 24fps, but are sourced from the 18fps films, so they play fast. It was mentioned I would need to do something to adjust the header to set it to play at 18fps, then use software to convert the frame rate.

So, I know Cornucopia has provided me with lots of useful information, but it's kind of been overwhelming. More than I'm familiar with. But, the gist I get is that it is felt I should start with the uncompressed. I suppose I may need to convert it to another uncompressed format, since the recommendations for adjusting frame rate were using programs for either mpg or avi. Also, using dgpulldown and avisynth was mentioned, programs I'm not familiar with. And something about flags and stored as 18fps on dvd but set to play at 29....

So, I dunno, is there a step by step guide to do this, starting with my blackmagic files and ending with dvd and bluray? I'm not asking someone to type out everything I need to do, but if a basic procedure could be listed that could point me in the right direction, I am capable of reading documentation and figuring certain things out, and I know probably enough has already been stated in this thread that I should be able to figure it all out, but I am still feeling overwhelmed and like I could use a bit more of a point in the right direction to be prepared.

Or is this worth bothering with? Will there really be a noticeable difference in doing all this with the uncompressed over using the prores files?

[Cornucopia]

I thought I mentioned it to you before, but ProRes is a Quicktime-Only codec and you might have problems with it. See my past post:

ProRes 422 Encoder only comes with FCP/FCS (mac only $$), or with DVS Clipster (turnkey only $$$$), or with Telestream's encoders (Vantage/Episode/etc - Mac & PC $$$), or using hardware encoders such as AJA or others ($$$).

Decoder now comes included free with QT as of version 7.6.6 or higher (PC & Mac).

ProRes 4444 is still a problem on the PC (no decoder only, just those costly options mentioned above.

Others could also give you good help here (perhaps better than me), but here's my take on it...

You take your source files - which should be truly 18fps now - and you create a modified Vegas timeline template that is 18fps.
THEN, load you files into the session. Edit them as you would for a nice, finished master.
THEN, (for DVD) Export/Render as 18fps MPEG2 (you'll probably have to create a custom render template modified from a standard DVD one, for this, too) - all other specs should be DVD compliant.
THEN, in DGpulldown, set the source FPS as 18 (if it doesn't already recognize it) and the target fps as 29.97/NTSC; it'll create a copy that - for all further purposes - looks to be a standard pulled-down 29.97 NTSC MPEG2. This should now be DVD-compliant.
THEN, author in the app of your choice (it ought to accept that modified MPEG2 file now!!??). Burn as a DVD. ENJOY

For BD, unless others on this site (or Doom9) know of a similar tool for h.264, or unless you want to fall back on encoding as MPEG2 (in which case you would do a similar thing with DGPullown), you will have to Export/Render your EDITED 18fps master to a nice, LARGE HD lossless codec.
THEN, import that file into a Standard, BD-compliant timeline template. Drop it onto the timeline.
THEN, export/render that to a standard BD-compliant h.264 file (like 1080p24), accepting whatever pulldown choices are given you by Vegas.
Then, author & burn your BD/AVCHD disc and Enjoy.

BTW, do you already have a Windows Blackmagic decoder? If not, that company is (kind of) obliged to give you a copy (Blackmagic supplies ALL their hardware customers with a codec that is specifically to be given out under these circumstances).

HTH,

Scott