convert video for project after composite

[christopher_2007]

hello everybody,

i have a, for me big, problem and I really hope, you guys can help me!

Some video files of mine are around 200GB (per file) in one project. I work with this orginal files in order to cut them, compose, render, ... and when the project is finished, I whant to archive the howl project!
But the orginal video files are way to big in order to keep all of them in the original file!

so I thought, that I maybe convert the videos and then just replace the orginal video files in the project with the converted ones. but only changing the source, so the cutting, the effects, the compose, ... is remaining and will not be loose!

but when I convert the video with the same fps rate, the exact same length and resolution (but an other container and bit rate), the video is a little off in the project. the video line is some millisecounds moved to one direction, I can tell this, because the audio is the original one in the comp and video+audio are not sync anymore ... and because the only thing i changed is the video file, it has to be the problem!

so what can I do? I really hope, you can help me!
I attached two files to this post, both are video file reports and the names are self-explanatory, I hope!
(For converting I simply used after effects ... but what settings should I use? bit-rate, frame-rate, container, compress, ...)

greetings,
Chriss

PS.: Sorry for my bad english, I hope, you can understand me

[Cornucopia]

You are talking about proxies. Except you are doing it in reverse. Most people do proxy editing this way:
1. Get real large source files.
2. Make smaller rez, lower quality, smaller size proxy files - named the same as source files (but in a different folder).
3. Swap the folders' files so the source files are somewhere else and the proxies are where the source files would be expected.
4. Load in the proxies to your editor.
5. Edit/Compose your program using the proxies.
6. Once the "composition" (aka session/timeline) is saved, exit the app, then swap the files BACK to their original folder places.
7. Restart the app, and it should either "assume" the real files are the proxies and use them instead, or you might have to run a "re-link" process to get them recognized.
8. Once that is done, you just export your finished, high-rez, high-quality product.

The problem with this is that when it is finished, you are STILL supposed to archive the FULL LARGE-SIZED files! Not the proxies. And that is where you want to cut corners. Plus, you've already edited with your full-sized files, and seemingly un-linked your audio from your video, so you can't really go back to using proxies at this stage, unless you were to go back to square-one and recreate the WHOLE project using the proxies from the very start. Plus, proxies that are encoded correctly don't have any sync issues themselves. That's a separate encoding problem you are having as well.

The way I see it, right now your only viable choice is the suck it up and foot the bill for a large enough storage to handle the full footage. And understand that THIS is how high-quality production works all the time - using of lots and lots of space. But it is only "way too big" from your newbie perspective. Think of them as normal, very large files that need to be stored and store them.

Scott

[christopher_2007]

hello^^

thanks for the answer in detail! there are a lot of new information for me to find!

but at the end, there is still this one question: there must be a way to switch to video-files which are mostly the same, without having slitly postponed video lines! ... i mean, it is mostly the same, if i switch one billion pictures from tif to jpg ... at the end, the resolution and the number are the same! but the format is an other and so the howl project is smaller.

So why are my video lines postponed? how can I fix it, because sometimes I have to render a video a secound time and when this video already is in an other project, I do not whant to change all project aboph! With which settings is this possible?

greetings,
Chriss

[poisondeathray]

Don't use AAC audio for intermediates. Many encoders pad ~40-60ms at the beginning

Use uncompressed audio

[christopher_2007]

this would be a possibility, but no solution, because video+audio in the compressed video file are ok, not postponed at all!

but the full video WITH the audio is postponed in the composition! I can tell this, because I cut the video exactly (with the original files), rendered it and everything was ok. but then I put the new compressed files in the folder, and at the edges of the cuts, some more frames of something, I cut away, appeared! so there must be a problem with the container, or at least with the video

greetings

[christopher_2007]

how can it be, that no one can help me with this problem? it is not possible that the switching of videofiles changing something in the edeting and the cut! greetings

[Cornucopia]

Look, we don't know how you arrived at your "originals" to begin with. You are using UTvideo lossless compression for the video codec. That's ALWAYS difficult to edit/composite using multiple video tracks if your system isn't already a powerhouse, heavy-duty system. Particularly if it's HD video (as yours is).
Plus you have the complicating factor that your original FPS is 30, while you are in Germany (a PAL-system country), so it should NATIVELY be 25 FPS. When you converted, your resulting video was 25Fps (contradicting your original statement that you didn't change the fps), but you don't say how you did the FRC (in AE?), so there's a good chance you have the de-synchronization going on there. There are many better ways of doing 30->25fps FRC, depending on what you want to accomplish (AVISynth giving probably the most high-quality options).
Add to that the fact that both your source and converted audio files have 2 streams each and they're ALL compressed. Plenty of opportunity for synch problems there.

However, since your originals are already not optimal, it would be better to re-create them optimally FIRST rather than to convert them (to something 1 generation removed from sub-optimal).

We can't help you if you don't help us. More info, more info, and more info.

Scott

[poisondeathray]

this would be a possibility, but no solution, because video+audio in the compressed video file are ok, not postponed at all!

but the full video WITH the audio is postponed in the composition! I can tell this, because I cut the video exactly (with the original files), rendered it and everything was ok. but then I put the new compressed files in the folder, and at the edges of the cuts, some more frames of something, I cut away, appeared! so there must be a problem with the container, or at least with the video

greetings

Now you have at least 2 problems

The 1st is more than a possibility. Yes, you started with MP3 in AVI container, but AAC in MP4 typically adds a padding muxing delay (at least many AAC encoders and MP4 muxers do). Each clip will be offset ~40ms . I repeat: it's terrible to use as an intermediate. Use search, this has been discussed before. If you don't believe me , test it yourself, zoom in to the timeline




The solution is to use the same container and audio, or better yet uncompressed audio. ie. AVI + copy the MP3 , or AVI and uncompressed pcm wave . The problem is you are probably using AVC with b-frames . This can cause many problems in AVI container - mixed up frame order, green or black frames, b-frame decoder lag. If you encode without b-frames that should solve some of the issues , but reduce compression effciency

Most people do the opposite to what you are doing ; they often encode their compressed videos to a lossless I-frame intermedate with uncompressed audio to avoid the problems you're getting

[christopher_2007]

hi,

I am very sorry for my short textes with so less information! ... my days are short and i am only around two or three hours per day at home, so i really apologise for my misbehavior! But now I am sitting in the train and so I have more then enouph time to write some more details

we wrote a program at my university for physic simulations. We then record the graphic card output stream in order to get video material. For that we use a programm calles "DxTory" ( https://www.videohelp.com/tools/Dxtory ). With that programm you can record most grafic card output streams, mostly for games, but also for all kind of other applications and programs.

the simulations are very resouces overusing, and so we are only able to record with 30fps 1920x1080p. (I guess, that the 30fps is not static, because it can be the case, that the grafic card produces less than 30 frames/secound, but I am not quite sure!)
More information about the container and the settings are in the "original.txt" from my first post.
(If you know a better container and better settings for the future, let me know! I am open for new knollage!)

Then we compose the video material in Adobe Premiere and Adobe After Effects, for quick previews we then use Adobe brige and for rendering the Adobe Media Encoder, all of the CS6 Suite.

We have no problems by using the original files, so we never thought of converting the material befor the cutting and edeting.

Now we have several projects and the problem is, that the memory size is very big and so we thought, that we maybe converte the files to a smaller container. In that case, the projects were still useable, the only thing would be, that the video material is bad, but we already exportet the videos and only whant to archiev the projects now. The only thing we might need in the future are some effects, or composite settings, but not the "real" exportable video material.

But still: we want to have a full intakt project WITHOUT andy shift of video and audio, that's the main problem I came here for!

In the meanwile, I rendered my original files in NTFC and not in PAL ... still the problem.
Also, I rendered without audio compression and also without audio at all ... still the problem.

The main problem could be descripted as following:
When I replace the orginal video files in the premiere project with a smaller export of the vieo (same length, same fps, other bit rate, same audio), the orginal cut/eccects/compositions/... should be remaining.
So far, it is the case. But the video has a slitly shift in the timeline! and so, all the cuts are on wrong places, and the audio/effects/pictures does not match with the video anymore. The rest is all on the right position, only the video line is shifted!

(I should have mentioned, that the orginal record has only one video line, but 4 audio lines.
We extracted the audio lines, and seperatly imported the orginal container (with 1 video and 4 audios), deleted the 4 audios and imported the 4 audio lines, we first extracted. So the only thing we are using of the original video container, is the video itselfe.
The audios are in own files and are NOT konverted! they are still the same as in the beginning!)

I am open for all new solutions and will try everything possible, you mention.
So I really hope, you can help me!

greetings,
Christopher

PS.: Sorry for the many mistakes ...

[smrpix]

Can you get back to your original timeline before you replaced the footage with the low resolution proxies?

You're probably better off just making a compressed version of your finished show. Do keep your originals if there's any way you can, because anything else will be lower quality forever.

[christopher_2007]

i have everything backed up, so it is no problem to rescue the project, if I make a mistake. and a lower quality forever is no problem! really, that is exactly, what i whant!
but i whant it without a postponed video line

greetings

[christopher_2007]

does someone have an other idea?

[smrpix]

Christopher,

Maybe if you can post a screengrab of your timeline. At this point we don't know what your actual original files were, what your project is, how long the show is, how many edits, how you are planning to view it, what it's for, etc. Technical problems don't exist without context, nor do solutions.

Unfortunately, you're not giving us very much to go on.

Do you have anyone local with more video experience you can call who can actually take a hands-on look? It sounds like you may have compounded issues.

[Cornucopia]

For it to really be done correctly, I would suggest you RE-EXPORT from your physics program at 1920x1080 @ 25FPS (or 50 Fields per second), so that you won't need to FrameRate Convert at all. DxTory should be able to do that (haven't tried it, but the screenshot certainly shows that it's possible)
You also make sure your video codec is a VISUALLY LOSSLESS, but LOSSY codec, like Cineform or DNxHD. There are free installers for those available if you don't already have them installed.
You also make sure your audio codec is UNCOMPRESSED (aka LPCM WAV).

Then, you re-edit your program all over again. This time, it shouldn't bog down (because your V.LL.b.L. codec as mentioned above is better with your available resources) and you can directly export without needing to do proxies first, and your DIRECT archive of your originals will now be more easy to manage (much smaller originals) so you want need to lose the originals nor make a further lossy version just to keep something.

Sure would help if you had given us MediaInfo info and detailed steps & apps listing like I original asked for...

<edit>Shoot, in the time you've been asking for this stuff, you could have already redone the thing!

Scott

[christopher_2007]

i attached two text files with the media info export to my original thread, and i told you all programms, i am using, so I don't know, what more infos you need and redo the thing is not so easy, all video projects have a total length around 30 hours or something, just the original recorded length! but thanks so far, i will try some steps and hope, that it will help. maybe you have an new idea till then. But one last question: why is it so important to record in 25 fps? i mean, if my project, my sequence and my original video are 30fps, it shouldn't be a problem, or do I misunderstand something? greetings

[poisondeathray]

Did you try an intermediate in the same container (AVI) ? , and without B-frames ?

Are the files interpreted as 30.0 FPS, not 29.97 ? Double check in premiere

[Cornucopia]

i attached two text files with the media info export to my original thread, and i told you all programms, i am using, so I don't know, what more infos you need and redo the thing is not so easy, all video projects have a total length around 30 hours or something, just the original recorded length! but thanks so far, i will try some steps and hope, that it will help. maybe you have an new idea till then. But one last question: why is it so important to record in 25 fps? i mean, if my project, my sequence and my original video are 30fps, it shouldn't be a problem, or do I misunderstand something? greetings

As I explained earlier, you live in Germany, which is a PAL country. That means the best target framerate should be the same that every video device in your country natively runs at (for compatibility & smoothness), which is 25 FPS. Otherwise, you will be continually swimming upstream.
You gave us 2 media info files and you mentioned AfterEffects and Dxtory, but not the original app, nor any of the session settings you used or a clear description of step-by-step actions.
Without some of these things being cleared up, there is little more we could suggest. And there does seem to be a little bit of a language barrier.

Scott