This article will give you an idea of what I'm trying to do:
https://batchloaf.wordpress.com/2017/02/12/a-simple-way-to-read-and-write-audio-and-vi...-part-2-video/
I'm trying to read video frames into a bitmap and write the bitmap out to a Prores file. I've got it working with mp4 on a Linux vm but am having trouble writing the frames out to Prores.
Opening an input pipe works fine:
The problem seems to lie in the output pipe:Code:FILE *pipein = popen("ffmpeg -i KodakChart.MP4 -f image2pipe -vcodec rawvideo -pix_fmt rgb24 -", "r");
The above code give me a playable Prores file with audio, but merely re-encodes the video from the original file and bypasses the bitmap.Code:FILE *pipeout = popen("ffmpeg -y -i KodakChart.MP4 -c:v prores -profile:v 3 -c:a copy output.mov", "w");
Here is the code that writes the bitmap named "frame":
I have tried the following without success:Code:fwrite(frame, 1, (H*W*3)/2, pipeout);
Code://FILE *pipeout = popen("ffmpeg -y -f rawvideo -vcodec rawvideo -c:v prores -profile:v 3 -c:a copy output.mov", "w");Any help is appreciated. Thank you.Code://FILE *pipeout = popen("ffmpeg -y -f rawvideo -vcodec rawvideo -pix_fmt yuv422p -c:v prores -profile:v 3 -c:a copy output.mov", "w");
Try StreamFab Downloader and download from Netflix, Amazon, Youtube! Or Try DVDFab and copy Blu-rays! or rip iTunes movies!
+ Reply to Thread
Results 1 to 30 of 91
Thread
-
-
Why would you want to do this ? What are you trying to do exactly ?
There is potential for problems with pixel format conversions; ie. whatever the source is , 8bit RGB bitmap , 10bit422 YUV prores ... Matrix issues, rounding errors
Why is it necessary to read from the physical bmp for this? Are you doing something to the bmp sequence in another program? why not read from pipe before the bmp and specify multiple outputs (eg if you wanted to include the rgb conversion before the prores write)? You could use filter_complex with 2 splits for example -
Why is it necessary to read from the physical bmp for this? Are you doing something to the bmp sequence in another program?
Ouch! Forgot about the 8- to 10-bit aspect of it. -
If you're performing operations in other programs, then there is a timing issue here as well; You can't access a bmp in the other program until it's written out, and presumably you don't want to read the bmp sequence until it's been modified in the other program (and the modified version written out by the other program)
So you might as well just read a bmp sequence separately later (assuming you don't care about the pixel format , bit depth, matrix conversions) . ie. just separate the 2 stages -
I'm modifying the pixels elsewhere in the same program.
-
I know this doesn't address your original question but it seems much easier
-
For the output to prores pipe, you have -i KodakChart.MP4 , which means take the original video as input
Usually stdin is just -
so
Code:-i -
Code:ffmpeg -f rawvideo -s 1920x1080 -pix_fmt rgb24 -r 24 -i - .....
-
Here is the complete code. It is strictly experimental and may not even work. It disregards the 8-bit 10-bit difference.
The idea is to clip off all luminance pixels > 235 to 235.Code:if (lum[y*W+x] > 235) lum[y*W+x] = 235;
Code:// Video processing example using FFmpeg // Written by Ted Burke – last updated 12-2-2017 // Now works in YUV space // Outputs to prores // To compile: gcc Ted2.c -o Ted2 // Note: make sure .MP4 file extension matches case #include <stdio.h> // Video resolution #define W 1280 #define H 720 // Allocate a buffer to store one frame unsigned char frame[((H)*(W)*3)/2]; int main(void) { int x, y, count; // Create a pointer for each component's chunk within the frame // Note that the size of the Y chunk is W*H, but the size of both // the U and V chunks is (W/2)*(H/2). i.e. the resolution is halved // in the vertical and horizontal directions for U and V. unsigned char *lum;//, *u, *v; unsigned char *u, *v; lum = frame; u = frame + H*W; v = u + (H*W/4); FILE *pipein = popen("ffmpeg -i KodakChart.MP4 -f image2pipe -vcodec rawvideo -pix_fmt rgb24 -", "r"); FILE *pipeout = popen("ffmpeg -y -f rawvideo -s 1280x720 -pix_fmt rgb24 -vcodec rawvideo -r 59.94 -i KodakChart.MP4 -c:v prores -profile:v 3 -c:a copy output.mov", "w"); // Process video frames while(1) { // Read a frame from the input pipe into the buffer // Note that the full frame size (in bytes) for yuv420p // is (W*H*3)/2. i.e. 1.5 bytes per pixel. This is due // to the U and V components being stored at lower resolution. count = fread(frame, 1, (H*W*3)/2, pipein); // If we didn’t get a frame of video, we’re probably at the end if (count != (H*W*3)/2) break; // Process this frame for (y=0 ; y<H ; y++) { for (x=0 ; x<W ; x++) { if (lum[y*W+x] > 235) lum[y*W+x] = 235; } } // Write this frame to the output pipe fwrite(frame, 1, (H*W*3)/2, pipeout); } // Flush and close input and output pipes fflush(pipein); pclose(pipein); fflush(pipeout); pclose(pipeout); }
-
Not sure about linux, but for windows piping, you'd also have to map the rawvideo pipe, and map which stream the audio is coming from
Another way to clip would be to use -vf lutrgb or lutyuv . The filter also works at >8 bit depths now -
Another way to clip would be to use -vf lutrgb or lutyuv . The filter also works at >8 bit depths now
I've tried that and itdoesn't work very well. It doesn't hard clip at the specified values. ffmpeg has issues with video levels. Also, when you re-encode to, let's say H.264 it puts the artifacts >235 back in. -
lutyuv and limiter filters do clip at specified values, perhaps you do not know how to use filters.
Non-lossless encoding can produce various artifacts you can not avoid that. -
If you wish to limit pixels like that use -vf scale=in_range=pcut_range=tv
Otherwise, command will do nothing.
If you just want to clamp pixels, use lut or limiter filter.Last edited by chris319; 14th May 2018 at 14:45.
-
I assure you, richardpl does know how to measure video levels. He is a ffmpeg developer
lutyuv does work, I've verified this in the past for both 8bit and 10bit
If there are problems , then document the issues or post a detailed bug report so they can be fixed. Rounding errors are expected for some operations, and lossy codecs can produce excursions as well -
If there are problems , then document the issues or post a detailed bug report so they can be fixed.
-
-
Have a look at the scope shot below. Here is the code used to generate it:
Code:ffmpeg -i stairstep.mp4 -y -vf lutyuv=y='clip(val,16,235)' outfile.mp4
http://www.chrisnology.info/videos/Cliptest.jpg -
The TV levels filter works as expected. It compresses the video levels to the range 16 - 235, thus altering the levels.
I'm looking to clip the video without altering the levels.
Code:bin\ffmpeg -i stairstep.mp4 -y -vf "scale=in_range=pc:out_range=tv" -c:a copy outfile.mp4
The problem is that these ringing artifacts pop up when you encode the video to a deliverable format, be it h.264, h.265 or even prores. They can be tamed to an extent by applying some "unsharpening" and removing some of the high-frequency content. -
Possible explanations: an issue with your source video, or testing method, or scopes ? It definitely works here
I can check your video if you want to upload it
Understood; then you want to use lutyuv or limiter
The problem is that these ringing artifacts pop up when you encode the video to a deliverable format, be it h.264, h.265 or even prores. They can be tamed to an extent by applying some "unsharpening" and removing some of the high-frequency content. -
If you didn't know ffmpeg have own waveform monitor: http://trac.ffmpeg.org/wiki/WaveformMonitor
-
an issue with your source video, or testing method, or scopes ?
Code:ffmpeg -i stairstep.mp4 -y outfile.mp4
Nearly perfect reproduction of the video levels. No problem with source video, testing method or scope.
So the problem is either with my ffmpeg filter syntax or with ffmpeg itself. -
-
Here is the code used to create the video from a bmp.
Code:bin\ffmpeg -y -loop 1 -t 10 -i stairstep.bmp -pix_fmt yuv420p -crf 17 -c:v libx264 -vf scale=out_color_matrix=bt709 -color_primaries bt709 -color_trc bt709 -colorspace bt709 -r 59.94 -c:a copy test_pattern.mp4
Please upload your video. Likely it's either a problem with your video, or you're not actually measuring YUV, but RGB in a standard range conversion
Please post the code used for your clipping and point out any differences between your code and mine.
http://www.chrisnology.info/videos/outfile.mp4
The scope is beyond reproach. It has been thoroughly and rigorously calibrated and tested with Matlab and it passes muster. Again, it perfectly reproduces the gray scale. -
The reason is there is nothing to clip . It' s not a full range video . Your RGB to YUV conversion for the test video uses limited range (RGB 0,0,0-255,255,255 gets "mapped" to Y 16-235, CbCr 16-240)
If nothing happens after lutyuv clipping, the most likely explanation is your video has nothing to clip - it is limited range Y (16-235) , and you are measuring standard range converted RGB values . That is exactly what is happening here
Technically , a Y' waveform is supposed to measure Y values, not their RGB converted values
If you look at a real Y' waveform, there are no values <16 or >235 in your video -
First problem: the ffmpeg scope is confining the levels to 16 - 235. That's not what's in the file. The stairstep ranges from 0 - 255.
http://www.chrisnology.info/videos/ffmpegscope.jpg
Here is the PureBasic code used to generate the stairstep. Now you've got all the evidence.
Code:Global screenWd, screenHt Procedure DrawGrayScale() StartDrawing(ImageOutput(1)) Box(0,0,screenWd,ScreenHt, 0) Steps = 16 X = 0 Lum.f = 0 LumStep.f = 255 / (Steps-1) XStep = ScreenWd/Steps For ct = 1 To Steps Box(X,0,XStep,ScreenHt, RGB(Lum,Lum,Lum)) Lum + LumStep X + XStep Next StopDrawing() EndProcedure CreateImage(1,1280,720) ExamineDesktops() ScreenWd = DesktopWidth(0):ScreenHt = DesktopHeight(0) OpenWindow(0,0,0, ScreenWd,ScreenHt, #Null$,#PB_Window_BorderLess) AddKeyboardShortcut(n, #PB_Shortcut_Escape, #PB_Shortcut_Escape) ;"ESCAPE" KEY TO QUIT PROGRAM DrawGrayScale(): ShowCursor_(#False) SaveImage(1,"stairstep.bmp") StartDrawing(WindowOutput(0)) DrawImage(ImageID(1),0,0) StopDrawing() Repeat event = WaitWindowEvent() If event = #PB_Event_Repaint DrawGrayScale() ElseIf event = #PB_Event_Menu ;KEYBOARD INPUT menuItem = EventMenu() Select menuItem Case #PB_Shortcut_Escape Break EndSelect EndIf ForEver End
-
Clearly you don't understand how the lum. component of a video signal is derived.
-
Your source video is limited range , it's already standard range ,with nothing to clip. (Just look at the command line you used, it converts RGB to YUV with limited range, that's already proof enough)
Your scope actually measures RGB values if it shows the 0 and 255 values.
You can verify this with multiple methods , not necessarily ffmpeg (I like using other methods to verify). For example , you can verify this with broadcast NLE or hardware SDI out
I am 100% certain of these facts. -
Code:
Steps = 16 X = 0 Lum.f = 0 LumStep.f = 255 / (Steps-1) XStep = ScreenWd/Steps For ct = 1 To Steps Box(X,0,XStep,ScreenHt, RGB(Lum,Lum,Lum)) Lum + LumStep X + XStep Next
Code:ffmpeg -y -loop 1 -t 10 -i stairstep.bmp -pix_fmt yuv420p -crf 17 -c:v libx264 -vf scale=out_color_matrix=bt709 -color_primaries bt709 -color_trc bt709 -colorspace bt709 -r 59.94 -c:a copy test_pattern.mp4
Code:it converts RGB to YUV with limited range
Your scope actually measures RGB values if it shows the 0 and 255 values.
Now view the video and count the steps (there are 16 steps, 16 decimal values apart. Now take an eyedropper program or use Colorzilla to measure the colors. They range all the way from 0 to 255.
Your ffmpeg scope is hiding everything outside the range of 16 - 235, but you're resistant to the notion that it's basically lying to you.
Where do you think the lum. component comes from? It's calculated from the R, G and B components. Kr, Kg and Kb are the lum. coefficients. I'm surprised you don't already know this.
Code:Y = R * Kr + G * Kg + B * Kb
-
Your RGB test pattern is probably OK. That's not what the concern is. The concern is the RGB=>YUV conversion, and your scopes
Code:ffmpeg -y -loop 1 -t 10 -i stairstep.bmp -pix_fmt yuv420p -crf 17 -c:v libx264 -vf scale=out_color_matrix=bt709 -color_primaries bt709 -color_trc bt709 -colorspace bt709 -r 59.94 -c:a copy test_pattern.mp4
Code:it converts RGB to YUV with limited range
Read up on ITU Rec 709 conversion. If I recall, you coded some RGB<=>YUV functions. Might be a good idea to revisit
https://www.itu.int/rec/R-REC-BT.709/en
https://en.wikipedia.org/wiki/Rec._709
Yes you started with a nice RGB test pattern 0-255. But your RGB conversion to YUV gets "mapped" to Y 16-235, CbCr16-240 . So you have no values Y<16 or Y>235. You have to use a "full range" conversion to map RGB0-255 to YUV0-255. It's know as a "full range" conversion.
Your scope actually measures RGB values if it shows the 0 and 255 values.
Your ffmpeg scope is hiding everything outside the range of 16 - 235, but you're resistant to the notion that it's basically lying to you.
This is your outfile.mp4 source. Notice it's limited range, there are no values >100 or <0 IRE. Compare to the full range one I posted above before applying lutyuv. Values are preset <0 IRE, and >100IRE (which correspond to Y=16, Y=235 in 8bit)
But in this case, the ffmpeg waveform is correct too (it's not always, some formats have decoding and levels issues, or certain flags can confuse the issue). This is a true full range YUV video, and with lutyuv clipping applied . Your outfile.mp4 shows no values <0 or >100. It's limited range
Again, I'm 100% certain about this.
Similar Threads
-
Newbie at using FFMPEG - DVD Rips to ProRes with AAC Audio?
By Brainiac in forum Video ConversionReplies: 4Last Post: 12th Apr 2017, 19:00 -
IS the prores much better then AVCHD? A debunking video test about prores.
By Stears555 in forum Video ConversionReplies: 8Last Post: 21st Mar 2017, 01:22 -
Problem at efficiency of prores encoding vias ffmpeg.
By zhangjie in forum Newbie / General discussionsReplies: 0Last Post: 15th Jun 2015, 03:46 -
Strange ffmpeg error when ProRes encoding
By SameSelf in forum Video ConversionReplies: 8Last Post: 25th Aug 2014, 21:34 -
mpeg to prores 422 HQ Itunes spec with ffmpeg
By vinbel in forum ffmpegX general discussionReplies: 21Last Post: 29th Jan 2014, 03:07