FIND / FINDSTRING now outputs extra lines that were not in the search strin

[JN-]

FIND / FINDSTRING now outputs extra lines that were not in the search string.

My search string is the word average.

I use FIND 3 times and now only 1 of the 3 works as expected ... The SSIM one.
FIND /i "Average" < "ffmpeg SSIM and PSNR.txt" > "PSNR ONLY.txt"
FIND /i "All:" < "ffmpeg SSIM and PSNR.txt" > "SSIM ONLY.txt"
FIND /i "score" < "ffmpeg VMAF.txt" > "VMAF ONLY.txt"

Up until now it worked. I used ffmpeg to output SSIM and PSNR values to text file "ffmpeg SSIM and PSNR.txt"

I then used this to output to file "PSNR ONLY.txt"
FIND /i "Average" < "ffmpeg SSIM and PSNR.txt" > "PSNR ONLY.txt"

The above "PSNR ONLY.txt" file typically only contained a single line containing the word average, pertaining to PSNR.

I then put this single line into the var "_PSNR_SINGLE_LINE" as next line below, then extracted the PSNR average value.
SET /p _PSNR_SINGLE_LINE=<"PSNR ONLY.txt"

This is the output to the PSNR ONLY.txt file, should be only a single line, the last here, not 7 lines.

frame= 73 fps=0.0 q=-0.0 q=-0.0 size=N/A time=00:00:00.00 bitrate=N/A speed= 0x
frame= 173 fps=168 q=-0.0 q=-0.0 size=N/A time=00:00:00.00 bitrate=N/A speed= 0x
frame= 284 fps=183 q=-0.0 q=-0.0 size=N/A time=00:00:00.00 bitrate=N/A speed= 0x
frame= 384 fps=186 q=-0.0 q=-0.0 size=N/A time=00:00:00.00 bitrate=N/A speed= 0x
frame= 494 fps=191 q=-0.0 q=-0.0 size=N/A time=00:00:00.00 bitrate=N/A speed= 0x
frame= 600 fps=193 q=-0.0 q=-0.0 size=N/A time=00:00:00.00 bitrate=N/A speed= 0x
[Parsed_psnr_9 @ 000001af0d8e9d80] PSNR y:42.194614 u:47.254519 v:48.699876 average:43.409878 min:37.558604 max:55.487915

I am using the latest version of ffmpeg. Newer versions sometimes means a minor change re: the location of the average PSNR/SSIM values on the single line.

With a version change I may have to change just the following ... SET _PSNR_RESULT=%_PSNR_SINGLE_LINE:~84,9%

Extracting just a single line for SSIM worked ok as previously ...
[Parsed_ssim_4 @ 000001af0d8e7380] SSIM Y:0.988755 (19.490456) U:0.986370 (18.655012) V:0.988336 (19.331407) All:0.988288 (19.313573)

The above single line was extracted from the same file "ffmpeg SSIM and PSNR.txt" see a small extract below.

frame= 73 fps=0.0 q=-0.0 q=-0.0 size=N/A time=00:00:00.00 bitrate=N/A speed= 0x
frame= 173 fps=168 q=-0.0 q=-0.0 size=N/A time=00:00:00.00 bitrate=N/A speed= 0x
frame= 284 fps=183 q=-0.0 q=-0.0 size=N/A time=00:00:00.00 bitrate=N/A speed= 0x
frame= 384 fps=186 q=-0.0 q=-0.0 size=N/A time=00:00:00.00 bitrate=N/A speed= 0x
frame= 494 fps=191 q=-0.0 q=-0.0 size=N/A time=00:00:00.00 bitrate=N/A speed= 0x
frame= 600 fps=193 q=-0.0 q=-0.0 size=N/A time=00:00:00.00 bitrate=N/A speed= 0x
[Parsed_psnr_9 @ 000001af0d8e9d80] PSNR y:42.194614 u:47.254519 v:48.699876 average:43.409878 min:37.558604 max:55.487915
[Parsed_ssim_4 @ 000001af0d8e7380] SSIM Y:0.988755 (19.490456) U:0.986370 (18.655012) V:0.988336 (19.331407) All:0.988288 (19.313573)
[out#0/null @ 000001af7bd5de40] video:580KiB audio:5060KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: unknown
frame= 675 fps=196 q=-0.0 Lq=-0.0 size=N/A time=00:00:00.00 bitrate=N/A speed= 0x

In this SSIM extract example the search string is "ALL" and it correctly output the 3rd last line ONLY.
So using 3 different search strings to extract the lines with PSNR, SSIM and VMAF now only outputs a single line for SSIM values.
I thought it might have something to do with the frame= entries, but they are in the same file to search that outputs a single line for SSIM.

I could do a workaround, namely reversing the contents of file "PSNR ONLY.txt" then using SET /P to extract only the 1st. line.
I would be better if someone can see a way of only initially outputting a single line with the search string "average".

I have the same issue with getting a single line with VMAF. This is the output ...
The line I require is at the bottom. The search string was "score".

frame= 26 fps=0.0 q=-0.0 size=N/A time=00:00:00.00 bitrate=N/A speed= 0x
frame= 65 fps= 63 q=-0.0 size=N/A time=00:00:00.00 bitrate=N/A speed= 0x
frame= 108 fps= 70 q=-0.0 size=N/A time=00:00:00.00 bitrate=N/A speed= 0x
frame= 149 fps= 73 q=-0.0 size=N/A time=00:00:00.00 bitrate=N/A speed= 0x
frame= 188 fps= 73 q=-0.0 size=N/A time=00:00:00.00 bitrate=N/A speed= 0x
frame= 231 fps= 75 q=-0.0 size=N/A time=00:00:00.00 bitrate=N/A speed= 0x
frame= 271 fps= 75 q=-0.0 size=N/A time=00:00:00.00 bitrate=N/A speed= 0x
frame= 312 fps= 76 q=-0.0 size=N/A time=00:00:00.00 bitrate=N/A speed= 0x
frame= 354 fps= 76 q=-0.0 size=N/A time=00:00:00.00 bitrate=N/A speed= 0x
frame= 394 fps= 76 q=-0.0 size=N/A time=00:00:00.00 bitrate=N/A speed= 0x
frame= 433 fps= 76 q=-0.0 size=N/A time=00:00:00.00 bitrate=N/A speed= 0x
frame= 472 fps= 76 q=-0.0 size=N/A time=00:00:00.00 bitrate=N/A speed= 0x
frame= 512 fps= 76 q=-0.0 size=N/A time=00:00:00.00 bitrate=N/A speed= 0x
frame= 555 fps= 77 q=-0.0 size=N/A time=00:00:00.00 bitrate=N/A speed= 0x
frame= 594 fps= 77 q=-0.0 size=N/A time=00:00:00.00 bitrate=N/A speed= 0x
frame= 631 fps= 77 q=-0.0 size=N/A time=00:00:00.00 bitrate=N/A speed= 0x
frame= 673 fps= 77 q=-0.0 size=N/A time=00:00:00.00 bitrate=N/A speed= 0x
[Parsed_libvmaf_4 @ 00000186de426b80] VMAF score: 99.032929

[JN-]

I tried 7 input files. One of the 7 output correct SSIM and PSNR (so a single output line) and the VMAF text file used to get values had only 2 lines, still not 1 line. A mystery for sure.

Everything works aok if I use my previous ffmpeg version 28-Nov-2023.

[JN-]

I think the problem lies with the lines that output the "frames" values have no line feed char only a CR char at the end of the line.

The previous version of ffmpeg had similar (no LF) but the difference is that the PSNR/SSIM output that is written to the text file with the newer version (my same ffmpeg syntax is used) is packed together, frame line after frame line. The older output spread those "frame" lines apart.

So I speculate that FIND / FINDSTR doesn't handle the lines with no LF properly when they occur one after the other ?

So removing all lines with "frame" in them didn't work. The following removes lines with frames in them but unfortunately also removed the next line that didn't have the word frames in it but DID have the PSNR value. Probably caused by the last frames line NOT having a LF char.

REM This will output all lines except those with frame in them
FINDSTR /V /i "frame" "ffmpeg-SSIM-and-PSNR.txt">"PSNR-ONLY-NO-FRAME.txt"

[JN-]

I have tried reversing and deleting and both don’t solve the issue mainly because of the proximity of the line with the psnr value to the “frames” lines.

So. As I see it the simplest and most elegant fix would be to read each line from the text file with the ssim and psnr values, add a LF line feed char where its not in place (all lines starting with “frames”, they only have CR at end instead of CRLF) and save to new text file.

Any takers, is beyond my capabilities.

[_Al_]

It can be parsed, avoiding using FIND or FINDSTR, using some typical windows batch hacks. Basically splitting line with known delimiter, both before and after that float value. In case of average, splitting by "average:" and then by " min:"

@echo off
set "average="
for /f "delims=" %%x in (text.txt) do call :search_in_line "%%x"
echo average=%average%
echo press any key to exit & pause > nul & exit

:search_in_line <line text>
set "line=%1"
if not _%line:average=% == _%line% call :parse_line "%~1"
goto :eof

:parse_line <line for parsing>
set "line=%1"
set "left=%line:average:=" & set "right=%"
rem those two lines have to be there, if not it does not work, again, some other weird windows batch hack
echo %left% > nul
echo %right% > nul
set "average=%right: min:=" & set "rest=%"
goto :eof

but rather, some proper language could be used, like python

[JN-]

Thanks for that _A_ Brilliant! It gets the value I wanted. I put it into a "split lines.bat" below, with a few notes!

Output is to "TEXT-OUTPUT.txt". Has 2 lines.

@echo off
mode CON:cols=228 lines=30
color 0A
SETLOCAL EnableDelayedExpansion

set "average="
for /f "delims=" %%x in (text.txt) do call earch_in_line "%%x"
echo average=%average%
echo press any key to exit & pause > nul & exit

earch_in_line <line text>
set "line=%1"
if not _%line:average=% == _%line% call arse_line "%~1"
goto :eof

arse_line <line for parsing>
set "line=%1"
set "left=%line:average:=" & set "right=%"
rem those two lines have to be there, if not it does not work, again, some other weird windows batch hack
REM echo %left% > nul
REM echo %right% > nul
echo %left%>TEXT-OUTPUT.txt
echo %right%>>TEXT-OUTPUT.txt
set "average=%right: min:=" & set "rest=%"
goto :eof
REM but rather, some proper language could be used, like python

REM TEXT-OUTPUT.txt contents is ...
REM 40.773202 min:37.123585 max:49.749442
REM Excellent, just what I wanted.

REM Empty files are output and need to be deleted afterwards ...
REM Stream
REM settb
REM pcm_s16le

REM This is always displayed on screen while processing of Split lines.bat ...
REM Is two lines. The 2nd line ... average=40.773202 is exactly what I want.
"frame= 74 fps=0.0 q=-0.0 q=-0.0 size=N/A time=00:00:00.00 bitrate=N/A speed= 0x frame= 170 fps=165 q=-0.0 q=-0.0 size=N/A time=00:00:00.00 bitrate=N/A speed= 0x frame= 269 fps=173 q=-0.0 q=-0.0 size=N/A time=00:00:00.00 bitrate=N/A speed= 0x frame= 384 fps=185 q=-0.0 q=-0.0 size=N/A time=00:00:00.00 bitrate=N/A speed= 0x frame= 482 fps=186 q=-0.0 q=-0.0 size=N/A time=00:00:00.00 bitrate=N/A speed= 0x frame= 600 fps=193 q=-0.0 q=-0.0 size=N/A time=00:00:00.00 bitrate=N/A speed= 0x [Parsed_psnr_9 @ 00000150d7b60240] PSNR y:39.420810 u:45.689526 v:47.423019 "
average=40.773202


RE: python.
SET _START=batch, SET PEAK=delphi 5, SET /A ega=3/4_cent+=?, SET _END=batch.

The input file used for testing, with it's "Average PSNR" and "SSIM ALL" values that need extracting is ...