play video with histogram display, avisynth+ ?

[butterw]

> AVSMeter64.exe LumaHistogram.avs
does give me the info about syntax errors.

A player with a built-in or plug-in histogram feature is the way to go to visualize the histogram of the video as played (performance, wide file-format support without indexing, ease of use).
The goal of my avs script is different: analyse source video files and then evaluate the processing done in the player (PC video players typically expand range).


I would like to analyse the source video files in a similar way to ColorYUV(analyze=true), and save the results to a txt file or get an aggregate result for the file. For this I'm currently looking at runtime function such as YPlaneMin(), which are evaluated at each frame of a clip. They have to be passed to runtime filters (ConditionalFilter, ScriptClip, FrameEvaluate). The subject seems somewhat advanced and I haven't found a simple/good example for this so far.
http://avisynth.nl/index.php/ScriptClip#ScriptClip

ScriptClip("subtitle(string(current_frame))")

is easy enough: shows the frame-number in a clip. But it crashes if I try with string(YPlaneMin(vid)).

[_Al_]

if you just start with Avisynth I recommend just to skip it, exactly because for your purpose. Proceed with Vapoursynth instead. Avisynth has for each thing some hack function and you want more, some change and it needs to be yet written by someone else again. In Vapoursynth you can write what you want right away.
I'm sure, this script below could be done in Avisynth too, but as described it is ready to be changed, as for formatting, printing, placing.

import vapoursynth as vs
from vapoursynth import core

def evaluate(n, clip):
    aver = clip.get_frame(n).props['MyPlaneStatsAverage']
    _min = clip.get_frame(n).props['MyPlaneStatsMin']
    _max = clip.get_frame(n).props['MyPlaneStatsMax']
    text = f'Average={aver:.2f}\nMin={_min:.2f}\nMax={_max:.2f}\n' #:.2f means two decimal places, \n means new line
    return clip.text.Text(text)

clip = core.avisource.AVISource('red.avi')
clip = clip.std.PlaneStats(prop='MyPlaneStats')
evaluated_clip = core.std.FrameEval(clip, functools.partial(evaluate, clip=clip))
evaluated_clip.set_output()

also you can let things evaluate and print it results or make a graph, just using couple of lines, whatever python can do. Not to mention that if you get comfortable, you just use portable versions, freeze the code and you just might use some exe file to analyze things etc.

[_Al_]

just realized, you wanted stats per channel:

import vapoursynth as vs
from vapoursynth import core

def evaluate(n, clip):
    text = []
    for plane in ['Y','U','V']:
        aver = clip.get_frame(n).props[f'Stats{plane}Average']
        _min = clip.get_frame(n).props[f'Stats{plane}Min']
        _max = clip.get_frame(n).props[f'Stats{plane}Max']
        text.append(f'{plane}\nAverage={aver:.2f}\nMin={_min:.2f}\nMax={_max:.2f}\n') #:.2f means two decimal places, \n means new line
    return clip.text.Text(''.join(text))

clip = core.avisource.AVISource('red.avi')
clip = clip.std.PlaneStats(plane=0, prop = 'StatsY' )\
           .std.PlaneStats(plane=1, prop = 'StatsU' )\
           .std.PlaneStats(plane=2, prop = 'StatsV' )
evaluated_clip = core.std.FrameEval(clip, functools.partial(evaluate, clip=clip))
evaluated_clip.set_output()

so here we go, this is the flexibility I talked about ...

[poisondeathray]

Other options are ffmpeg/ffplay -vf signalstats to print a text file , or avisynth writefile

eg. signalstats

https://ffmpeg.org/ffmpeg-filters.html#signalstats-1
ffprobe -f lavfi movie=input.ext,signalstats="stat=tout+vrep+brng" -show_frames 1>signalstats.txt



e.g write file for YPlaneMin YPlaneMax
http://avisynth.nl/index.php/WriteFile

#whateversource
colon = ": "
WriteFile(last, "log.txt", "current_frame", "colon", "YPlaneMin(last)", "colon", "YPlaneMax(last)" )

But there seems to be an issue with current avs+ x64 WriteFile , not sure if some syntax changed or something. It works with older x86 version

[_Al_]

To analyze video and write stats for all channels in some log in vapoursynth, again, just using Python functions,
separating tasks into threads, in our case each plane is analyzed in its own thread,
then writing result on disk as a log file, again using python function:

import vapoursynth as vs
from vapoursynth import core
from concurrent import futures   #standard python library module

def analyze_channel(clip, plane_name): 
    text = []
    for frame in range(clip.num_frames):   
        aver = clip.get_frame(frame).props[f'Stats{plane_name}Average']
        _min = clip.get_frame(frame).props[f'Stats{plane_name}Min']
        _max = clip.get_frame(frame).props[f'Stats{plane_name}Max']
        text.append(f'{frame}: {plane_name}Average={aver:.2f} {plane_name}Min={_min:.2f} {plane_name}Max={_max:.2f}\n')
    return ''.join(text)

clip = core.avisource.AVISource('red.avi')
clip = clip.std.PlaneStats(plane=0, prop = 'StatsY' )\
           .std.PlaneStats(plane=1, prop = 'StatsU' )\
           .std.PlaneStats(plane=2, prop = 'StatsV' )

print('starting to analyze planes\nanalyzing wait ...')
with futures.ThreadPoolExecutor(max_workers=4) as analyze:
    #starting three jobs, one for each plane , each in separate thread
    jobs = [analyze.submit(analyze_channel, clip, plane_name) for plane_name in ['Y','U','V']]
    texts = [job.result() for job in jobs]

print('analyzing done')
print('writing log file ...')
with open('planes_log.log', 'w') as log:
    for text in texts:
        log.write(text) 
print('writing done')

[butterw]

@poisondeathray: I was looking at the same WriteFile example, but it wasn't working well with AvsPmod, vdub2, mpc-be.
The solution for me was to run it with: >> AVSMeter64.exe luma_stats.avs

The following code is clean/compact enough.

/* luma_stats.avs
tested on Avisynth+ v3.6.1 x64. Run with: AVSMeter64.exe luma_stats.avs 
Saves luma.csv with full Y stats 
perf: 107fps on 1080p
*/ 
fname = "b:\Videos\monster-1_ed.mp4" 
LSmashVideoSource(fname, prefer_hw=2)
fname_stats = "C:\temp\luma.csv"
sep = "; "
header = "frame" +sep+ "Ymin" +sep+ "Ymin(0.4)" +sep+ "Ymed" +sep+ "Yavg" +sep+ "Ymax(0.4)" +sep+ "Ymax"
WriteFileStart(fname_stats, "header", append=false)
WriteFile(fname_stats, "current_frame", "sep", "YPlaneMin", "sep", "YPlaneMin(0.4)", "sep", "YPlaneMedian", "sep", "AverageLuma", "sep", "YPlaneMax(0.4)", "sep", "YPlaneMax", flush=false, append=true)

[jagabo]

The the runtime WriteFile() function works for me (AviSynth+ 64 bit):

WhateverSource()

ylabel = "    Y "
ulabel = "    U "
vlabel = "    V "
dash = "-"
thresh = 0.4
WriteFile(last, "Info.txt", "current_frame", \
    "ylabel", "YPlaneMin(last, threshold=thresh)", "dash", "YPlaneMax(last, threshold=thresh)", \
    "ulabel", "UPlaneMin(last, threshold=thresh)", "dash", "UPlaneMax(last, threshold=thresh)", \
    "vlabel", "VPlaneMin(last, threshold=thresh)", "dash", "VPlaneMax(last, threshold=thresh)")

Open in VirtualDub2 (64 bit), preview input, log:

0    Y 20-216    U 109-158    V 98-145
0    Y 20-216    U 109-158    V 98-145
1    Y 19-216    U 109-158    V 98-145
2    Y 20-216    U 109-158    V 99-145
3    Y 20-216    U 109-158    V 99-145
4    Y 19-216    U 109-158    V 99-145
5    Y 20-216    U 109-157    V 98-145
6    Y 20-216    U 109-158    V 98-145
7    Y 20-216    U 109-159    V 98-145
8    Y 20-216    U 109-159    V 98-145
9    Y 19-216    U 109-158    V 97-145
10    Y 19-216    U 109-158    V 98-145
11    Y 19-216    U 109-158    V 98-145
12    Y 20-216    U 109-158    V 98-145

The first line appears twice because virtualdub displays the first frame when you open the script, then displays it again (and the rest of the frames) when you preview.

Though I find a waveform monitor to be much more useful...

[_Al_]

Those stats are needed for some visual later?

from concurrent import futures
import matplotlib.pyplot as plt   #python -m pip install -U matplotlib

clip = core.avisource.AVISource('red.avi')

def analyze_channel(clip, plane_name): 
    _min, _max = [],[]
    for frame in range(clip.num_frames):   
        #aver.append(clip.get_frame(frame).props[f'Stats{plane_name}Average'])
        _min.append(clip.get_frame(frame).props[f'Stats{plane_name}Min'])
        _max.append(clip.get_frame(frame).props[f'Stats{plane_name}Max'])
    return [_min, _max]

clip = clip.std.PlaneStats(plane=0, prop = 'StatsY' )\
           .std.PlaneStats(plane=1, prop = 'StatsU' )\
           .std.PlaneStats(plane=2, prop = 'StatsV' )

print('starting to analyze planes\nanalyzing wait ...')
with futures.ThreadPoolExecutor(max_workers=4) as analyze:
    #starting three jobs, one for each plane , each in separate thread
    jobs = [analyze.submit(analyze_channel, clip, plane_name) for plane_name in ['Y','U','V']]
    y_axis = [job.result() for job in jobs]
print('analyzing done')

x_axis    = list(range(clip.num_frames))
for plane, plane_color in enumerate(['yellow', 'blue', 'red']):  
    plt.title(f"Min & Max values for Y-yellow, U-blue, V-red")
    plt.xlabel("frames")
    plt.ylabel("min & max per channel")
    [plt.scatter(x_axis, y_axis[plane][min_max], s=10, c=plane_color) for min_max in [0,1]]
    plt.show()

[butterw]

You can look at other non avisynth options too, like ffplay or mpv with -vf histogram and waveform . Maybe overlay (partial transparency) or stack the graphic

I've finally got round to installing mpv video player.

mpv -vf histogram "filename.mp4"
https://ffmpeg.org/ffmpeg-filters.html#histogram-1

The command displays stacked YUV histograms, but it doesn't show the video (how to display the video side by side ?).
Like avisynth it's an input method meaning it's the source histogram rather than the histogram of the displayed video output .
The main benefit vs avisynth would be that indexing of non-mp4 is not required.

[butterw]

On the subject of the avisynth frame analysis script with csv output: it would be desirable to be able to perform the analysis much faster. (ex: 10 or 20x)

The solution seems to be to not process all frames but just one frame per scene.

How to select frames for processing ? A fast/naive way would be to just sample 2% of frames at regular intervals. Is there a more advanced way of doing this in avisynth (keyframes only, or fast scene detection algorithm) ?

[poisondeathray]

You can look at other non avisynth options too, like ffplay or mpv with -vf histogram and waveform . Maybe overlay (partial transparency) or stack the graphic

I've finally got round to installing mpv video player.

mpv -vf histogram "filename.mp4"
https://ffmpeg.org/ffmpeg-filters.html#histogram-1

The command displays stacked YUV histograms, but it doesn't show the video (how to display the video side by side ?).
Like avisynth it's an input method meaning it's the source histogram rather than the histogram of the displayed video output .
The main benefit vs avisynth would be that indexing of non-mp4 is not required.

You can stack or overlay -vf histogram using a filter chain with split. See the example at the end of this link
https://trac.ffmpeg.org/wiki/WaveformMonitor

FFmpeg's waveform has many display options
https://ffmpeg.org/ffmpeg-filters.html#waveform

On the subject of the avisynth frame analysis script with csv output: it would be desirable to be able to perform the analysis much faster. (ex: 10 or 20x)

The solution seems to be to not process all frames but just one frame per scene.

How to select frames for processing ? A fast/naive way would be to just sample 2% of frames at regular intervals. Is there a more advanced way of doing this in avisynth (keyframes only, or fast scene detection algorithm) ?

You can use Select functions in avisynth to process every nth frame, or groups/combinations of frames
http://avisynth.nl/index.php/Select

[jagabo]

For one frame per scene change you can use WriteFileIf() and one of the scenechange detectors or the runtime functions like YDifferenceFromPrevious.

http://avisynth.nl/index.php/Internal_functions/Runtime_functions#Runtime_functions

I don't see this as useful though. The first frame of a scene change isn't necessarily reflective of the rest of the scene.

[butterw]

ffplay -vf "split=2[a][b],[b]waveform=g=green[bb],[a][bb]overlay" "b:\Videos\monster2.mp4" does work

but mpv --vf="split=2[a][b],[b]waveform=g=green[bb],[a][bb]overlay" "b:\Videos\monster2.mp4" fails "Error parsing option vf (option parameter could not be parsed)".

If you can get them to work those ffmpeg video filters are not bad (with less customization freedom than avisynth) but the filter chain syntax isn't easy...

[poisondeathray]

ffplay -vf "split=2[a][b],[b]waveform=g=green[bb],[a][bb]overlay" "b:\Videos\monster2.mp4" does work

but mpv --vf="split=2[a][b],[b]waveform=g=green[bb],[a][bb]overlay" "b:\Videos\monster2.mp4" fails "Error parsing option vf (option parameter could not be parsed)".

If you can get them to work those ffmpeg video filters are not bad (with less customization freedom than avisynth) but the filter chain syntax isn't easy...

split does not work with mpv afaik

The mpv workaround is to load file twice: once normally, and again as --external-file, and using --lavfi-complex. It's not ideal, takes more resources


eg.

mpv --lavfi-complex="[vid1]waveform=g=green[bb],[vid2][bb]overlay[vo]" input.mp4 --external-file=input.mp4

You can change opacity of overlay, or configure it differently such as using stacking horizontal or vertical instead of overlay

[butterw]

For one frame per scene change you can use WriteFileIf() and one of the scenechange detectors or the runtime functions like YDifferenceFromPrevious.

http://avisynth.nl/index.php/Internal_functions/Runtime_functions#Runtime_functions

I don't see this as useful though. The first frame of a scene change isn't necessarily reflective of the rest of the scene.

Regular frame sampling certainly seems a lot simpler.
Maybe you would need to select a frame in the middle of the scene (as detected by YDiff).

I haven't tried vapoursynth yet, but it looks like it could be a better bet outside the core use cases of avisynth. The fact that you can use python libraries makes it very attractive for custom developments.

Edit: I've added YDifferenceFromPrevious to my luma_stats.avs script.
According to the docs, it returns the absolute difference of pixel value between the current and previous frame of clip.
I'm assuming that means avg(abs(pixel_Ydiff))) ?
Detected scene change would be when Ydiff>user defined threshold.