Automatic change/scene detection for video game videos

[onewaydown]

So, essentially, I'm trying to pluck out all the kills in a video from an FPS game. There's a on-screen text feed in a certain spot that displays when the player gets a kill. I'm trying to see if there is some sort of software that can scan a selected area for any changes. More specifically, whenever some red text is displayed. Most search results I can find usually are basic scene detection or full frame change comparisons. I was hoping there might be a tool that lets me select a certain portion of the frame to monitor for change - preferably looking for any change to or inclusion of red. Is this a pipe dream?

Thanks!

[jagabo]

An AviSynth script can probably do this. A sample video would allow someone to write a script for you since AviSynth is hard to get started with.

[onewaydown]

An AviSynth script can probably do this. A sample video would allow someone to write a script for you since AviSynth is hard to get started with.

Well, right now they're all saved PS4 streams on Youtube. Sample vid is here - https://www.youtube.com/watch?v=ChIA5gmAkRI

Quality isn't the best, due to the lack of bitrate options on the PS4. However, attached is a sample, where the personal kill feed is in the middle, slight right, and left justified, where it indicates the [weapon] in white and then the gamer name of the person killed in red text. The exact starting placement of the red text will vary due to difference in weapon name lengths but it's still the same basic area.


[Attachment 47642 - Click to enlarge]

It would be cool to be able to automatically detect any red pixels in that area and create a list of timestamps for me to go review later. I mean, it would be real awesome if it could cut/trim that section of the vid to a new file but all I really need are the timestamps. I have dozens of 5-6hr streams that I don't want to rewatch completely just to find some of my kills.

[pandy]

ffmpeg can detect scene changes, and as you know where change is expected then crop video to this area and analyse it with ffmpeg, perhaps R plane only - should not be to difficult.

Bellow example without crop, full video processed, assumed video framerate is 25 fps so to get correct fps it should be placed manually accordingly to your source framerate, tile 2x2 (mosaic) image is created. Scene change sensitivity must be input as asked, lower value higher scene change sensitivity, 0.25 - 0.1 seem to be OK.

@setlocal
@echo off
@echo "script"

@rem folder where ffmpeg.exe is located
@set FF=C:\FF
@set PATH=%FF%;%PATH%

@set filename=%~1

@set /p sens=Enter Scene Detection Sensitivity (0..1 e.g. 0.25): 
@echo Scene Detection Sensitivity is: %sens%
@ffmpeg -hide_banner -v 32 -stats -y -i "%filename%" -vf "drawtext=timecode='00\:00\:00\:00':r=25:x=(w-tw)/2: y=(2*lh):fontsize=64:fontcolor=black:box=1:boxcolor=0xFFFFFFFF@1,scale=iw/4:-4:out_range=full:sws_flags=spline:sws_dither=0,select='gt(scene\,%sens%)',tile=2x2" -vsync 0 %1_preview_%%03d.png

@endlocal
@timeout 30
@exit

[jagabo]

I downloaded part of the 720p video and came up with this:

LWLibavVideoSource("Star Wars Battlefront PS4.mkv") 
Crop(580,240,120,24)
SelectEvery(180) # every 180'th frame, 3 seconds at 60 fps
MaskHS(startHue=90.0, endHue=105.0, MinSat=30)
WriteFileIf(last, "log.txt", "AverageLuma>20", "current_frame*180" ) # multiply by 180 to get original frame number

That opens the video, crops it to just the area where the kill messages appear, looks at that area (only every 3 seconds) for saturated reds, and writes the frame number to log.txt if reds are found. That means it will generate false positives if other red objects appear in that area. It may miss some kills if the message lasts for less than 3 seconds.

You'll have to change the file name and the crop area if the frame size is different. Open the script in an editor and run through it from beginning to end to force the entire file to be searched (I like to use VirtualDub, File -> Run Video Analysis Pass). Note that log.txt is appended to each time you run the script -- so delete it before you run the script. It took about 30 seconds to search through a 12 minute clip.

I had to use a fairly wide hue range for detecting the red text. That was because the message is is semi transparent so the color shifts a bit depending on the background. You may also need to adjust the average luma threshold. Higher if you get too many false positives, lower if you miss some kill messages.

Sample log.txt for the first 12 minutes of the youtube video (720p):

8820
10620
14040
24120
24300
34380
39420
43740

[onewaydown]

I downloaded part of the 720p video and came up with this:

LWLibavVideoSource("Star Wars Battlefront PS4.mkv") 
Crop(580,240,120,24)
SelectEvery(180) # every 180'th frame, 3 seconds at 60 fps
MaskHS(startHue=90.0, endHue=105.0, MinSat=30)
WriteFileIf(last, "log.txt", "AverageLuma>20", "current_frame*180" ) # multiply by 180 to get original frame number

That opens the video, crops it to just the area where the kill messages appear, looks at that area (only every 3 seconds) for saturated reds, and writes the frame number to log.txt if reds are found. That means it will generate false positives if other red objects appear in that area. It may miss some kills if the message lasts for less than 3 seconds.

Thanks for this! Now... what do I need to install/use to run that? I have Vdub64, downloaded a ffmpeg plugin so I can open mp4/mkv files, installed AviSynth and downloaded the L-SMASH-Works plugin (extracted to the avisynth plugin folder) - as I think that's what you're referencing. I also downloaded QuEnc, since I searched and saw some people using that to run the scripts. However, when I try and open the script, I get a variable not found error in Vdub and an AviSynth error saying no function for LWLibavVideoSource in QuEnc. So I think I'm missing... something.

Side note, what do you use to download the video from YT? Due to copyright from game music, I can't dl it from my account. I've used some websites like KeepVid but didn't know if there was an easier/better way.

[jagabo]

Now... what do I need to install/use to run that? I have Vdub64, downloaded a ffmpeg plugin so I can open mp4/mkv files, installed AviSynth and downloaded the L-SMASH-Works plugin (extracted to the avisynth plugin folder) - as I think that's what you're referencing.

It sounds like your general setup is correct. Install AviSynth, put LSMASHSource.dll in AviSynth's plugins folder so it's available automatically every time AviSynth is used.

I also downloaded QuEnc, since I searched and saw some people using that to run the scripts. However, when I try and open the script, I get a variable not found error in Vdub and an AviSynth error saying no function for LWLibavVideoSource in QuEnc. So I think I'm missing... something.

The error in VirtualDub may be because you used the wrong method of opening the script. Don't use File -> Run Script. Use File -> Open Video File.

The error with QuEnc probably indicates you have mismatched bitness -- for a 32 bit editor/encoder/player you need to use 32 bit AviSynth and 32 bit AviSynth filters. For a 64 bit editor/encoder/player you need 64 bit AviSynth and 64 bit AviSynth filters. Since you got as far as the AviSynth error message about the missing LWLibavVideoSource filter, AviSynth and QuEnc bitness matches, LSMASH must be the mismatch.

One note: you'll probably have to change the name of the file loaded by LWLibavVideoSource.

Oh, one other thing. I just noticed that the output of the script is Y800 -- 8 bit greyscale. Some programs may not accept that. If you have a problem try adding ConvertToRGB() or ConvertToYV12() at the end of the script.

Side note, what do you use to download the video from YT? Due to copyright from game music, I can't dl it from my account. I've used some websites like KeepVid but didn't know if there was an easier/better way.

I used youtube-dl. A command line driven downloader:

youtube-dl https://www.youtube.com/watch?v=ChIA5gmAkRI

By default it downloads the highest quality version. I aborted the download before it finished which left me with a somewhat corrupt mp4 file. So I remuxed the video into an MVK file. And used that for the AviSynth script.

[onewaydown]

The error in VirtualDub may be because you used the wrong method of opening the script. Don't use File -> Run Script. Use File -> Open Video File.

The error with QuEnc probably indicates you have mismatched bitness -- for a 32 bit editor/encoder/player you need to use 32 bit AviSynth and 32 bit AviSynth filters. For a 64 bit editor/encoder/player you need 64 bit AviSynth and 64 bit AviSynth filters. Since you got as far as the AviSynth error message about the missing LWLibavVideoSource filter, AviSynth and QuEnc bitness matches, LSMASH must be the mismatch.

One note: you'll probably have to change the name of the file loaded by LWLibavVideoSource.

Oh, one other thing. I just noticed that the output of the script is Y800 -- 8 bit greyscale. Some programs may not accept that. If you have a problem try adding ConvertToRGB() or ConvertToYV12() at the end of the script.

Ah, it must have been AviSynth. I know I got x64 for everything else and I didn't think about it when dling the normal installer for it.

Sooo... tried ASynth+/Vdub64 - Vdub hangs for a few seconds, then get "does not have a video stream" error. Tried the script with and without the path to the video in it. Put all the files in the Vdub folder too. Uninstalled AS+, went back to 32bit AS. Regular Vdub doesn't hang, just pops that same error.

[jagabo]

tried ASynth+/Vdub64 - Vdub hangs for a few seconds, then get "does not have a video stream" error. Tried the script with and without the path to the video in it. Put all the files in the Vdub folder too. Uninstalled AS+, went back to 32bit AS. Regular Vdub doesn't hang, just pops that same error.

That indicates everything is working but LWlibavVideoSource() isn't able to identify the video stream in the source file. What is the source? Post a MediaInfo report. If it's an AVI file try AviSource().

[onewaydown]

tried ASynth+/Vdub64 - Vdub hangs for a few seconds, then get "does not have a video stream" error. Tried the script with and without the path to the video in it. Put all the files in the Vdub folder too. Uninstalled AS+, went back to 32bit AS. Regular Vdub doesn't hang, just pops that same error.

That indicates everything is working but LWlibavVideoSource() isn't able to identify the video stream in the source file. What is the source? Post a MediaInfo report. If it's an AVI file try AviSource().

I was using an MP4 download from before at first but I deleted it and redownloaded it with that command line tool.
edit - to confirm, I downloaded this for the plugin - https://sourceforge.net/projects/virtualdubffmpeginputplugin/- and the vid opens fine in Vdub on its own.

General
Complete name                            : D:\Videos\VirtualDub\SWBF.mp4
Format                                   : MPEG-4
Format profile                           : Base Media / Version 2
Codec ID                                 : mp42 (isom/mp42)
File size                                : 8.08 GiB
Duration                                 : 6 h 33 min
Overall bit rate mode                    : Variable
Overall bit rate                         : 2 937 kb/s
Encoded date                             : UTC 2018-11-17 11:48:03
Tagged date                              : UTC 2018-11-17 11:48:03
Writing application                      : Google
gsst                                     : 0
gstd                                     : 23634454

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L3.1
Format settings                          : CABAC / 1 Ref Frames
Format settings, CABAC                   : Yes
Format settings, RefFrames               : 1 frame
Format settings, GOP                     : M=1, N=60
Codec ID                                 : avc1
Codec ID/Info                            : Advanced Video Coding
Duration                                 : 6 h 33 min
Bit rate                                 : 2 806 kb/s
Width                                    : 1 280 pixels
Height                                   : 720 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Variable
Frame rate                               : 30.000 FPS
Minimum frame rate                       : 18.286 FPS
Maximum frame rate                       : 54.662 FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.102
Stream size                              : 7.72 GiB (96%)
Title                                    : ISO Media file produced by Google Inc. Created on: 11/17/2018.
Encoded date                             : UTC 2018-11-17 11:48:03
Tagged date                              : UTC 2018-11-17 11:48:03
Color range                              : Limited
Color primaries                          : BT.709
Transfer characteristics                 : BT.709
Matrix coefficients                      : BT.709
Codec configuration box                  : avcC

Audio
ID                                       : 2
Format                                   : AAC LC
Format/Info                              : Advanced Audio Codec Low Complexity
Codec ID                                 : mp4a-40-2
Duration                                 : 6 h 33 min
Bit rate mode                            : Variable
Bit rate                                 : 128 kb/s
Channel(s)                               : 2 channels
Channel layout                           : L R
Sampling rate                            : 44.1 kHz
Frame rate                               : 43.066 FPS (1024 SPF)
Compression mode                         : Lossy
Stream size                              : 361 MiB (4%)
Title                                    : ISO Media file produced by Google Inc. Created on: 11/17/2018.
Encoded date                             : UTC 2018-11-17 11:48:03
Tagged date                              : UTC 2018-11-17 11:48:03

[jagabo]

Did you change the filename in the source line of the script to read:

LWLibavVideoSource("D:\Videos\VirtualDub\SWBF.mp4")

By the way, with mp4 files you can also use:

LSMASHVideoSource("D:\Videos\VirtualDub\SWBF.mp4")

instead of LWlibavVideoSource. That's is a little faster since it doesn't have to build an index file.

[onewaydown]

Did you change the filename in the source line of the script to read:

LWLibavVideoSource("D:\Videos\VirtualDub\SWBF.mp4")

By the way, with mp4 files you can also use:

LSMASHVideoSource("D:\Videos\VirtualDub\SWBF.mp4")

instead of LWlibavVideoSource. That's is a little faster since it doesn't have to build an index file.

I tried both with and without the D:\Videos\VirtualDub\ path. The script file and the vid are in the same folder. I tried the LSMASH version too - same error.

[onewaydown]

Man, this is annoying. AviSynth sample scripts run fine. VDub opens the video fine. Tried uninstalling/reinstalling AS/AS+ and LSMASH. Made sure all the bitnesses matched. Can't figure out what I'm doing wrong to get the 'no video stream' error. QuEnc seems to give me a module not found error, whether that helps. Are there any other tests or anything I can try?

[jagabo]

Try opening some other video files from other sources with LWlibavVideoSource(). Does it work with any of them? Try installing ffmpegSource and using that instead with ffVideoSource("D:\Videos\VirtualDub\SWBF.mp4").

http://avisynth.nl/index.php/FFmpegSource

Try remuxing the mp4 to MKV with MkvToolNix. Open the MKV with LWlibavVideoSource() or ffVideoSource(). Any luck?

As a last resort, try DirectShowSource("D:\Videos\VirtualDub\SWBF.mp4").

Oh, one other possibility, make sure you have write access to the folder with the video file.

[onewaydown]

First, thank you very much for your help on this. You're a gentleman and a scholar. Happy New Year to you!

Try opening some other video files from other sources with LWlibavVideoSource(). Does it work with any of them? Try installing ffmpegSource and using that instead with ffVideoSource("D:\Videos\VirtualDub\SWBF.mp4").

http://avisynth.nl/index.php/FFmpegSource

Try remuxing the mp4 to MKV with MkvToolNix. Open the MKV with LWlibavVideoSource() or ffVideoSource(). Any luck?

So, I tried an MP4 recorded direct from OBS, as well as remuxed MKV of the YT DL, and both gave me the same error for both L modules.

ffVideoSource seems to work with either MP4, the original YT DL or the OBS one. At least it opens and the crop sized boxes show for input and output windows. When I hit Space, it seems to be taking about 3 seconds to process each frame. I don't see any sort of 'preview' in those windows. They seem to stay black. Should I see the frame being processed? Checking the log.txt for the frames - I opened the MP4 directly in VDub and checked the frames, they didn't seem to match up.

Does that identify some sort of issue with how I've installed LSMASH? I downloaded the 32bit archive and extracted what was in the AviSynth folder into the AviSynth/plugins folder. Made sure x86 ver of Visual C++ Redist was installed (I repaired it, since it was already installed). I don't *think* I've messed up. =)

(side note, the downloaded MP4 seems to be 30fps -- I presume I should adjust the 180 values to 90 in the script?)

[jagabo]

Some progress, at least.

ffVideoSource seems to work with either MP4, the original YT DL or the OBS one. At least it opens and the crop sized boxes show for input and output windows... I don't see any sort of 'preview' in those windows. They seem to stay black.

Yes, you only see the small crop. It will be black where no red is detected, white where red is detected. If your videos aren't 1280x720 (like the youtube sample I downloaded) you will have to change the Crop() to reflect where the text appears in your video. The first two values are the location (in pixels) of the top left corner of the crop (x, y). The next two are the width and height of the box.

When I hit Space, it seems to be taking about 3 seconds to process each frame.

Pressing space plays the video in real-time. SelectEvery(180) discards 179 of every 180 frames, but it also reduces the frame rate to maintain the running time. So playing an hour long video with SelectEvery(180) will still take 3 hours. use File -> Run Video Analysis Pass to process the video more quickly -- VirtualDub will run through the video as quickly as AviSynth can process it.

Checking the log.txt for the frames - I opened the MP4 directly in VDub and checked the frames, they didn't seem to match up.

The MediaInfo report you posted showed a variable frame rate video. That may be the cause of the problem. VFR can be difficult to work with. You may be able to fix that by specifying the frame rate:

ffVideoSource("Star Wars Battlefront PS4.mkv", FpsNum=30, FpsDen=1) # or 30000 and 1001 for 29.97 fps

Does that identify some sort of issue with how I've installed LSMASH? I downloaded the 32bit archive and extracted what was in the AviSynth folder into the AviSynth/plugins folder. Made sure x86 ver of Visual C++ Redist was installed (I repaired it, since it was already installed).

It sounds like you did everything correctly. I don't know what the problem might be.

(side note, the downloaded MP4 seems to be 30fps -- I presume I should adjust the 180 values to 90 in the script?)

Yes. In both SelectEvery() and WriteFileIf().

[onewaydown]

Some progress, at least.

Yes, you only see the small crop. It will be black where no red is detected, white where red is detected. If your videos aren't 1280x720 (like the youtube sample I downloaded) you will have to change the Crop() to reflect where the text appears in your video. The first two values are the location (in pixels) of the top left corner of the crop (x, y). The next two are the width and height of the box.

The MediaInfo report you posted showed a variable frame rate video. That may be the cause of the problem. VFR can be difficult to work with. You may be able to fix that by specifying the frame rate:

ffVideoSource("Star Wars Battlefront PS4.mkv", FpsNum=30, FpsDen=1) # or 30000 and 1001 for 29.97 fps

I'll try those parameters. Otherwise, remuxing should set a constant framerate and I'll try that file instead.

Crop looks good. Ran an analysis pass and saw some name, or portions of names, showing up. Is there a way to set a threshold of the amount of pixels to determine a hit? I wondered if there was, maybe could ignore <10-20 pixels/percent or something to filter out false positives. (figured you would have thought of that already but doesn't hurt to ask, right?)

Thanks again, my good fellow!

[jagabo]

Ran an analysis pass and saw some name, or portions of names, showing up. Is there a way to set a threshold of the amount of pixels to determine a hit? I wondered if there was, maybe could ignore <10-20 pixels/percent or something to filter out false positives. (figured you would have thought of that already but doesn't hurt to ask, right?)

That's what the "AverageLuma>20" parameter is about. The output of MaskHS is black (luma = 0) and white (luma = 255) pixels. AverageLuma is the average luma value of all the pixels (0 to 255) within the cropped area. The higher the threshold the more red there has to be to trigger the log output.

The MaskHS parameters will also effect the red detection. As I noted earlier, the message box is semi transparent. So some of the background color blends with the red text, changing its hue and saturation a bit. So I had to use a fairly wide range of colors (startHue to endHue) and lowish saturation (MinSat) parameters: startHue=90.0, endHue=105.0, MinSat=30. Also, the fact that YV12 video stores the colors at half the resolution of the luma makes this problem worse.

Here's a UV plot of the chroma channels when the red text falls on a black background:


[Attachment 47707 - Click to enlarge]

The colorful graph at the lower left is the UV plot (the whole image has its saturation increased to make the colors more intense for demonstration purposes). The white spots indicate colors that appear in the frame. I've labeled the hue angles (0, 90, 180, 270). The center of the graph represents low saturation, the outer edges high saturation. The black background has low saturation so it produces most of the white dots near the center. The dots closer to the top of the graph along the ~110 degree line are the red text. The text is antialiased and blended with the black background producing all the dots in between along that line.

When the red text appears over a cyan background (pretty much the opposite of red) the reds are changed:


[Attachment 47708 - Click to enlarge]

the dots in the graph are shifted to the right and down. So I couldn't look just for dots at 110 degrees with high saturation. I had to widen the range to accommodate different backgrounds.

[pandy]

Perhaps Cr plane can be processed (it carry mostly red part) end eventually complementary Cb with negated value.

[onewaydown]

That's what the "AverageLuma>20" parameter is about. The output of MaskHS is black (luma = 0) and white (luma = 255) pixels. AverageLuma is the average luma value of all the pixels (0 to 255) within the cropped area. The higher the threshold the more red there has to be to trigger the log output.

The MaskHS parameters will also effect the red detection. As I noted earlier, the message box is semi transparent. So some of the background color blends with the red text, changing its hue and saturation a bit. So I had to use a fairly wide range of colors (startHue to endHue) and lowish saturation (MinSat) parameters: startHue=90.0, endHue=105.0, MinSat=30. Also, the fact that YV12 video stores the colors at half the resolution of the luma makes this problem worse.

Thanks for the detailed samples. It makes sense that the background would pull the hue one way or the other, so I get the idea here. Not fully, but I get what you're saying.
Sort of the same as Photoshop and masks and alpha channels -- another thing that I've never dived into using enough to understand but get the concept.

I'm still struggling to find the frames in the log where it picks up the hits, though. I ran the script, updated to 90 vs 180 frame adjustment, and got this as the first batch:

4410
5310
7020
12150
17190
19710
21870
24030
28890
36540
36720

Pretty much what you got, just halved because of the different framerate calculation. However, if I look via VDub, these seem to be the first frames with red kill text:

8301-8341
10192  (red laser bolt in that region, so possible hit)
11155-11192
11439-11475
11806-11845
11984-12020

At least going by the Frame status in VDub. There may be a couple other hits of red from player outlines or names above their character but I didn't really notice too many. Either way, there doesn't seem to be frames in the log that match up to that streak of 4 in about 30 seconds. I definitely see it working in VDub, seeing the usually semi-legible names popping up; however, I can't seem to figure out how the numbers line up. I've attached a clip, trimmed just after the last frame I was looking at. If you have a moment, could you check it out - see what you see/get?

[jagabo]

Using either ffVideoSource() or LWlibavVideoSource() with your cut.mkv I get the following list:

4410
5310
7020
12150

Opening the script minus all the processing I get a 12161 frame video with the following ranges for the kill messages:

4379 - 4478
5266 - 5366
6970 - 7068
12064 - 12160

All the log file entries fall within the observed ranges.

Opening the mkv file directly with VirtualDub I get 12150 frames with these ranges:

4375 - 4474
5261 - 5361
6963 - 7061
12052 - 12149

So VirtualDub and AviSynth are drifting further out of sync as the video progresses.

The AVS script shows the frame rate as 30.000 fps. AviSynth and MediaInfo show the frame rate as 29.97 fps. Changing ffVideoSource to:

ffVideoSource("cut.mkv", fpsNum=30000, fpsDen=1001)

Nearly syncs it to VirtualDub (within a frame or so) and shows 12149 frames with the following:

4410
5310
7020
12060

So I guess you need to force the frame rate in AviSynth.

[onewaydown]

Aahhhh, I see what's wrong. Aside from the drifting frames that might be happening, you're looking at the wrong on-screen message. You're cropping and looking at the big message in the middle of the screen -- which is the person killing me. =) I should have picked that up when you had the "YOURSELF" message in your UV samples. I'm looking for the smaller message of the person I've killed. If you look back at the pic in one of the firsts posts, it's that little message a little more down and to the right. I'll mess with the crop settings and then force the frame rate in the script and see how that works out.

Thanks a ton, and sorry for any confusion or anything!

edit - I think Crop(820,336,200,16) might be good. The hue settings might have to change, though, since this message text isn't as large or red as the one you've been looking at.

[jagabo]

Sorry about the confusion! Yes, the correct text is smaller and hence less saturated and more shifted in hue. Its location is more variable so you have to use a bigger box to make sure you get it. And it appear for shorter durations. I came up with this:

LWLibavVideoSource("Star Wars Battlefront PS4.mkv", fpsNum=60000, fpsDen=1001) 

TOP_LEFT_X = 828
TOP_LEFT_Y = 336
BOX_WIDTH = 172
BOX_HEIGHT = 16

Crop(TOP_LEFT_X, TOP_LEFT_Y, BOX_WIDTH, BOX_HEIGHT)
SelectEvery(60)
Sharpen(1.0)
MaskHS(startHue=75.0, endHue=105.0, MinSat=30)
WriteFileIf(last, "log.txt", "AverageLuma>10", "current_frame*60")

That generates several false positives but I think it gets all the kill messages.

[onewaydown]

Sorry about the confusion! Yes, the correct text is smaller and hence less saturated and more shifted in hue. Its location is more variable so you have to use a bigger box to make sure you get it. And it appear for shorter durations. I came up with this:

LWLibavVideoSource("Star Wars Battlefront PS4.mkv", fpsNum=60000, fpsDen=1001) 

TOP_LEFT_X = 828
TOP_LEFT_Y = 336
BOX_WIDTH = 172
BOX_HEIGHT = 16

Crop(TOP_LEFT_X, TOP_LEFT_Y, BOX_WIDTH, BOX_HEIGHT)
SelectEvery(60)
Sharpen(1.0)
MaskHS(startHue=75.0, endHue=105.0, MinSat=30)
WriteFileIf(last, "log.txt", "AverageLuma>10", "current_frame*60")

That generates several false positives but I think it gets all the kill messages.

I sort of expect false positives, so that's fine. Thanks for the tweaks. For some reason, I'm getting 'no video stream' errors with my old script files or the one with this new script. Not sure what I broke - ASynth samples still work. Gotta head to work so I'll have to sort that out and test this later on.

Thx again!

[jagabo]

Here's a further modification that shows the original image as well as the map of red:

LWLibavVideoSource("Star Wars Battlefront PS4.mkv", fpsNum=60000, fpsDen=1001) 

TOP_LEFT_X = 828
TOP_LEFT_Y = 336
BOX_WIDTH = 172
BOX_HEIGHT = 16

Crop(TOP_LEFT_X, TOP_LEFT_Y, BOX_WIDTH, BOX_HEIGHT)
SelectEvery(60)
Sharpen(1.0) # increases saturation of small text
img = last # remember image
MaskHS(startHue=75.0, endHue=105.0, MinSat=30) # find red
WriteFileIf("log.txt", "AverageLuma>10", "current_frame*60") # write to log file if more than 10 of 255 are red
bigmap = PointResize(BOX_WIDTH, BOX_HEIGHT) # restore original width/height -- MaksHS halved it
ConvertToYUY2(img).VideoScope("both", true, "U", "V", "UV") # add UV plot

bigmap = bigmap.AddBorders(0,0,last.width-bigmap.width,0).ConvertToYUY2() # extend red map to right for stacking
StackVertical(bigmap,last) # stack the red map and the image+graph
Tweak(sat=2.0, bright=-30) # more saturated colors for viewing

Note the wider range in MaskHS() and lower threshold in WriteFileIf().


[Attachment 47763 - Click to enlarge]

At the top left you can see map of reds found by MaskHS. Right below that you can see the original crop from which the map was built. At the bottom right is the UV plot. Log.txt:

8460
8520
16620
19920
22320
22860
22920
23640
24000
29100
38880
41040

'no video stream' errors...

Are you using LSMASH or ffVideoSource().

[onewaydown]

'no video stream' errors...

Are you using LSMASH or ffVideoSource().

Ah, yup. Forgot you were working with LS and didn't catch that. All good, works great. =) I appreciate your expertise, my good man. *tips fedora* Again, your thorough explanation goes a tad over my head ATM but you've given me the tools I need, once I wrap my head around it. One thing I'm an expert at in life is learning the things I need as I need them. Never needed to work with video to this level - so it's a good learning experience.

Next stop on my pipedream list -- seeing how I can automatically direct stream trim and export X number of seconds/frames before and after these keyframes. I assume it's as easy as setting like variable1 as frame-500 and variable2 as frame+300, then use a TRIM(variable1, variable2). Can that be done during this script processing (so rather than WriteFileIf, do a TRIM), or better to do it separate after having the list of frames (like do I have to load the video differently)?