How to have multiple videos at different framerates inset within 1 frame

[archiveresearch]

I have a couple hours of footage from two different cameras. JVC GZ-MG130u and Nikon Coolpix 3200. The JVC gives me 29.97 fps, and the Coolpix 15 fps.

Mediainfo for both follows.

JVC:

G: MPEG-PS, 45.6 MiB, 39 s 552 ms, 9 662 kb/s

V: MPEG Video, Main@Main, 720x480, 29.970 FPS, 9 086 kb/s

A: AC-3, 384 kb/s, 2ch, 48.0 kHz

M: Menu


General

Complete name                   : MOV07A.MOD
Format                          : MPEG-PS
File size                       : 45.6 MiB
Duration                        : 39s 552ms
Overall bit rate mode           : VBR
Overall bit rate                : 9662 Kbps
FileExtension_Invalid           : mpeg mpg m2p vob vro pss evo

Video

ID                              : 224 (0xE0)
Format                          : MPEG Video
Format version                  : Version 2
Format profile                  : Main@Main
Format settings                 : CustomMatrix / BVOP
Format settings, BVOP           : Yes
Format settings, Matrix         : Custom
Format settings, GOP            : M=3, N=15
Format settings, picture structure: Frame
Duration                        : 39s 540ms
Bit rate mode                   : VBR
Bit rate                        : 9086 Kbps
Maximum bit rate                : 8900 Kbps
Width                           : 720 pixel
Height                          : 480 pixel
Display aspect ratio            : 4:3
Frame rate                      : 29.970 (30000/1001) fps
Standard                        : NTSC
Color space                     : YUV
Chroma subsampling              : 4:2:0
Bit depth                       : 8 bit
Scan type                       : Interlaced
Scan order                      : TFF
Compression mode                : Lossy
Bits/(Pixel*Frame)              : 0.877
Time code of first frame        : 00:00:00:00
Time code source                : Group of pictures header
GOP, Open/Closed                : Closed
Stream size                     : 42.8 MiB (94%)
Color primaries                 : BT.601 NTSC
Transfer characteristics        : BT.601
Matrix coefficients             : BT.601

Audio

ID                              : 189 (0xBD)-128 (0x80)
Format                          : AC-3
Format/Info                     : Audio Coding 3
Commercial name                 : Dolby Digital
Muxing mode                     : DVD-Video
Duration                        : 39s 552ms
Bit rate mode                   : CBR
Bit rate                        : 384 Kbps
Channel(s)                      : 2 channel
Channel layout                  : L R
Sampling rate                   : 48.0 KHz
Frame rate                      : 31.250 fps (1536 SPF)
Compression mode                : Lossy
Stream size                     : 1.81 MiB (4%)
Service kind                    : Complete Main

Coolpix:

G: QuickTime, 10.3 MiB, 20 s 800 ms, 4 145 kb/s

V: JPEG, 640x480, 15 FPS, 4 060 kb/s

A: English, PCM, 64.0 kb/s, 1 channel, 7 875 Hz


General

Complete name                   : DSCN3598.MOV
Format                          : QuickTime
Format/Info                     : Original Apple specifications
File size                       : 10.3 MiB
Duration                        : 20s 800ms
Overall bit rate mode           : VBR
Overall bit rate                : 4145 Kbps
Movie_More                      : NIKON DIGITAL CAMERA
Encoded date                    : UTC 2021-05-04 19:46:24
Tagged date                     : UTC 2021-05-04 19:46:24
Writing library                 : Nikon
Origin                          : Digital Camera

Video

ID                              : 1
Format                          : JPEG
Codec ID                        : jpeg
Duration                        : 20s 800ms
Bit rate mode                   : VBR
Bit rate                        : 4060 Kbps
Width                           : 640 pixel
Height                          : 480 pixel
Display aspect ratio            : 4:3
Frame rate mode                 : CFR
Frame rate                      : 15.000 fps
Color space                     : YUV
Chroma subsampling              : 4:2:2
Bit depth                       : 8 bit
Compression mode                : Lossy
Bits/(Pixel*Frame)              : 0.881
Stream size                     : 10.1 MiB (98%)
Language                        : English
Encoded date                    : UTC 2021-05-04 19:46:24
Tagged date                     : UTC 2021-05-04 19:46:24

Audio

ID                              : 2
Format                          : PCM
Format settings                 : Little / Unsigned
Codec ID                        : raw
Duration                        : 20s 800ms
Bit rate mode                   : CBR
Bit rate                        : 64.0 Kbps
Channel(s)                      : 1 channel
Sampling rate                   : 7875 Hz
Bit depth                       : 8 bit
Stream size                     : 160 KiB (2%)
Language                        : English
Encoded date                    : UTC 2021-05-04 19:46:24
Tagged date                     : UTC 2021-05-04 19:46:24

If I want, say, JVC footage as a background with an inset of Coolpix footage, would my only options be to find a way to effectively double the 15fps footage (which I'm sure would be absolutely jarring, unless someone knows some QTGMC script I don't) or reducing the framerate of the JVC?

I'm guessing there's no way to let the footage coexist, within the same frame, in its native form?

I'm using Shotcut for my NLE. I refuse to adopt the Adobe model of monthly payments, and I prefer open source software anyway.

[poisondeathray]

You can try motion interpolation of 15fps to 29.97fps using RIFE or mvtools2 or similar - basically "optical flow" methods . They synthesize "inbetween" frames , and can sometimes look ok, but other times can produce distracting artifacts. It depends on the source content

[Bwaak]

Shooting at 15 fps in 2021? Your MOD file is 30i, so you need to quadruple Coolpix's frame rate if you want frame rates to match, not just to double.

would my only options be to find a way to effectively double the 15fps footage (which I'm sure would be absolutely jarring, unless someone knows some QTGMC script I don't)...

QTGMC is a deinterlacer. Why would you want to deinterlace 15p footage?

...or reducing the framerate of the JVC?

You can reduce 30i to 30p or 15p, or increase 15p to 30p or 30i or 60p, whichever.

I'm guessing there's no way to let the footage coexist, within the same frame, in its native form?

Why would not you try instead of guessing? You don't need to change frame rate simply to have an inset, your NLE will adjust frame rate for you. There may be settings, like either simply duplicate frames or interpolate them.

[jagabo]

If I want, say, JVC footage as a background with an inset of Coolpix footage, would my only options be to find a way to effectively double the 15fps footage (which I'm sure would be absolutely jarring, unless someone knows some QTGMC script I don't) or reducing the framerate of the JVC?

The simplest way would be to use ChangeFPS(30000, 1001) on the 15 fps clip. That will double each frame (except every 1000th) and make a 29.97 video out of the 15 fps video. Then you Overlay() the (previously) slower clip onto the faster clip. Of course the slower clip will be jerky.

slowclip = WhateverSource("15fpsclip.ext")
fastclip = WhateverSource("29.97fpsclip.ext")

slowclip = slowclip.ChangeFPS(30000, 1001) # or fastclip.framerate

Overlay(fastclip, slowclip, x=?, y=?)

# encode interlaced

Or you could use AssumeFPS(30000, 1001) on the slower clip to make it run ~twice as fast. And Overlay. The slow clip will now be smoother but twice as fast.

slowclip = WhateverSource("15fpsclip.ext")
fastclip = WhateverSource("29.97fpsclip.ext")

slowclip = slowclip.AssumeFPS(30000, 1001) # or fastclip.framerate, twice as fast, half as long

Overlay(fastclip, slowclip, x=?, y=?)

# encode interlaced

Or, as mentioned earlier, you can try using motion interpolation. That can work well for panning shots but may deliver distortions with complex motions.

slowclip = WhateverSource("15fpsclip.ext")
fastclip = WhateverSource("29.97fpsclip.ext")

slowclip = slowclip.FrameRateConverter(NewNum=30000, NewDen=1001)

Overlay(fastclip, slowclip, x=?, y=?)

# encode interlaced