Scanned 16mm Optical Soundtrack Direct to Digital Audio?

[mtt1853]

I was wondering if it were technically possible to convert a scanned image of a 16mm optical audio track (example) directly to a digital audio file. I figure this would give the best possible conversion and the highest audio fidelity.

The advice I've heard thus far from others is just to run the film through a projector, record the audio, and manually synch it with the video afterward, but this seems like it would lose a lot of quality in the process. Any ideas?

Thanks,
Mike

[jagabo]

Technically possible? Of course. Would it sound any better? I'm Doubtful. I suppose it would depend on the quality of the scan vs the quality of the projectors audio circuitry.

[filmboss80]

Agree with jagabo. The optical soundtrack, as printed along the side of the film, is going to have its signal range limits that cannot really be surpassed. Plus, you still have artifacts from dust spots and film scratches, whether the waveform is scanned or run through the standard optical reader.

You're not the first person to consider the idea:
http://osdir.com/patents/Optics-motion/Correction-optical-film-soundtrack-deficiencies-07053978.html

But I doubt you'll find a practical scanning system readily available.

Consider this: Before being printed to an optical soundtrack, all 16mm film audio tracks were recorded to magnetic fullcoat. Perhaps you can investigate to see if the film's fullcoat still exists. Audio readers are not that difficult to find. However, if that is not practical, your best bet is to record a high quality wav file of the film's projected soundtrack, and just try to EQ and clean it up the best you can with a good audio editor. (That's what I do.)

[mtt1853]

At least with a scanned image, I would be able to clean up any scratches or dust spots before running the process. I would think defects like that would be much harder to correct after they are converted to audio without adversely affecting the sound.

Not sure any magnetic audio tracks exist for these films, since many of them date back to the '30s. Let me know if anyone has any tips on software that might be able to do this.

Thanks again,
Mike

[filmboss80]

Films of the '30s did not have magnetic sound recordings; so, no, there would not be a fullcoat audio track. (I didn't realize your films were that old.)

You can still digitally clean up pops, crackles, and hisses once you digitize the soundtrack. It can be a time consuming process, and absolute perfection cannot be achieved. Still, you can improve what you have if you work hard enough at it.

Start out with Audacity (as it is free, works well, and can use a variety of filters).

[jagabo]

A quick back-of-the-napkin calculation: The resolution of film is about 2000 dpi. A 16mm film frame is about half an inch tall. So the audio for each frame has a bandwidth of roughly 1000 Hz.

[mtt1853]

So I take that to mean the audio fidelity is junk to begin with, so don't bother going through so much trouble to try and squeeze a tiny bit more out of it?

[jagabo]

Yes.

[Khaver]

You might try taking a feed directly from the optical reader on the projector into a mic or line level input on your sound card. This way you're skipping the projectors preamps.

[Des]

I don't have any advice to give here, but I am very curious to learn how you would insert the scanned waveform into an audio program to clean it up and play etc?

I don't mean to hijack this thread, so feel free to PM me.

--dES

[Cornucopia]

A quick back-of-the-napkin calculation: The resolution of film is about 2000 dpi. A 16mm film frame is about half an inch tall. So the audio for each frame has a bandwidth of roughly 1000 Hz.

Actually, I think it's more like 12,000Hz.

2000 dpi for 1" = 2000 "Dots" or Samples, for 1/2" = 1000 Samples (for 1 Frame).
24 Frames go by the scanner per second = 24,000 Samples/second.
According to the Nyquist theorem, that would give you ~12,000Hz bandwidth. About what I would expect for Analog 16mm optical.
(I'd be curious as to what the dynamic range would be...)

You might try taking a feed directly from the optical reader on the projector into a mic or line level input on your sound card. This way you're skipping the projectors preamps.

I'm guessing optical readers DON'T output the appropriate range & voltage for what's needed for Line level input to an A-to-D. That's what pre-amps are for.

Scott

[filmboss80]

I don't have any advice to give here, but I am very curious to learn how you would insert the scanned waveform into an audio program to clean it up and play etc?

That's a point I didn't initially have time to make to the OP, who seems to be under the impression that common audio software somehow reads a graphic interpretation of a sound waveform. That's not to say that some sort of waveform interpretation software--similar to optical film readers--may be developed (as I alluded to in an earlier link). I just haven't seen anything available--or reasonably feasible. With most audio software, digital sound recordings can be represented visually as waveforms, but the software is not reading some sort of waveform graphic, per se.

[jagabo]

I made a mistake in my earlier calculation. I forgot to account for 24 frames per second. So the correct calculation is: About 2000 dpi, about 1/2 inch per frame, and 24 frames per second -- so there are 24000 samples per second. So a direct optical transfer might be a little better than a projector's output.

I am very curious to learn how you would insert the scanned waveform into an audio program to clean it up and play etc?

You can convert the width of the white portion of each line to an analog voltage. Those become your sequential audio samples. I trimmed the image from mtt1853's link down to a 128x2271 grayscale image of just the audio track, then wrote a quick program to sum* the pixels across each scanline, and wrote the new samples out as 16 integers. I then imported those samples into Audacity as raw data. In the original image each frame was about 500 scan lines tall. The equates to about 12 KHz samples. In Audacity I normalized the waveform and resampled to 44.1 KHz. The sample is only about 1/5 second long but you can tell it's some sort of orchestral (brass band?) piece. Which make sense since it's from the closing credits of a video. Sample attached.

* Basically you have white signal pixels against a black background across each scanline. So summing them is largely equivalent to finding the width of the white portion and a lot easier. The background wasn't entirely black so this leaves some DC offset which was accounted for in Audacity's Normalize filter. This is probably very similar to the way the optical to audio converter works on the projector. A thin slit allows the lamp to project a line of the waveform onto an optical sensor that converts the amount of light received to a voltage. That voltage is then amplified and sent to the speaker.

[Des]

Thank you for the feedback, filmboss80.
Jagabo, I say this with admiration, you and many others here amaze me!

Am looking forward to learning how this project turns out for the OP.

--dES

[elmuz]

Oh my god jagabo this is amazing!

[filmboss80]

Oh my god jagabo this is amazing!

Indeed! You may be onto something, jagabo. I wonder about the estimated time involved in creating a wav file for something like a 1200ft 16mm film using this process, and whether the finished output is any better than what you would get from a projector's optical sound reader. Even if it is not practical for whole films, it may still provide a means to clean up small sections of poor audio in a film.

Hope you have time to do some more experimenting.

[Khaver]

I'm guessing optical readers DON'T output the appropriate range & voltage for what's needed for Line level input to an A-to-D. That's what pre-amps are for.

That's why I said MIC or line level input

[jagabo]

I wonder about the estimated time involved in creating a wav file for something like a 1200ft 16mm film using this process

Yes, with your typical flatbed scanner it would be quite a pain scanning 14 inches at a time, then having to align all the individual scans, etc. And most flatbeds don't have the kind of resolution you need for this. A purpose-built scanner would be better. And in that case you wouldn't save the scanned data, just convert directly to PCM samples, line by line.

Hope you have time to do some more experimenting.

I don't have access to any other samples to work with. The one that was provided was low resolution, blurry at the edges, very noisy, and highly JPG compressed. I don't think it's worth doing anything more with it. I just thought it was a fun little proof of concept project.

[olyteddy]

I'm guessing optical readers DON'T output the appropriate range & voltage for what's needed for Line level input to an A-to-D. That's what pre-amps are for.

From my experience with the local film society they are 'Microphone Level' and heavily bass biased. Our first 'Surround Sound' festival we fed a borrowed stereo optical pickup into a mixer (balanced mic inputs) and out to a Sony Pro-Logic receiver.

[Spiny Norman]

My guess is, it's possible but doesn't exist, so it would have to be built from scratch! Wouldn't it be better to just use a conventional solution but to make sure you use good playback equipment?

Out of interest, what kind of films are they? In the late twenties films had sound on disc, which reportedly was better quality.

[mtt1853]

Well everybody, I've been busy since my first post. I tried something similar to jagabo's solution, adding up the amount of white in any given line and making that an individual sample. I started with a very large (35000 x 3000 pixel) TIFF image of 24 frames of film. I'm including 2 frames of it here in JPG format so you can see what I'm working with. I was lucky on this particular clip to have a duplicated soundtrack, more on that in a second.

I wrote a program that would assemble all the samples and create a sound clip from them. As you can see from the image, there is kind of a lot of dust on there obscuring the soundtrack in places. Since the top and bottom soundtracks are identical to each other (they are not stereo), I overlaid one on top of the other, did a lighten mask much like in Photoshop, and all the dust magically disappeared!

I used the open source NAudio library to generate a WAV file out of my result, making it 35kHz (since it was already 24 frames and 35000 pixels wide), and the result actually sounds pretty good! I think it can be cleaned up further in an audio program but at least I'm starting with as good of a source material as I can.

I think I will try to do this for all films I digitize since many of them are in poor shape with broken sprocket holes and cannot be run through a projector at all.

Thanks for your input everybody, this has been pretty fun so far.

Mike

[Edit: the audio sample is of Louis Armstrong saying "and we gonna swing"]

[jagabo]

How does it sound compared to playing the film in a projector and recording that?

[mtt1853]

I'm not sure... This is one of those films that has broken sprocket holes, so running it through my projector only damages the film further. I will ask a friend into film restoration to see what they say. In the meantime I'll try to scan a different film with healthy sprocket holes and compare to the projector-recorded track.

[jagabo]

Thanks, I'd like to hear how the two methods compare. Scanning the film manually seems like an awful lot of work!

[mtt1853]

Actually I am planning on making an automated film advancer so I don't have to manually sit there for hours on end. I may need a little bit of luck, but this will definitely be a fun project!

[roeltje]

Hi everyone,

This is really amazing. Hearing Jagabo's Orchestral sample and Mike's Louis Armstrong saying "and we gonna swing".

I have a frame-by-frame scanner which makes an uncompressed image of each film frame (8mm, 9,5mm, 16mm, and even 17,5mm).
When I zoom out the digital camera I can scan the whole picture frame, even with perforation holes and soundtrack. Or I can scan just the sound past. Our framescanner scans at aroud 15-27 frames per second. I use virtualdub to join all the pictures together and make an uncompressed AVI film.

Because old film tends to schrink and be unstable, this results in the issue that all the images might not be in the exact absolute center of the picture which means that the visual audio track might not be 100% aligned in each sequential picture. There can therefore be some vertical and/or horizontal movement between two sequential frames.

I'd be willing to scan your whole 16mm film for you to test with.

Best regards,
Roel

[davealtson]

There is actually an open-source project for doing this:
http://www.aeosound.org/
With tech details here:
http://www.neh.gov/files/grants/university_of_south_carolina_open-source_application_f...of_optical.pdf

[netmask56]

I had a job, in a public broadcasting organisation, to restore the dialog as best I could of an important 1930's optical soundtrack that had been badly recorded in the first place and quite badly damaged. We were presented with a new print so we didn't have to deal with broken sprockets etc. Playback was via a MagnaTech machine. Normal equalisation did next to nothing but increase noise and distortion. What I finally did out of desperation use an UREI low pass filter 40db/octave set at 3Khz and I think from memory we knocked off the lows below 120Hz followed by a gentle setting downwards expander (a Kepex) and then as an experiment used an Eventide Harmoniser. Fiddled with the controls creating faux harmonics above 3Khz based on the fundamental structure below that and we ended up with an acceptable track for broadcast on the evening news.

Now from a legit archival perspective what we did was a no no but we did have something that was intelligible and with clear dialog. The item was only about 5 minutes long. Normally harmonisers are used in the music world to brighten up strings and woods that treble boost just doesn't cut it and mostly in the pop genre - anyway if you can lay your hand on one it's worth a try.

[Cornucopia]

Harmonizers are great, but you don't need to go to that expense: There are a number of pitch changers/doublers out there, either based on principle of tempo-adjusted sample conversion, or of double ring modulation, or of harmonic distortion, or of (my personal trick) rectified waveforms, or a combo of the 4. Some are freeware. Just have to know how to use them.
For bandwidth extension (like the famous "Aphex Aural Exciter"), I also suggest you precede the effect with a sidechain that bandpass filters the top existing octave, so that what gets generated in ONLY in the extended octave and doesn't intermodulate with existing frequencies.
Note: a little goes a LONG way.

@netmask56, titles like those it makes sense to get both a "clean" capture, for archival's sake, and a processed, optimized capture for the task at hand. Although it sounds like that master was very usable, so many of them aren't, so in their case, it makes the most sense to get MORE than necessary the first time around - the masters might not hold up for multiple passes.
One can always digitally filter down later.

Scott

[netmask56]

I was relating an experience from well over 20 years ago when the Eventide Harmoniser was the only one on the market and indeed the Kepex noise gate had just been released - anyone remember "Hi I'm Alison!" from their promotional 45rpm disc? LOL now... and being a public broadcaster we had no budget problems in those days. And a little goes a long way couldn't agree more - always a light touch on equalisation. I have to laugh when I see home setups with the bass and treble turned all the way up - pity there isn't a midrange that could be turned all the way down - once had a valve amp and preamp (local Sydney company) that had an "absence" control.
I used to put a 3~4db midrange dip in the backing tracks so that the vocal could sit nicely there without having to use too much eq on the voice.... lots of old techniques work in lieu of advanced technology...