Colorizing black & white movie

[poisondeathray]

Wow, I sure am glad I posted.

I'm am completely surprised by both the tone and the negativism of the replies posted above. I am especially puzzled because even traditional B&W colorization is pretty awful and, IMHO, a complete waste of time. In fact, I avoid all colorized movies because they look so bad. To criticize this method by saying that it sometimes chooses the wrong color is completely ludicrous because the "real" colorization method gets it wrong all the time. What's more, while the colors produced by this method seem to have a purplish cast, and don't perfectly track the object, that has been my experience with colorized movies as well.

The differences are a matter of degree.

So, for the third time, for those who apparently didn't read my two previous posts or simply chose to ignore them for some reason, I am making no claim that this is some sort of ultimate colorization technology, and it has tremendous limitations and shortcomings . Unfortunately I have a sneaking suspicion that the same people will once again ignore this statement, even though I have shouted it this time by putting it in boldface.

Finally, to get back to the main point, this is almost certainly the only method available to some guy (the OP) who wants to colorize his own home movies. What else can he do on a zero budget, and with little time to spend? Certainly no one else in this thread , including those who just posted above, have suggested any other technique that he is likely to be able to use, including the two who seem to have gotten upset, for reasons that are completely unclear to me.

So, calm down, and let the OP take a look and decide for himself. If he too thinks the technology is useless, let him make that decision. I actually don't understand the point of your "warnings" for newbies. Are you saying that they'll try this and think it is great? If they do actually try it, and if they like the results, isn't that a good thing? To say otherwise -- that you know better than they do about what they should like -- is the definition of snobbery.

I'm just trying to help the guy, so please, give us all a break!!

This is a discussion board, we are discussing things. Everything is calm, what makes you think otherwise ?

Not everything has to be viewed through rose colored glasses. What you call "negativity" is I would call "realistic".

Yes, it's good to provide more options . But realistically it's useless for real colorization work. It's way off the mark. This is nothing more than a toy. Yes, let him decide if it's something worth investigating. Maybe you're confusing the wrong thread but the OP didn't say anything about zero budget. He asked for shot by shot, like professional colorization tools. This is neither of those .

I would like to colorize one of my w&b movies.
I'm looking for a software dedicated to doing that job.

However, I do not want to colorize the fames one by one.
I'm looking for a software that does the job shot by shot.
This is how the pros do it these days, but not sure this is available to the general public.

The script itself is pretty interesting because a few lines of code can "guess" the colors. But for actual work - not very useful

I agree colorization is terrible in general, it's not for everybody. If it's not done well it's much worse than B&W . That script produces results in that terrible category. Just my opinion

[bmick23]

Truth be told, you're not going to find anything (even professional grade) that will automate the process to the extent you are asking. "The pros" don't do it that way, either. Footage is usually outsourced to labs in India where every object in the frame is individually rotoscoped and sent back to the studio where an artist adds color. Automation is going to give you messy results and if you want a clean image on a budget, you're going to have to do it yourself. The colorization scripts are interesting and cool to play with, but they do not provide consistent, controllable or accurate results.

Although some here disagree, the best method for colorization on a consumer level is After Effects and Mocha. Look it up and don't just bash the method without seeing how it works. I use this program almost every day and it is pretty incredible to be able to perform full motion tracking and mask creation on a laptop with no lag. Yes, it is time consuming, but look at what you are asking for. You get out of it what you put into it, plain and simple. There is not a magic button or secret trick for this. You're trying to create footage that tricks your eyes into believing you are looking at real, color footage and doing that means you will be doing a lot of detail work.

[MWilson]

Wow, I sure am glad I posted.

I'm am completely surprised by both the tone and the negativism of the replies posted above. I am especially puzzled because even traditional B&W colorization is pretty awful and, IMHO, a complete waste of time. In fact, I avoid all colorized movies because they look so bad. To criticize this method by saying that it sometimes chooses the wrong color is completely ludicrous because the "real" colorization method gets it wrong all the time. What's more, while the colors produced by this method seem to have a purplish cast, and don't perfectly track the object, that has been my experience with colorized movies as well.

The differences are a matter of degree.

So, for the third time, for those who apparently didn't read my two previous posts or simply chose to ignore them for some reason, I am making no claim that this is some sort of ultimate colorization technology, and it has tremendous limitations and shortcomings . Unfortunately I have a sneaking suspicion that the same people will once again ignore this statement, even though I have shouted it this time by putting it in boldface.

Finally, to get back to the main point, this is almost certainly the only method available to some guy (the OP) who wants to colorize his own home movies. What else can he do on a zero budget, and with little time to spend? Certainly no one else in this thread , including those who just posted above, have suggested any other technique that he is likely to be able to use, including the two who seem to have gotten upset, for reasons that are completely unclear to me.

So, calm down, and let the OP take a look and decide for himself. If he too thinks the technology is useless, let him make that decision. I actually don't understand the point of your "warnings" for newbies. Are you saying that they'll try this and think it is great? If they do actually try it, and if they like the results, isn't that a good thing? To say otherwise -- that you know better than they do about what they should like -- is the definition of snobbery.

I'm just trying to help the guy, so please, give us all a break!!

This is a discussion board, we are discussing things. Everything is calm, what makes you think otherwise ?

Not everything has to be viewed through rose colored glasses. What you call "negativity" is I would call "realistic".

Yes, it's good to provide more options . But realistically it's useless for real colorization work. It's way off the mark. This is nothing more than a toy. Yes, let him decide if it's something worth investigating. Maybe you're confusing the wrong thread but the OP didn't say anything about zero budget. He asked for shot by shot, like professional colorization tools. This is neither of those .

I would like to colorize one of my w&b movies.
I'm looking for a software dedicated to doing that job.

However, I do not want to colorize the fames one by one.
I'm looking for a software that does the job shot by shot.
This is how the pros do it these days, but not sure this is available to the general public.

The script itself is pretty interesting because a few lines of code can "guess" the colors. But for actual work - not very useful

I agree colorization is terrible in general, it's not for everybody. If it's not done well it's much worse than B&W . That script produces results in that terrible category. Just my opinion

As author of said script, I take offense to the "useless" and "toy" statements. Actually most of your post. I wrote the script because I was in the OPs situation. I've said time and time again elsewhere that it is far from perfect, so in that we agree. But a useless toy it's not.

[poisondeathray]

As author of said script, I take offense to the "useless" and "toy" statements. Actually most of your post. I wrote the script because I was in the OPs situation. I've said time and time again elsewhere that it is far from perfect, so in that we agree. But a useless toy it's not.

Sorry I don't mean to offend. Don't get all worked up and make it personal, I'm giving an honest review of the tool. Not you. You're not the "tool" (at least I hope...)



My comments are in the context of professional colorization work. If you've done real (accurate) colorization work with other programs, you'd realize this is hindering, not helping.

If the masks were accessible in other programs, then it would might shave of a few % of work as a starting point. As it is, it makes work harder, not easier - you have to spend time fixing problems. Maybe it can be improved, I hope it can.

Please re-read what the OP was really looking for. It's 100% definitely not this filter. To reiterate "He asked for shot by shot, like professional colorization tools." This is neither of those . As such, it's "useless" and a "toy" in that context . It really is.

This is fine for something fun and quick, pretty amazing what a few lines of code can do. But that' s not what the OP was looking for

[MWilson]

"However, I do not want to colorize the fames one by one."

Sounds like it to me.

And FWIW, it has improved.

[poisondeathray]

And FWIW, it has improved.

Great, thanks. I'll check it out again soon

[TFLeonard]

I read a very interesting article in an nVidia newsletter that described what I think is a neural network approach to having AI make an "educated guess" as to what color grayscale pixels are in an image based on similar color scenes. This is basically a training method for the AI where human input "teaches" the program that the sky is blue, the grass is green or a post box is red.

I was especially interested since I am trying to restore badly faded 16mm color movies. Since I have examples of what the color should be in many scenes, this method sounds promising.

Unfortunately I can no longer find any reference to this research and I seem to have lost the original article. Anyway if anyone has further info on this technique, or hopefully information on software that'll do this, please share!

[themaster1]

I was especially interested since I am trying to restore badly faded 16mm color movies. Since I have examples of what the color should be in many scenes, this method sounds promising.

Have you tried the rgbadapt plugin ? see here
original french topic (with examples) see here

[TFLeonard]

Thanks for the links, at the very least it was good at exercising my French reading.

I use the Adobe CC programs so I'm a bit hesitant to invest a lot of time and effort in the software they're using. Can these things be done in After Effects?

In my situation most of the 16mm Kodacolor films have lost their Cyan and Yellow dyes leaving mainly magenta. I've fiddled around in Speedgrade, After Effects, Premiere Pro and finally Photoshop with weak results. The main problem I find is that there simply isn't enough Cyan and Yellow information left for the tools to work with. Manual colorization may be a way to re-introduce obvious colors, but I hate the tinted look and I really want it to look realistic.

I do also have a Super 8 copy of the 16mm film that was made when the dyes were in better shape, so although fading, its closer to the full color look. Unfortunately this transfer is severely cropped by around 10% around the edges of the frame.

Long story short I'm trying to blend the color from the Super 8 version with the grayscale(luminosity) of the 16mm film transfer and the results are pretty good. The tedious part is manually cloning parts of the super 8 frame to fill in the cropped areas frame by frame in Photoshop.

I'm hoping to get at least a more full color rendering at the end of this exercise, which may be better suited to more traditional color grading techniques.

[jagabo]

AviSynth's ColourLike() filter may be useful for making the 16mm film colors more like the 8mm film colors.

http://forum.doom9.org/showthread.php?t=96308

You would build histograms for the parts of the frame that match, then apply the adjustments to the whole frame.

[manono]

It looks like you might have to learn some AviSynth in spite of yourself. Colourlike and other similar filters do exactly what you want:

http://forum.doom9.org/showthread.php?t=96308

http://avisynth.nl/index.php/ColourLike#Levels_and_Chroma

Edit: Oops, too late.

[magnu]

I've been hearing about software called Deoldify which does auto-colourizing via the web. The only problem I see with something like this is that you can't really judge the quality of the actual resulting video, since you have to upload the video to a website.
Is there any software, such as Mocha Pro, that can do it automatically without using the internet?

[lingyi]

You're jumping from A to C. Deoldify is a deep learning project that is far from being able to match what the best human colorists can do and on par with what's available to the general public. Which still looks like someone took a watercolor set to the original. Read this interview with the developer (90% of which goes over my head): https://hackernoon.com/interview-with-the-creator-of-deoldify-fast-ai-fellow-jason-ant...c-c0437670059b Particularly his response about the project above the pic of the woman. His "wow" moment was the that colors on the cup were cleanly rendered (though no living person probably knows what the original colors were) and acknowledging the "zombie arm" (which isn't colorized) of the woman.

Are there better colorizing programs available. Possibly. But unlike Deoldify, they're not Open Source and have a price tag to match.

[Cornucopia]

For me to start getting on the automated colorization bandwagon, or even consider it more than a dalliance, I would need to have aspirants to use a large Variety series of known & publicly recognized color images (for "ground truth" reference), their completely desaturated grayscale versions as the starting point, and the results, both from their methods and from competitors. WITH reference/learning samples, so it could be recreated & tested.
Scientific method with peer review. Otherwise, it's totally subjective bias & marketed hyperbole.

Scott

[TFLeonard]

I think a compromise between the fully automated "wish" and the manual way, is to have a solution which uses reference frames for each scene. In that way a human could make the decisions about how to colorize objects, applying those changes to the reference frame and the tedium of manually rotoscoping and creating masks would be up to the computer. What is interesting to me in the automated approaches is how the algorithms can separate the objects in the scene and track them automatically.

For my personal project of restoring 1930's Kodacolor movies, I have made some progress with the movies I had duped to Super 8 back in the 70's. Using Adobe After Effects CC, I was able to frame match each scene and replace the cropped areas with the content aware fill. I then use the color blending mode to restore the faded color in my HD transfer.

The huge advantage here was having the less faded copy to work from. Without that, I'm back to square one.

[manono]

If there's some color still, you might try Gammac to see if it improves your videos.

[lingyi]

I think a compromise between the fully automated "wish" and the manual way, is to have a solution which uses reference frames for each scene. In that way a human could make the decisions about how to colorize objects, applying those changes to the reference frame and the tedium of manually rotoscoping and creating masks would be up to the computer. What is interesting to me in the automated approaches is how the algorithms can separate the objects in the scene and track them automatically.

For my personal project of restoring 1930's Kodacolor movies, I have made some progress with the movies I had duped to Super 8 back in the 70's. Using Adobe After Effects CC, I was able to frame match each scene and replace the cropped areas with the content aware fill. I then use the color blending mode to restore the faded color in my HD transfer.

The huge advantage here was having the less faded copy to work from. Without that, I'm back to square one.

This was discussed and detailed in previous posts in this thread. Your process is how colorization is currently done and more restoration than colorization since some of the original color data is still there. The OP is asking about completely B&W sources.

[lingyi]

Just thought of something. All the samples with people I've seen are caucasian. How does manual colorization (much less automated) handle non-caucasian skin color? Are there any 'good' examples? What if there's a group of people in the shot? Do they all have the same skin tone? What color skin tone would Warner Oland who played Charlie Chan in the '40's sport?

Do B&W sources contain enough (or any) information about skin tone? How does the manual or automated process work? Does the person or automated process doing the colorization determine the skin color based on facial attributes? What if the person is an albino? What about eye and hair color?

[DB83]

This clip on yt may go some way to answer the question of 'color'

https://www.youtube.com/watch?v=SM9YwN_Dvv0

It also illustrates quite well everything that is good and bad about colorization techniques.

Now I have the following on dvd (it actually looks better there than here) but it also shows, without spilling the beans on the software used, how 'Holiday Inn' was colorized.

https://www.youtube.com/watch?v=DtmQvOuMfgU

[AndyK]

There have been some developments in this field and now there is a web based service that will actually colorize black and white videos:

https://newlify.io

[DB83]

There's a distinct smell of spam in here right now.

[johnmeyer]

There's a distinct smell of spam in here right now.

You are almost certainly correct, although when I Googled the company name, I found this very nice "Bathroom Toilet Mat." Perhaps that means something.

Newlify Shaggy Flocking Feet Bathroom Toilet Mat Green 18.9x26.38 Inch

[AndyK]

Hi there,
Sorry that you got the impression that this is spam.
Maybe if you will know me better it will make sense.

My name is Andy and I am the creator of Newlify.
I have developed Newlify after looking for a service that colors videos myself and seeing that non existed.

The reason that you cannot find Newlify on google right now is that I have released it only yesterday and it takes time for websites to rank high on google.
I have been searching a bit around the web today for people who could benefit from Newlify when I stumbled across this question, and so I posted it here.

I would love to answer any question regarding the service, or machine learning based video coloring.

[DB83]

So you followed the forum rules - this one


Advertising
You must contact the webmaster and get permission before you attempt to post solicitations or advertising. Anything that appears to be advertising or solicitation will be removed immediatly. This does not apply to LEGITIMATE news or user guides.


Before you resurrected a topic that is several years old ?

[Cornucopia]

@AndyK,

Do yourself (and the rest of us) a favor:
Take a well-known full, rich COLOR video clip (or do more than one), DESATURATE it so it is only grayscale, then attempt YOUR process on the grayscale version, and then COMPARE those 2 (the actual ground truth color original + the attempt results) side by side. That will most clearly demonstrate how well (or NOT) your process really works.
Anybody can apply color, but demonstrating that you are applying the Appropriate colors, in the Appropriate spots, Dynamically (where the color's hue basically doesn't change just because it's grayscale does), yet Consistently throughout a clip, is how one tells whether you are being accurate or not.

I'll be waiting...


Scott

[DB83]

Put a carrot in front of a donkey and the donkey typically takes the carrot


So (unless this service is inundated with conversion) it will be quite easy to determine that a quite recent conversion was sourced by me.


I made it quite easy since I am familiar with what this really should look like and tomorrow (since it is now quite late) I will, for the benefit of the community at large, dissect the results in some detail - some are quite good and others really bad. But that, I guess, is to be expected.


For now I will advise that the 31 sec sample clip(charged per minute) took approx. 30 mins to convert (timed by the email advise)

[johnmeyer]

Here is a link to a doom9.org discussion five years ago about a script that attempted to automatically colorize B&W movies, without user input or intervention:

Automated Colorization Script (Now ChromaJig)

The author did exactly what Scott suggested, and then provided the before (the original color) and after (the colorized version via the desaturated B&W version of the original). No one could figure out how the script did what it did. The colors were a little odd, but sometimes it did seem to make the right choices. However, since no one here has ever heard of this script, it is obvious that it wasn't a barn burner.

The only automatic colorization technique of which I'm aware that actually works is the trick discovered that PAL B&W Kinescopes or videos made from video that was originally broadcast as color often contained dot crawl. That hatched pattern contains hints as to the color and intensity of the original colors in each part of the video. Here's a Wikipedia article describing this process:

Colour recovery

[Cornucopia]

Put a carrot in front of a donkey and the donkey typically takes the carrot


So (unless this service is inundated with conversion) it will be quite easy to determine that a quite recent conversion was sourced by me.


I made it quite easy since I am familiar with what this really should look like and tomorrow (since it is now quite late) I will, for the benefit of the community at large, dissect the results in some detail - some are quite good and others really bad. But that, I guess, is to be expected.


For now I will advise that the 31 sec sample clip(charged per minute) took approx. 30 mins to convert (timed by the email advise)

Thanks @DB83, I appreciate your efforts to provide something objective for us all! I look forward to seeing it/them. Though, the burden of proof wasn't on us but on the OP who made the claims.


Scott

[DB83]

@Scott. I fully agree that the burden of proof is not with any of us. Maybe he thinks that he has already proved the service with the sample on the site. Unfortunately that sample is, IMHO, much too 'clean'.


I attach the 'before' and 'after' samples. The original is taken from a BBC (Born Before Color ) television program first broadcast in 1966. It may not be the best source to test this but I chose it due to my familiarity of the location and the subject.


On reflection I could have added a little more with some close up skin tones but my experience of these things overdo it with everyone having a perfect tan - in these parts you only see the sun on tv


The bad: At the start of the clip, the vehicle's color changes. No I did not expect the correct one but certainly not Pink !!. The background mountains are also washed out.


The final exterior shot of the people entering the chapel is all wrong. Check the longer exterior shot just before that with the reddish tone of the building now. The gent's suit's color is also totally out of place.


The good: The final interior shot is quite representative of how the chapel would have looked. Maybe a little too 'new and shiney' for a building then over 60 years old yet I have been inside other local ones which, even older than that, look just as good.


Any way. I do not regret the 3 bucks spent. Would I spend more ? Probably not since my general opinion of colorization has not changed.

[AndyK]

Thank you for checking out the service!
If there would be interest from customers I will probably create a version that lets users tweak parameters to make the final conversion better.
In any case if people from this forum want to try out the service you can talk to me through this forum or through the website and I can give you a special discount for longer form material.
I will also release some longer videos for demos soon.