Sony Vegas / Adobe After Effects (transparency problems with overlay)

[deadlys]

Hi guys,

So I created a simple Music Visualizer in Adobe After Effect(link1), no problems there.. if I render it as an AVI the raw file is way to large (15Gig) for a 2.45min song. Instead I use H.264(.mp4) to keep the file size small.

The problem comes when I import the visualizer into my Sony Vegas Project. I want the music visualizer to act as an overlay for another video.

Unfortunately even if I tweak the alpha channels of the video in AE* before rendering it, when later imported in my Sony Vegas project the visualizer's background creates the illusion of transparency(link2) ONLY if it's used to cover the resolution of the main video(refer to link2). When I shrink the size of the visualizer video and move it to one of the corners(which is what I want), the visualizer video is half transparent and you can clearly see the borders of it(refer to link2).
Please note that because the video of the music visualizer isn't truly transparent, when it's used as full-screen overlay it washes the colors of the main video - this is because the visualizer video isn't fully transparent.

Question: How can I keep the music visualizer video as an overlay but get rid of the separation in transparency and make it fully transparent, so that when you watch the video.. ONLY the visualizer appears in the overlay instead of this annoying half transparent/half white background along with it? (link2)

link1: http://puu.sh/na0nP/1f68b70422.png
link2: http://puu.sh/na2Ba/b63c1c8381.jpg

* AE: refering to "Adobe After Effects"

[smrpix]

You can create an alpha channel in AE to make the part you want transparent transparent.
You can increase the contrast of the overlay in AE so that you can do a luma key in Vegas and key out the black.
You can increase the contrast in Vegas and then key out the black.
You can adjust your key in Vegas to make the background of your current overlay invisible -- your blacks are in video scale, no reason not to use RGB (and your "whites" are medium-light grey.)

Also, when you're keying don't worry about a large file size. You want a clean image. When you render your final the file size will shrink.

[Cornucopia]

If you are exporting to mp4/h264 from AE, the custom transparency won't be retained. You need either to export as 4:4:4:4 (large file sizes!), or export your alpha channel separately as a grayscale file and use it as a separate controlling mask.

Scott

[deadlys]

Solution Found.
When creating the overlay video in Adobe After Effects use Shift Channels (under Effects menu) and change the "Take Alpha From" to Red,Green,Blue or Alpha. Go to Effects -> Channel-> Remove Color Matting. Then Render your image( you can use H.264 as the output format but make sure to change the Channels to Alpha in the "Video Ouput" section. NOTE: When using the video as overlay, change it's "compositing mode" to Screen and more importantly add Chroma Keyer to the video through the Video Event FX and adjust the colors and contrast to match the color set by you in AE(Shift Channels).

Thank you for taking your time and answering Cornucopia,smrpix. I appreciate it !