Tools needed:

AVI-Mux_GUI
AC3Machine
BeSweet GUI
BeSweet
VirtualDubMod

To increase the volume of an AC3 soundtrack, you need an avi without audio & a separate AC3 soundtrack. I've been using AutoGK to convert VOBs into xvid movies. AutoGK leaves an AC3 file in the gk_tmp directory, so all I need is the audio-less avi. There are other ways to create these files, quicker ones, I'm sure, but I'm familiar with AutoGK, like the quality & it creates the AC3 file by default.

To create an audio-less avi, start AVI-Mux_GUI.
Right-click in the open file area & browse to your avi. Highlight it & click the Generate Data Source From Files button.
Check the No Audio box & click the Start button. Select where to save the file.

If you need to de-mux the AC3 soundtrack from an avi, start VirtualDubMod & open the avi. From the Streams pulldown, select Stream List. It will list the audio track. Pick the Demux button & VirtualDubMod will save a separate AC3 file.

Now you can add gain to the AC3 file. I like using AC3Machine because it's straightforward. BeSweet GUI can be used, but it kept re-encoding the audio to an MP3 file if I wasn't careful. If you want to use it, the AC3 settings can be set by clicking on the AZID 1 button.
Open AC3Machine. In the Locations area, double-click in the long box beside BeSweet. Browse to the BeSweet executable. You only need to do this the first time you use AC3Machine. I couldn't get it to work with BeSweet V1.4, but it did work with V1.5b28.
Double-click in the box beside Input & select the AC3 file.
Double-click in the Output box & select where to save the new file.
In the Azid Downmix area, make sure nothing is checked.
In the Overall Output Config area, check Channels Mode & set it to 5.1. Check the Gain & set it. I've gotten good results by adding 10db.
Set the bitrate. I've been using 320.
Click the Give me AC3! button & save the new file.

Now you need to mux the new AC3 file with the audio-less avi.
Start AVI-Mux_GUI.
Right-click in the open file area & browse to your audio-less avi. Highlight it & click the Generate Data Source From Files button.
Right-click in the open file area again & browse to your new AC3 file.
Make sure that the All Audio Streams box is checked & click Start.
I haven't experienced any sync problems, but that doesn't mean that it won't happen. If it does, you'll need to futz with the delay setting in AVI-Mux_GUI.

The resulting file plays fine on my computer, but freezes & locks up when played over my modded xbox. To fix this, I open the avi in VDubMod, set it to Direct Stream Processing in the Video pulldown & save a new avi.

GSpot shows the audio to be 5-channel AC3 & my receiver, an HK 325, identifies it the same, too.