VideoHelp Forum
+ Reply to Thread
Results 1 to 28 of 28
Thread
  1. Member
    Join Date
    Mar 2008
    Location
    United States
    Search Comp PM
    I would like to convert my DVD collection to a compressed format for efficient storage on my HTPC. In addition, I would like to retain decent picture quality for viewing on a 1080p HDTV. I have been experimenting with several video conversion programs (AVS video converter, DVDPean, AutoMKV & RipBot264), encoders (Xvid & h264), encoder configuration settings and filters. In all instances, the foreground portion of the video looks crisp & clear but the background portion looks blotchy & pixelated. Is this to be expected or is there a way to clean up the background? If so, how? Here's an example (see circled areas)

    [/img]
    Quote Quote  
  2. Mod Neophyte Super Moderator redwudz's Avatar
    Join Date
    Sep 2002
    Location
    USA
    Search Comp PM
    Probably too low of a bitrate from your encoder, if the source was a DVD.
    Quote Quote  
  3. Some of that could be in your source and therefore could be reduced with a deblocking filter. And some if it will be unaviodable if you require low bitrates. One of the methods used by low bitrate encoders is to remove details, especially in dark areas. The result is macroblock artifacts like you're seeing.
    Quote Quote  
  4. Always Watching guns1inger's Avatar
    Join Date
    Apr 2004
    Location
    Miskatonic U
    Search Comp PM
    Like it or not, this is almost unavoidable when using mpg-4 for compression. I have never seen an Xvid or H264 file that didn't have it to some degree, even when high bitrates are used. I believe you are pretty much stuck with it if you use Xvid. H264 at bitrates of around half the source DVD might give you better results.
    Read my blog here.
    Quote Quote  
  5. Member vhelp's Avatar
    Join Date
    Mar 2001
    Location
    New York
    Search Comp PM
    I am experimenting in this same area -- background giblets or floating chisel chodd is what I
    mainly call them when I see them in the background. In fact, I'm running some encoding
    scenarios on a short clip (the mummy returns) for posting, cause due to my ananlog cabletv,
    there is much noise to attend to first, and then design a unique filter-chain around *this*
    particular capture source and then finally encode it to a low-bitrate codec of choice. I was
    hoping for h264, preferebly the raw .mp4 version, but i'm not sure yet on the howto parts.
    There's still much to iron out before I finalize to something acceptable. I mean, this project
    will not be from an already clean dvd/mpeg "digital" source, but rather from a noisy analog one.

    What you have to do is find a good (high enough) bitrate to start with, and then observe the
    video for various details. You want to find out how low you can reduce the bitrate (or quality
    control value) to but still maintain detail.

    But, before you begin something like this, you really need to review your dvd collection for its
    background attributes to determine if it already has them. I mean, there is no point in going
    all out crazy to debug all this if your original (dvd) ource has these artifacts in them aready.
    If you've determined that original source has them then you might want to consider reviewing
    ways to eliminate or reduce them *before* attempting your xvid/h264 encoding possibilities.

    -vhelp 4622
    Quote Quote  
  6. As others have suggested, a higher bitrate should eliminate those blocks. For a given bitrate, you will find less blocking with h264 than XviD.

    If you want, you can adjust h264's inloop blocking filter, but I doubt this would be necessary for a good DVD source and a decent bitrate
    Quote Quote  
  7. Member vhelp's Avatar
    Join Date
    Mar 2001
    Location
    New York
    Search Comp PM
    You know, I think his trouble is prob the source, or his dvd he said. I think the artifacts
    are already there to begin with. I finished yet another encode of that sample clip in h264
    (w/out audio unfortunately) and for the noisy source that it is (analog cabletv) in the
    background scenes, they do not have the artifact mentioned in this discussion. So it must
    be this users source that is the root of the problem here

    -vhelp 4625
    Quote Quote  
  8. Always Watching guns1inger's Avatar
    Join Date
    Apr 2004
    Location
    Miskatonic U
    Search Comp PM
    Xvid, and to a lesser degree, H.264, work best with detailed or noisy sources if given enough bitrate. They both suffer from blocking (Xvid/Divx are the worst) on large areas of low tonal range - sky, single colour walls etc. It is inherent in the way the compression works. H.264 tries to address it with it's built in deblocking. I have encoded good quality source at stupidly high bitrates (2 - 3K for 640 pixel width source) using both Xvid and Divx and still ended up with blocky clusters in areas of sky and shadowed backgrounds, yet on lower birates I can get crisp images wherever there is patterns or detail. The detail stops the compressor seeing larger flat areas and over compressing them.
    Read my blog here.
    Quote Quote  
  9. Member
    Join Date
    May 2007
    Location
    reality
    Search Comp PM
    I have been running an HTPC for just over 3 years now and have seen it go through many incarnations of what it would be used for and how it would do it...

    I originally built my HTPC to do much the same as you want: to store movies in Divx or Xvid format on the system, ready to be played at a push of the remote. I went through much the same as you, never impressed with the quality of the compressed movies. I would guess that once you have become accustomed to the quality of the original DVD a compressed, lossy conversion of the same movie is somewhat dissapointing or else I am just too damn demanding. I played with many settings and software solutions. The higher the bit rate went, the larger the file got, meaning less could be stored on the hard drive. Adding an additional drive became the solution but then I came to the realization that I was moving in the wrong direction. The HTPC was to be a simple and QUIET system. More drives and their associated noise and heat would not help the situation. Rethink time...

    I have now settled my HTPC into what has become a very workable and versatile system:

    It is used as a music jukebox. All of my audio CD's (approximately 1,200) are ripped to MP3 format and stored on the hard drive, sorted by genre, artist and CD title. I find MP3 to be quite acceptable as a lossy compression format. Assembled playlists are a huge benefit.

    It is used as a PVR to record/timeshift cable TV shows in DVD quality MPEG2 format. Shows can be edited and burned to DVD as required. Connects to the internet to obtain up-to-date local TV listings. Makes programing TV shows to record as easy as right clicking and selecting "record this show".

    It is used to view live TV and much like a Tivo, can pause live TV or record on demand.

    It is used to store digital photograph collections and much like slide projectors of years past can be used to bore the hell out of guests by forcing them to relive a vacation in Eugene, OR.

    It connects to digital cameras, MP3 players, Pocket PC's and smart phones to transfer music, video and pictures.

    The dilema was movies. What to do? I finally settled on playing the original DVD's. I organized them into a database on the HTPC and keep the discs well organized for easy locating. I play the DVD's on the HTPC and have eliminated the standalone DVD player from my primary home theatre. My entire home theatre now consists of the TV, surround amplifier, HTPC and speakers. The DVD's now occupy an area once used for CD storage. By ripping the CD's to MP3 it solved the storage issues. IMHO CD's seem much more agreeable to compression than DVD.

    My 2 cents...
    Quote Quote  
  10. Mod Neophyte Super Moderator redwudz's Avatar
    Join Date
    Sep 2002
    Location
    USA
    Search Comp PM
    I'm experimenting with H.264 encoding from DVD rips at present. I did notice the blocking problem on a snow scene but the built in de-blocking filter seems to help a lot. Adding bitrate didn't have as much affect.

    For my DVDs, I use a 400 disc Sony changer. The rest of my compressed videos are on a server computer in the back room, accessed by the HTPC over a LAN, and all is fed to a video projector and surround sound system.
    Quote Quote  
  11. Banned
    Join Date
    Jun 2007
    Location
    UNREACHABLE
    Search Comp PM
    I suppose that choosing a suitable quantize matrix might be
    part of the solution to this problem. However the QMs for H264-encoding
    are even more mysterious than the "well-known" mpeg-1/2 matrices.
    So far, I have not found the courage for starting to understand them.
    Quote Quote  
  12. Xvid and Divx (the Windows codecs) have playback deblocking options. They don't work quite as well as h.264 deblocking though.
    Quote Quote  
  13. Member
    Join Date
    Mar 2008
    Location
    United States
    Search Comp PM
    Thanks for all of the input everyone. Converting to Xvid used to work great until I recently upgraded to a HDTV. Solve one problem and create 3 new ones.

    I tried bitrates from 1500-5000 kbps. That didn't solve the problem.

    The source video does look a bit noisy. Would you try preprocessing the video with something like VirtualDub before encoding? That was where I was headed next. If that doesn't work I'll probably just store my favorites in original format on the HTPC and the rest on disc.
    Quote Quote  
  14. Don't use B frames if you don't want macroblocks. B frames are encoded with higher quantizers so they get more macroblock artifacts. Encode in single pass target quantizer mode with a quantizer of 2. But if you do this the resulting AVI file won't be much smaller than the source MPEG. And you'll still get an additional round of DCT ringing artifacts.

    To remove macroblock artifacts from your source, use AviSynth's Deblock() filter. Of course, what you're doing smearing out the blocks to smooth the edges, not adding back the detail that was lost in the original compression. You'll still get creepy crawly artifacts because the way the individual frames are smeared varies from frame to frame.
    Quote Quote  
  15. Member vhelp's Avatar
    Join Date
    Mar 2001
    Location
    New York
    Search Comp PM
    I just got a great idea ...

    RagingSloth, can you upload a small vob, say around 10 megs (I'm still on dial-up) and we
    can all have a go at it in encoding it to an Xvid and/or H264 and we can all review each others
    methods/results/and things. I think that this would be a great challenge for everybody to help
    fine-tune or resolve the these types of artifacts.

    (Post the link to an unprocessed dvd snipped vob as soon as possible, thanks)

    This will keep us busy figuring this all out, but it will also serve as an adventure.. oh yeah, and fun

    The challenge is on..

    -vhelp 4527
    Quote Quote  
  16. Member vhelp's Avatar
    Join Date
    Mar 2001
    Location
    New York
    Search Comp PM
    Stupid is, stupid ain't, though I gotta stupid question anyway..

    B frames are encoded with higher quantizers so they get more macroblock artifacts.
    Well, can you vary the matrice values for just those B frames in an effort to give less quantized
    macroblocks ??

    -vhelp 4628
    Quote Quote  
  17. Member vhelp's Avatar
    Join Date
    Mar 2001
    Location
    New York
    Search Comp PM
    --- A memory unblock of a deblock from a bi-product off part of an algo's intended purpose..

    To remove macroblock artifacts from your source, use AviSynth's Deblock() filter. Of course, what you're doing smearing out the blocks to smooth the edges, not adding back the detail that was lost in the original compression. You'll still get creepy crawly artifacts because the way the individual frames are smeared varies from frame to frame.
    I thought I'd add this snipplet in becuase it jarred a memory of something close to the issue at hand.

    During my exploration of the Median filter I was developing from scratch, in one of my algorithems for
    processing the image fields, I stumbled on a positive bi-product of the kaos. And that was a form of
    deblocking. In some of my test videos using various algorithms, I found this out. I remember snipping
    the routine and dumpping it at the bottom of the (pascal) source code, a place where I like to dump
    things for later (and easier) retreaval when time allows and memory serves

    --> Image Processing: Temporal / Spacial Median filter -- by vhelp; -- Dec 23, 2007

    I would love to look into that routine and see what might/might not come out of it.. only.. the only
    problem with that is finding it through all the beta versions I made of the filter. It's a maze of versions
    and spin-off to other project designs, etc.

    --- regarding codec version installed

    I would also suggest (to the originator of this topic) to look into the version of the codec installed
    on the computer.. the xvid and h264 codecs. Maybe its the codec or the player that links to the
    codec for decoding these videos.

    -vhelp 4629
    Quote Quote  
  18. Originally Posted by RagingSloth
    Converting to Xvid used to work great until I recently upgraded to a HDTV.
    Have you tried calibrating the TV set? If the problem is most obvious on the new TV, then the out-of-the-box brightness and contrast settings could be at least partly to blame.

    At the very least run the TV set through the THX Optimizer found on all the Star Wars DVDs, as well as on dozens of others. Also, as Midzuki suggests, using better quantiser matrices can help, but will require larger file sizes for the same quality.
    Quote Quote  
  19. Member
    Join Date
    Mar 2008
    Location
    United States
    Search Comp PM
    vhelp: I'll upload a clip asap. Can you recommend a tool to snip the vob file?

    manono: The macroblocking is visible on my HDTV and a separate PC monitor. I calibrated my HDTV the best I could using Digital Video Essentials. I'll take a look at the THX Optimizer as well.
    Quote Quote  
  20. DGIndex: mark-in, mark-out, Save Project and Demux Video
    Quote Quote  
  21. Member
    Join Date
    Mar 2008
    Location
    United States
    Search Comp PM
    I would also suggest (to the originator of this topic) to look into the version of the codec installed
    on the computer.. the xvid and h264 codecs. Maybe its the codec or the player that links to the
    codec for decoding these videos.

    -vhelp 4629
    According to Gspot, AVS Video Converter is using XviD 1.1.0 Final and H.264/MPEG-4 AVC codecs for encoding. I have been using either J River Media Center 12 or AVS DVD Player 2.4 for playback.

    Here is a test clip per your request:

    test_clip.demuxed.m2v
    Quote Quote  
  22. Member
    Join Date
    May 2007
    Location
    reality
    Search Comp PM
    Originally Posted by redwudz
    I'm experimenting with H.264 encoding from DVD rips at present. I did notice the blocking problem on a snow scene but the built in de-blocking filter seems to help a lot. Adding bitrate didn't have as much affect.

    For my DVDs, I use a 400 disc Sony changer. The rest of my compressed videos are on a server computer in the back room, accessed by the HTPC over a LAN, and all is fed to a video projector and surround sound system.
    Very interested to hear how your H.264 experiment works out, redwudz. I have thought about H.264 but have put so much time into organizing my DVD's, both physicaly and on the HTPC, that I haven't budgeted any more time for this project.

    I would need to daisy-chain 3 of those Sony changers together!
    Quote Quote  
  23. Originally Posted by RagingSloth
    IVTC'd, Xvid, target quantizer = 2, no B frames, about half the size:

    xvid2.avi
    Quote Quote  
  24. oops, duplicate post.
    Quote Quote  
  25. -I chose ~2500kbps (about 1/2 bitrate of original)
    -720x480 (not resized)
    -DAR 16:9
    -2-pass high quality encode - Virtualdub for XviD, MeGUI for x264 (I can't figure out proper bitrate/sizing for XviD in MeGUI)
    -Same .avs script used for both, no filters, default settings
    -x264 r808, XviD [version 1.2.0-dev build 2007-07-25]
    -Screenshots show more differences as .bmp, but for hosting purposes, reduced to .jpg with 100 quality

    XviD
    http://www.megaupload.com/?d=0Z01H47H

    x264
    http://www.megaupload.com/?d=6KHATCDU

    Comments: I think x264 is the best quality, and jagabo's magic on the XviD looks better than my stock XviD settings - where you can clearly see blocking similar to the original post


    ORIGINAL



    XviD-2500kbps



    Jagabo Xvid 2800kbps



    x264-2500kbps

    Quote Quote  
  26. Banned
    Join Date
    Jun 2007
    Location
    UNREACHABLE
    Search Comp PM
    To whom this may interest: 8)

    Sharktooth's EQM AVC-HR rev.1

    This matrix was made for medium-high bitrate backups with High-Profile
    AVC encoders that support custom matrices and 8x8 discrete cosine
    transform. It also reduces blocking.

    Usage:
    - high quality backups of DVD material at full resolution (anamorphic)
    and bitrates of at least 1300-1400kbps (for average motion and not
    noisy/old movies).
    - non anamorphic/vertically resized encodes at bitrates of at least
    1000kbps.

    Standard version (JM, x264 & Encavc beta2-2 compatible):

    INTRA4X4_LUMA =
    6, 9,13,19,
    9,14,20,27,
    13,20,28,35,
    19,27,35,42

    INTRA4X4_CHROMAU =
    6,10,15,20,
    10,16,21,27,
    15,21,28,33,
    20,27,33,42

    INTRA4X4_CHROMAV =
    6,10,15,20,
    10,16,21,27,
    15,21,28,33,
    20,27,33,42

    INTER4X4_LUMA =
    8,11,15,20,
    11,16,21,27,
    15,21,28,35,
    20,27,35,42

    INTER4X4_CHROMAU =
    8,11,16,21,
    11,17,22,27,
    16,22,28,35,
    21,27,35,42

    INTER4X4_CHROMAV =
    8,11,16,21,
    11,17,22,27,
    16,22,28,35,
    21,27,35,42

    INTRA8X8_LUMA =
    6, 7, 8,10,12,14,16,18,
    7, 9,11,13,15,16,18,20,
    8,11,14,16,17,19,21,22,
    10,13,16,18,20,22,24,26,
    12,15,17,20,23,25,28,30,
    14,16,19,22,25,29,34,38,
    16,18,21,24,28,34,46,52,
    18,20,22,26,30,38,52,72

    INTER8X8_LUMA =
    8, 9,10,12,14,16,19,21,
    9,11,13,15,17,19,21,23,
    10,13,16,18,20,22,23,25,
    12,15,18,21,23,24,26,32,
    14,17,20,23,25,27,33,40,
    16,19,22,24,27,34,41,52,
    19,21,23,26,33,41,53,64,
    21,23,25,32,40,52,64,80
    P.S.: As for the file "test_clip.demuxed.m2v", here goes its "weird" QM,
    according to Rocka's MatrixEditor

    08 05 07 09 11 13 14 17
    08 08 09 11 13 13 14 17
    08 08 11 12 13 14 17 94
    09 11 13 13 14 17 94 94
    11 11 13 13 14 94 94 94
    13 13 14 16 94 94 94 94
    13 13 14 94 94 94 94 94
    13 14 94 94 94 94 94 94

    12 12 13 14 15 16 22 26
    12 13 14 15 16 22 26 32
    13 14 15 16 22 26 32 94
    14 15 16 22 26 32 94 94
    15 16 22 26 32 94 94 94
    16 22 26 32 94 94 94 94
    22 26 32 94 94 94 94 94
    26 32 94 94 94 94 94 94
    Quote Quote  
  27. Member vhelp's Avatar
    Join Date
    Mar 2001
    Location
    New York
    Search Comp PM
    --> filename: vhelp.test_clip.demuxed.[v1.0].mp4

    EDIT: ahh, screw it.. the link doesn't seem to want to work and I'm out of ideas and energy. So I give up.
    Have a good night and weekend, everyone..

    -vhelp 4630
    Quote Quote  
  28. Member
    Join Date
    Nov 2007
    Location
    United States
    Search Comp PM
    Greets,

    A bit late to the party here but, RagingSloth if you are still working on this... I would second Midzuki's suggestion of trying Sharktooth's EQM AVC-HR matrix. It is a great matrix. Stumbled across it a few days back - thanks in part to Midzuki's suggestion here.

    I was having problems converting my dvd of Alien with AutoMKV and getting severe blocking in the opening scene as they pan across the planet while spelling out the title. Even while using Constant Quality of 17 and having sufficient bitrate. I realize it is part of h264's nature to scavage heavily from low detail areas. Yet, with the EQM AVC-HR matrix it cleared all the blocking artifacts up. I was very pleased to see the improvement.

    Good luck on your project.

    Cheers,
    Rick
    Rene: Could you not just wound him a little bit?
    Hans: Well now, with a 25 pound shell that is not easy.

    'Allo 'Allo
    Quote Quote  



Similar Threads

Visit our sponsor! Try DVDFab and backup Blu-rays!