So, I'm trying to encode an anime with x264, but can't preserve lines satisfyingly for the life of me. It's not that the encoded anime looks bad (compared to the source), but lines in darker areas are distorted.
What are the things I could try to make x264 allocate more bits to these lines and preserve them better? (Not using any filtering is preferable)
Try StreamFab Downloader and download from Netflix, Amazon, Youtube! Or Try DVDFab and copy Blu-rays!
+ Reply to Thread
Results 1 to 11 of 11
Thread
-
-
Use more bitrate, higher quality settings (for example the slow preset vs. the veryfast preset), aq-mode=2 or 3, --no-psy.
-
Lower aq strength should put more bits to the edges. (--tune animation lowers it from default 1.0 to 0.6.)
Is x265 an option for you? IMHO it's much better for your use case if speed and compatibility aren't problems. -
@jagabo
As I said the general quality is good so I don't think I'm using too low a bitrate, It's not high but not low either. My problem is that I want x264 to allocate more bits to these areas without increasing the bitrate, making it distribute more bits to areas I want basically.
I'm using "veryslow film 10bit". Mind you, I've done some 20 test encodes on this particular anime till now, testing aq-mode 3, other aq methods, aq strengths etc. and I've been able to improve the quality, but it's still not distributing bits exactly where I want it to.
Could you please explain your reason for suggesting --no-psy? Also I think I've read on doom9 that psy-rd 0 should be used instead of --no-psy.
@sneaker
I did test Animation tuning, but it looked worse, I'm gonna test tune film with lowered aq strength next. These are lines within flat areas that get distorted or disappear while other areas look almost identical to the source.
If I were doing 1080p I'd go x265, but I'm doing 720p, and x265 looks too blurred for my taste for 720p, that was at least my experience with it in the past. -
Low contrast detail in dark areas is the first place you'll see problems with too little bitrate in x264.
If your source is noisy apply some noise reduction before encoding. The less bitrate is used to encode noise the more is available for real details. -
-
It's been some time since I tested it. Back then old/grainy anime (Totoro) were difficult with x265. With modern/clean anime (One Punch Man, Attack on Titan, ...) x265 was much better than x264.
There is even some x265 modding by the Anime community with fan-made tunings for anime.
https://down.7086.in/x265-Yuuki-Asuna/
https://github.com/msg7086/x265-Yuuki-Asuna
I haven't tested them personally but they look worth checking them out.Last edited by sneaker; 18th Jun 2019 at 09:56.
-
OK, but how would go about improving those dark areas without increasing the bitrate, I mean take bits from other areas?
I don't think anything I do achieves this in a significant way.
Good idea, have to try this. Care to suggest an appropriate de-noiser for this type of content? One that doesn't destroy much detail. -
That's what I'm testing, maybe I'll give x265 another chance after all.
There is even some x265 modding by the Anime community with fan-made --tune animation.
https://down.7086.in/x265-Yuuki-Asuna/
https://github.com/msg7086/x265-Yuuki-Asuna
I haven't tested them personally but they look worth checking them out.
What about modded x264? Has anybody tested them? -
Probably most discussion/documentation is either Chinese or Japanese. If you run --help:
Code:-t/--tune <string> Tune the settings for a particular type of source or situation: (mid bitrate anime) littlepox/lp, (slower) littlepox++/lp++, (high bitrate anime BD / film) vcb-s/vcbs, (slower) vcb-s++/vcbs++, psnr, ssim, grain, zerolatency, fastdecode
Code:else if (!strcmp(tune, "animation")) { if (param->bframes + 1 < param->lookaheadDepth) param->bframes++; if (param->bframes + 1 < param->lookaheadDepth) param->bframes++; param->psyRd = 0.4; param->rc.aqStrength = 0.4; param->deblockingFilterBetaOffset = 1; param->deblockingFilterTCOffset = 1; } else if (!strncmp(tune, "littlepox", 9) || !strncmp(tune, "lp", 2) || !strncmp(tune, "vcb-s", 5) || !strncmp(tune, "vcbs", 4)) { param->searchRange = 25; //down from 57 param->bEnableAMP = 0; param->bEnableRectInter = 0; param->rc.aqStrength = 0.8; //down from 1.0 if (param->rdLevel < 4) param->rdLevel = 4; param->rdoqLevel = 2; //force rdoq to be effective param->bEnableSAO = 0; param->bEnableStrongIntraSmoothing = 0; if (param->bframes + 1 < param->lookaheadDepth) param->bframes++; if (param->bframes + 1 < param->lookaheadDepth) param->bframes++; //from tune animation if (param->tuQTMaxInterDepth > 3) param->tuQTMaxInterDepth--; if (param->tuQTMaxIntraDepth > 3) param->tuQTMaxIntraDepth--; if (param->maxNumMergeCand > 3) param->maxNumMergeCand--; if (param->subpelRefine < 3) param->subpelRefine = 3; param->keyframeMin = 1; param->keyframeMax = 360; param->bOpenGOP = 0; param->deblockingFilterBetaOffset = -1; param->deblockingFilterTCOffset = -1; param->maxCUSize = 32; param->maxTUSize = 32; param->rc.qgSize = 8; param->cbQpOffset = -2; //better chroma quality to compensate 420 subsampling param->crQpOffset = -2; //better chroma quality to compensate 420 subsampling param->rc.pbFactor = 1.2; //down from 1.3 param->bEnableWeightedBiPred = 1; if (tune[0] == 'l') { // Mid bitrate anime param->rc.rfConstant = 20; param->psyRd = 1.5; //down param->psyRdoq = 0.8; //down if (strstr(tune, "++")) { if (param->maxNumReferences < 2) param->maxNumReferences = 2; if (param->subpelRefine < 3) param->subpelRefine = 3; if (param->lookaheadDepth < 60) param->lookaheadDepth = 60; param->searchRange = 38; //down from 57 } } else { // High bitrate anime (bluray) or film param->rc.rfConstant = 18; param->psyRd = 1.8; //down param->psyRdoq = 1.0; //same if (strstr(tune, "++")) { if (param->maxNumReferences < 3) param->maxNumReferences = 3; if (param->subpelRefine < 3) param->subpelRefine = 3; param->bIntraInBFrames = 1; param->bEnableRectInter = 1; param->limitTU = 4; if (param->lookaheadDepth < 60) param->lookaheadDepth = 60; param->searchRange = 38; //down from 57 } } }
Last edited by sneaker; 18th Jun 2019 at 09:55.
-
I haven't actually seen the content but something like SMDegrain() in AviSynth:
https://forum.videohelp.com/threads/393441-Grain-reduction#post2553114
Similar Threads
-
what are the best x264 settings to preserve grain in anime?
By TheNeverhood in forum Newbie / General discussionsReplies: 9Last Post: 18th Feb 2019, 08:21 -
[I need help] Old anime lines enhancing/filtering
By BeyondTheEnergy in forum RestorationReplies: 17Last Post: 14th Nov 2018, 20:46 -
Anime series upscaled to make lines look better?
By zanzar in forum Newbie / General discussionsReplies: 20Last Post: 4th Dec 2016, 16:41 -
Trying to Restore anime lines, need help with avisyth scripts linedarkening
By kairukun in forum RestorationReplies: 3Last Post: 25th Nov 2015, 14:20 -
encoding some anime
By Tylerr in forum Video ConversionReplies: 21Last Post: 22nd Jan 2015, 09:48