I use MediaInfo to check metadata info, and compare before and after encodes. Occasionally, I notice for some audio encodes in HandBrake, I get a small -5 ms audio delay relative to video. If I choose audio passthrough, there isn't a delay. I also noticed some times unaltered audio tracks (extracted from same source) muxed into the encoded mkv file of that source using MKVToolnix, will also produce a -5 ms audio delay.

I compared the original and encoded mkv files to see if the audio is synced with the video, and I think they sound synced. Is this -5 ms a common number which occurs after an encode or remux? Why does this occur? Can I just safely ignore it for future encodes?