I already wrote a post about my problem, but maybe posting it here is more adequate:

I bought this unit with 1TB of capacity. I'm having problems on playing some MKV files with x264 encoded video stream, specially with 1080p files.

I have one 720p (Pulp Fiction) for which mediainfo output is:

cabac=1 / ref=5 / deblock=1:0:0 / analyse=0x3:0x113 / me=umh / subme=7 / psy_rd=1.0:0.0 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / chroma_qp_offset=-2 / threads=6 / nr=0 / decimate=1 / mbaff=0 / bframes=3 / b_pyramid=1 / b_adapt=1 / b_bias=0 / direct=1 / wpredb=1 / keyint=250 / keyint_min=25 / scenecut=40(pre) / rc=2pass / bitrate=5853 / ratetol=1.0 / qcomp=0.60 / qpmin=10 / qpmax=51 / qpstep=4 / cplxblur=20.0 / qblur=0.5 / ip_ratio=1.40 / pb_ratio=1.30 / aq=1:1.00

And then, a 1080p (Avatar) with these options:
cabac=1 / ref=4 / deblock=1:0:0 / analyse=0x3:0x133 / me=umh / subme=7 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=12 / sliced_threads=0 / nr=0 / decimate=0 / interlaced=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=3 / wpredb=1 / wpredp=2 / keyint=240 / keyint_min=24 / scenecut=40 / intra_refresh=0 / rc_lookahead=30 / rc=2pass / mbtree=1 / bitrate=8171 / ratetol=1.0 / qcomp=0.60 / qpmin=10 / qpmax=51 / qpstep=4 / cplxblur=20.0 / qblur=0.5 / ip_ratio=1.40 / aq=1:1.00

The 720p file plays without problems. The 1080p not.

I also have got another 1080p (Starship Troopers) which the Iomega wouldn't play saying it had an invalid container. I reencoded it and played with the same problems as Avatar.

I made some more tests, like raising the quantiser up to 21 (CRF mode with Avidemux) and disabling all advanced options (b-pyramid, weighted p-frames, b-frames as references, etc.) and it seemed to play well.
The problem is that I want to know what are the constraints of this device, because it's very tedious to test one by one every single option that Avidemux lets you choose. For instance I made a test deactivating weighted p-frames (problematic on some devices) and it still stuttered and showed artifacts.

Maybe is just peak bitrate and I have to raise more the quantiser (the artifacts appeared more frequently during shaky scenes).

I only want to know what options to use on my x264 encodings so the device can play them. Meanwhile, I'm testing more combinations in order to achieve the maximum playable quality.

Thanks in advance.