Camcorder fell – trying to properly import MTS clip without metadata

[abolibibelot]

A few days ago, I was recording a video with my Panasonic SD90 camcorder on a small tripod, in precarious balance on a small surface (don't do dat) ; at some point it fell, and then a message appeared on the screen saying “Control data error has been detected” (actually it was in french, but from what I could find doing a research with the most likely keywords this seems to be the exact formulation of the error message in english with that range of devices – with the french wording I only found three results, two of them being PDF manuals for Panasonic camcorders of different models – and even in english all I could find were threads about hardware failures or user errors – like deleting video files from a computer – so nothing relevant to that particular issue). Then the message “Repairing” appeared. As it was taking too long and I was in a hurry to continue shooting, and as I couldn't stop the process, even by pressing the power-on / power-off button, I had to remove the battery, then put another SD card, figuring that I would try to figure it out later on. (Besides, I have read when searching about this that the “repairing” process could completely delete a clip if it didn't manage to properly recognize it, which is quite scary.)

Then, yesterday, I first opened the problematic SD card on the computer, copied everything from it to a HDD as a safety measure, and examined the “STREAM” video folder : that clip I was shooting when the camera fell is there, named 00017.MTS (along with another short clip named 00016.MTS), and seems to be flawless all the way through, until the very moment when it fell (I can see a slow upward panning at the very end). But of course the metadata is missing, and the software HDWriter which I use to import videos from this camera doesn't recognize that clip. Then I put that memory card in the camcorder, hoping that it would simply resume the repairing process, but it didn't ; it didn't find a problem with the data structure either, it was ready to shoot. Then, hoping that I could force it to redo the repair, I started recording a new clip, then deliberately removed the memory card, then put it back in : it did display the same messages, and this time I let it finish the repair process, but it was finished very quickly anyway, and it only repaired the clip I had just taken (meaning that it added the correct AVCHD metadata), not the one I want to fix. And then, more puzzling : the camera in view mode now only recognizes that last clip, everything else seems to be gone, even the still pictures. However, if I access the card from the computer, I can still see the still pictures, and the three MTS clips (the two that were there earlier plus the new one). If I compare with the structure I backed up earlier with WinMerge, I can see these differences :
– a new 00018.CPI file in “BDMV\CLIPINF” (the other one being 00016.CPI corresponding to the first short clip, so there's no 00017.CPI corresponding to 00017.MTS),
– a new 00003.MPL file in “BDMV\PLAYLIST”,
– a new 00003.VPL file in “IISVPL”
– the THUMB.TDT and THUMB.TID files in “AVCHDTN” are different,
– the INDEX.BDM and MOVIEOBJ.BDM files in “BDMV” are different.
Then I restored the files from the backup when different, deleted the newly added files, which should have re-generated the structure of the AVCHD folder exactly as it was before I made those attempts – but then the device in view mode recognized no file at all. Now I'm at a loss...

Back to the main issue : is there a way to generate the missing metadata in a situation like this, either by forcing the camera to do so, or by using computer software ? Failing that, where could I find a description of each one of the relevant files, detailed enough so that I could try to generate the missing ones by editing the existing ones in order to properly import that clip ? Or is there nothing else I can do except copy the MTS file as-is and use it as-is for editing purposes ?

Thanks.

[smrpix]

What's wrong with using the MTS files as-is? Were they spanned clips?

While I'm as strong an advocate as you will find of importing the whole card structure -- you don't HAVE to do it in this case.

Try downloading on of the TMPGenc tools like Video Mastering works and use its advanced muxer. Just run your bad file through it. If it can recognize the file at all, it tends to do a good job of repair.

[abolibibelot]

What's wrong with using the MTS files as-is? Were they spanned clips?

While I'm as strong an advocate as you will find of importing the whole card structure -- you don't HAVE to do it in this case.

In this case no it's not a spanned clip (size is ~1.5GB), so yes it should work as-is... but I'd still like to get it done properly, if only because my OCD-ness level is through the roof...

As a side note, the problem with importing the whole AVCHD structure is that no sorting is allowed (thematic or by date or otherwise), and it means keeping a gazillion of files with similar nondescript names like 00005.MTS and so on, resulting in a huge mess after a while... I already complained about that as you will find.
Currently I do both – sort of : I import the files with the dedicated softwares, which changes their names according to the time they were made (and their extension from .MTS to .m2ts although no transcoding is involved, the files are exactly the same), and makes it obvious which ones are spanned (same base name with added (1), (2)... or (001), (002)... numerotations for each segment) ; and I also save the native structure, with empty MTS files (no need to have two copies on the same drive), using Robocopy.

robocopy.exe %source% %destin% /E /B /DCOPY:T /TIMFIX /FFT /DST /XJ /R:0 /W:1 /CREATE
robocopy.exe %source% %destin% /E /B /DCOPY:T /TIMFIX /FFT /DST /XJ /R:0 /W:1 /XF *.mts *.mp4 *.jpg

The first command creates the complete file tree with empty files (0 byte), the second copies the actual contents of files except for the large MTS / MP4 / JPG files, so it effectively copies only the metadata files. That way if I ever need the original structure later on, I will be able to re-create it, by copying the imported MTS files back to the “STREAM” folder, or better yet by creating hard links with LinkShellExtension, changing the name to match the original one, based on the identical modification date (that's one reason why it's important to preserve the timestamps when importing, which Panasonic's HD Writer doesn't do, see the above linked thread for a guide on how to restore the original timestamps from the “STREAM” folder to the HD Writer import folder, again using Robocopy).

Try downloading on of the TMPGenc tools like Video Mastering works and use its advanced muxer. Just run your bad file through it. If it can recognize the file at all, it tends to do a good job of repair.

Again, the file itself doesn't seem to be “bad”, it plays fine in regular media players and doesn't seem to have glitches, even at the end (it only cuts abruptly, and it seems to cut slightly before the actual fall, probably because the split second recorded during the fall was still being processed or stored in a buffer memory). Only metadata files are missing for the camcorder (or HD Writer) to recognize it.

I tried a tool called AVCCAM Restorer : by selecting the second option, “Restore the specified AVCHD folder”, it doesn't work, displaying an error message, “A file necessary for the restoration is not found. The specified restoration object cannot be restored.” By selecting the third option, “Restore the AVCHD folder from the stream files in the specified folder”, it does work (but only if that folder is not contained within an AVCHD structure, as mentioned in the included PDF guide), and recreates a folder structure with metadata files, but those files seem to be different from those created by that model of camcorder. It's not clear what specific devices it's supposed to be compatible with (and AVCHD is supposed to be a standard, go figure...). The PDF is from 2008... I haven't tried the first option yet as it seems to modify files directly on the memory card (if it's indeed not compatible with that device's format it's certainly not a good idea).

[abolibibelot]

Well... nothing else ?

How can I explain that the camcorder no longer recognizes any file on that memory card (it's an irrelevant detail but it's actually a µSD card connected through a µSD to SD adapter), including the still pictures, even though they're still perfectly readable on the computer ?

[smrpix]

Not much to add because you are obsessing over low-priority things. You dropped your camera yet you managed to recover all the video, many people are not so lucky.

MediaInfo can probably give you the metadata you need regarding timestamps. Rename the files (i.e C00018-2019-09-30-175527.mts,) put them in an appropriately named folder and move on.

If you want to restore what should be on the card so it looks neat in explorer -- best of luck to you. You don't have to convince me of the importance of metadata -- but if I have to choose between losing that or losing the video, there's no contest.

Your bigger issue is (in my view) can you continue to use the camera to make recordings? get a new card and see.

[TomLes]

I guess, the file system just failed to mark up free space clusters where the last few seconds of the video was written to as such that are occupied by the data. This is quite a common case when the power is lost unexpectedly. To recover the video, you would have to allocate those clusters and then use a data recovery tool. This is not an easy task, so I don't think worth trying...