Hex editor to recover damaged GoPro video files (Hero3)

[snooger]

All,
I have an SD card with 40 mp4 files on it. After file 20, all the video is corrupted. I've used numerous tools to try and "recover" the video but to no avail. I strongly believe the video is still on the SD card, but somehow pointers to the files are messed up. I decided to use a hex editor and when I do a search for some common strings and sure enough, I did bump into them. In my ignorance, I decided I'd just plop a known good header from a working file on top of what appears to be a corrupted header. Problem is, it didn't work.

Below is the header of a good file. I am thinking that maybe things fail because there's filesizes and addresses that are different for each video file. When I paste this known good header on top of a different file, it won't work.

Any thoughts on this technique. It seems to me it should work, but I don't know file formats well enough to decipher what this is precisely telling me.


00000000 | 00 00 00 20 66 74 79 70 | ... ftyp | . 瑦灹
00000008 | 61 76 63 31 00 00 00 00 | avc1.... | 癡ㅣ..
00000010 | 61 76 63 31 69 73 6F 6D | avc1isom | 癡ㅣ獩浯
00000018 | 00 00 00 00 00 00 00 00 | ........ | ....
00000020 | 00 00 39 8A 6D 6F 6F 76 | ..9Šmoov | .訹潭癯
00000028 | 00 00 00 6C 6D 76 68 64 | ...lmvhd | .氀癭摨
00000030 | 00 00 00 00 D1 F3 70 CE | ....ÑópÎ | ..칰
00000038 | D1 F3 70 CE 00 01 5F 90 | ÑópÎ.._� | 칰Ā遟
00000040 | 00 13 85 2E 00 01 00 00 | ..…..... | ጀ⺅Ā.
00000048 | 01 00 00 00 00 00 00 00 | ........ | ....
00000050 | 00 00 00 00 00 01 00 00 | ........ | ..Ā.
00000058 | 00 00 00 00 00 00 00 00 | ........ | ....
00000060 | 00 00 00 00 00 01 00 00 | ........ | ..Ā.
00000068 | 00 00 00 00 00 00 00 00 | ........ | ....
00000070 | 00 00 00 00 40 00 00 00 | ....@... | ..@.
00000078 | 00 00 00 00 00 00 00 00 | ........ | ....
00000080 | 00 00 00 00 00 00 00 00 | ........ | ....
00000088 | 00 00 00 00 00 00 00 00 | ........ | ....
00000090 | 00 00 00 04 00 00 01 80 | .......€ | .Ѐ.老
00000098 | 75 64 74 61 00 00 00 14 | udta.... | 摵慴.᐀
000000A0 | 46 49 52 4D 48 44 33 2E | FIRMHD3. | 䥆䵒䑈⸳
000000A8 | 31 31 2E 30 33 2E 30 30 | 11.03.00 | ㄱ〮⸳〰

[davexnet]

Why don't you post a couple of the bad files to the forum and one good file from the same device.
Somebody will take a look

[snooger]

Why don't you post a couple of the bad files to the forum and one good file from the same device.
Somebody will take a look

Thank you. I will do that. I know I can't use the actual files themselves (the corrupt ones) because when I use the hex editor and open the files, they are typically all FFs... once again I'm assuming some pointers are messed up somewhere. However, I can look at a good file and since they all start the same way, I can then find the same reference on "bad files". At this time, I don't quite know how much of a bad file to grab so here is what I did. Each video file, I believe, begins with ftypavc1. I did a search for that string on the SD card and came up with around 40 matches. That makes sense to me because there are 40 videos. The good file (gopro512.mp4) is a copy of the exact file off the SD card. The bad file file10.mp4 is a copy of the data between matches of the string ftypavc1 off the SD card. I can 'play" this file, but no video is available. I do note some significant difference between the good file and this bad file.

I've done this test several times and it seems to be consistent. The bad file plays, but is black. If someone wants to take a look at it, please do . It is video from a motorcycle trip down highway 1 in CA and then through vegas and then Colorado. Bucket list stuff. I lost all my vegas videos as well as Colorado.

Thank you very much!

[aedipuss]

if all the data is meaningless ff's there is nothing to recover. you can't repair it if there's nothing there to work with.

[snooger]

if all the data is meaningless ff's there is nothing to recover. you can't repair it if there's nothing there to work with.

Agreed. If I look at the file it is all ff. That's why I looked for mp4 atoms. Something got screwed up with file pointers.

[Cornucopia]

Or it just reserved the record filespace but never actually recorded any data.

Haveto ask: did you use the GoPro itself to format the card? (You should ALWAYS)

Scott

[theewizard]

file header data will contain a file ID (name, file number etc what ever the devices uses), the file size, an identifier for the file type, and typically the last (2) bytes in the header point to the location address of the next memory block address of the file, something the hex editor or other software will use when displaying the file

i haven't done any HEX editing since i quit hacking files on 'dumb' aka 'feature' cell phones, to add custom backgrounds and other features, like laptop usb tethering, custom ringtones etc..

thats about 8 years or more

[davexnet]

I'm working in the Hex editor. Can you post one more bad file, I want to compare with the other one

[snooger]

Or it just reserved the record filespace but never actually recorded any data.

Haveto ask: did you use the GoPro itself to format the card? (You should ALWAYS)

Scott

Yep it was a brand new card and I did format it with the go pro. One thing I didn't mention is how the corruption occurred. There were 41 videos on the card. I could see them all. There was one video that was basically me messing around with the camera... Maybe 20 seconds. I just deleted that file and in the process of deleting the camera battery died. When I put the other battery in, I now had 40 files and every file after the one I deleted would not play.anymore. this is why I feel the video is still on the card. The GoPro recorded it and
I saw them. My theory is that in the process of deleting that one file the index pointers to all the files after the deleted ones got corrupted.

[snooger]

I'm working in the Hex editor. Can you post one more bad file, I want to compare with the other one

I certainly will. I only posted one because I was having a hard time finding possible candidate under 500 meg. Will probably get another one up in a few hours. Thank you!

[DB83]

Have you tried ?

https://www.videohelp.com/software/recover-mp4-to-h264

Relies on that the bad files are exactly the same parameters as a good one. Noted that the bad file in your sample was not identical, parameter wise, as a good one.

But there is data. I do not see all zero or 'ff' bytes and they may be reason for some hope. Not tested in the samples provided since the bad one may not be a representative of the full file.

[snooger]

file header data will contain a file ID (name, file number etc what ever the devices uses), the file size, an identifier for the file type, and typically the last (2) bytes in the header point to the location address of the next memory block address of the file, something the hex editor or other software will use when displaying the file

i haven't done any HEX editing since i quit hacking files on 'dumb' aka 'feature' cell phones, to add custom backgrounds and other features, like laptop usb tethering, custom ringtones etc..

thats about 8 years or more

I was hoping I'd see something like that and it would lead me to the actual video. However, I couldn't decpyher it. I'm not sure that taking a "snapshot" of a region on the SD card between the ftypavc1 is of any value, but it's all I have right now.

[snooger]

I'm working in the Hex editor. Can you post one more bad file, I want to compare with the other one

Got to my computer this morning and the hex editor was closed down. Looks like windows rebooted. Ugh! It took about two days to search the whole SD cards for strings. I'll try a few different ways to see if I can get it to work faster. I really appreciate you looking.

[snooger]

Have you tried ?

https://www.videohelp.com/software/recover-mp4-to-h264

Relies on that the bad files are exactly the same parameters as a good one. Noted that the bad file in your sample was not identical, parameter wise, as a good one.

But there is data. I do not see all zero or 'ff' bytes and they may be reason for some hope. Not tested in the samples provided since the bad one may not be a representative of the full file.

I've not used this specific tool but now that I'm learning more, I will give it a shot. Most tools look at the files themselves and not the disk. If you look at the files, they are junk.

[snooger]

I used Stellar Phoenix Video Repair Software for repairing damaged video files. If the video file is still on the SD card, you try a demo version of stellar phoenix video repair. It shows the preview of repair files.

This was one of the first tools I used. I was not able to get it to recover video. I should probably try it again now that I've learned a bit more.

[Budman1]

@Snooger
If you still want the file 10.mp4 recovered, you can download the result at https://files.secureserver.net/0szuptDbtjfitz. It's still a little damaged in the middle but most of it is there.
You may be able to clean it up a bit with re-encoding or cutting the bad parts. I used the Recover_MP4 program from this site, on the 10.MP4 file and the GoPro.MP4 you supplied as a good file.
Thanks
Budman1

[Dr.Zack]

Hi snooger, did you solve your problem and fix the vid?
I'm facing the same problem with a very important video. I can't post it because I dont own all the copyrights (video from a research project) and also for privacy concerns.
I'm looking for any tutorial to recover this file using Hex editor. I am ready to spend as much time as necessary to recover at list a part of it but I do not know where to start...
thanks in advence for help or any advice.

[abolibibelot]

Each video file, I believe, begins with ftypavc1. I did a search for that string on the SD card and came up with around 40 matches. That makes sense to me because there are 40 videos.

Got to my computer this morning and the hex editor was closed down. Looks like windows rebooted. Ugh! It took about two days to search the whole SD cards for strings. I'll try a few different ways to see if I can get it to work faster.

You could do that automatically with Photorec. It won't repair the unreadable files, but will find all the MP4 file signatures and extract the complete files on the fly (so long as they're not fragmented).
Or with WinHex, you can run a string search with the “list search hits” box enabled, that way you don't have to run the same command 40 times, you can do other things in the mean time, and at the end you can save the result so you don't have to start all over again in case there's any kind of SNAFU.

Below is the header of a good file.

Could you show the header of a bad file ? Is the “ftyp” signature there ? Some fields in the header can only be written once the file is finalized, and are required for the file to be recognized correctly by most players / encoders. It would make sense that they are first recorded as placeholder "FF" values, and then updated when the recording is stopped. If that step is not performed, typically because of a shock or because the batteries are drained, the file ends up corrupted.

Repairing the volume with CHKDSK could correct wrong pointers, but do a full image of the card before attempting that. In some cases, after a CHKDSK /F scan, a wrongly allocated file will disappear and be moved to a (hidden) found.000 directory in the volume's root, containing files named file0000.chk, file0001.chk... Renaming those files with the correct name / extension makes them readable, if the header is correct and there's no missing index.

To repair MP4 files with a missing index and/or incomplete header, I've had success with Grau Video Repair. The free version only allows to recover 50% of the recoverable portion of the file, but there's a trick allowing to get the complete recoverable portion (by appending three copies of the same files with the Windows shell command copy /B, or with a hexadecimal editor).

I have an SD card with 40 mp4 files on it. After file 20, all the video is corrupted.

If all files after 20 are corrupted, there's the possibility that the memory card is counterfeit. A counterfeit card can have a displayed size of 32GB, but an actual size of 4GB, and everything past 4GB is written to a non-existant space. If the card has been emptied, you can test it, either manually, by copying a 1GB file repeatedly and checking its MD5 checksum : if it's indeed a counterfeit 4GB card, the first 3 or 4 copies will be correct, but all copies beyond that will be corrupted ; or you can test it with a specific tool like H2testw.


(Sorry, I hadn't noticed that the original question was asked in April...)


@Dr.Zack :
Do you know how this file ended up corrupted ? If you open it with a hexadecimal editor, what does the header look like ?
And if you don't own the copyrights, can't you, well, obtain it again the same way you obtained it the first time around ?

[Dr.Zack]

@abolibibelot
The header is identical to that of another file (from the same cam, with a same profile)... ftypavc1.... etc., but the editor showed a huge part of zero blocks (maybe more than 99% of the file)... at the end there are only 14 lines of blocks. Otherwise, I can't find "mdat" which should contain data intimations...
For copyrights I mean I'm not only the one on this research project. The video also contains testimonies of persons and we've signed a privacy charter to use those testimonials only for research purposes so I avoid to broadcast them on the web.

[JVRaines]

Mostly 00h bytes and no mdat atom means you probably have no video in the file.

[abolibibelot]

For copyrights I mean I'm not only the one on this research project. The video also contains testimonies of persons and we've signed a privacy charter to use those testimonials only for research purposes so I avoid to broadcast them on the web.

But, doesn't anyone have a copy of that file, if it's so damn important ?
Because otherwise, as has been said above, no amount of magic can recreate a working video file out of "more than 99%" of 00s. Something must have gone wrong when it was created, or transfered, or stored, or re-transfered, whatever, either someone screwed up at some point or there was some kind of hardware failure, in any case, nothing can be done with the file as it is. There could be a small possibility of recovery if it was copied from another storage device and said storage device hasn't been used at all ever since, which is unlikely, and if that hypothetical other storage device was defective – especially if it was a counterfeit memory card, see previous post – it's unlikely that the whole file could be recovered anyway. Donc ça semble bel et bien foutu, désolay...

[Dr.Zack]

edit: I found mdat atom but with 1% of bytes ...

I have accidentally deleted the file (shift del) on my SD card and recover it. I take it myself ... so no duplicate...
thank you for all your advices.

[abolibibelot]

I have accidentally deleted the file (shift del) on my SD card and recover it. I take it myself ... so no duplicate...
thank you for all your advices.

That's different if it's not the original file which you analyzed... What did you use to recover the one you have ? Some recovery softwares are more reliable than others. If you haven't used the card since (or foolishly recovered that file onto the same card – see below), you may have more luck with Photorec (free – part of TestDisk), or R-Studio (not free but more convenient and versatile). Both are very efficient with "raw" file detection, meaning, finding files when their allocation information (which exact clusters belong to which file) is lost or corrupted. You could also try Recuva (free), which is less complete and powerful than R-Studio but provides a very useful information about the files it finds : it indicates if they have been partially or totally overwritten with another file (or not), which allows to quickly verify the extent of the damage if that other file is available (by searching a string from file A in file B or the other way around, with a hexadecimal editor like WinHex, which has a “synchronize and compare” feature which displays the contents of both files and allows to scroll them simultaneously) ; it also has a "raw" file recovery feature, although it's less complete than R-Studio it could be enough as it detects MP4 files (select “deep scan” in the “Action” menu to activate the “raw” search, “raw” files have a name like “[012545].mp4”, because the metadata is lost the original name or timestamps or attributes can't be retrieved but the actual contents can be complete – if the file was not fragmented, see below).

Be sure to never export the recovered files to the same volume from which you're trying to recover from ! (A few months ago I tried to help someone remotely – using TeamViewer – in a case like this : a woman who was working on a documentary, had made interviews, had copied all files from the memory card to her computer except one which was somehow unselected, then deleted them all on the memory card... when she realized it – she hadn't used the card since so at that point the file should have been recoverable – she posted a request for help on a classified ads website, which I found, but before I could reach her, she had handed over the memory card to a friend working in computer networking, who should have known better, yet he made that very mistake : he extracted all the deleted files from the memory card... to the memory card ! And of course, Murphy's Law was in full force and effect, the one file she needed was overwritten with another one among those she had already copied, at least I could show her that using WinHex... I spent more than an hour on it with zero result, whereas, if dat dumb dude hadn't messed wid it, it would have been solved in a few minutes !)

If the files were fragmented (which can happen on a memory card if some files are deleted but other are kept, leaving "holes" on the volume, and when a file larger than such a hole is recorded afterward it will be stored in different places, with those other files in between), it makes such a method less efficient (without the file allocation information there's no way of knowing where the different portions are scattered, although some tools try to make educated guesses – Photorec at a very rudimentary level, CnW is supposed to be much more sophisticated but I haven't tried it), especially with MP4 files which often have their index located at the end, if the last clusters are missing then the whole file is unreadable. But since you see large portions completely empty, it would seem unlikely.

On SSDs, there's a “Trim” feature which automatically wipes deleted files to optimize performance, but as far as I know there's no such thing on memory cards, so if you did not full-format that card or use some “wipe” feature (CCleaner has such a feature for instance, but it's a crap cleaner just like Saul Goodman is a criminal lawyer), and (again) haven't recorded anything on that card since that accidental deletion, the file should still be there somewhere.

I was puzzled recently when I did the following :
– copied JPG pictures and MP4 files from a 32GB SD card to a HDD ;
– deleted (most of) those files ;
– copied MTS / AVCHD files to a HDD using a dedicated software ;
– imaged the SD card with WinHex, for future reference (some files are not extracted in the “AVCHD” directory when using a dedicated software like HDWriter or PhotoFunStudio, and on specialized forums it's often recommanded to copy the whole directory so as to preserve the native AVCHD structure, which some softwares require) ;
– wiped the MTS files with the “Wipe securely” function in WinHex (they were already copied, and strictly identical, I always verify MD5 or CRC before deleting, the idea is to compress the image as much as possible by wiping the large files which I already transferred) ;
– then I wanted to also wipe the deleted JPG and MP4 files, but, when pointing at them in WinHex, I realized that most of them appeared empty or with invalid contents, although I hadn't touched those files yet...
I found out that the allocation information was wrong, the actual files were located way further on the partition (most of them with a constant offset of 30064771072 bytes which is exactly 28GB, I haven't yet figured out what that means). I mounted the image, analyzed it with Recuva : same result. But somehow R-Studio managed to get the correct location for almost all the deleted files. (This is using the file allocation table, which ideally recovers files with all their metadata ; in “raw” mode all the tools mentioned would have found them – at least the non-fragmented ones – albeit with a random name / date.)
I don't know if that's relevant to your situation, since you say that at least the header looks correct (in the situation described above the whole files appeared completely invalid, the whole contents were shifted by the same offset value), but it could be a hint in the right direction.

Good day to you.

[Dr.Zack]

@abolibibelot
Thank you for all those precious information.
First I would like to apologize for not being word-perfect in English
So basically, I know there is a very poor chance that I repair this file, but it is very interesting to me to learn more about that ... Maybe it will be useful in the future for me or for someone else arround me. I've tried many of tools you listed above but in fact the story is a little bit more complicated. I recorded the video with my GoPro and copied it to my secondary internal drive (SSD 500gb) then I formated the SD card as usual :/ a few days later, I deleted several files to put my SSD in order. When I looked for the recording I didn't find if. So I remembered that I deleted it accidentally Then I downloaded several softwares to try to recover it ( recuva recuva pro onetrack, easeus, Hetman, Remo and many others. Oddly I found the video only on the formated SD card ( 1.8 gb, 20m25s, the exact featuresr of the deleted one) but nothing in the SSD. I dont have used SSD since that day. I unfortunataly formated the SD card but never wrote on it an never save anything in the sdd drive too. So the only software which gave me something quite correct is recuva. Onetrack and others softs sometimes find raw data but size, duration and extension didn't match even when the files are stored in MP4 folders or mp4 filter when I perform the scan. For eg. Easeus gives me huge mp3 files (20gb 10 times the size of the deleted video) I recovered them, in the header there is a ftypavc1 atom, but when I try to repair them ... nothing. I also tried to repair the file (recovered from formated SD card) with onetrack and mp4 recorery ... No results. I also tried a new Chinese software (MDG I think) and always got the same result ... Just the first frame of the video during 2-3 seconds and nothing after. Since that, I formated the ssd and install Ubuntu on it so no more chance with the SSD. On the other hand, the recovered file and the sd card still intact... maybe I will attempt later. I will check out your message slowly and deeply. I think there are many things to learn. Thank you so much

[abolibibelot]

First I would like to apologize for not being word-perfect in English

Apparently we're both french... (Toulouse here)

then I formated the SD card as usual :/ [...] I unfortunataly formated the SD card but never wrote on it

After a quick format, files' contents are not overwritten, recovery is usually still possible.

Oddly I found the video only on the formated SD card ( 1.8 gb, 20m25s, the exact featuresr of the deleted one) but nothing in the SSD. I dont have used SSD since that day.

As I wrote earlier, this is most likely because of the “Trim” feature, which reduces the odds of recovering deleted files on a SSD, even if it hasn't been used at all after the deletion ; just letting it idle a few minutes can be enough to have the wanted file completely wiped. (Hard disk drives don't require that feature, and the contents of deleted files stay on them until they get overwritten by other files.)

So the only software which gave me something quite correct is recuva. Onetrack and others softs sometimes find raw data but size, duration and extension didn't match even when the files are stored in MP4 folders or mp4 filter when I perform the scan. For eg. Easeus gives me huge mp3 files (20gb 10 times the size of the deleted video) I recovered them, in the header there is a ftypavc1 atom, but when I try to repair them ... nothing. I also tried to repair the file (recovered from formated SD card) with onetrack and mp4 recorery ... No results.

And so is it the file recovered by Recuva that you mentioned earlier, which has a large portion of empty data ? Was it recovered with its original name (meaning that the recovery was based on the file allocation table), or with a random name (meaning that it was detected as a “raw” file and the original file allocation information has been lost) ?
For the Easus software, you mean MP4 files I presume ? (MP3 wouldn't have the "ftyp" signature.)
Have you tried Photorec yet ? A disadvantage with this tool is that it extracts *everything* it finds, so even if only one file is needed, it requires a large amount of free space on the destination volume. The list of file types it detects can be changed before the scan, in “file options” ; if you're only looking for a MP4 file, you should check MP4 and uncheck everything else, to avoid recovering unwanted files, and prevent false positives, which can sometimes truncate a valid file in “raw” recovery mode (for instance : sometimes inside a valid MP4 video file, there can be a random string of characters which happens to be a typical file signature, for instance "ÿØÿ" which is the begining of a JPG picture ; in such a case most recovery softwares will stop extracting the MP4 video at this point and start extracting what appears to be a JPG file, but is in fact still a part of the video data ; if the detection of JPG files is disabled, the extraction of the MP4 file continues and can be complete if all the original data is still there).