VideoHelp Forum

+ Reply to Thread
Results 1 to 25 of 25
Thread
  1. Hello,

    I need help!!

    I have just finished filming a film with a 19 days schedule in Russia. We used 2 cameras, Blackmagic4k and FujifilmX. The DOP backed up the data to 3 harddisks, one original and 2 backups (harddisks 1, 2, 3). Since the data was more than 5 TBs, he backed up days 17, 18 & 19 on different harddisks (hardisks 4, 5, 6).

    We played back all footage on location in the cameras, and everything played fine.

    The problem is that when we started edit, all footage from the Fujifilm Camera from days 17, 18 & 19 on harddisks 4, 5, 6 was corrupt. (Footage from Blackmagic and Fujifilm for days 1-16 on harddisks 1, 2, 3 is fine. Footage from Blackmagic on harddisks 4, 5, 6 is fine.)

    Days 17, 18 and 19 were the most expensive as they had action scenes, and the Fujifilm camera filmed most of the moving shots.

    Here are examples of files that are corrupt. The videos start playing and then most frames are corrupted -

    ------------------------------------------
    https://drive.google.com/file/d/1-M7e9wTomzuH_BQMSy-AqT2gZydARouE/view?usp=sharing
    https://drive.google.com/file/d/112G40f8KmMWNXW3knPiOO4RH3B4a0S5T/view?usp=sharing
    https://drive.google.com/file/d/1vR0HLgarxn1CnoCw43PrW867tK-iGbtw/view?usp=sharing
    https://drive.google.com/file/d/1XbaSMdYThCrBiAUyzOR_J6gPihbnme9n/view?usp=sharing
    ------------------------------------------

    Is there any way to UN-Corrupt the files?

    (I tried some video repair software I found on google but they did not help!)
    (Re-Filming is NOT an option due to budget constraints!)

    Any help appreciated!

    Neole
    Quote Quote  
  2. You could try untrunc
    https://github.com/anthwlock/untrunc
    You have to provide a working sample from the same camera with the same specs.
    Quote Quote  
  3. Member
    Join Date
    Aug 2004
    Location
    PA USA
    Search Comp PM
    The fact that the camera playback was fine leads me to the corruption could have been on your 4, 5, 6 hard drives. It could be drive data corruption, I would try isobuster, it's not free but, I've had success with it in the past. If the failure occurred between the camera and the drives I'm afraid it's not recoverable.
    It's not important the problem be solved, only that the blame for the mistake is assigned correctly
    Quote Quote  
  4. aBigMeanie aedipuss's Avatar
    Join Date
    Oct 2005
    Location
    666th portal
    Search Comp PM
    hopefully you aren't stupid and still have the original files from the fujifilmx on sd cards. no? too bad, hopefully a lesson learned.
    --
    "a lot of people are better dead" - prisoner KSC2-303
    Quote Quote  
  5. Originally Posted by aedipuss View Post
    no?
    But, yes, of course he does have them!
    He just wanted to waste his time googling software, wasting our time, opening a thread here, explaining it thouroughly, asking for help... of course he still has them on SD.
    Quote Quote  
  6. aBigMeanie aedipuss's Avatar
    Join Date
    Oct 2005
    Location
    666th portal
    Search Comp PM
    Originally Posted by [ss]vegeta View Post
    Originally Posted by aedipuss View Post
    no?
    But, yes, of course he does have them!
    He just wanted to waste his time googling software, wasting our time, opening a thread here, explaining it thouroughly, asking for help... of course he still has them on SD.


    and that wouldn't surprise you? anyone with a big enough budget film can afford sd cards for each day's shoot... if not they're just idiots.[edit] then again they did use a fujifimx.....
    --
    "a lot of people are better dead" - prisoner KSC2-303
    Quote Quote  
  7. Originally Posted by Neole1 View Post
    all footage from the Fujifilm Camera from days 17, 18 & 19 on harddisks 4, 5, 6 was corrupt.
    ...
    Footage from Blackmagic on harddisks 4, 5, 6 is fine.)
    So it's most unlikely that the harddisks have a problem. How did you copy the information from the Fuji cam to harddisk? 100% the same as with the Blackmagic cam? Simply copied SD cards? Via laptop? Windows? Mac?
    Quote Quote  
  8. Member TomLes's Avatar
    Join Date
    Jun 2019
    Location
    Chicago, US
    Search PM
    Ping guys from restore.media. They deal with such issues.
    Quote Quote  
  9. Originally Posted by ProWo View Post
    You could try untrunc
    https://github.com/anthwlock/untrunc
    You have to provide a working sample from the same camera with the same specs.
    Thanks, I tried it, but it simply removed the corrupt frames, shortening the videos. That does not help.
    Quote Quote  
  10. Originally Posted by sum_guy View Post
    The fact that the camera playback was fine leads me to the corruption could have been on your 4, 5, 6 hard drives. It could be drive data corruption, I would try isobuster, it's not free but, I've had success with it in the past. If the failure occurred between the camera and the drives I'm afraid it's not recoverable.
    I will try it. Though it does not look like what is required.
    Quote Quote  
  11. Originally Posted by Quint View Post
    Originally Posted by Neole1 View Post
    all footage from the Fujifilm Camera from days 17, 18 & 19 on harddisks 4, 5, 6 was corrupt.
    ...
    Footage from Blackmagic on harddisks 4, 5, 6 is fine.)
    So it's most unlikely that the harddisks have a problem. How did you copy the information from the Fuji cam to harddisk? 100% the same as with the Blackmagic cam? Simply copied SD cards? Via laptop? Windows? Mac?
    The DOP copied the files from Blackmagic and Fuji the same way, to desktop PC. First copy was to Harddisk 4, then from 4 to Harddisks 5 & 6. This was done the same way to Harddisks 1, 2, 3, but there was no error there.
    Quote Quote  
  12. Originally Posted by TomLes View Post
    Ping guys from restore.media. They deal with such issues.
    Thanks, this looks helpful. I will post the result here.
    Quote Quote  
  13. Originally Posted by TomLes View Post
    Ping guys from restore.media. They deal with such issues.
    The site says - "Note: Restore.Media repairs only damaged video and audio files which you can't open in a media player. If you have a playable video with distortions, freezes, or artifacts, we can't help fixing that, unfortunately."

    Mine plays, but with distortions, freezes, or artifacts. I will try it anyway.
    Quote Quote  
  14. Member TomLes's Avatar
    Join Date
    Jun 2019
    Location
    Chicago, US
    Search PM
    Originally Posted by Neole1 View Post
    Originally Posted by TomLes View Post
    Ping guys from restore.media. They deal with such issues.
    The site says - "Note: Restore.Media repairs only damaged video and audio files which you can't open in a media player. If you have a playable video with distortions, freezes, or artifacts, we can't help fixing that, unfortunately."

    Mine plays, but with distortions, freezes, or artifacts. I will try it anyway.
    This is how their online tool works. But they also offer manual recovery/checkup. I'd suggest to contact them via email and describe the issue. They will check your files and let you know if they can fix it.
    Quote Quote  
  15. Originally Posted by Neole1 View Post
    The DOP copied the files from Blackmagic and Fuji the same way, to desktop PC. First copy was to Harddisk 4, then from 4 to Harddisks 5 & 6. This was done the same way to Harddisks 1, 2, 3, but there was no error there.
    Desktop-PC with Windows I suppose. We had several problems in the past with copying very large files with Windows Explorer that sometimes causes biterrors, only if files are bigger than about 200GB. It is said this happens only with windows 7, but there is no guarantee. What software did you use for copying? Win10? Explorer?
    I am not sure I can help, only try to recapitulate the whole thing, maybe some possibily pops up when we know the very exact course of events.
    Inside the files do not look corrupted, at least no longer zero-chains.
    Quote Quote  
  16. OP, it would help if you included more details about the hardware and software that you are using.

    Were any of the hard drives (4, 5 and 6) Western Digital drives? I ask because some of them had a very annoying powersaving feature programmed into their firmware.

    When copying very large files to the drive, say 4GB+, Windows does not immediately move the data across. Instead, it does some internal house keeping (like calculate time needed, allocating space etc). During this delay the hard drive would timeout and enter hardware powersaving mode so that when Windows does start to write data to the drive it does not spinup and turn on fast enough to receive the data. Data corruption would usually occurs due to this.

    Western Digital supplied a tool called WDIDLE3 to turn off powersaving - it can found on the Ultimate BootCD utility.
    Quote Quote  
  17. Member
    Join Date
    Oct 2021
    Location
    At Doom9
    Search PM
    That's what you get by letting the DOP do IT tasks.
    Before doing backups you need to create the checksums files, then double check them after copying is done, then backups are safely done and originals can be deleted.
    There is a small possibility that files can be repaired, "Harddisk 4" needs to be inspected, but again, that is a task for the IT guy.
    Last edited by VoodooFX; 4th Dec 2021 at 03:33.
    Quote Quote  
  18. Originally Posted by TomLes View Post
    Originally Posted by Neole1 View Post
    Originally Posted by TomLes View Post
    Ping guys from restore.media. They deal with such issues.
    The site says - "Note: Restore.Media repairs only damaged video and audio files which you can't open in a media player. If you have a playable video with distortions, freezes, or artifacts, we can't help fixing that, unfortunately."

    Mine plays, but with distortions, freezes, or artifacts. I will try it anyway.
    This is how their online tool works. But they also offer manual recovery/checkup. I'd suggest to contact them via email and describe the issue. They will check your files and let you know if they can fix it.
    Thanks I will email them details.
    Quote Quote  
  19. Originally Posted by Quint View Post
    Originally Posted by Neole1 View Post
    The DOP copied the files from Blackmagic and Fuji the same way, to desktop PC. First copy was to Harddisk 4, then from 4 to Harddisks 5 & 6. This was done the same way to Harddisks 1, 2, 3, but there was no error there.
    Desktop-PC with Windows I suppose. We had several problems in the past with copying very large files with Windows Explorer that sometimes causes biterrors, only if files are bigger than about 200GB. It is said this happens only with windows 7, but there is no guarantee. What software did you use for copying? Win10? Explorer?
    I am not sure I can help, only try to recapitulate the whole thing, maybe some possibily pops up when we know the very exact course of events.
    Inside the files do not look corrupted, at least no longer zero-chains.
    Windows 10 Explorer I guess. The files are less than 1GB. The blackmagic files are larger, which had no errors after copying.
    Quote Quote  
  20. Originally Posted by sageone View Post
    OP, it would help if you included more details about the hardware and software that you are using.

    Were any of the hard drives (4, 5 and 6) Western Digital drives? I ask because some of them had a very annoying powersaving feature programmed into their firmware.

    When copying very large files to the drive, say 4GB+, Windows does not immediately move the data across. Instead, it does some internal house keeping (like calculate time needed, allocating space etc). During this delay the hard drive would timeout and enter hardware powersaving mode so that when Windows does start to write data to the drive it does not spinup and turn on fast enough to receive the data. Data corruption would usually occurs due to this.

    Western Digital supplied a tool called WDIDLE3 to turn off powersaving - it can found on the Ultimate BootCD utility.
    Yes SeaGate and Western Digital. But wouldn't the problem have happened on some of the files? Right now it is on all the files!
    Quote Quote  
  21. Originally Posted by VoodooFX View Post
    That's what you get by letting the DOP do IT tasks.
    Before doing backups you need to create the checksums files, then double check them after copying is done, then backups are safely done and originals can be deleted.
    There is a small possibility that files can be repaired, "Harddisk 4" needs to be inspected, but again, that is a task for the IT guy.
    The same DOP has been doing it for 4 films for me! This is the first time the error has occurred.

    Good tip about the checksums files, I have seen it being done when I filmed with Red.

    I will get a tech guy to check the harddisk too.
    Quote Quote  
  22. So you copied files (that are even larger than the corrupted) without problems on the same harddisk as the ones that are corrupted.
    Also the error didn't occur on the camera, because you watched and tested there without problems.
    The copying process also was exactly the same.
    The only possible problems remaining seem to me:

    -SSD cards (different?)
    -Handling of SSD cards before copying
    -Reading of the SSD cards (problems with one kind, none with the other kind)

    I know, that doesn't help much, just trying.
    Quote Quote  
  23. Originally Posted by Quint View Post
    So you copied files (that are even larger than the corrupted) without problems on the same harddisk as the ones that are corrupted.
    Also the error didn't occur on the camera, because you watched and tested there without problems.
    The copying process also was exactly the same.
    The only possible problems remaining seem to me:

    -SSD cards (different?)
    -Handling of SSD cards before copying
    -Reading of the SSD cards (problems with one kind, none with the other kind)

    I know, that doesn't help much, just trying.
    That's also weird, because the SSD cards on the camera were the same that were used from Day 1-16. They worked fine then.
    Quote Quote  
  24. Originally Posted by TomLes View Post
    Originally Posted by Neole1 View Post
    Originally Posted by TomLes View Post
    Ping guys from restore.media. They deal with such issues.
    The site says - "Note: Restore.Media repairs only damaged video and audio files which you can't open in a media player. If you have a playable video with distortions, freezes, or artifacts, we can't help fixing that, unfortunately."

    Mine plays, but with distortions, freezes, or artifacts. I will try it anyway.
    This is how their online tool works. But they also offer manual recovery/checkup. I'd suggest to contact them via email and describe the issue. They will check your files and let you know if they can fix it.

    Restore.Media did get back very quickly -

    ---------------------------
    Hello,

    We have analyzed your files.

    They have valid headers and indices.

    But the media data is damaged too much.

    Some parts of files are filled with a junk data.

    It seems your storage devices just failed.

    Unfortunately it is impossible to reconstruct missing/damaged frames. Sorry.

    We can remux these files to drop damaged frames and keep only valid frames.

    But these files will be useless.


    Sincerely,

    Your Restore.Media Team​​

    ---------------------------

    I think I will have to just give up on these files and work with the footage from the Blackmagic camera for the scenes filmed on the last 3 days. The static shots will not have the desired impact, but I hope we can get usable edits.
    Quote Quote  
  25. Member Cornucopia's Avatar
    Join Date
    Oct 2001
    Location
    Deep in the Heart of Texas
    Search PM
    In future, I recommend NOT using straight OS tools to copy, but rather something dedicated like TeraCopy which can do checksums and verification as part of the task.

    The other thing to think about that might work well is not copying merely files, but the disc image. It is handy, specifically in referencing clips by source media id during editing.

    These obviously won't help for these unfortunate clips, but could help for subsequent productions.

    Also, having had similar production challenges, I have found it prudent to do backup disc integrity checks daily, perhaps even more often, and SSD/SD card tests EVERY time those are to be rotated back into service. If you do a round-robin of media, a 3:1 or 4:1 ratio works well (recording, uploading+verifying, reformatting+testing, staging).

    Scott
    Last edited by Cornucopia; 4th Dec 2021 at 14:18.
    Quote Quote