VideoHelp Forum
+ Reply to Thread
Page 1 of 2
1 2 LastLast
Results 1 to 30 of 57
Thread
  1. Member
    Join Date
    Aug 2005
    Location
    Europe
    Search PM
    Under Windows 7 64-bit when copying large compressed files to an external usb3 hdd I get no error messages, but after copying verification/extraction of the compressed files fails. The problem happens with all usb3 ports on the motherboard and using different (2.5" and 3") hard disks which are virus free and have been checked for problems. The latest Intel drivers are installed, but problem seems to happen also if drivers are removed and when antivirus and firewall are disabled. The problem can be reproduced easily and small files do not seem to have the same problem when copying. The problem seems to be a known one and a workaround is to use third party utilities for copying, but I am looking for a real solution. Since the motherboard and CPU are new I am thinking if I should return them. The problem is discussed in more detail in the following link:

    http://social.technet.microsoft.com/Forums/windows/en-US/13a7426e-1a5d-41b0-9e16-19437...al-ntfs-drives

    Any advice?
    Quote Quote  
  2. Member budwzr's Avatar
    Join Date
    Apr 2007
    Location
    City Of Angels
    Search Comp PM
    Try 7zip.
    Quote Quote  
  3. DECEASED
    Join Date
    Jun 2009
    Location
    Heaven
    Search Comp PM
    Originally Posted by budwzr View Post
    Try 7zip.
    WTF

    Back on topic:

    First of all, many thanks for the link, I was unaware of that serious design flaw of Windows 7...

    Assuming the (stupid) file indexing service is enabled, disable it and see whether that makes some difference.

    Another option --- since the problem seems to be NTFS-specific, you could format the external drives to exFAT.
    Quote Quote  
  4. Member
    Join Date
    Feb 2004
    Location
    Australia
    Search Comp PM
    1: Always confirm the hard drives identity (model) and search online for reported issues ... if results poor its either rma or replace.

    2: Never rely on manufacturers drive fitness utilities ... I've seen failing drives pass tests

    Take the drive with no data on it and recreate the partitions and reformat it ... use paragon partition manager (free edition) ... AVOID ms format utility.

    Do two tests:

    1: Perform file copy using windows transfer method ... check for error

    2: Use an external application to copy files (roadkills unstoppable copier) ... check for errors.

    Now move onto next stage:

    Not all cables are equal ... a usb cable can operate poorly ... retest with known good cable

    Not all external enclosures control boards are equal ... some are down outright garbage ... retest with known brand unit.

    -----------

    Have you ever on the system experienced any no matter what it was issues with usb ports ?

    If system has ever indicated "usb reset" warning use pci addon ... onboard usb has been comprised ... it'll still work but large file transfers will suffer corruption.
    Quote Quote  
  5. Member
    Join Date
    Aug 2005
    Location
    Europe
    Search PM
    I prefer to use NTFS and this is not a hard disk problem. I have several different USB2/USB3 external disks from Toshiba, Verbatim, Buffalo and also have the same problem. Windows copy shows no error messages but produces resulting files (zip, rar, 7z etc) fail verification/extraction. Cables are fine. This is a new system and the first problem I have found with usb ports. I have never came across the a "usb reset" warning.
    Quote Quote  
  6. DECEASED
    Join Date
    Jun 2009
    Location
    Heaven
    Search Comp PM
    Well, assuming the actual problem is in the Windows 7 64-bit edition(s), we the end-users don't have many choices...

    a) switch to a 32-bit edition of Windows 7, or downgrade to Windows Vista, or upgrade to Windows 8

    b) always use Teracopy as a workaround

    c) use a different file system (FYI, exFAT does not have the limitations of FAT32, and it's not "for pendrives only").
    Quote Quote  
  7. Member
    Join Date
    Aug 2005
    Location
    Europe
    Search PM
    Switching to Windows 7 32-bit is no a solution as I want to use more than 4GB of RAM. Only using Teracopy is not very practical. I might try exFAT, but I already have several NTFS hard disks with data that I need to use. And just to clarify things the problem appeared after I upgraded my system. I actually kept everything and replaced only the motherboard and the cpu. I am sure the problem is not the cpu and I was about to return the motherboard suspecting of a hardware issue, but I then found several posts from others about the problem. I have disabled the indexing and windows search services without luck, but the problem seems to go away by changing the removal policy for each hdd from quick removal to better performance. I still can't figure out why I am having the problem with the new motherboard and cpu and not with the previous one with the exact same windows configuration. And I don't know if by returning the motherboard I will get a replacement or a refund if the problem is only happening in Windows 7 64-bit.
    Last edited by kyrcy; 2nd Oct 2014 at 13:27.
    Quote Quote  
  8. A Member since June, 2004 Keyser's Avatar
    Join Date
    Feb 2013
    Location
    Westernmost point of Europe
    Search Comp PM
    Well. At last I see this issue addressed. I've ALWAYS had this problem. With different computers, different external hard drives/enclosures and even different operating systems.

    It is stated tha the problem occurs with Windows 7 64-bit, and yes, I can confirm this, but it also happened to me when I used Windows XP. The one thing I can't confirm is if it also happened with FAT32 partitions.

    It has been driving me crazy for years. Everytime I copy a large file I have to perform a verify operation. And it happens with compressed and uncompressed files alike.
    "The greatest trick the Devil ever pulled was convincing the world he didn't exist."
    Quote Quote  
  9. DECEASED
    Join Date
    Jun 2009
    Location
    Heaven
    Search Comp PM
    Often the problem is in the cheapo chipsets used in the USB-to-IDE/SATA adapters:

    http://bigacid.wordpress.com/2008/12/08/jm20337-read-data-corruption-solution/

    --- but "often" is not the same as "always"
    Quote Quote  
  10. I've had that happen too. What I found was that the rear panel ports were fine, but the front ones would consistently corrupt large files. The cause was the internal cable between the MB and the front panel. With USB you should never discount a bad cable as the source of your problem (especially with the crap being sold now), the only way you can tell a cable is good is to test it on another PC, preferably one with the same USB chipset. Another thing to verify is the memory, do a full run with Memtest.
    Quote Quote  
  11. Member
    Join Date
    Aug 2005
    Location
    Europe
    Search PM
    I have some enclosures using Asmedia chipset and have the problem so is not a JMicron specific issue. I am sure this is not a cable problem either. Memory was already checked with memtest with no errors.
    Quote Quote  
  12. Member budwzr's Avatar
    Join Date
    Apr 2007
    Location
    City Of Angels
    Search Comp PM
    Originally Posted by nic2k4 View Post
    With USB you should never discount a bad cable as the source of your problem (especially with the crap being sold now)...
    Absolutely true. I just had that issue today.
    Quote Quote  
  13. Member
    Join Date
    Aug 2005
    Location
    Europe
    Search PM
    I have reproduced the problem on three different brand hard disks, both 2.5" and 3.5", using different cables and all are having the same problem. Same hard disks were used with my previous setup with no problem. It's not a cable problem.
    Quote Quote  
  14. For want of something else to try, I'd probably stick a USB3 PCI-e card in the PC to see what happens. They're pretty cheap these days. If the problem goes away when connected to the USB card, maybe returning the MB would be an idea.

    A quick look through the thread you linked to seemed to indicate most of the people with the problem had Dell computers. Given you said the MB and CPU are new I assume that's not the case here?

    You mentioned the problem occurs when using the USB3 ports, but what about the USB2 ports? Maybe I missed where you mentioned them.

    Given the USB ports are built into the MB, maybe checking for a BIOS update wouldn't be too silly. And resetting the BIOS too.

    You might want to try is installing Windows from scratch and testing copying before anything else is installed, aside from drivers. No Windows updates, nothing else.
    Quote Quote  
  15. Member
    Join Date
    May 2014
    Location
    Memphis TN, US
    Search PM
    Good advice, hello_hello. In my own experience with many IT departments and friends' PC's over the years, I have come to this conclusion:

    Dell =
    - My sister Ann's brother
    Quote Quote  
  16. Member
    Join Date
    Aug 2005
    Location
    Europe
    Search PM
    It's not a Dell. The problem occurs with both USB2 and USB3 ports. I already flashed the latest available bios. I will try a USB3 PCI-e card, before re-installing Windows.
    Last edited by kyrcy; 3rd Oct 2014 at 17:38.
    Quote Quote  
  17. Originally Posted by nic2k4 View Post
    I've had that happen too. What I found was that the rear panel ports were fine, but the front ones would consistently corrupt large files. The cause was the internal cable between the MB and the front panel. With USB you should never discount a bad cable as the source of your problem (especially with the crap being sold now), the only way you can tell a cable is good is to test it on another PC, preferably one with the same USB chipset. Another thing to verify is the memory, do a full run with Memtest.
    That pretty much covers my experience with this issue.

    1) Cabling. Did you check the *internal* cables from mobo to ports?
    2) Flaky enclosure controller. That doesn't seem to be the case here.
    3) Memory. How long did you run the test? I've seen it go through a few passes before bombing out. Also, one time memtest found nothing and I used the mobo's onboard tester instead, which finally found an error. Reseating the memory fixed it that time.
    4) Maybe the mobo has a defect?
    Pull! Bang! Darn!
    Quote Quote  
  18. Member
    Join Date
    Aug 2005
    Location
    Europe
    Search PM
    The problem occurs with the USB ports on the back of the motherboard that have no internal cabling. I have four RAM sticks and the problem occurs even when each one is installed separately so they can't be all faulty. The problem occurs with a USB3 PCI-e card too (less frequently but it still occurs) something that indicates that this is not a problem with the USB ports on the motherboard, but I am not sure if it rules out completely a problem with the motherboard.
    Last edited by kyrcy; 4th Oct 2014 at 02:29.
    Quote Quote  
  19. Did you try copy with Teracopy to see if problem related to Windows?
    Quote Quote  
  20. Member
    Join Date
    Aug 2005
    Location
    Europe
    Search PM
    Teracopy seems to copy with no problems, it's just not very practical to only use that for copying.
    Quote Quote  
  21. So high possibility its windows
    You have option to check by test installing windows 8 or windows 10 on different drive, disconnect other drivers before install
    Windows 8.1 Enterprise 90 day free trial Iso file : http://www.microsoft.com/en-us/evalcenter/evaluate-windows-8-1-enterprise
    Windows 10 Technical Preview you download for free, its 3.81 GB file: http://windows.microsoft.com/en-us/windows/preview-iso
    Last edited by roma_turok; 4th Oct 2014 at 11:04.
    Quote Quote  
  22. Member
    Join Date
    Nov 2014
    Location
    N. Calif
    Search Comp PM
    Upon seeing the file corruption with win7x64 on two different motherboards and with Seagate 5TB USB3 drive, I then bought a NexStar-HX USB3 drive enclosure and used that with a WD 1TB black sata drive installed. I saw the same type of corruption with the WD / enclosure setup. The corruption only occurs when writing to the external USB3 drive, not when reading from it. I do not see corruption on the internal sata interfaces.

    The USB3 ports where the corruption occurs are the ones on the backs of the motherboards.

    To test for file corruption after copying a TB or more of files, I use the program "exactfile" which reads and computes an md5 checksums for each file I copy. I compute checksums for the orignal files, and then exactfile applies these checksums to test if the copies are valid or corrupted. This is faster than comparing each file.

    The particular files I'm copying in this case are huge tables of specialized compressed data. In this case the files are about 100GB each.

    After seeing a checksum error, since the checksum doesn't reveal which bytes are bad, I use the cmd program "FC /b file1 file2 >logfile.txt" for a binary compare of only the particular file that had a checksum miscompare. Using "FC," I can see all the error miscompare bytes and location.

    On the Seagate 5TB drive, the miscompares are a 16K byte block. On the WD 1TB drive / enclosure setup, the miscompare is a 4K byte block. Perhaps this points to where the problem is, since the size of the error block is different on the two drives.

    This is not due to PC memory, since PC memory error would usually be a few bits wrong, and not a large block as I am seeing.

    The rate at which these errors occur for me is about once per TB copied, or once for every 10 of these 100GB files copied.

    It's helpful that others are testing and reporting seeing similar errors to mine.

    If this is due to hardware, it could be described as a metastability bad chip design. These are rare these days due to well established design rules for hardware.

    The error rate is low, but there should be no errors.

    I've not yet tried other software besides win7x64 file copying via windows explorer.
    Last edited by E<>MC^2; 14th Nov 2014 at 04:25.
    Quote Quote  
  23. Member Cornucopia's Avatar
    Join Date
    Oct 2001
    Location
    Deep in the Heart of Texas
    Search PM
    Originally Posted by kyrcy View Post
    Teracopy seems to copy with no problems, it's just not very practical to only use that for copying.
    Why is it not very practical, when it can integrate into the shell and replace Windows' own file copy (if you so choose)? I have 2 machines where I give it the Option to do that for every file copy operation (though it could also be set to automatically do it without choice). Seems about as easy as one would need.

    Scott
    Quote Quote  
  24. Something else to check, is your external HDD on a power-protected circuit? I was getting many errors copying files before I realized the AC/Heater turning on and off momentarily drained power from the drive.
    Quote Quote  
  25. Member
    Join Date
    Jul 2007
    Location
    United States
    Search Comp PM
    Originally Posted by kyrcy View Post
    Switching to Windows 7 32-bit is no a solution as I want to use more than 4GB of RAM. Only using Teracopy is not very practical. I might try exFAT, but I already have several NTFS hard disks with data that I need to use. And just to clarify things the problem appeared after I upgraded my system. I actually kept everything and replaced only the motherboard and the cpu. I am sure the problem is not the cpu and I was about to return the motherboard suspecting of a hardware issue, but I then found several posts from others about the problem. I have disabled the indexing and windows search services without luck, but the problem seems to go away by changing the removal policy for each hdd from quick removal to better performance. I still can't figure out why I am having the problem with the new motherboard and cpu and not with the previous one with the exact same windows configuration. And I don't know if by returning the motherboard I will get a replacement or a refund if the problem is only happening in Windows 7 64-bit.
    When you swapped out the MB and CPU did you do a complete fresh install or did you just reuse your original install. Even if the MB is the same model as the original there may be a different BIOS or other components that may be causing the errors. If you did a fresh install with the "exact same windows configuration", have you updated ALL the latest hardware drivers (you mention you installing and uninstalling the lastest Intel drivers, but the latest isn't always the greatest). Again, something that may have worked in your old system configuration may not be the best for your new MB and CPU.

    Is your PSU strong enough to power your new MB and CPU. PSUs lose power and efficiency over time. while it may be rated as powerful enough to run your new components, it may not be enough.

    Also, how is your power to your external drives. Internal drives are less prone to being affected by poor power conditions because the PSU regulates the power. If you have a UPS or line conditioner, try plugging the external drives into that to even out voltage spikes and drops.
    Quote Quote  
  26. Member
    Join Date
    Nov 2014
    Location
    N. Calif
    Search Comp PM
    Originally Posted by El Heggunte View Post
    Well, assuming the actual problem is in the Windows 7 64-bit edition(s), we the end-users don't have many choices...

    a) switch to a 32-bit edition of Windows 7, or downgrade to Windows Vista, or upgrade to Windows 8

    b) always use Teracopy as a workaround

    c) use a different file system (FYI, exFAT does not have the limitations of FAT32, and it's not "for pendrives only").
    At El Heggunte's suggestion, I used the latest freeware version of teracopy to determine if the errors could be eliminated.

    Results: I copied 30 Terabytes with teracopy and had zero file corruption, no errors.
    After the above, on the same hardware and software machine, I did a 1TB copy with Windows explorer and had a corrupted block in one file, as I had previously experienced.

    The teracopy has a lower error rate, and per my testing, zero errors.

    Copying large amounts of data to USB3 destination drives with Win7x64 and native file copying with windows explorer produces occasional random file corruption.

    I note that an area of my professional expertise is "test engineering." I have not yet tried Win7-32 bit or Win8 for this task.

    I have no affiliation with Microsoft or the vendor that offers teracopy.

    These errors have been observed on Win7x64 on two machines.
    Asus z77 motherboard core i7 16MB memory
    MSI z68 motherboard corei5 8mb memory

    The errors are that a 4K or 16K block contains entirely wrong data.

    Win7x64 might have a bug that is not seen when using teracopy.
    Quote Quote  
  27. Originally Posted by E<>MC^2 View Post
    These errors have been observed on Win7x64 on two machines.
    Asus z77 motherboard core i7 16MB memory
    MSI z68 motherboard corei5 8mb memory

    The errors are that a 4K or 16K block contains entirely wrong data.
    That certainly is troubling. You tested on 2 Intel chipsets, to rule out a hardware metastability issue it should be tested on AMD. If the problem is in the Windows API the error should still be there. But if the problem is at the driver level you would have to test with a completely different implementation like with a Ubuntu Live Disc.
    Quote Quote  
  28. Member
    Join Date
    Nov 2014
    Location
    N. Calif
    Search Comp PM
    Originally Posted by nic2k4 View Post
    Originally Posted by E<>MC^2 View Post
    These errors have been observed on Win7x64 on two machines.
    Asus z77 motherboard core i7 16MB memory
    MSI z68 motherboard corei5 8mb memory

    The errors are that a 4K or 16K block contains entirely wrong data.
    That certainly is troubling. You tested on 2 Intel chipsets, to rule out a hardware metastability issue it should be tested on AMD. If the problem is in the Windows API the error should still be there. But if the problem is at the driver level you would have to test with a completely different implementation like with a Ubuntu Live Disc.
    Although an AMD board is not at my immediate disposal, the two Intel based boards have differently named USB3 controllers. Asus has ASMedia XHC1, while the MSI has Renesas. The drivers have different versions. Windows 7 update indicates these are all up to date.

    At this point I do not need to test an AMD board in order to publicize it, since the problem is proven to be across different hardware vendors versions. It is possible that the problem is still due to a metastability, which only Microsoft's method in their software causes to surface. Although I have duplicated the problem with different vendors of USB3 drives and motherboards, the "failing" element may be a common or equivalently metastable chip. If so, that chip appears to be in widespread use. AMD may or may not also fail, but that would not affect the needs to start a more serious investigation of the bug.

    At this point it would be logical to enlist Microsoft's assistance, and publicize the results.

    This is a problem that affects the integrity of our data systems. The typical user, including for example a user working with Win7x64 computers at the US Department of Homeland security, would not know this problem occurred until it was too late, and the affect on data is roughly 64K bytes per terabyte copied to the destination drive. I note that copying a terabyte of data with windows explorer sometimes produces more than one 64k byte error, and sometimes none, so my sampling is not sufficient to say, more precisely, how after the corruption occurs.

    I have seen these kind of error in software before, and the obvious way to make the problem go away is to use teracopy instead of Microsoft Windows Explorer copying, but I doubt that Microsoft would publicly announce and agree that all users should switch to teracopy.

    What if a "no-fly" list was corrupted because of this error, and a terrorist allowed to fly because of this bug?
    Quote Quote  
  29. Member
    Join Date
    Nov 2014
    Location
    N. Calif
    Search Comp PM
    Additional information, based on reading the thread linked above by the person that started this thread, and repeated here below.

    My teracopy program has the "use system write cache" box NOT checked.

    The drive policies for the destination drives have the "Quick removal (default) option selected, which makes the "write caching policy" check box on-checked and inaccessible, i.e. the section is greyed out.

    I am going to do some additional testing with the drive policies set to "Better Performance" instead of "Quick Removal" for the destination drives, but haven't done that yet as of this post.

    https://social.technet.microsoft.com/Forums/windows/en-US/13a7426e-1a5d-41b0-9e16-1943...al-ntfs-drives
    Quote Quote  
  30. Do you use Microsoft Security Essentials antivirus by any chance?

    I had exactly the same problem more than a year ago and it really annoyed me as it corrupted a 'seemingly random' selection of my valuable backup files, and without verifying afterwards, it was impossible to tell if the file was corrupted or not as Windows did not complain about any errors. There were no logs or errors about disk controller faults or anything like that. It was only because the files were compressed, and the fact that they failed the checksum checks that I knew the files were corrupted.

    In my case, it occurred with both USB2 (4GB) pen drives, and also portable USB2 portable hard drives (between 320 MB and 1TB) on Windows 7 (64 bit), and all using the 'safe removal' option. I was 95% sure it was not the drives or the cables as they would work fine on other machines.

    The problem only seemed to be occur more when copying large file (at least 256MB, but usually larger when using hard drives). Maybe MSE locks the file while scanning and this interferes with the copying process, just a wild stab in the dark 'no-evidence-kind-of-guess'.

    After some experimenting, I discovered that removing MSE solved the file corruption problem copying using Windows Explorer. That was enough for me. After this point, I stopped using MSE, and since then no more problems ever with file corruption! Just to round it off, no problems before using MSE, no problems after removing it, but problems while it was installed.

    I don't think it's solely MSE's fault, as I can't believe the millions of other users would not have encountered the problem, but maybe the combination of that with my disk controller perhaps is responsible. Not suggesting your issue will be the same, but thought it was worth sharing.

    BTW, while experimenting I did try TeraCopy and the copies were verified to be good. I just preferred to use the built in explorer.
    Last edited by Zero123; 24th Nov 2014 at 17:04.
    Quote Quote  
Visit our sponsor! Try DVDFab and backup Blu-rays!