Major problems copying files to a new My Book 8TB WD External Hard Drive

[CaptainCatholic587]

Hello everyone!

I have been purchasing WD external hard drives for my PC for about 10 years now. I have always been extremely happy with their hard drives. Every two years or so, I buy a new WD hard drive (always upgrading space), and I just copy all of my old files to the new hard drive, and store the old hard drive away in my closet.

In any case, I recently bought a My Book 8TB drive. I ended up having major problems with it, problems I have never experienced in the 10 years I've been buying WD hard drives.

After opening the hard drive, I copied all of my files (all digital pictures) to it from my old hard drive (about 319 GB of pictures). I copied it straight to the main hard drive "folder" itself. Like usual, it turned out there were 2 or 3 files that had names that were too big, so it gave me an error message stating something like, "this file is too big to be copied." This was no big deal. So I changed the name of these two or three files on the original hard drive, and then copied them over to my new hard drive. All was good.

However, then I ran into some significant problems. After copying all files over, I then created a new folder called "Family Pictures" on the new hard drive. I moved all copied files into this new folder. The move completed without any error messages whatsoever. I checked to see how many files were under "Family Pictures" (under "Properties"), and it was the correct number.

However, when I was scrolling through the pictures, I noticed that the thumbnails did not populate for a few pictures. The file was there - the correct name was there too. But when I clicked on it, it was blank and I got the error message, "Windows Photo Viewer can't open this picture because either Photo Viewer doesn't support this file format, or you don't have the latest updates to Photo Viewer." I knew right away that this was a "bogus" error message; I don't need any updates, because the same exact picture worked just fine on my original hard drive when I opened it. I knew the copy to the new hard drive didn't work correctly.

This was highly disturbing to me, because I wouldn't have known in any way that certain files did not copy over correctly if I hadn't checked - I never received an error message about this (like I usually always do), and the file appeared in the "file count" accurately. I never had this problem before. Now I'm worried that I might have other files that didn't copy over correctly. It would take forever for me to check each and every individual file (there are thousands and thousands). How can I fix this? (And more importantly, how do I prevent this from happening in the future?). I don't know what to call these, but to me they are "ghost" files, because they appear to be there but they are not.

Additionally, I tried to delete the "ghost" files from my hard drive (so that I could rename the original files on my other hard drive with shorter names and then recopy them to the new hard drive). But it wouldn't let me! I received the error message, "The source file name(s) are larger than is supported by the file system. Try moving to a location which has a shorter path name, or try renaming to shorter name(s) before attempting this operation." I can't delete these files, I can't rename them, I can't move them - they're just stuck there. These "ghost" files are now cluttering my hard drive. This is very obnoxious! I googled how to fix this, but I cannot get the "open a command line" solution to work. I also tried disabling the recycling bin, but that doesn't work either. Does anyone know how I can delete these files???

Finally, I have another problem - when I go to "Properties" under the new hard drive, I see that I have 405 GB of used space on the hard drive. This is insane, because I only copied 319 GB of pictures to the hard drive. What?!?! Where is that extra 86GB of used space coming from? Is that because of the "ghost" files? When I click on the folder I created, "Family Pictures," it states that this folder takes up 330 GB of space on the hard drive, but the "size on disk" is 388GB. I have no idea what that means, but it seems like I got completely ripped off of 58 GB for no reason (again, is it because of the "ghost" files?).

All of these issues are extraordinarily frustrating, and I have never once encountered any of these issues with a WD hard drive. I am tempted to return this (it seems defective) and switch to another brand if these issues aren't solved. If anyone knows how to solve any of these problems, I would greatly appreciate it!!

Additionally - even if I do solve these problems, I am now extremely paranoid that I'm going to "think" all of my files copy over to my new hard drive, but in reality some of them might be missing. Is there anyway to absolutely guarantee that all files copy over correctly? I never would have known that some files didn't copy over correctly if I hadn't spent 10 minutes randomly clicking a few files.

Thanks so much! Any help is greatly appreciated!!
Tony

[lingyi]

I recommend reformatting the drive and running chkdsk before retrying to copy your files over. As I've stated many times before, external drives (especially 8tb) are the true bargain right how. However, the weak point is always the USB interface (which are highly prone to failure).

Next, add these utilities to your PC (they're all free):

Unlocker http://www.snapfiles.com/get/Unlocker.html
This will unlock almost any file that's locked by Windows. Be sure to doublecheck and unlock only the file(s) you want.

Teracopy https://www.codesector.com/teracopy
Highly recommended here and on other forums. Speeds up copying files and you can have it verify that the copied files are identical to the original.

ViceVersa http://www.tgrmn.com/free/.
Gives you a side by side view of the files in two folders and can be to set to run a CRC check to verify both folder/files are identical. The only real limitation of the free version is it doesn't recognize Unicode characters.

[Luke M]

I believe My Books come formatted in exFAT with 128K cluster size. That explains the wasted space (an average of 64K is wasted for each file). It's good for large files like video, though, because it improves performance when files are fragmented.

[lingyi]

Just checked and you're right. I never noticed because I always do a format and chkdsk before I start using my drives.

[pandy]

I would check if your operating system support 4KiB sector size - almost sure that 8GB HDD's are using 4KiB sector internally and it is well known that some older OS's may face some issues with this. Additionally if i can recommend something - always perform copy with verification if possible and after successful verification you may delete source files - never relay on move itself.

[CaptainCatholic587]

Hi everyone! Thank you so much for all of the responses, this has been incredibly helpful!

Lingyi, I am unfamiliar with chkdsk, as I have never done it before today, but I believe I just did it correctly. I just right-clicked the drive under "My Computer," clicked on "Properties," then "Tools," and then under "Error-checking" I clicked on "Check now." I then checked the box next to "Automatically fix file system errors," (but I did not check the box next to "Scan for and attempt recovery of bad sectors"), and then I clicked "Start."

It scanned the hard drive, and then I received the message, "No problems were found on the device or disk. It is ready to use." So I'm guessing that means my hard drive is in good shape and there shouldn't be any problems with it? (Is the purpose of chkdsk to make sure your external hard drive is "good to go" and doesn't contain any errors?").

Additionally, I can definitely reformat the drive if that's what you'd recommend! I am very unfamiliar with how to do this though - is there a particular format I should select? (NTFS?)

Luke M, it looks like you're correct! It looks like my new 8TB hard drive is indeed set to exFAT, and the Allocation unit size is set to 1024 kilobytes. That is way too big!!! No wonder I have all that wasted space. Thanks so much for your help in discovering this!

Should I just re-format the whole hard drive to NTFS? If so, can I do this by just right-clicking the hard drive (in "My Computer"), clicking "Format," changing "File system" to "NTFS (Default)," and changing the "Allocation unit size" to "4096 bytes" (the smallest available)? The box next to "Quick Format" is checked. Then I can just click "Start?"

Let me know if you guys think this would be the best way to do it! I don't mind losing all of the data on my new hard drive, as I still have all my original data on my original hard drive. I'd rather just have it be in the correct format.

By the way lingyi, you mentioned that USB interfaces are prone to failure. Are there any particular issues I should be aware of? (I actually have 4 total external hard drives to back up all of my data - I have one plugged into my computer at all times, and then I have one in the same room which is always unplugged except for when I back up files to it. I also have another one on another floor of my house. Finally, my Dad keeps an external hard drive of all my files at his house, which is miles away, and I back up all my data to that drive every 6 months). I figure that with all these backup drives in multiple locations my data is pretty safe.

Thank you all very, very much for the help again, I appreciate it!!
Tony

[CaptainCatholic587]

I recommend reformatting the drive and running chkdsk before retrying to copy your files over. As I've stated many times before, external drives (especially 8tb) are the true bargain right how. However, the weak point is always the USB interface (which are highly prone to failure).

Next, add these utilities to your PC (they're all free):

Unlocker http://www.snapfiles.com/get/Unlocker.html
This will unlock almost any file that's locked by Windows. Be sure to doublecheck and unlock only the file(s) you want.

Teracopy https://www.codesector.com/teracopy
Highly recommended here and on other forums. Speeds up copying files and you can have it verify that the copied files are identical to the original.

ViceVersa http://www.tgrmn.com/free/.
Gives you a side by side view of the files in two folders and can be to set to run a CRC check to verify both folder/files are identical. The only real limitation of the free version is it doesn't recognize Unicode characters.

PS: About these utilities, thank you very much for the suggestions lingyi! Regarding unlocker, are you recommending that I use that to "unlock" the "ghost" files that I can't seem to be able to delete?

Regarding Teracopy, that sounds pretty awesome! I will see if I can try that out. Regarding how it speeds up copying files though, does that ever have any negative effect on copying files? (Sometimes I get paranoid when using external software that it might "mess up" something). It's strange - I have literally never experienced problems like this before where I copied files to an external hard drive, and they just weren't there and no error message popped up to warn me (is this typical, or is my example a rare occurrence? I used to always get an error message if a copy wasn't successful, usually because of a long file name). But if you think it's reliable and the added speed won't "mess up" the copy, I will definitely try that out!! It would be great to verify that anything I copy from one drive to the next is identical.

Thanks again, much appreciated!!

[CaptainCatholic587]

I would check if your operating system support 4KiB sector size - almost sure that 8GB HDD's are using 4KiB sector internally and it is well known that some older OS's may face some issues with this. Additionally if i can recommend something - always perform copy with verification if possible and after successful verification you may delete source files - never relay on move itself.

Hi Pandy! Awesome, thanks for the information! Do you happen to know how to check if your operating system supports 4KiB sector size? I googled it a bit but can't seem to find out how (I use Windows 7).

Additionally, I would definitely like to perform a copy with verification - do you know how I can do this? (Do I need an external program, possibly one of the ones lingyi mentioned like TeraCopy?).

Thanks again!

[Luke M]

Luke M, it looks like you're correct! It looks like my new 8TB hard drive is indeed set to exFAT, and the Allocation unit size is set to 1024 kilobytes. That is way too big!!! No wonder I have all that wasted space. Thanks so much for your help in discovering this!

Ah, interesting. I mentioned 128K because that's the most commonly used cluster size for exFAT (it supports up to 32MB). 1MB is big!

Should I just re-format the whole hard drive to NTFS? If so, can I do this by just right-clicking the hard drive (in "My Computer"), clicking "Format," changing "File system" to "NTFS (Default)," and changing the "Allocation unit size" to "4096 bytes" (the smallest available)? The box next to "Quick Format" is checked. Then I can just click "Start?"

Yes, that should work and at least solve the wasted space problem.

[lingyi]

Hi everyone! Thank you so much for all of the responses, this has been incredibly helpful!

Lingyi, I am unfamiliar with chkdsk, as I have never done it before today, but I believe I just did it correctly. I just right-clicked the drive under "My Computer," clicked on "Properties," then "Tools," and then under "Error-checking" I clicked on "Check now." I then checked the box next to "Automatically fix file system errors," (but I did not check the box next to "Scan for and attempt recovery of bad sectors"), and then I clicked "Start."

It scanned the hard drive, and then I received the message, "No problems were found on the device or disk. It is ready to use." So I'm guessing that means my hard drive is in good shape and there shouldn't be any problems with it? (Is the purpose of chkdsk to make sure your external hard drive is "good to go" and doesn't contain any errors?").

Additionally, I can definitely reformat the drive if that's what you'd recommend! I am very unfamiliar with how to do this though - is there a particular format I should select? (NTFS?)

Luke M, it looks like you're correct! It looks like my new 8TB hard drive is indeed set to exFAT, and the Allocation unit size is set to 1024 kilobytes. That is way too big!!! No wonder I have all that wasted space. Thanks so much for your help in discovering this!

Should I just re-format the whole hard drive to NTFS? If so, can I do this by just right-clicking the hard drive (in "My Computer"), clicking "Format," changing "File system" to "NTFS (Default)," and changing the "Allocation unit size" to "4096 bytes" (the smallest available)? The box next to "Quick Format" is checked. Then I can just click "Start?"

Let me know if you guys think this would be the best way to do it! I don't mind losing all of the data on my new hard drive, as I still have all my original data on my original hard drive. I'd rather just have it be in the correct format.

By the way lingyi, you mentioned that USB interfaces are prone to failure. Are there any particular issues I should be aware of? (I actually have 4 total external hard drives to back up all of my data - I have one plugged into my computer at all times, and then I have one in the same room which is always unplugged except for when I back up files to it. I also have another one on another floor of my house. Finally, my Dad keeps an external hard drive of all my files at his house, which is miles away, and I back up all my data to that drive every 6 months). I figure that with all these backup drives in multiple locations my data is pretty safe.

Thank you all very, very much for the help again, I appreciate it!!
Tony

Chkdsk performs a scan of your hard drive for corrupted, misplaced files or parts of files. There are better utilities for a more thorough scan, but since your drive is new and passed the logical scan, you should be good to go (after a re-format).

The default for Windows is NTFS and 4K sectors. Do a quick format and you're good.

9 out of 10 times when someone posts here and other forums that an external drive has failed, the advice is to remove the drive from the case. And in 8+ of those 9 cases, the drive is fine, but the interface failed.

I apologize to the regulars who have seen me post his dozens of times, but it's simple economics. An external HDD can go for nearly half the price of an identical bare HDD. So either the HDD manufacturers are making a huge profit on bare drives or they're cutting costs on the case and USB interface.

I've owned dozens of external HDDs, both desktop and portable and I've given up on using them as designed (I remove the drives from the case for use as an internal) as I've had almost all of them fail, usually at the worst moment. I suspect constantly plugging and unplugging and the cheap transformer power supplies included with drives are subjecting the USB to an electrical shock, eventually killing it.

[lingyi]

I recommend reformatting the drive and running chkdsk before retrying to copy your files over. As I've stated many times before, external drives (especially 8tb) are the true bargain right how. However, the weak point is always the USB interface (which are highly prone to failure).

Next, add these utilities to your PC (they're all free):

Unlocker http://www.snapfiles.com/get/Unlocker.html
This will unlock almost any file that's locked by Windows. Be sure to doublecheck and unlock only the file(s) you want.

Teracopy https://www.codesector.com/teracopy
Highly recommended here and on other forums. Speeds up copying files and you can have it verify that the copied files are identical to the original.

ViceVersa http://www.tgrmn.com/free/.
Gives you a side by side view of the files in two folders and can be to set to run a CRC check to verify both folder/files are identical. The only real limitation of the free version is it doesn't recognize Unicode characters.

PS: About these utilities, thank you very much for the suggestions lingyi! Regarding unlocker, are you recommending that I use that to "unlock" the "ghost" files that I can't seem to be able to delete?

Regarding Teracopy, that sounds pretty awesome! I will see if I can try that out. Regarding how it speeds up copying files though, does that ever have any negative effect on copying files? (Sometimes I get paranoid when using external software that it might "mess up" something). It's strange - I have literally never experienced problems like this before where I copied files to an external hard drive, and they just weren't there and no error message popped up to warn me (is this typical, or is my example a rare occurrence? I used to always get an error message if a copy wasn't successful, usually because of a long file name). But if you think it's reliable and the added speed won't "mess up" the copy, I will definitely try that out!! It would be great to verify that anything I copy from one drive to the next is identical.

Thanks again, much appreciated!!

I've used all three of these programs for years and highly recommend them, especially Teracopy. Windows Copy does a poor job of reporting copy or move errors. What you think may be a successful copy or move often isn't. When you use Teracopy, just click the Verify box and it will run a CRC check on the files once the copying is done. Don't be scared of the estimated completion time (for copy/move and verify) which may show hours or days for completion. It will change on a file by file basis. Keep in mind that moving multiple small files will take longer than multiple large files.

The only catch is that AFAIK, Teracopy will only let you verify your files if you've checked the verify box before starting or check the verify box while the Window is still open. This is where ViceVersa comes in as it allows you verify files after they've been copied/moved.

As for Unlocker. Yes, it will allow you to unlock and delete "ghost files" that are locked by some unknown Windows process. I notice Win 10 files more often then previous versions. Just be careful if you're unlocking hidden files as it's possible to delete some critical files that shouldn't be deleted (that's why they're hidden!).

[pandy]

I would format HDD with NTFS, i would change cluster size to 64KiB (good trade-off between space loss and speed, you should also reduce MFT size and its fragmentation with this) of course if you have many small files (smaller than 64KiB) then this may be not a best idea. Support for large (big) 4KiB sector on Windows https://support.microsoft.com/en-us/help/2510009/microsoft-support-policy-for-4k-secto...ves-in-windows .

[CaptainCatholic587]

Luke M, it looks like you're correct! It looks like my new 8TB hard drive is indeed set to exFAT, and the Allocation unit size is set to 1024 kilobytes. That is way too big!!! No wonder I have all that wasted space. Thanks so much for your help in discovering this!

Ah, interesting. I mentioned 128K because that's the most commonly used cluster size for exFAT (it supports up to 32MB). 1MB is big!

Should I just re-format the whole hard drive to NTFS? If so, can I do this by just right-clicking the hard drive (in "My Computer"), clicking "Format," changing "File system" to "NTFS (Default)," and changing the "Allocation unit size" to "4096 bytes" (the smallest available)? The box next to "Quick Format" is checked. Then I can just click "Start?"

Yes, that should work and at least solve the wasted space problem.

Luke M I did this and it worked perfectly - thanks for helping me solve that part of the problem!!

[CaptainCatholic587]

Hi everyone! Thank you so much for all of the responses, this has been incredibly helpful!

Lingyi, I am unfamiliar with chkdsk, as I have never done it before today, but I believe I just did it correctly. I just right-clicked the drive under "My Computer," clicked on "Properties," then "Tools," and then under "Error-checking" I clicked on "Check now." I then checked the box next to "Automatically fix file system errors," (but I did not check the box next to "Scan for and attempt recovery of bad sectors"), and then I clicked "Start."

It scanned the hard drive, and then I received the message, "No problems were found on the device or disk. It is ready to use." So I'm guessing that means my hard drive is in good shape and there shouldn't be any problems with it? (Is the purpose of chkdsk to make sure your external hard drive is "good to go" and doesn't contain any errors?").

Additionally, I can definitely reformat the drive if that's what you'd recommend! I am very unfamiliar with how to do this though - is there a particular format I should select? (NTFS?)

Luke M, it looks like you're correct! It looks like my new 8TB hard drive is indeed set to exFAT, and the Allocation unit size is set to 1024 kilobytes. That is way too big!!! No wonder I have all that wasted space. Thanks so much for your help in discovering this!

Should I just re-format the whole hard drive to NTFS? If so, can I do this by just right-clicking the hard drive (in "My Computer"), clicking "Format," changing "File system" to "NTFS (Default)," and changing the "Allocation unit size" to "4096 bytes" (the smallest available)? The box next to "Quick Format" is checked. Then I can just click "Start?"

Let me know if you guys think this would be the best way to do it! I don't mind losing all of the data on my new hard drive, as I still have all my original data on my original hard drive. I'd rather just have it be in the correct format.

By the way lingyi, you mentioned that USB interfaces are prone to failure. Are there any particular issues I should be aware of? (I actually have 4 total external hard drives to back up all of my data - I have one plugged into my computer at all times, and then I have one in the same room which is always unplugged except for when I back up files to it. I also have another one on another floor of my house. Finally, my Dad keeps an external hard drive of all my files at his house, which is miles away, and I back up all my data to that drive every 6 months). I figure that with all these backup drives in multiple locations my data is pretty safe.

Thank you all very, very much for the help again, I appreciate it!!
Tony

Chkdsk performs a scan of your hard drive for corrupted, misplaced files or parts of files. There are better utilities for a more thorough scan, but since your drive is new and passed the logical scan, you should be good to go (after a re-format).

The default for Windows is NTFS and 4K sectors. Do a quick format and you're good.

9 out of 10 times when someone posts here and other forums that an external drive has failed, the advice is to remove the drive from the case. And in 8+ of those 9 cases, the drive is fine, but the interface failed.

I apologize to the regulars who have seen me post his dozens of times, but it's simple economics. An external HDD can go for nearly half the price of an identical bare HDD. So either the HDD manufacturers are making a huge profit on bare drives or they're cutting costs on the case and USB interface.

I've owned dozens of external HDDs, both desktop and portable and I've given up on using them as designed (I remove the drives from the case for use as an internal) as I've had almost all of them fail, usually at the worst moment. I suspect constantly plugging and unplugging and the cheap transformer power supplies included with drives are subjecting the USB to an electrical shock, eventually killing it.

Thanks so much again, lingyi!! The chkdsk turned out great, no errors. However, I decided to run it one more time while checking the 2nd box ("Scan for and attempt recovery of bad sectors"). 6 hours went by and it wasn't even 1% of the way through (I guess because it's a massive 8TB hard drive), so I was worried it'd take over a week to finish up - I ended up canceling it. I had a lot of hesitation, as I didn't want to mess up my hard drive (looks like canceling it can sometimes cause errors) - but fortunately everything seems to have turned out fine!

I'll definitely be sure to run the chkdsk each time I get a new hard drive though - I'll just keep the 2nd box unchecked. Thanks again!

[CaptainCatholic587]

I recommend reformatting the drive and running chkdsk before retrying to copy your files over. As I've stated many times before, external drives (especially 8tb) are the true bargain right how. However, the weak point is always the USB interface (which are highly prone to failure).

Next, add these utilities to your PC (they're all free):

Unlocker http://www.snapfiles.com/get/Unlocker.html
This will unlock almost any file that's locked by Windows. Be sure to doublecheck and unlock only the file(s) you want.

Teracopy https://www.codesector.com/teracopy
Highly recommended here and on other forums. Speeds up copying files and you can have it verify that the copied files are identical to the original.

ViceVersa http://www.tgrmn.com/free/.
Gives you a side by side view of the files in two folders and can be to set to run a CRC check to verify both folder/files are identical. The only real limitation of the free version is it doesn't recognize Unicode characters.

PS: About these utilities, thank you very much for the suggestions lingyi! Regarding unlocker, are you recommending that I use that to "unlock" the "ghost" files that I can't seem to be able to delete?

Regarding Teracopy, that sounds pretty awesome! I will see if I can try that out. Regarding how it speeds up copying files though, does that ever have any negative effect on copying files? (Sometimes I get paranoid when using external software that it might "mess up" something). It's strange - I have literally never experienced problems like this before where I copied files to an external hard drive, and they just weren't there and no error message popped up to warn me (is this typical, or is my example a rare occurrence? I used to always get an error message if a copy wasn't successful, usually because of a long file name). But if you think it's reliable and the added speed won't "mess up" the copy, I will definitely try that out!! It would be great to verify that anything I copy from one drive to the next is identical.

Thanks again, much appreciated!!

I've used all three of these programs for years and highly recommend them, especially Teracopy. Windows Copy does a poor job of reporting copy or move errors. What you think may be a successful copy or move often isn't. When you use Teracopy, just click the Verify box and it will run a CRC check on the files once the copying is done. Don't be scared of the estimated completion time (for copy/move and verify) which may show hours or days for completion. It will change on a file by file basis. Keep in mind that moving multiple small files will take longer than multiple large files.

The only catch is that AFAIK, Teracopy will only let you verify your files if you've checked the verify box before starting or check the verify box while the Window is still open. This is where ViceVersa comes in as it allows you verify files after they've been copied/moved.

As for Unlocker. Yes, it will allow you to unlock and delete "ghost files" that are locked by some unknown Windows process. I notice Win 10 files more often then previous versions. Just be careful if you're unlocking hidden files as it's possible to delete some critical files that shouldn't be deleted (that's why they're hidden!).

Nice! I just downloaded TeraCopy. It is AMAZING, I absolutely love it - thank you very much!!! Being able to verify that each and every file copied correctly is a life-saver. Great tool! Thanks again!!

[CaptainCatholic587]

I would format HDD with NTFS, i would change cluster size to 64KiB (good trade-off between space loss and speed, you should also reduce MFT size and its fragmentation with this) of course if you have many small files (smaller than 64KiB) then this may be not a best idea. Support for large (big) 4KiB sector on Windows https://support.microsoft.com/en-us/help/2510009/microsoft-support-policy-for-4k-secto...ves-in-windows .

Awesome, the reformat seems to have worked great - thanks again!!

[Nelson37]

First, a few mb of wasted space on an 8 TB drive are not something I would worry about.

Multiple files which failed a simple copy operation would be a major concern to me. I would invest some time to Isolate and Identify the problem.

Copy the suspect files to the primary hard drive, and to the external, several times. If there is no failure, on MULTIPLE attempts with MULTIPLE files, then that indicates that the external drive and/or the USB interface failed, randomly, intermittantly, and unpredictably. That would be the problem that I would be concerned with.

USB interfaces for external drives SUCK, and they SUCK EVEN WORSE when power is supplied through the USB interface. First, use an internal drive for any reliable copy, period, no exceptions. Next, use an external drive with a separate power plug, to eliminate the most common problem with such drives, and keep multiple copies for the day when the interface suffers catastrophic failure and the data is totally lost.

Also, checking an 8 TB drive for errors takes a long time, especially through a USB interface. If you don't want to spend the necessary time to check for errors, either get smaller drives, get internal drives, or simply accept the occasional data loss that comes from drive errors.

" I have a problem, but I do not want to spend the required amount of time to fix it, so I will attempt to address other, completely unrelated problems and ignore the primary issue."

[CaptainCatholic587]

First, a few mb of wasted space on an 8 TB drive are not something I would worry about.

Multiple files which failed a simple copy operation would be a major concern to me. I would invest some time to Isolate and Identify the problem.

Copy the suspect files to the primary hard drive, and to the external, several times. If there is no failure, on MULTIPLE attempts with MULTIPLE files, then that indicates that the external drive and/or the USB interface failed, randomly, intermittantly, and unpredictably. That would be the problem that I would be concerned with.

USB interfaces for external drives SUCK, and they SUCK EVEN WORSE when power is supplied through the USB interface. First, use an internal drive for any reliable copy, period, no exceptions. Next, use an external drive with a separate power plug, to eliminate the most common problem with such drives, and keep multiple copies for the day when the interface suffers catastrophic failure and the data is totally lost.

Also, checking an 8 TB drive for errors takes a long time, especially through a USB interface. If you don't want to spend the necessary time to check for errors, either get smaller drives, get internal drives, or simply accept the occasional data loss that comes from drive errors.

" I have a problem, but I do not want to spend the required amount of time to fix it, so I will attempt to address other, completely unrelated problems and ignore the primary issue."

Hi Nelson! Thanks so much for the post! Haha I hear you about the "I have a problem but don't want to spend the required amount of time to fix it" mentality - I try not to have it, but can succumb to it on occasion. Your suggestion to copy the problem files to other hard drives is a great idea, though unfortunately I won't be able to do that now as I have renamed all the problem files, giving them much shorter names. Fortunately, I think I may have solved the problem (I sure hope so anyway - any additional feedback is greatly appreciated).

As you said, "Multiple files which failed a simple copy operation would be a major concern to me," and yes, that was definitely my biggest concern. I should specify - the problem files actually did not fail the initial copy - they copied to the new hard drive just fine. But, when I moved the same files on the new hard drive to a brand new folder - that was when I encountered the problem. I determined that this was because the file names/folders were entirely too large (well over 255 limit). They must have not been "over the limit" to copy to the new hard drive, but they were over the limit when I then created a new folder (on the new hard drive) and then moved the files to the new folder. I believe this is what caused the error. The only files I had a problem with were files that had extremely long names (or were saved in subfolders with extremely long names).

However, what was alarming was that Windows Explorer did not give me any kind of error message to let me know that my files didn't copy over correctly. I only stumbled on this by chance. So I believe it was Windows Explorer that failed, not my new hard drive. I did run a chkdsk on my new hard drive, which said the hard drive was fine (though I admit I didn't select the 2nd option on the chkdsk, just the first).

To counter this problem, I'm going to start using smaller names when labeling folders and files. However, the biggest change I'm making is that I'm now using TeraCopy to copy all of my files from now on (I'm also erasing all the data on my new hard drive, and recopying all the data over to the new hard drive via TeraCopy to ensure it all copies over correctly). Nelson, in your opinion, do you think this is enough to prevent such a problem from occurring again? As long as TeraCopy verifies that every file is copied over correctly, I should be good, right? (Or has anyone encountered errors with TeraCopy, i.e. it "verifies" that a file copied over correctly but it didn't really? I would be concerned if TeraCopy has reliability issues, though it sounds like everyone here trusts it and has nothing but good things to say about it)

To elaborate on my hard drives, I have always been very careful with my data (at least I think!). I am in the process of digitizing all of my pictures/videos from my childhood, as well as my Dad's childhood (tons of 8mm videos from the 1950s and 1960s), and now my kids' childhood (so I have videos from 1955 through 2018, which is pretty cool - though a lot of data!). I enjoy having all of it on one single hard drive, as it's really cool to click around and go from 1963 to 1997 to 2014 - my kids really enjoy this. This is why I have 8TB hard drives as opposed to let's say four 2TB hard drives.

I have one 8TB hard drive plugged into a WDTV on our 60" TV in our family room, so we can watch home videos from any year at any time on our TV with a couple of simple clicks. Then, I have two external hard drives in my upstairs office with all the same data (one of these hard drives is plugged into my computer at all times, whereas the 2nd one is unplugged at all times except when backing up new data). Finally, my Dad (who lives several miles away) has a hard drive stored at his house with all this data. So I've felt very good about maintaining all my data, as several terrible things would have to happen all at the same time in order for me to lose all of it, haha, and now with TeraCopy I feel even better about it. Though if anyone has any suggestions, feel free to let me know! (Note: All my external hard drives have their power supplied via a separate power cord that plugs into the wall).

I was, however, totally unaware that people have problems with USB interfaces and external hard drives. I have been using WD external hard drives since 2007 and literally never once experienced any sort of problem until now - though again, I think it was caused by the combination of Windows Explorer and my file names being far too long (also, I was copying from an NTFS hard drive to an exFAT hard drive - I reformatted the new one to an NTFS hard drive to match the original, just in case that caused problems too). I don't believe the problem at anything to do with the hard drive itself.

Anyway, thanks so much everyone again for all your help! And thanks for the additional information Nelson! Any other advice or things I should be aware of would be greatly appreciated.

[Nelson37]

The long file and folder names were likely the problem.

Do not forget that the actual name of the file INCLUDES both the file, AND the folder name.

The USB drive interface is just not long-term reliable. Have seen many, many failures over the decades. I personally hate them with a passion, and do not use them at all. Plug the drive in internally for a reliable, safe, and much faster data transfer.

[CaptainCatholic587]

The long file and folder names were likely the problem.

Do not forget that the actual name of the file INCLUDES both the file, AND the folder name.

The USB drive interface is just not long-term reliable. Have seen many, many failures over the decades. I personally hate them with a passion, and do not use them at all. Plug the drive in internally for a reliable, safe, and much faster data transfer.

Awesome, thanks Nelson, really appreciate it! (And yes, these files were actually saved in a sub-folder with a long name, under a folder with a long name, and the file itself had a long name - not very smart on my end!).

Thanks again!!

[CaptainCatholic587]

Hello all!

A quick update (and another quick question!). I am finally just about finished with copying all of my video and picture files to several new hard drives I bought over the summer. I've been using TeraCopy and I absolutely love it. I haven't experienced any problems with it yet.

However, I do have one last minor issue that is a bit puzzling. I've been using a 4TB WD external hard drive for the past 3 years. I just bought three new 8TB WD external hard drives to upgrade my storage needs. I'll refer to the three new 8TB drives as "hard drive A" and "hard drive B" (I haven't copied anything on to "hard drive C" yet). I copied everything from my 4TB drive to hard drive A using TeraCopy. It came out great - every last file was verified, and nothing failed or skipped. Then, I copied everything directly from hard drive A straight to hard drive B using TeraCopy. Once again, it came out great - every last file was verified, and nothing failed or skipped. I went through several files on both hard drives manually - everything I checked looked great (though I obviously didn't look at every last file manually).

When I go to Windows Explorer and right-click on the various folders on hard drive A and hard drive B, everything looks the same (i.e. when I right-click on the "Family Videos" folder on drive A, and compare it to when I right-click on the "Family Videos" folder on drive B, it all looks the same - they each take up exactly 2.28TB (2,516,974,885,952 bytes), containing 31,859 Files and 2,998 Folders). So that's excellent. And every last folder on both hard drives matches up perfectly and is identical. In fact, when I select multiple folders on both hard drives (every last folder - there are 5 of them total), they also come out identical. So that tells me they match perfectly.

However, when I simply right-click the external hard drives themselves in Windows Explorer (i.e. I right click "My Book (A" and "My Book (B"), they don't match up as perfectly. They both have 3.74TB of used space, and they both have 3.53TB of free space. However, hard drive A has "4,117,918,371,840 bytes" of used space, while hard drive B has "4,117,899,665,408 bytes" of used space. So I'm not sure why there's a slight difference in the amount of used space each one has - they should have the exact same files on them, nothing more, nothing less, and TeraCopy verified this.

Both hard drives are formatted the same way (NTFS with an allocation unit size of 4096 bytes), so I don't understand what would cause this slight difference.

More than likely, I'm probably just being paranoid haha, because like I said, when I compare each individual folder, they are identical on each drive. It's just when I compare the entire drives to each other, there's a slight difference - I'm guessing it's something over my head haha, such as maybe every external hard drive has a slightly different amount of space on it?

If anyone could comment with a reasonable explanation to extinguish my digital anxieties hahaha, that would be fantastic.

Thank you all again so much for all of your help!
Tony

PS: There also isn't anything saved "outside" of folders (i.e. it's not like "hard drive A" has five folders but also a couple of random files saved outside the five folders, while "hard drive B" has all five folders but does not have the couple of "random files" saved outside the five folders - from what I see, both drive A and B should be identical).

[Bernix]

Hi,
didn't read all the thread and I bet somebody sugested it already. If you have for sure on both disc identical folders, try throught properties check size. There are 2 size. Size and size on disc. I think you are interested in normal size, they should be identical. This doesn't work for whole HDD of course.


Bernix

[lingyi]

Run ViceVersa and it will show the hidden files/folders and any difference, including date between the files. You may be able to copy the files to make both drives match exactly. If you're unable copy the files or want to delete them, post the filenames and size here as some may be critical system files.

Edit: Once you've copied/deleted the extra files, make sure to empty the Recycle Bin as these will still show. Ideally, once you've done this, ViceVersa will report that both drives are identical. I switched to ViceVersa Pro because it allows ignoring user selected files/folders.

Note that WDTV does (and other media players may) add files to the drive. The WDTV adds an index file (which can be large) and bookmarks whenever you view a file. These can be safely deleted, but will be recreated (taking time to index) when you connect the drive and view videos.

[Bernix]

Hi,
not sure if win10 is supported, but according to http://manual.winmerge.org/Compare_dirs.html you can compare folders too. It is called WinMerge. Its main purpose is compare text files difference, but can be used as far as in manual, for comparing directories too. I hope it is free. It is open source and it is bit old. But can help you probably to be sure the files are identical.


Bernix

[CaptainCatholic587]

Run ViceVersa and it will show the hidden files/folders and any difference, including date between the files. You may be able to copy the files to make both drives match exactly. If you're unable copy the files or want to delete them, post the filenames and size here as some may be critical system files.

Edit: Once you've copied/deleted the extra files, make sure to empty the Recycle Bin as these will still show. Ideally, once you've done this, ViceVersa will report that both drives are identical. I switched to ViceVersa Pro because it allows ignoring user selected files/folders.

Note that WDTV does (and other media players may) add files to the drive. The WDTV adds an index file (which can be large) and bookmarks whenever you view a file. These can be safely deleted, but will be recreated (taking time to index) when you connect the drive and view videos.

Hey lingyi, thanks so much for the advice! I just downloaded ViceVersa, it looks like a great program. When I used ViceVersa to compare my "T" and "E" drives, a message immediately came up that stated, "The two folders are identical (excluding filtered files and sub-folders)." Does this mean that ViceVersa doesn't scan sub-folders? (Or do I need to upgrade to the paid version for that?). My main hard drives only have 5 main folders, but each folder has hundreds of sub-folders.

However, I did use ViceVersa to compare two other hard drives that I just copied over. The hard drives I referred to in my previous post store all of my "home videos." However, I have another hard drive (my "G" drive) that houses all my Sony Vegas Pro editing files. I recently copied everything from my "G" drive to a new hard drive (my new "F" drive) via TeraCopy. TeraCopy said it copied all of my files correctly, nothing skipped or failed, and it verified everything.

However, when comparing the two hard drives via Windows Explorer afterward, Windows Explorer said that my "G" drive had one extra file compared to my "F" drive. ViceVersa actually noticed that not 1, but 6 files were not copied over. According to ViceVersa, my "G" drive had 6,726 files and folders, but my "F" drive had only 6,720 files and folders. However, when looking at the missing files (the ones my "G" drive had but my "F" drive did not have), they were all insignificant files to my knowledge and included the following:

1) AlbumArt_{08B69FAE-82BD-4F07-988F-CE9B8C6AFFBD}_Large.jpg
2) AlbumArt_{08B69FAE-82BD-4F07-988F-CE9B8C6AFFBD}_Small.jpg
3) AlbumArtSmall.jpg
4) autorun.inf
5) desktop.ini
6) Folder.jpg

So all of these files were supposedly on my "G" drive, but not on my new "F" drive. Oddly enough, I did not recognize any of these files (perhaps they are all something installed on the actual hard drives with the software? Maybe even just "thumbnail" pictures for the hard drive itself?). I even searched my "G" drive for #1-#3 of these files (by typing their names into the "Search" button on Microsoft Explorer), and NONE of them came up on my "G" drive. Strange!

I then searched for #4. At first, I thought that I did find that. I thought it was the "autorun" button for Sony Acid Music Studio Version 9. I installed Sony Acid directly on to my "G" hard drive (instead of my Desktop - I don't remember why I did this, but I did, haha). This file was saved under the folder "Sony Acid Music Studio Version 9.0" which was directly on my "G" hard drive. Every single other file from the "Sony Acid Music Studio Version 9.0" folder copied over correctly to my "F" drive except for this one. However, I then checked my "F" drive and this file IS there. So TeraCopy did copy it over. So perhaps the Sony Acid Music Studio "autorun" file is NOT the file that ViceVersa recognized as not being copied over from the "G" drive to the "F" drive. Perhaps the "autorun" file ViceVersa is referring to is another "invisible" file that's just part of the "G" drive's software?

When I searched for #5, "desktop.ini," two files came up on both the "G" drive and the "F" drive. So according to Windows Explorer, both hard drives have two desktop.ini files and should match (but according to ViceVersa, the "G" drive has one that the "F" drive doesn't have, even though it's not visible on my "G" drive via Windows Explorer).

When I searched for #6, "Folder.jpg," one file came up on both the "G" drive and the "F" drive. So according to Windows Explorer, both hard drives have a file labeled "folder_up" and should match (but according to ViceVersa, the "G" drive has a "Folder.jpg" file that the "F" drive doesn't have, even though it's not visible on my "G" drive via Windows Explorer). Again, very strange.

I used ViceVersa to copy the 6 files from the "G" drive to my "F" drive. After I did that, according to ViceVersa, both drives now have exactly 6,726 files and folders, and the "Total Size" of both drives is exactly "322,792,221,639." Oddly enough though, Windows Explorer still says that my "G" drive has "97,811 Files" and my "F" drive has "97,810" files (one less file). Though Windows Explorer says both drives have the same exact number of Folders (7,593). So Windows Explorer has not detected any changes since I used ViceVersa to copy the 6 missing files. Very strange. I am assuming that ViceVersa's information is much more accurate than Windows's information, correct?

So basically... it doesn't appear that TeraCopy failed to copy any of my files over from my "G" drive to my "F" drive! But for some reason, both Windows Explorer and ViceVersa tell me that my "G" drive and "F" drives have a different number of files on them, but they appear to be "invisible" files that I can't see even on my original "G" drive. Very strange!

Going back to the issue from my previous post, is it probable that Windows Explorer notices a different number of "used bytes" between my "A" and "B" drives due to similar "invisible" files? (And ViceVersa isn't catching this because the problem there isn't visible on my main folders, but in sub-folders?).

I am pretty happy to see that no "major" file was skipped via my TeraCopy copy. However, I am still confused about the differences in files among the drives. Any light that anyone can shed on this strange situation would be awesome, hahaha.

Thank you all very much again for your help! (And PS: Bernix, thank you very much for your help as well! You are correct, I am more interested in "exact size" as opposed to "size on disk," although unfortunately Windows Explorer labels both figures as being different. I will try WinMerge later tonight though!!! Thanks again!)

[lingyi]

I'm a bit confused by the numerous drive names in your post, but I think I'm following what you're saying.

You don't need to get the Pro version of ViceVersa. The message you got with the free version is good. Is this the "A" and "B" drives you refer to in your other post? There are some advanced file sync and scheduling modes in the Pro version, but I prefer manually using Teracopy for syncing since I can view the progress.

I'm not 100% sure (hopefully others will chime in to confirm or correct), but I believe the difference in size displayed by Explorer is because when a file is copied it may use an additional sector or sectors when written. I just checked two drives that ViceVersa shows as identical and there's a 50KB difference between the drives. I suspect if I were to defrag the drives, the difference would be smaller or zero. On a 8TB drive, this is less than nothing and nothing to be concerned about.

The extra file in one of your drives is likely a hidden 0 byte file and ViceVersa ignores it. Again nothing to be concerned about*, but you can see it by turning showing hidden files in Explorer. Go to View and check Hidden Files. Be very careful as this will show system files that shouldn't be deleted!

*Some viruses disguise themselves as 0 byte files or change your files to show 0 byte. Your antivirus should catch this type of behavior.

The AlbumArt and other .jpg files are automatically generated by Windows Media Player and possibly other media players when you play an audio file. These are safe to delete, but will come back whenever you play an audio file. I don't use WMP, but I believe it's possible to turn off the AlbumArt search function. I believe if you view the files as thumbnails in Explorer, these also get saved on your drive for quick future reference.

The desktop.ini file is generated by Windows when you view your directories and files. It can be deleted, but will come back as soon as you open a file or directory. Best to leave it alone. It's safe to copy.

If you open the autorun.inf file in Notepad, it should have some info about what generated it. It's probably safe to delete, but will likely be generated by whatever program created it.

Since you like comparing folders, another nice addition to your PC would be Q-Dir http://www.snapfiles.com/get/qdir.html. This allows up to 4 view windows so it's easy to compare directories/folders. I've stopped using Windows Explorer since getting this.

Another program you might find useful is VVV (Virtual Volumes View) http://vvvapp.sourceforge.net/. This allows you to create an offline view of your drives so you don't have to connect them to view/search your files. You still have to connect the drive to actually access the files. Another nice function is you can export the directory/file details to an .csv file for cataloging. The one major catch is it displays file size in bytes, so you'll have to convert it to MB or GB. Q-Dir also allows exporting to .xls, but I like VVV's formatting better.

[lingyi]

Just thought of something else.

If Chkdsk finds lost or corrupted files, it may create Found.000 and File0000.chk folders and files. They're hidden files and may contain recoverable data (text files are usually recoverable, video and audio files less so as they're usually truncated). If you don't need the folders/files, they can be deleted.

[CaptainCatholic587]

I'm a bit confused by the numerous drive names in your post, but I think I'm following what you're saying.

You don't need to get the Pro version of ViceVersa. The message you got with the free version is good. Is this the "A" and "B" drives you refer to in your other post? There are some advanced file sync and scheduling modes in the Pro version, but I prefer manually using Teracopy for syncing since I can view the progress.

I'm not 100% sure (hopefully others will chime in to confirm or correct), but I believe the difference in size displayed by Explorer is because when a file is copied it may use an additional sector or sectors when written. I just checked two drives that ViceVersa shows as identical and there's a 50KB difference between the drives. I suspect if I were to defrag the drives, the difference would be smaller or zero. On a 8TB drive, this is less than nothing and nothing to be concerned about.

The extra file in one of your drives is likely a hidden 0 byte file and ViceVersa ignores it. Again nothing to be concerned about*, but you can see it by turning showing hidden files in Explorer. Go to View and check Hidden Files. Be very careful as this will show system files that shouldn't be deleted!

*Some viruses disguise themselves as 0 byte files or change your files to show 0 byte. Your antivirus should catch this type of behavior.

The AlbumArt and other .jpg files are automatically generated by Windows Media Player and possibly other media players when you play an audio file. These are safe to delete, but will come back whenever you play an audio file. I don't use WMP, but I believe it's possible to turn off the AlbumArt search function. I believe if you view the files as thumbnails in Explorer, these also get saved on your drive for quick future reference.

The desktop.ini file is generated by Windows when you view your directories and files. It can be deleted, but will come back as soon as you open a file or directory. Best to leave it alone. It's safe to copy.

If you open the autorun.inf file in Notepad, it should have some info about what generated it. It's probably safe to delete, but will likely be generated by whatever program created it.

Since you like comparing folders, another nice addition to your PC would be Q-Dir http://www.snapfiles.com/get/qdir.html. This allows up to 4 view windows so it's easy to compare directories/folders. I've stopped using Windows Explorer since getting this.

Another program you might find useful is VVV (Virtual Volumes View) http://vvvapp.sourceforge.net/. This allows you to create an offline view of your drives so you don't have to connect them to view/search your files. You still have to connect the drive to actually access the files. Another nice function is you can export the directory/file details to an .csv file for cataloging. The one major catch is it displays file size in bytes, so you'll have to convert it to MB or GB. Q-Dir also allows exporting to .xls, but I like VVV's formatting better.

Wow, thank you SO much for the abundance of information lingyi, I sincerely appreciate it!! Your detailed explanation of what some of my .inf and and AlbumArt files are was very helpful. I will download the programs you mentioned over the weekend and take a look, they sound great!!

Thank you very much again, all such great information! I will stop being paranoid now haha, sounds like there was a very reasonable explanation for the discrepancies.

[abolibibelot]

– There are quite a few tools to check long path names on Windows, this one works well :
http://www.softpedia.com/get/System/File-Management/Path-Scan.shtml
There is a trick in Windows 7+ (well, at least Windows 7, haven't seriously used subsequent versions) which allows the explorer to manage files with a too long path by relying on short names : if you check the properties for such a file, you may see the path as, for instance, “D:\FOLDER~1\FOLDER~2\THIS_I~1\name of the file.ext” instead of the full path which would be too long according to this 260 characters rule, “D:\folder 1\folder 2\This is a very very long folder name\name of the file.ext”. If the total path + name length is within a certain limit (I don't know exactly which it is), you can still access it from Windows Explorer, rename it, move it, view it with the default picture viewer if it's a picture, but third-party softwares may have trouble accessing them anyway. Beyond that certain limit, even with this trick, Explorer will choke. I haven't done extensive tests, or read specific information regarding this behaviour, so I couldn't guarantee that all this is accurate.
Another thing is : in Windows Explorer, there's a warning when copying or moving files or folders full of files which have to be written anew, if this results in too long path names, but there's no warning when moving a folder full of files into another folder on the same partition (meaning that the files don't have to be written anew, only their paths are updated) if it results in the same outcome, so you only know it when trouble comes, or by checking for long paths with a dedicated tool.

– CHKDSK only reports and fixes NTFS file system inconsistencies, too long paths are not considered as such, it's an issue for Windows Explorer, not as far as NTFS is concerned.

– By the same token, there's no point in running CHKDSK on a brand new empty drive : it will not find file system inconsistencies, and there are better tools than CHKDSK /R to check for potential bad sectors (with something like HDTune or HD Sentinel you get a map to visualize the advancement of the process, and identify the location of bad sectors, if any, whereas CHKDSK acts as a “black box” and barely reports anything useful).

– I haven't used TeraCopy recently, I don't know how it performs, but checking the CRC after each file copy must considerably slow down the process. And I'm wary of relying on the same tool to copy and verify, as I've seen some bugs wherein a given tool reported a copy as correct when in fact it was not. I mostly use the command line tool Robocopy, or the GUI tool SynchronizeIt, both of which have the rare particularity of preserving all timestamps (FastCopy does that as well, it's a good alternative to Robocopy if you don't feel comfortable using a command line program, although it has less options, and it's reportedly even faster than TeraCopy), including directory timestamps (with Robocopy use the /DCOPY:T switch), then verify in a second step with WinMerge (similar to ViceVersa, which I've never tried, but 100% freeware, I actually use WinMerge2011 which is a fork with some added features), or with md5deep, a command line MD5 calculation tool (the advantage is that the lists of MD5 checksums can be stored for future reference).
I must warn that SynchronizeIt has a bug when dealing with “sparse” files ; it's a rarely used NTFS feature which allows to store empty sectors without actually allocating them, thus reducing the occupied size for large files which have a lot of empty sectors ; some download managers make use of it to avoid allocating the full size of files being downloaded (that's how I got such files years ago without knowing anything about this feature), I reported it to the author, who eventually provided me with a fixed version, but for some reason he never released it on his website and the version publicly available is still the 3.5 from 2009. I can provide the updated 3.6b from 2015 (which deals perfectly with “sparse” files) by private message for anyone interested. Aside from keeping timestamps, I like its ease of use and streamlined interface.
With md5deep I use this command :
md5deep64.exe -r -z -t -e -j1 "F:\folder path\*" > "G:\folder path - md5deep.txt"
Where -r stands for recursive (scans all sub-directories), -z adds the file size, -t adds the timestamp (modified date), -e displays the estimated time for checking each file (too bad there's no estimation for the whole process), and most importantly -j1 runs the process on 1 thread only ; with the default settings, it uses all the cores available, which dramatically decreases the performance : the MD5 calculation is not limited by CPU power on modern computers, but by the drive's throughput, so with an 8-threads CPU, 8 files are checked at the same time, the drive has to grind like crazy, the average speed is poor and it needlessly stresses the drive ; by using 1 thread only it proceeds at the maximum speed allowed by the drive, and you can check the MD5 on two drives simultaneously by opening two CMD prompts. A plus side effect is that with one thread the files are checked in the same order, which makes it much easier to compare the lists : when all is finished, I open one of the lists with Notepad2 and replace in each line the drive letter (or the main path if the files were copied to a specific folder with a different name) with that of the other list, then save a modified copy, then check the MD5 of both lists (original list for drive A and modified list for drive B), if it matches it means that every single file contained in the scanned folders has a matching MD5 and the copy was entirely flawless, that way I don't even have to look at the lists in detail ; if there's a mismatch, then I use WinMerge to compare the lists and locate the discrepancies. Later on if I have a suspicion that a file could be corrupted (for instance if I see a video or audio glitch which I hadn't noticed before), I can go back to those lists and verify if the MD5 matches (meaning that the glitch was there originally) or not (meaning that the file was indeed corrupted, most likely by bad sectors).

– When copying from a drive A to both a drive B and a drive C, I usually copy from A to B without checking, then from B to C without checking, and then I thoroughly compare C with A before deleting the files on A. If all goes well, I know that the copy is flawless on B and C and only have to check once.

"The two folders are identical (excluding filtered files and sub-folders)." Does this mean that ViceVersa doesn't scan sub-folders?

No, it probably means that filtered sub-folders are excluded (the adjective applies to both).

However, when comparing the two hard drives via Windows Explorer afterward, Windows Explorer said that my "G" drive had one extra file compared to my "F" drive. ViceVersa actually noticed that not 1, but 6 files were not copied over. According to ViceVersa, my "G" drive had 6,726 files and folders, but my "F" drive had only 6,720 files and folders. However, when looking at the missing files (the ones my "G" drive had but my "F" drive did not have), they were all insignificant files to my knowledge and included the following:

Either those files were created after the copy, as has been said above, or they were skipped by TeraCopy because of their “hidden” attribute. If you used a program which preserves the folders' timestamps, you could check the modified date of the folders which contain those files, which would be newer on the drive containing them if they were generated afterwards. Otherwise, check the creation date of those files, and compare it to the date of the copy.

I even searched my "G" drive for #1-#3 of these files (by typing their names into the "Search" button on Microsoft Explorer), and NONE of them came up on my "G" drive. Strange!

The search function in Windows 7 is horrendous (it was much better in Windows XP, it's one of the reasons why it took me so long to stop using the good old WinXP). I now use the search module of Total Commander for that purpose. There may be better dedicated tools, but Total Commander does many things and does them well enough to replace a bunch of dedicated tools (for instance its folder comparison module is almost as good as WinMerge).

I'm not 100% sure (hopefully others will chime in to confirm or correct), but I believe the difference in size displayed by Explorer is because when a file is copied it may use an additional sector or sectors when written. I just checked two drives that ViceVersa shows as identical and there's a 50KB difference between the drives. I suspect if I were to defrag the drives, the difference would be smaller or zero.

No, the “size on disk” (which should now be renamed as storage units no longer necessarily come as “disks”) only varies according to the cluster size, and to the use of special NTFS features, i.e. compression and “sparse” writing. And the number of files reported when checking the properties of a folder can vary if some files inside it have a too long path/name, as per the first issue mentioned in this thread.

Another program you might find useful is VVV (Virtual Volumes View) http://vvvapp.sourceforge.net/.

Thanks for this suggestion, I was looking for something like that a looong time ago... I'll check that right away.