I have been purchasing WD external hard drives for my PC for about 10 years now. I have always been extremely happy with their hard drives. Every two years or so, I buy a new WD hard drive (always upgrading space), and I just copy all of my old files to the new hard drive, and store the old hard drive away in my closet.
In any case, I recently bought a My Book 8TB drive. I ended up having major problems with it, problems I have never experienced in the 10 years I've been buying WD hard drives.
After opening the hard drive, I copied all of my files (all digital pictures) to it from my old hard drive (about 319 GB of pictures). I copied it straight to the main hard drive "folder" itself. Like usual, it turned out there were 2 or 3 files that had names that were too big, so it gave me an error message stating something like, "this file is too big to be copied." This was no big deal. So I changed the name of these two or three files on the original hard drive, and then copied them over to my new hard drive. All was good.
However, then I ran into some significant problems. After copying all files over, I then created a new folder called "Family Pictures" on the new hard drive. I moved all copied files into this new folder. The move completed without any error messages whatsoever. I checked to see how many files were under "Family Pictures" (under "Properties"), and it was the correct number.
However, when I was scrolling through the pictures, I noticed that the thumbnails did not populate for a few pictures. The file was there - the correct name was there too. But when I clicked on it, it was blank and I got the error message, "Windows Photo Viewer can't open this picture because either Photo Viewer doesn't support this file format, or you don't have the latest updates to Photo Viewer." I knew right away that this was a "bogus" error message; I don't need any updates, because the same exact picture worked just fine on my original hard drive when I opened it. I knew the copy to the new hard drive didn't work correctly.
This was highly disturbing to me, because I wouldn't have known in any way that certain files did not copy over correctly if I hadn't checked - I never received an error message about this (like I usually always do), and the file appeared in the "file count" accurately. I never had this problem before. Now I'm worried that I might have other files that didn't copy over correctly. It would take forever for me to check each and every individual file (there are thousands and thousands). How can I fix this? (And more importantly, how do I prevent this from happening in the future?). I don't know what to call these, but to me they are "ghost" files, because they appear to be there but they are not.
Additionally, I tried to delete the "ghost" files from my hard drive (so that I could rename the original files on my other hard drive with shorter names and then recopy them to the new hard drive). But it wouldn't let me! I received the error message, "The source file name(s) are larger than is supported by the file system. Try moving to a location which has a shorter path name, or try renaming to shorter name(s) before attempting this operation." I can't delete these files, I can't rename them, I can't move them - they're just stuck there. These "ghost" files are now cluttering my hard drive. This is very obnoxious! I googled how to fix this, but I cannot get the "open a command line" solution to work. I also tried disabling the recycling bin, but that doesn't work either. Does anyone know how I can delete these files???
Finally, I have another problem - when I go to "Properties" under the new hard drive, I see that I have 405 GB of used space on the hard drive. This is insane, because I only copied 319 GB of pictures to the hard drive. What?!?! Where is that extra 86GB of used space coming from? Is that because of the "ghost" files? When I click on the folder I created, "Family Pictures," it states that this folder takes up 330 GB of space on the hard drive, but the "size on disk" is 388GB. I have no idea what that means, but it seems like I got completely ripped off of 58 GB for no reason (again, is it because of the "ghost" files?).
All of these issues are extraordinarily frustrating, and I have never once encountered any of these issues with a WD hard drive. I am tempted to return this (it seems defective) and switch to another brand if these issues aren't solved. If anyone knows how to solve any of these problems, I would greatly appreciate it!!
Additionally - even if I do solve these problems, I am now extremely paranoid that I'm going to "think" all of my files copy over to my new hard drive, but in reality some of them might be missing. Is there anyway to absolutely guarantee that all files copy over correctly? I never would have known that some files didn't copy over correctly if I hadn't spent 10 minutes randomly clicking a few files.
Thanks so much! Any help is greatly appreciated!!
+ Reply to Thread
Results 1 to 20 of 20
I recommend reformatting the drive and running chkdsk before retrying to copy your files over. As I've stated many times before, external drives (especially 8tb) are the true bargain right how. However, the weak point is always the USB interface (which are highly prone to failure).
Next, add these utilities to your PC (they're all free):
This will unlock almost any file that's locked by Windows. Be sure to doublecheck and unlock only the file(s) you want.
Highly recommended here and on other forums. Speeds up copying files and you can have it verify that the copied files are identical to the original.
Gives you a side by side view of the files in two folders and can be to set to run a CRC check to verify both folder/files are identical. The only real limitation of the free version is it doesn't recognize Unicode characters.
Last edited by lingyi; 20th Jun 2018 at 23:53.
I believe My Books come formatted in exFAT with 128K cluster size. That explains the wasted space (an average of 64K is wasted for each file). It's good for large files like video, though, because it improves performance when files are fragmented.
Just checked and you're right. I never noticed because I always do a format and chkdsk before I start using my drives.
I would check if your operating system support 4KiB sector size - almost sure that 8GB HDD's are using 4KiB sector internally and it is well known that some older OS's may face some issues with this. Additionally if i can recommend something - always perform copy with verification if possible and after successful verification you may delete source files - never relay on move itself.
Hi everyone! Thank you so much for all of the responses, this has been incredibly helpful!
Lingyi, I am unfamiliar with chkdsk, as I have never done it before today, but I believe I just did it correctly. I just right-clicked the drive under "My Computer," clicked on "Properties," then "Tools," and then under "Error-checking" I clicked on "Check now." I then checked the box next to "Automatically fix file system errors," (but I did not check the box next to "Scan for and attempt recovery of bad sectors"), and then I clicked "Start."
It scanned the hard drive, and then I received the message, "No problems were found on the device or disk. It is ready to use." So I'm guessing that means my hard drive is in good shape and there shouldn't be any problems with it? (Is the purpose of chkdsk to make sure your external hard drive is "good to go" and doesn't contain any errors?").
Additionally, I can definitely reformat the drive if that's what you'd recommend! I am very unfamiliar with how to do this though - is there a particular format I should select? (NTFS?)
Luke M, it looks like you're correct! It looks like my new 8TB hard drive is indeed set to exFAT, and the Allocation unit size is set to 1024 kilobytes. That is way too big!!! No wonder I have all that wasted space. Thanks so much for your help in discovering this!
Should I just re-format the whole hard drive to NTFS? If so, can I do this by just right-clicking the hard drive (in "My Computer"), clicking "Format," changing "File system" to "NTFS (Default)," and changing the "Allocation unit size" to "4096 bytes" (the smallest available)? The box next to "Quick Format" is checked. Then I can just click "Start?"
Let me know if you guys think this would be the best way to do it! I don't mind losing all of the data on my new hard drive, as I still have all my original data on my original hard drive. I'd rather just have it be in the correct format.
By the way lingyi, you mentioned that USB interfaces are prone to failure. Are there any particular issues I should be aware of? (I actually have 4 total external hard drives to back up all of my data - I have one plugged into my computer at all times, and then I have one in the same room which is always unplugged except for when I back up files to it. I also have another one on another floor of my house. Finally, my Dad keeps an external hard drive of all my files at his house, which is miles away, and I back up all my data to that drive every 6 months). I figure that with all these backup drives in multiple locations my data is pretty safe.
Thank you all very, very much for the help again, I appreciate it!!
Regarding Teracopy, that sounds pretty awesome! I will see if I can try that out. Regarding how it speeds up copying files though, does that ever have any negative effect on copying files? (Sometimes I get paranoid when using external software that it might "mess up" something). It's strange - I have literally never experienced problems like this before where I copied files to an external hard drive, and they just weren't there and no error message popped up to warn me (is this typical, or is my example a rare occurrence? I used to always get an error message if a copy wasn't successful, usually because of a long file name). But if you think it's reliable and the added speed won't "mess up" the copy, I will definitely try that out!! It would be great to verify that anything I copy from one drive to the next is identical.
Thanks again, much appreciated!!
Additionally, I would definitely like to perform a copy with verification - do you know how I can do this? (Do I need an external program, possibly one of the ones lingyi mentioned like TeraCopy?).
The default for Windows is NTFS and 4K sectors. Do a quick format and you're good.
9 out of 10 times when someone posts here and other forums that an external drive has failed, the advice is to remove the drive from the case. And in 8+ of those 9 cases, the drive is fine, but the interface failed.
I apologize to the regulars who have seen me post his dozens of times, but it's simple economics. An external HDD can go for nearly half the price of an identical bare HDD. So either the HDD manufacturers are making a huge profit on bare drives or they're cutting costs on the case and USB interface.
I've owned dozens of external HDDs, both desktop and portable and I've given up on using them as designed (I remove the drives from the case for use as an internal) as I've had almost all of them fail, usually at the worst moment. I suspect constantly plugging and unplugging and the cheap transformer power supplies included with drives are subjecting the USB to an electrical shock, eventually killing it.
The only catch is that AFAIK, Teracopy will only let you verify your files if you've checked the verify box before starting or check the verify box while the Window is still open. This is where ViceVersa comes in as it allows you verify files after they've been copied/moved.
As for Unlocker. Yes, it will allow you to unlock and delete "ghost files" that are locked by some unknown Windows process. I notice Win 10 files more often then previous versions. Just be careful if you're unlocking hidden files as it's possible to delete some critical files that shouldn't be deleted (that's why they're hidden!).
I would format HDD with NTFS, i would change cluster size to 64KiB (good trade-off between space loss and speed, you should also reduce MFT size and its fragmentation with this) of course if you have many small files (smaller than 64KiB) then this may be not a best idea. Support for large (big) 4KiB sector on Windows https://support.microsoft.com/en-us/help/2510009/microsoft-support-policy-for-4k-secto...ves-in-windows .
I'll definitely be sure to run the chkdsk each time I get a new hard drive though - I'll just keep the 2nd box unchecked. Thanks again!
First, a few mb of wasted space on an 8 TB drive are not something I would worry about.
Multiple files which failed a simple copy operation would be a major concern to me. I would invest some time to Isolate and Identify the problem.
Copy the suspect files to the primary hard drive, and to the external, several times. If there is no failure, on MULTIPLE attempts with MULTIPLE files, then that indicates that the external drive and/or the USB interface failed, randomly, intermittantly, and unpredictably. That would be the problem that I would be concerned with.
USB interfaces for external drives SUCK, and they SUCK EVEN WORSE when power is supplied through the USB interface. First, use an internal drive for any reliable copy, period, no exceptions. Next, use an external drive with a separate power plug, to eliminate the most common problem with such drives, and keep multiple copies for the day when the interface suffers catastrophic failure and the data is totally lost.
Also, checking an 8 TB drive for errors takes a long time, especially through a USB interface. If you don't want to spend the necessary time to check for errors, either get smaller drives, get internal drives, or simply accept the occasional data loss that comes from drive errors.
" I have a problem, but I do not want to spend the required amount of time to fix it, so I will attempt to address other, completely unrelated problems and ignore the primary issue."
As you said, "Multiple files which failed a simple copy operation would be a major concern to me," and yes, that was definitely my biggest concern. I should specify - the problem files actually did not fail the initial copy - they copied to the new hard drive just fine. But, when I moved the same files on the new hard drive to a brand new folder - that was when I encountered the problem. I determined that this was because the file names/folders were entirely too large (well over 255 limit). They must have not been "over the limit" to copy to the new hard drive, but they were over the limit when I then created a new folder (on the new hard drive) and then moved the files to the new folder. I believe this is what caused the error. The only files I had a problem with were files that had extremely long names (or were saved in subfolders with extremely long names).
However, what was alarming was that Windows Explorer did not give me any kind of error message to let me know that my files didn't copy over correctly. I only stumbled on this by chance. So I believe it was Windows Explorer that failed, not my new hard drive. I did run a chkdsk on my new hard drive, which said the hard drive was fine (though I admit I didn't select the 2nd option on the chkdsk, just the first).
To counter this problem, I'm going to start using smaller names when labeling folders and files. However, the biggest change I'm making is that I'm now using TeraCopy to copy all of my files from now on (I'm also erasing all the data on my new hard drive, and recopying all the data over to the new hard drive via TeraCopy to ensure it all copies over correctly). Nelson, in your opinion, do you think this is enough to prevent such a problem from occurring again? As long as TeraCopy verifies that every file is copied over correctly, I should be good, right? (Or has anyone encountered errors with TeraCopy, i.e. it "verifies" that a file copied over correctly but it didn't really? I would be concerned if TeraCopy has reliability issues, though it sounds like everyone here trusts it and has nothing but good things to say about it)
To elaborate on my hard drives, I have always been very careful with my data (at least I think!). I am in the process of digitizing all of my pictures/videos from my childhood, as well as my Dad's childhood (tons of 8mm videos from the 1950s and 1960s), and now my kids' childhood (so I have videos from 1955 through 2018, which is pretty cool - though a lot of data!). I enjoy having all of it on one single hard drive, as it's really cool to click around and go from 1963 to 1997 to 2014 - my kids really enjoy this. This is why I have 8TB hard drives as opposed to let's say four 2TB hard drives.
I have one 8TB hard drive plugged into a WDTV on our 60" TV in our family room, so we can watch home videos from any year at any time on our TV with a couple of simple clicks. Then, I have two external hard drives in my upstairs office with all the same data (one of these hard drives is plugged into my computer at all times, whereas the 2nd one is unplugged at all times except when backing up new data). Finally, my Dad (who lives several miles away) has a hard drive stored at his house with all this data. So I've felt very good about maintaining all my data, as several terrible things would have to happen all at the same time in order for me to lose all of it, haha, and now with TeraCopy I feel even better about it. Though if anyone has any suggestions, feel free to let me know! (Note: All my external hard drives have their power supplied via a separate power cord that plugs into the wall).
I was, however, totally unaware that people have problems with USB interfaces and external hard drives. I have been using WD external hard drives since 2007 and literally never once experienced any sort of problem until now - though again, I think it was caused by the combination of Windows Explorer and my file names being far too long (also, I was copying from an NTFS hard drive to an exFAT hard drive - I reformatted the new one to an NTFS hard drive to match the original, just in case that caused problems too). I don't believe the problem at anything to do with the hard drive itself.
Anyway, thanks so much everyone again for all your help! And thanks for the additional information Nelson! Any other advice or things I should be aware of would be greatly appreciated.
The long file and folder names were likely the problem.
Do not forget that the actual name of the file INCLUDES both the file, AND the folder name.
The USB drive interface is just not long-term reliable. Have seen many, many failures over the decades. I personally hate them with a passion, and do not use them at all. Plug the drive in internally for a reliable, safe, and much faster data transfer.