Management of > 4TB HDDs

[Seeker47]

I mostly run older hardware, and prefer to stay away from ex-FAT and GPT organizational schemes. However, most of the larger HDDs (and even some flashdrives) are turning up right out of the packaging as being ex-FAT; the HDDs will read as being GPT. I've been converting them to NTFS, with MBR type for the HDDs. Just got a Seagate 5 TB portable HDD, which in real-world reckoning is more like 4.5 TB actual. It came with an 88 MB EFI partition at the front. Did it come set as GPT due to unavoidable mechanics of the extra platters ? I used Acronis Disk Director to make it MBR, with most of the rest reformatted as NTFS. It took two operation cycles with DD to get there, but this left things as two volumes, with the front 88 MB one no longer registering as "EFI" but as Unallocated Space instead. I'm just guessing that that EFI business had something to do with retaining some option to allow booting from this drive ? Or maybe it was needed for users who wanted to use the drive for Apple Mac ? But it's not looking like this space can be reclaimed and merged into the main NTFS partition ? I'm hoping that I have not damaged anything via making these changes. But this is going to be a recurring issue, to the extent I continue doing anything with the larger drive sizes.

[usually_quiet]

I mostly run older hardware, and prefer to stay away from ex-FAT and GPT organizational schemes. However, most of the larger HDDs (and even some flashdrives) are turning up right out of the packaging as being ex-FAT; the HDDs will read as being GPT. I've been converting them to NTFS, with MBR type for the HDDs. Just got a Seagate 5 TB portable HDD, which in real-world reckoning is more like 4.5 TB actual. It came with an 88 MB EFI partition at the front. Did it come set as GPT due to unavoidable mechanics of the extra platters ? I used Acronis Disk Director to make it MBR, with most of the rest reformatted as NTFS. It took two operation cycles with DD to get there, but this left things as two volumes, with the front 88 MB one no longer registering as "EFI" but as Unallocated Space instead. I'm just guessing that that EFI business had something to do with retaining some option to allow booting from this drive ? Or maybe it was needed for users who wanted to use the drive for Apple Mac ? But it's not looking like this space can be reclaimed and merged into the main NTFS partition ? I'm hoping that I have not damaged anything via making these changes. But this is going to be a recurring issue, to the extent I continue doing anything with the larger drive sizes.

Yes, the EFI partition is required for UEFI secure boot and requires that the drive uses GPT formatting. Most newer computers are set up to use UEFI instead of Legacy BIOS but your drive was also GPT formatted because of the partition size. To use MBR formatting, all partitions must be 2TB or smaller.

[Cornucopia]

The world has already moved on to UEFI (vs. Legacy bios), and 64bit (vs. 32 or older), and GPT (vs. MBR). Especially for drives that one intends to boot with. MacOS and Win10/11 both expect the above. Specifically, for drives, 2TB is the limit for MBR. Anything over that is either lost, or the means to access it is a hack/kludge that renders access dubious at best, and often unstable.

Drive logical filesystem formatting is more flexible, you can use NTFS or exFat, or get drivers for other formatting types (Mac, Linux). FAT otoh also has a 2TB hard limit, though you could partition a larger drive up into multiple logical volumes in order to use the available space.

My suggestion to you is "keep the hot hot, and the cold cold". IOW, for legacy systems, continue to use things as you would...but only with truly smaller drives (<=2TB). And for newer systems, embrace the new paradigm, which works quite well if you are following the rules.
For transfer drives that might move back and forth, keep them smaller and legacy compatible (they will of course still work with new systems), just don't expect them to be bootable.


Scott

[usually_quiet]

If memory serves, UEFI began to gain acceptance while Windows 8 was still the latest version of Windows. I think UEFI was the preferred option for Windows 10 systems but some motherboards would still allow using Legacy BIOS as an option for Windows 10. Windows 11 does officially require UEFI but I have read there is a workaround to install it on a system that's using a legacy BIOS.

[Seeker47]

The world has already moved on to UEFI (vs. Legacy bios), and 64bit (vs. 32 or older), and GPT (vs. MBR). Especially for drives that one intends to boot with. MacOS and Win10/11 both expect the above. Specifically, for drives, 2TB is the limit for MBR. Anything over that is either lost, or the means to access it is a hack/kludge that renders access dubious at best, and often unstable.

Drive logical filesystem formatting is more flexible, you can use NTFS or exFat, or get drivers for other formatting types (Mac, Linux). FAT otoh also has a 2TB hard limit, though you could partition a larger drive up into multiple logical volumes in order to use the available space.

My suggestion to you is "keep the hot hot, and the cold cold". IOW, for legacy systems, continue to use things as you would...but only with truly smaller drives (<=2TB). And for newer systems, embrace the new paradigm, which works quite well if you are following the rules.
For transfer drives that might move back and forth, keep them smaller and legacy compatible (they will of course still work with new systems), just don't expect them to be bootable.


Scott

Thanks for your reply. The drive in question was never intended for booting purposes, but only as backup archival storage.
And, at least for the time being, it won't be accessed by anything later that Win 10 systems. (One of the reasons I'm avoiding ex-FAT is that I want contents to be seen if I connect such a drive to any of my existing playback devices. ex-FAT would not be seen or supported for that. Don't know about GPT.) As mentioned, I had to revert this drive to GPT in order to make the 4.5 TB NTFS partition. It does seem to be stuck with that first 88 MB partition though, which was originally showing as "EFI", but now shows as "Unallocated Space."

Before I put that much onto this drive, is its current status as is apt to remain stable and reliable ?

[The system I'm writing this on has had a 3 TB (2nd. internal HDD) in it, set up the old legacy way, for over 2 years now, and with nary a hiccup. All of my partitions that are not boot partitions are Logical Volumes.]

I'll have to double check this, but I'm pretty sure that I've put 4 TB HDDs into computers, as a second, internal data work / storage drive, and with them being straight up MBR and NTFS. May well have had to divide them into dual NTFS partitions of 2 TB, however. Offhand, I had thought that the upper limit for individual NTFS partitions was something like 16 or 32 TB ?

[usually_quiet]

I have a 1TB portable SSD drive that provides additional storage for my Windows 11 laptop. It uses MBR formatting and the NTFS file system. It is working perfectly.

I have four MBR formatted data drives up to 2TB installed in one computer running Windows 10 and three 4TB GPT formatted data drives in another computer running Window 10. All these drives use the NTFS file system. All these drives are working perfectly.

[Cornucopia]

I have heard of a number of users who have had no issues with MBR using an extension driver (possibly from the mfr?) for years, but I have also heard of a number of users who have had data rendered inaccessible. That's why I said dubious.
Always have backups!

NTFS itself as a filesystem is capable of Zetabytes of data capability (I think).

You might be able to recover the beginning partition with the right partition tool, but for 88MB, I would not worry about it.


Unless you are using cheapo or old embedded devices, most modern devices should be capable of supporting exFat.

Scott

[Seeker47]

One thing I meant to say in #5 above, if it was less than clear, is that every boot drive (1st. system HDD) that I have currently is a C: Primary (MBR) for the OS. Those would all be no larger than 2 TB. Some programs may be installed on the 2nd. HDD, which could be larger.

The 5 TB. storage backup HDD I described seems to be working fine thus far, in early tests. There was another of the same Seagate model that was put into service about a year ago, but I'll need to revisit it to be sure how I set it up.

I have at least 6 streaming / player boxes, including the (now effectively orphaned) Skystream One & Skystream Two, Amazon Fire (original model), WD Live TV (still used frequently for local playback), Roku Ultra etc., going back to a Minix U-something_or_other. This does not include the USB port playback option with the Oppo BR deck. I don't think any of these fell into the "cheapo devices" category, and most of them I've tested for ex-FAT in the past. I think the only one that stands a chance of seeing ex-FAT media might be the Nvidia Shield. And that, being a 2017 model (?), also maybe not. It's not about what's modern or not. This gear continues to serve my needs well, so no reason to change it.

I have a humongous video library at this point. Some would choose the route of NAS + Plex, and maybe I'll go for that at some point. But the established methodology in use here is fully portable, easy, and super-convenient. It does not rely upon any computer or other hardware running 24/7.

[Seeker47]

As mentioned, I had to revert this drive to GPT in order to make the 4.5 TB NTFS partition. It does seem to be stuck with that first 88 MB partition though, which was originally showing as "EFI", but now shows as "Unallocated Space."

I've continued ditching ex-FAT in favor of NTFS on HDDs like this, > 2 TB , for the reasons mentioned in my post #8 of this thread. The only anomaly I've run into so far has been that for one of these, when I plug it into a computer's USB port I get a very transitory Win-10 error saying that there is a problem with the drive that needs to be fixed. But when I then use CHKDSK or SCAN on the EFI partition &/or the NTFS storage partition, the resulting report is "No Errors Were Found." And access, reads, or writes remain unaffected. So this becomes just a mystery and a minor nuisance. If I could make the spurious error go away for this HDD -- without any major changes or risk -- I might be inclined to do so. That seems to have been the only such HDD so affected, and I keep adding more of them. (Storage is never sufficient, in the case of a LARGE film library.)

[The_Doman]

I also always use NTFS on my external disks.
Because it is much more robust (power/unexpected disconnections) and the success rate in case of a needed recovery are so much better.
Also more/better tools available for a NTFS recovery.
Ex-FAT i use rarely only for compatibility on certain devices and not important stuff.

UFS Explorer: Chances of data recovery depending on the file system
SysDev Laboratories: Filesystem type and data recovery chances

As mentioned, I had to revert this drive to GPT in order to make the 4.5 TB NTFS partition. It does seem to be stuck with that first 88 MB partition though, which was originally showing as "EFI", but now shows as "Unallocated Space."

Drives bigger then 2TB need GPT or you end up with problems/incompatibilities/workarounds soon or later.
New external drives coming in Ex-Fat you should just re-partition/format in NTFS immediately and using GPT when bigger then 2TB.

The only anomaly I've run into so far has been that for one of these, when I plug it into a computer's USB port I get a very transitory Win-10 error saying that there is a problem with the drive that needs to be fixed. But when I then use CHKDSK or SCAN on the EFI partition &/or the NTFS storage partition, the resulting report is "No Errors Were Found." And access, reads, or writes remain unaffected. So this becomes just a mystery and a minor nuisance.

Possible the disk has the "Better Performance" option set instead of the "quick removal"?
Google: Better Performance vs Quick Removal USB policy
But then i often still use the USB eject with big external/important drives.

One thing I meant to say in #5 above, if it was less than clear, is that every boot drive (1st. system HDD) that I have currently is a C: Primary (MBR) for the OS. Those would all be no larger than 2 TB. Some programs may be installed on the 2nd. HDD, which could be larger.

Same here, not using sizes bigger then 2TB for the main OS drives, even then often split in extra partition for easy image backup/restores.
UEFI/GPT yes, for some systems here but then I still prefer to have the really big drives separate from the main system.

[Seeker47]

. . .
Possible the disk has the "Better Performance" option set instead of the "quick removal"?
Google: Better Performance vs Quick Removal USB policy
But then i often still use the USB eject with big external/important drives.

Thanks for your detailed reply, and for those citation links. I don't know as to that setting, have to find out where to check on that. I've been using USB Safely Remove on most systems for several years, as opposed to the built-in Windows function.

[Cornucopia]

One reason I see for NOT using NTFS filesystem on disks, especially transfer disks, is the issue of PERMISSIONS.

Often, you will have user "PC1\JohnDoe" accessing his portable NTFS disk on PC2. He logs in as JohnDoe, but user "PC1\JohnDoe" DOES NOT EQUAL "PC2/JohnDoe". You will have to go through all the Admin-level, recursive taking control of the drives' file & folder ownership, and then allowing of access. This is a doable workaround, but not inconsequential, not trivial to do, and not quick to complete.


Scott

[Seeker47]

One reason I see for NOT using NTFS filesystem on disks, especially transfer disks, is the issue of PERMISSIONS.

Often, you will have user "PC1\JohnDoe" accessing his portable NTFS disk on PC2. He logs in as JohnDoe, but user "PC1\JohnDoe" DOES NOT EQUAL "PC2/JohnDoe". You will have to go through all the Admin-level, recursive taking control of the drives' file & folder ownership, and then allowing of access. This is a doable workaround, but not inconsequential, not trivial to do, and not quick to complete.


Scott

That's interesting . . . but I'm glad to say I haven't run afoul of that. I rotate between about 4 computers currently, all running Win-10 x64. No one else has access to them, and I'm generally unconcerned about access / security issues. (Not fond of anything that strikes me as impeding my efficiency and ease of use.) So I turned UAC Off from the start, and eschew any Logins rigamarole. Portable external HDDs and flash drives get connected to them all at will -- frequently. At this point, I don't recall for sure whether there is any other universal Admin level status for everything on said computers, but if there was I must have enabled it.

[The_Doman]

One reason I see for NOT using NTFS filesystem on disks, especially transfer disks, is the issue of PERMISSIONS

Just set the the ownership/permissions for the whole drive for everyone to full permission, then i have no problems with that.


[Attachment 87311 - Click to enlarge]

[Seeker47]

One reason I see for NOT using NTFS filesystem on disks, especially transfer disks, is the issue of PERMISSIONS

Just set the the ownership/permissions for the whole drive for everyone to full permission, then i have no problems with that.


[Attachment 87311 - Click to enlarge]

That's a Settings screen in Windows I've never seen before. Might be a parallel route to whatever it was that I did . . . which I thought was limited to turning UAC Off ?