I recently decided to upgrade the ReadyNAS Ultra to make room for some new storage requirements.  The ReadyNAS remains a surprisingly powerful and flexible device so things went well overall, but there was some weirdness that is worth documenting in case others run into it as well and are wondering if it is normal or will cause issues.  To review, my current ReadyNAS Ultra is configured as follows:

 

• Ultra 6
• RAIDiator-x86 4.2.26
• All bays full 6 x 2TB (mix of WD and Seagate “green” series drives)
• X-RAID2, single redundancy mode

For anyone not aware, X-RAID is a clever protection scheme which brings an added layer of flexibility while still providing standard RAID protection.  As a quick primer, the benefits are:

• allows a mixture of disk sizes
• allows for dynamic expansion of an array
• provides single disk redundancy (RAID 5 analog) or dual disk redundancy (RAID 6 analog) while maintaining the above benefits

Some caveats to X-RAID are:

• volume can only grow by an order of 8TB from it’s original size (for all of these caveats it is usable space being measured, not raw… so measure post protection volume size)
• volume cannot be larger than 16TB without initiating a factory reset.  So for example, if you started with a 10TB volume, even though technically you could go to 18TB without violating the “no larger than +8TB from inception” rule, you would be stopped because the final volume would be larger than 16TB.  Factory reset is data destructive so, while not a show stopper, this restriction is definitely one to watch as it can turn a simple expansion project into a very complex one requiring full backup/restore and a companion device that can absorb a potentially massive amount of data
• drives are grouped into layers by size.  What this means is that if you add 4TB drives in as replacements for 2TB drives, you create a 4TB disk layer and a 2TB disk layer that are, transparently to you, contributing to a single virtual volume.  The minimum require disk count per spindle size is 2 in order to retain protection.  Disks are replaced one at a time.  So in the case of 6 2TB drives, 1 is replaced by a 4.  Once sync’d, a second has to be replaced before the array is protected again.  Once 2 4TB drives are part of the volume, protection will be in sync again.  At this point the space sacrificed to volume protection will jump from a single 2TB spindle to one of the 4TB spindles (space inefficient, but at least it allows mixing).  In dual disk redundancy modes, the spindle counts are doubled.
• drive sizes can only go up, not down.  So if you jump from 2TB disks to a mix of 2TB and 4TB, you can no longer add a 3TB disk

With all of the above in mind, I decided to move forward with the 2TB to 4TB scenario, replacing 2 of my Seagate disks with the new Hitachi HST efficiency series 5400 RPM 4TB disks.  Following the process as prescribed by Netgear went well.  I pulled one of the 2TB disks, swapped the 4TB into the carrier (this took longer than the required 10 seconds you need to wait before swapping back in) and then installed the 4TB disk into the array.  The Netgear immediately flipped to “unprotected” and “disk fault” upon removal of the 2TB disk and switched over to “resyncing array” about 5 minutes following installation of the 4TB disk.  The first resync took 26 hours.  This is on a 9.25TB usable array which was about 33% full.  After resync, I did a reboot just for good measure.

After reboot I repeated the procedure with the second disk.  This time the process took about 18 hours.  So things improved which is good!  Upon completion of this resync, the array status flipped back to “protected”, but each of the 4TB disks was only utilized at the 2TB level (1875GB usable).  This is because the 4TB disks were added into the 2TB layer as 2TB disks.   At this point, a reboot is required in order to get the array to actually resize.  Following this reboot, the status of the ReadyNAS switched to “array expansion” and the GUI started updating progress against the eventual target size.  This is where things got weird:

As you can see, the GUI was reporting that the new size would be 10TB usable – a mere 750GB up from the old array size of 9.25TB.  Some quick math shows that this is either incorrect, or something is wrong:

ORIGINAL ARRAY:

• 6 x 2TB = 12TB RAW
• 6 x 1875GB usable = 11,250GB usable
• 1 disk sacrificed to protection = 9375GB usable
• 100GB for snapshot storage = 9275GB usable as expected

Now let’s consider the new array:

• 4 x 2TB = 8TB RAW
• 2 x 4TB = 8TB RAW
• 4 x 1875GB usable = 7,500GB usable
• 2 x 3750 usable = 7,500 usable
• Total RAW = 16TB, Total usable = 15TB
• 1 4TB disk sacrificed to protection = 11,250GB
• 100GB for snapshot storage = 11,150GB usable

This is pretty far off from the flat 10TB being reported.  Binary/decimal translation aside (10TB vs 10TiB), we’re looking at over 1TB “missing”.  So what gives?  Well before panicing, I decided to have a look at the console.  Check out what a quick df in Linux reported:

 

Ah ha!  11,645,691,704 1K blocks so, in other words, 11.6TB!  Much better.  The good news is that as I copy about 5TB up to the array, df is, as expected, reporting spot on accurate usage whereas the GUI is staying very fuzzy and very wrong.  The conclusion?  Something is up with the GUI post expansion (and post reboot as I rebooted twice to attempt to remediate).

So some final notes:

• be mindful when expanding of the 8TB and 16TB limits
• note that the minimum spindle size to maintain protection is 2 and that you will sacrifice a full 1 of those to reach the new protection size requirement
• reboot as many times as you want during resync.  it won’t cause any issue
• do not reboot during expansion as it might cause an issue
• expect that the GUI might not report size correctly

This is where things stand so far.  As the situation develops or changes, I will update!