For Part 1 of my DIY Home NAS Build, click here.
UPDATE: Power consumption figure for all drives (9 HDD) on – 80W. With 8 drives spun down, power reduces to 65W. This equates to around $103 yearly cost of running this machine at the current rate of 15.6c/kwh.
Now that we’ve assembled the hardware for the NAS, how do we go about setting it up and making sure everything is humming along nicely? There are an absolute plethora of options available in the RAID software arena, which is just as well, because the right choice is crucial in deciding the upgrade path of the NAS in the future. The common options for NAS build OS’s are everything from the enterprise-ready rpath-based Openfiler (free), Microsoft’s Windows Home Server (paid), the open-source FreeBSD-based FreeNAS, all the way to more ‘normal’ OS’s such as Canonical’s Ubuntu 10 and Microsoft Windows 7 or Server 2003/2008 doubling as NAS servers. Any of these are solid options, with the NAS-specific options generally having lower system requirements than running a full-fledged operating system.
To decide, first we have to analyse our criteria:
* Must combine drives of different sizes as one volume. Due to hardware and cost limitations, we have a bunch of various sized drives lying around, without the budget to dispose of these and replace them all with 2TB drives. This automatically eliminates RAID-5 as a storage option (unless you want to have lots of wasted space).
* Must be remotely administered, through web interface, without the need for a keyboard or monitor in the machine itself. All the above solutions allow this either through web, VNC or RDP.
* Must use space efficiently. This automatically eliminates RAID-1 (and consequently WHS) as a storage option as well, as it uses a full 50% of your space for parity, this is huge.
* Must allow for easy addition of storage space of various sizes (for example if a 3TB drive became available) in future, without having to re-format the entire volume. This eliminates FreeNAS as it’s volumes are not easily and seamlessly expandable. Note that many off-the-shelf solutions do allow for seamless expansion, this nearly always comes down to software choice rather than hardware setup with DIY setups.
* Must be easy to use and administer, for non-technical people like me.
In the end, as more and more options were eliminated from the above criteria, one option stood out. A freemium product by company Lime Technology named Unraid, based on FreeBSD. It satisfied all of the above criteria, looked fairly straightforward to setup, and was aimed squarely at the home/SOHO user with it’s easy setup and robust storage method. It uses a modified version of RAID 4: basically a JBOD, but with an entire drive dedicated to parity. The parity drive held check information based on the contents of all the other drives, so it has to be equal to or larger than the largest disk in the rest of the array.
A catastrophic (and extremely rare) case where two drives would simultaneously fail would render a RAID-5 array destroyed, since the data is striped, but with an Unraid array, the data on the non-affected drives can be salvaged if required (even on a Windows-based system with the appropriate plugin). The bonus is the entire OS runs entirely off a USB stick. I purchased a $15 Kingston stick and ensured the BIOS was set to boot from USB (which most motherboards should be able to do). System requirements are also extremely slim, meaning if you have a really old machine lying around, chances are you can make that into a NAS.
Installation was fairly seamless, the software is free to download and a batch-file install on a Windows system took less than a minute to copy the required files to the USB stick. On first bootup, the NAS boots to a terminal, and from there, you can immediately connect to it via web-browser on the network (or the internet if you have forwarded the appropriate ports) to begin administration.
Unraid detected my hardware perfectly, including all the drives. After a quick initialisation of the drives, they were ready to be written to. Thank goodness for gigabit networking, though if you don’t have gigabit access, you can plug in USB docking stations and mount them under Unraid. Unraid is free for 3 or less drives (including 1 parity), if you want more, there is a $69-$119 one-off lifetime license fee, which I felt was not unreasonable for the peace of mind, and still keeps the price of the entire project very reasonable.
I found that write-performance to the array with the parity drive on and activated suffered noticably (down to around 25MB/s from 60-70MB/s), so I switched off parity-checking until the bulk of my data had been moved across. Once this is done, a full-array parity check is calculated and from that point on, the data is protected. On consulting with others, I noted that ‘some’ of the Netgear and QNAP systems had write speeds in RAID-5 arrays of less than 10MB/s, due in most part due to the CPU bottlenecking operatings, due to being under-specced. Running the linux ‘top’ command in terminal shows that less than 15% of this E5300 CPU was being used on large sequential writes. I also noted that the majority of file transfers in the future I will be undertaking are over the Wireless-N network, which achieves around 8-9MB/s on a good day. Note that Unraid can also utilise a dedicated cache drive for maximum performance.
Mapping the Windows shares. The free space is pooled amongst all NAS volumes.
Now that the array was running well, I made sure that the appropriate Windows shares were mapped, all desktop devices, media players, mobile phones, tablets had the network name mapped and set about exploring some of the more detailed options. Unraid arranges files in shared volumes, which each can be spread over multiple disks. The user who is browsing the files sees only one share with the files they want, instead of having to browse and search over the (in the case 9) disks. It was relatively straightforward to set up some read-only shares and master shares and create a few volumes. The free space is shared amongst all volumes, as they are essentially just different ‘views’ on the same space.
One might wonder at this point, that if the disks spin down after 1 hour of inactivity (by default) to save power, then what if you were browsing a large directory and some files were on one disk and the rest were on another disk, which was spun down. This would constantly create a short delay while the second disk spun up, right? Fortunately, Unraid caters to this by writing to disks with respect to directory levels. For example, if you kept all your movies in a folder called ‘/usr/mnt/Movies‘ it would attempt to write all files within that folder to the same disk (or with respect to a directory depth you specify), as the chances are good they need to be accessed sequentially. Secondarily, it fills disk by default with a ‘high water’ method, which is basically filling up the most empty disk above a minimum criteria from the pool. The idea is that it leaves some space free on each drive, WHILE keeping files likely to be read sequentially together, thus minimising disk spin and reducing delays. Very clever.
The web administration interface is quite easy to use, allowing you to view your RAID condition, monitor network/disk performance and view detailed system logs / information. The package manager (ala many Linux distributions) allow for easy addition and maintenance of add-on plugins. Things like e-mail notifications, auto-shutdowns, periodic parity checking, additional modules for Apple devices, torrent clients (I chose Transmission), DDNS, lighttpd, PHP, Java and much more. The torrent client means I no longer have to leave my main desktop PC on all the time, though I miss the elegance and features of uTorrent (Transmission on Unraid is very bare bones).
The NAS has been running well for nearly two weeks now, with disk temperatures hovering around the 28-32C mark, well within maximum tolerance. Reliability has been top notch, the customisable e-mail status updates are useful for checking on the server if required. Apart from a directory monitoring issue with Transmission (probably mis-configured on my part), everything seems to going well. On occasion I need to wait around 5 seconds for a sleeping disk to spin up, though this is not a huge deal. I also did a test by removing three drives, writing a bulk of files to the array at the beginning, then reinstalling the drives to see if it would seamlessly integrate, which it did.
Unraid needs work in the area of duplicate file finding and possibly improvement in configuring basic options such as FTP access, but it does have a huge community with plenty of effective solutions for tweaking and fixing issues. In short, it’s nowhere near as polished or featured as the glossy interfaces seen on devices such as the QNAP boxes, and there may be a bit of fiddling involved to get things running just right. I admittedly have not yet had time to look into the more advanced features such as DLNA media sharing or automatic workgroup backups, maybe there will be a Part 3.
So there you have it, a DIY Home NAS Build, built from scratch out of all sorts of bits, a ton of expandability and coming in at a very reasonable cost. It’s a project I would highly recommend to anybody, who wants more storage on their network (you can even run multiple servers). See the Unraid FAQ to get started.
All comments/suggestions/feedback welcomed!