[Guide] Home NAS Build – Part 2: Software and Setup

For Part 1 of my DIY Home NAS Build, click here.

UPDATE: Power consumption figure for all drives (9 HDD) on – 80W. With 8 drives spun down, power reduces to 65W. This equates to around $103 yearly cost of running this machine at the current rate of 15.6c/kwh.

Now that we’ve assembled the hardware for the NAS, how do we go about setting it up and making sure everything is humming along nicely? There are an absolute plethora of options available in the RAID software arena, which is just as well, because the right choice is crucial in deciding the upgrade path of the NAS in the future. The common options for NAS build OS’s are everything from the enterprise-ready rpath-based Openfiler (free), Microsoft’s Windows Home Server (paid), the open-source FreeBSD-based FreeNAS, all the way to more ‘normal’ OS’s such as Canonical’s Ubuntu 10 and Microsoft Windows 7 or Server 2003/2008 doubling as NAS servers. Any of these are solid options, with the NAS-specific options generally having lower system requirements than running a full-fledged operating system.

To decide, first we have to analyse our criteria:

* Must combine drives of different sizes as one volume. Due to hardware and cost limitations, we have a bunch of various sized drives lying around, without the budget to dispose of these and replace them all with 2TB drives. This automatically eliminates RAID-5 as a storage option (unless you want to have lots of wasted space).

* Must be remotely administered, through web interface, without the need for a keyboard or monitor in the machine itself. All the above solutions allow this either through web, VNC or RDP.

* Must use space efficiently. This automatically eliminates RAID-1 (and consequently WHS) as a storage option as well, as it uses a full 50% of your space for parity, this is huge.

* Must allow for easy addition of storage space of various sizes (for example if a 3TB drive became available) in future, without having to re-format the entire volume. This eliminates FreeNAS as it’s volumes are not easily and seamlessly expandable. Note that many off-the-shelf solutions do allow for seamless expansion, this nearly always comes down to software choice rather than hardware setup with DIY setups.

* Must be easy to use and administer, for non-technical people like me.

In the end, as more and more options were eliminated from the above criteria, one option stood out. A freemium product by company Lime Technology named Unraid, based on FreeBSD. It satisfied all of the above criteria, looked fairly straightforward to setup, and was aimed squarely at the home/SOHO user with it’s easy setup and robust storage method. It uses a modified version of RAID 4: basically a JBOD, but with an entire drive dedicated to parity. The parity drive held check information based on the contents of all the other drives, so it has to be equal to or larger than the largest disk in the rest of the array.

A catastrophic (and extremely rare) case where two drives would simultaneously fail would render a RAID-5 array destroyed, since the data is striped, but with an Unraid array, the data on the non-affected drives can be salvaged if required (even on a Windows-based system with the appropriate plugin). The bonus is the entire OS runs entirely off a USB stick. I purchased a $15 Kingston stick and ensured the BIOS was set to boot from USB (which most motherboards should be able to do). System requirements are also extremely slim, meaning if you have a really old machine lying around, chances are you can make that into a NAS.


The main screen of the Unmenu plug-in for Unraid, easy to monitor status. Note how two drives are currently sleeping.

Installation was fairly seamless, the software is free to download and a batch-file install on a Windows system took less than a minute to copy the required files to the USB stick. On first bootup, the NAS boots to a terminal, and from there, you can immediately connect to it via web-browser on the network (or the internet if you have forwarded the appropriate ports) to begin administration.

Unraid detected my hardware perfectly, including all the drives. After a quick initialisation of the drives, they were ready to be written to. Thank goodness for gigabit networking, though if you don’t have gigabit access, you can plug in USB docking stations and mount them under Unraid. Unraid is free for 3 or less drives (including 1 parity), if you want more, there is a $69-$119 one-off lifetime license fee, which I felt was not unreasonable for the peace of mind, and still keeps the price of the entire project very reasonable.

I found that write-performance to the array with the parity drive on and activated suffered noticably (down to around 25MB/s from 60-70MB/s), so I switched off parity-checking until the bulk of my data had been moved across. Once this is done, a full-array parity check is calculated and from that point on, the data is protected. On consulting with others, I noted that ‘some’ of the Netgear and QNAP systems had write speeds in RAID-5 arrays of less than 10MB/s, due in most part due to the CPU bottlenecking operatings, due to being under-specced. Running the linux ‘top’ command in terminal shows that less than 15% of this E5300 CPU was being used on large sequential writes. I also noted that the majority of file transfers in the future I will be undertaking are over the Wireless-N network, which achieves around 8-9MB/s on a good day. Note that Unraid can also utilise a dedicated cache drive for maximum performance.

Mapping the Windows shares. The free space is pooled amongst all NAS volumes.

Now that the array was running well, I made sure that the appropriate Windows shares were mapped, all desktop devices, media players, mobile phones, tablets had the network name mapped and set about exploring some of the more detailed options. Unraid arranges files in shared volumes, which each can be spread over multiple disks. The user who is browsing the files sees only one share with the files they want, instead of having to browse and search over the (in the case 9) disks. It was relatively straightforward to set up some read-only shares and master shares and create a few volumes. The free space is shared amongst all volumes, as they are essentially just different ‘views’ on the same space.

One might wonder at this point, that if the disks spin down after 1 hour of inactivity (by default) to save power, then what if you were browsing a large directory and some files were on one disk and the rest were on another disk, which was spun down. This would constantly create a short delay while the second disk spun up, right? Fortunately, Unraid caters to this by writing to disks with respect to directory levels. For example, if you kept all your movies in a folder called ‘/usr/mnt/Movies‘ it would attempt to write all files within that folder to the same disk (or with respect to a directory depth you specify), as the chances are good they need to be accessed sequentially. Secondarily, it fills disk by default with a ‘high water’ method, which is basically filling up the most empty disk above a minimum criteria from the pool. The idea is that it leaves some space free on each drive, WHILE keeping files likely to be read sequentially together, thus minimising disk spin and reducing delays. Very clever.

The web administration interface is quite easy to use, allowing you to view your RAID condition, monitor network/disk performance and view detailed system logs / information. The package manager (ala many Linux distributions) allow for easy addition and maintenance of add-on plugins. Things like e-mail notifications, auto-shutdowns, periodic parity checking, additional modules for Apple devices, torrent clients (I chose Transmission), DDNS, lighttpd, PHP, Java and much more. The torrent client means I no longer have to leave my main desktop PC on all the time, though I miss the elegance and features of uTorrent (Transmission on Unraid is very bare bones).


An example of a status update email. Yes, I know there are sync errors :P

The NAS has been running well for nearly two weeks now, with disk temperatures hovering around the 28-32C mark, well within maximum tolerance. Reliability has been top notch, the customisable e-mail status updates are useful for checking on the server if required. Apart from a directory monitoring issue with Transmission (probably mis-configured on my part), everything seems to going well. On occasion I need to wait around 5 seconds for a sleeping disk to spin up, though this is not a huge deal. I also did a test by removing three drives, writing a bulk of files to the array at the beginning, then reinstalling the drives to see if it would seamlessly integrate, which it did.

Unraid needs work in the area of duplicate file finding and possibly improvement in configuring basic options such as FTP access, but it does have a huge community with plenty of effective solutions for tweaking and fixing issues. In short, it’s nowhere near as polished or featured as the glossy interfaces seen on devices such as the QNAP boxes, and there may be a bit of fiddling involved to get things running just right. I admittedly have not yet had time to look into the more advanced features such as DLNA media sharing or automatic workgroup backups, maybe there will be a Part 3.

So there you have it, a DIY Home NAS Build, built from scratch out of all sorts of bits, a ton of expandability and coming in at a very reasonable cost. It’s a project I would highly recommend to anybody, who wants more storage on their network (you can even run multiple servers). See the Unraid FAQ to get started.

All comments/suggestions/feedback welcomed!

Advertisements

11 thoughts on “[Guide] Home NAS Build – Part 2: Software and Setup

  1. It does seem shitty that you can’t easily add storage to ZFS storage pools, but I don’t let that stop me. I figure I’ll just wipe out the old array, build the new one, and then restore from a backup. Or just have 2 pools, rather than adding a new zdev to an existing pool. I don’t like the idea of having multiple zdevs per pool, since if you lose one, you lose the whole pool. Think about that. If you have a double parity zdev in the same pool as a single disk zdev, if you lose the single disk, you also lose all the data in the array as well! Sometimes, it’s better to have some degree of independence rather than putting all your eggs in one basket (or pool).

    Unraid does sound like the way to go with mismatched drives, but it also doesn’t sound very quick. ZFS stripes the parity data over all the drives, so the writes aren’t bottlenecked by the parity drive, and the reads should be quick as well. I would install the free version on a spare computer to try it out, but a 3 drive array doesn’t sound very exciting. 33% or more of your data is lost to single parity.

    I’m new to this whole array concept though. It seems a little scary. Having data spread out on multiple drives means that you could lose everything with just one drive failure! Rebuilding from parity data sounds interesting, but it’s not quite a guarantee, since there could be errors on the other drives you’re rebuilding from. Of course, I’ll still backup my important stuff, but double parity sounds like an acceptable compromise if it saves me a headache. In the array I’m building, I’m giving 33% to parity, but it’s double parity.

  2. I really like the features of ZFS based NAS products (be it Nexenta, Open Indiana, SmartOS, FreeNAS, EON Nas, whatever) especially data integrity. The BIG issue with Home use/Media storage/streaming is that you can not just add disks to a pool.

    So lets say you start out with a RaidZ (raid 5 equivalent) and use the minimum 3 disks. When it comes time to add more storage you can’t just add another disk to that pool you need to add another 3 disk raidz! Gets worse if you start out with larger number of disks. You are also stuck with performance that is the equivalent of just one disk until you start adding vdevs.

    If you could just expand a pool by adding another disk then ZFS would be a no brainer, but just that one feature is a killer. Unraid while not perfect is most suitable for home media server especially if you use a flash disk for caching.

  3. Seriously like the idea. Hardware, easy. NO idea about all that Raid/parity/transmission setup stuff. Initially I only want 2 drives(Raid 1/mirror?). Should I still consider this, or pay the (considerable) extra and buy a store bought NAS.
    Cheers
    Tony

  4. Can’t say I’ve even heard of Nexenta. I find Unraid to be suitable to my needs and it’s been going strong so far, but thanks for the tip.

  5. Trying to figure out what torrent client to use myself. I see that you used transmission. Would you care to state how you did that.

    how you got it installed that is. Did you get it through something like unMenu or did you install it manually?

    Best regards,
    Adam

  6. Extremely helpful, love to read your input.
    You goals are close to my own. Didn’t even know about unRAID until I stumble on this article.

    Thanks

    1. I wonder if anyone has tried running ZFS on a home NAS using OpenIndiana?
      Maybe the performance will be better…

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s