↓ Archives ↓

How to build an 8 TB RAID5 encrypted time capsule for 500 Euros

So I wanted to buy a NAS that can act as a time capsule for Apple computers and run a proper Linux at the same time. I also wanted to be able to run the occasional Windows or Linux VM and I wanted to have a lot of storage. As I knew the thing was going to be in our coworking space, it also needed to have disk encryption.

Here’s how I built this for just under €500.00 using standard components and free open source software.

Selecting the hardware components

I found the HP ProLiant MicroServer (see Review and more Picures) to deliver great value for the price. At the time of writing, you can buy it for €209.90 if you’re in Germany like me.

The N36L (which I bought) comes with a single 250GB hard drive which obviously did not meet my “a lot of storage” requirement. So I bought 4 identical Seagate Barracuda Green 2000GB SATA drives which would add another €229.92 to the bill if you bought them today. I am not an expert in hard drives, but the Seagate Barracuda brand was familiar and “Green” sounds good as well.

If you don’t want your new server to host virtual machines at some point, you can probably get out your credit card and check out right now. If you’re like me though, you’d add another 2 bars of 4GB Kingston ValueRAM PC3-10667U CL9 (DDR3-1333) to your cart. The two of them together are just €44.24, so it’s no big deal anyways.

All components together will set you off €484.06. The rest is based on open source software (Debian mostly) which is free as in beer. More about that after the break.

I won’t explain how to actually assemble the components. Please read the manuals if you’ve never installed RAM or a hard drive. It’s pretty easy anyways. The HP even comes with screws and a sort of screw driver which can both be found in the inner side of the hard drive door.

Installing the base system

Now, I want my servers to run a proper operating system, so I went for Debian. You can most probably use Ubuntu or CentOS and get the same results. It’s just a matter of taste, I guess. To get things going, I downloaded a small image from debian.org, put it on a USB drive and booted up the HP.

Considerations

For the most part, the installation is pretty straight forward. You can set up everything from within the Debian installer. There are a couple of small gotchas to consider though:

  • You’re going to need a very small (1MB should be largely enough) primary partition at the beginning of all disks. In order to handle the large 2TB disks, Debian is going to use GPT which in turn relies on that tiny partition.
  • We’re going to encrypt the entire drive in order for your box to be really secure. As far as I know, Debian cannot boot from an encrypted partition, so you’ll need a small partition for the /boot filesystem. Personally, I’ve put that one on a RAID1 array, so I don’t need to worry about rebooting the machine in case of a disk failure.
  • I remember having had a hard time booting from disk for the first time. For some reason, I had to set the proper boot device in the BIOS to make things work. Your mileage may vary.
  • The HP has an on-board RAID controller. Usually, I am more comfortable with things I can fully control, so I went for software RAID which is built-in in Debian. Things might be a bit more performant using the hardware controller and you might still get the same level of robustness and security. I just didn’t use it, so keep that in mind when reading my instructions.

Step by Step

So, basically, here’s what you need to do to get your Debian up and running on your new box:

  • On each drive, create the small (~ 1MB) GPT partition and flag it bios_grub.
  • Again on each drive, create another slightly larger (~100MB) partition and flag it raid.
  • Then, still on each drive, create a large (~2TB) partition taking up all the remaining space and flag it raid.
  • Now, set up a RAID1 using the four smaller 100MB partitions, should be called /dev/md0.
  • Then, set up a RAID5 using the four large 2TB partitions, should be called /dev/md1.
  • Next, set up a crypted partition on top of the RAID5 (/dev/md1), should be called md1_crypt
  • I recommend using LVM, so next, set up a physical volume on the crypted partition (md1_crypt) using up all its space.
  • Now, create a number of logical volumes. I’d recommend to set up one each for the usual suspects /, /home, /var, and /opt at least, but again, your mileage may vary. The nice thing about LVM is that you can add and change logical volumes at any time.
  • Finally set the moint point of /boot to your RAID1 (/dev/md0) and continue the installation. I’ll leave the package selection entirely up to you. Usually, I like to install a bare minimum during setup and get more stuff as I need it later on.
  • If everything went well, your new HP should boot up a couple of minutes later, asking for the password for your crypted disk and present you with the initial login prompt.
  • I usually start by installing ssh and sudo which enables me to do everything else from my own workstation. But you can continue the setup in your closet or wherever your HP sits, no problem.

AFP and Apple Time Machine

Disclaimer: The following is based on a great article by Chris Boot. I read it there first.

Enabling Time Machine support is actually much easier than I first thought and it doesn’t even require serious hacking, compiling or even using non-standard repositories if you’re on Debian. Here’s what you have to do:

  • If you plan on using Time Machine with Lion, you’ll need a newer version of netatalk than is currently available in Debian Squeeze, so you’ll have to add Wheezy to your /etc/apt/sources.list. Edit the file and add the following lines at the bottom:
    # testing
    deb http://ftp2.de.debian.org/debian/ wheezy main
    deb-src http://ftp2.de.debian.org/debian/ wheezy main
    deb http://security.debian.org/ wheezy/updates main
    deb-src http://security.debian.org/ wheezy/updates main
  • Also create a file called /etc/apt/preferences.d/00pinning and add the following lines:
    Package: *
    Pin: release a=stable
    Pin-Priority: 700
    
    Package: *
    Pin: release a=testing
    Pin-Priority: 650
  • Get Netatalk and Avahi using apt: apt-get update ; apt-get install -t wheezy netatalk avahi-daemon
  • In order to export a share using Netatalk, edit /etc/netatalk/AppleVolumes.default and add a line that reads /srv/timemachine "Time Machine" options:tm and be sure to create /srv/timemachine or change it to something you want to use as a location for your shared folder.
  • Next, edit the file /etc/netatalk/afpd.conf and add - -tcp -noddp -uamlist uams_dhx.so,uams_dhx2.so -savepassword -nodebug -icon at the bottom. It will configure netatalk to use the authentication methods required by Lion and some other stuff I found useful.
  • Now, doing a /etc/init.d/netatalk restart is actually already enough to export the folder using AFP ready for Time Machine goodness.
  • However, if you’re an Apple user you probably want to see your new server in the finder instead of just accessing it via its IP address. Enter Avahi. Create the file /etc/avahi/services/afpd.service and paste the following code:
    <?xml version="1.0" standalone='no'?><!--*-nxml-*-->
    <!DOCTYPE service-group SYSTEM "avahi-service.dtd">
    <service-group>
    <name replace-wildcards="yes">%h</name>
    
    <service>
     <type>_afpovertcp._tcp</type>
     <port>548</port>
    </service>
    
    <service>
     <type>_device-info._tcp</type>
     <port>0</port>
     <txt-record>model=Xserve</txt-record>
    </service>
    
    <service>
     <type>_adisk._tcp</type>
     <port>9</port>
     <txt-record>sys=waMA=YOUR-MAC-ADDRESS,adVF=0x100</txt-record>
     <txt-record>dk0=adVF=0x83,adVN=Time Machine</txt-record>
    </service>
    
    </service-group>

    Be sure to replace YOUR-MAC-ADDRESS with the actual MAC address of your HP box and to use the same string for Time Machine as you used in AppleVolumes.default.

And that’s it. Your Mac computers should start seeing your HP box and you should be able to use it as a time capsule for your wireless backups.

Please note: I wrote this guide a couple of weeks after I set this up at the LAUNCH/CO office. So most stuff is from memory. If you find any mistakes or happen to get stuck somewhere in the process, please leave a comment and I’ll try to help.

Update: You may have noticed that your shiny new DIY time capsule has stopped working since you upgraded your Macs to Lion. This blog post looks promising – I will check it out and update this post as soon as I find the time.

Update 2: If you have set up your N36L using these instructions before August 13th, 2011, your box won’t work with OS X Lion. I have just upgraded the instructions to use the latest version of netatalk. You should be able to make Time Machine support work again by following the new AFP and Apple Time Machine instructions adapting your existing config.


Update 3: After almost exactly 3 years running 24×7, two of the Barracuda drives failed at nearly the same time, rendering the RAID 5 unaccessible and losing all data. Fortunately, the machine was only used for backups, so no big deal. @meineerde recommended a Backblaze article that has some nice statistically relevant insight. Turns out, the Barracudas were a bad choice. Also, ordering them all from the same supplier probably meant receiving drives from the same production batch which again increases the chance of them dying at the same time. So now we’re replacing the drives with 4 Western Digital WD20EFRX Red 2TB, ordered from 4 different Amazon merchants. Next update hopefully coming in 3+ years 🙂 (N.b. the price for the 4 WD’s was €366 – which is significantly more than the initial 4 drives I bought)

16 Comments

  • Jul 8th 201123:07
    by Mads

    Hi Jan,

    Nice setup. I’ve been looking for a good and cheap NAS setup and this looks very nice. One thing I can’t see is how you manage to connect the 4 drives to the system, as it only seems to have one internal SATA port? Did you add a SATA controller and if so, what did you choose?

    Again nice setup and good instructions on the debian installation.

    Regards,
    Mads

  • Jul 25th 201115:07
    by Jan Schulz-Hofen

    @Mads: Thanks! I didn’t use any additional hardware. The N36L has 4 hdd slots/ports. You just insert the hard discs and you’re done. No cables, adapters, controllers needed 🙂

  • Jul 28th 201118:07
    by Tristan Waddington

    Thanks for the mention! This is a nicely thought-out tutorial.

  • Aug 1st 201114:08
    by Florian

    Hey Jan,

    also like the setup, just thinking about buying the hp box. Any new insights?

    Did you ever find the time to benchmark the achieved bandwidth? Seems especially interesting with encryption enabled. Also, what are your thoughts on server side encryption vs. easy retrieval of data in case of a real crash?

    Cheers,
    Florian

  • Aug 10th 201117:08
    by Jens krämer

    works like a charm!

    here’s another post about manually compiling netatalk 2.2 for Lion support: http://goeri.homeip.net/blog/?p=395

  • Aug 13th 201111:08
    by Jan Schulz-Hofen

    @Florian: No, I haven’t done any benchmarking yet. I always meant to but as it’s just a backup device I don’t really care about read/write performance. Also, for us, security is far more important than “easy retrieval”. The machine is just a backup, so if it fails and it’s so bad that data cannot be retrieved, we’d just wipe it. If one of our primary systems (the Macs which are being backed up to the N36L) fails, hopefully the backup system’s still up and running, so encryption would not be a problem for restoring the backups to the other machine.

    @Jens: I didn’t have to compile netatalk to make it work for Lion, I am just using the package (2.2~beta4-1) from wheezy – works perfectly.

  • Sep 25th 201120:09
    by Michael Hagmann

    thanks for the update, I just use FreeNAS 8rc2 which has everything included. But in a Memorystick with it, 4 disks and you are finished 😉

    Michael

  • Nov 2nd 201116:11
    by Christian Scherpe

    Where did you get 4x 2TB drives for only 230,- EUR? Please tell me!

  • Nov 4th 201114:11
    by another Florian

    Hi!
    I’m more or less torn whether I’ll build a n36l or buy some NAS-Box (like Synology). Have you any numbers on the power-consumption of the n36l? That’s sort of an important point to me, since I won’t be using it more than 5-6h/week but still would like to keep it runnign 24/7 (just in case… :-D)

  • Dec 4th 201111:12
    by Jan Schulz-Hofen

    @Christian: The Barracudas were dead-cheap when I got them. Looks like they’ve almost doubled since. Maybe you’ll have to wait until after the holiday season?

    http://geizhals.at/deutschland/?plive=588865

    @Florian: I’m sorry – I haven’t gotten around to measure power consumption, yet.

  • Dec 12th 201103:12
    by Sergio

    Thank you very much for this article, it was very helpful when I was trying to decide whether I should buy this or not. I went for the N40L because the price difference was unnoticeable and it comes with a faster CPU and 2GB of RAM.

    Unfortunately, I just started my “lots of TB and RAIDn” project at a bad time, as HDD prices went crazy after last October’s flooding in Thailand. As you said, the prices have doubled since then. I used to buy WD 2TB Green HDDs for 65 euros, but now they cost (depending on the retailer) between 120 and 180 euros. While other brands haven’t been as affected by the flood as WD, the high HDD demand has made their prices increase too. I guess everything will stabilize gradually with time, as Thailand recovers from the flood. But it’s also going to take more than this holiday season.

    Regarding power consumption, while doing my research on this machine I saw this video: http://www.youtube.com/watch?v=2zbjGywEkTA (amongst others, I think) where power consumption is measured during standby and also during a boot process. For extra power savings, there are tutorials on the web on how to replace the stock 200w PSU for a 90w PicoPSU. Power consumption drops from 36w to 18w when idle and to 0w in standby.

  • Apr 3rd 201201:04
    by Jens Krämer

    In case you have problems using netatalk 2.2 from the wheezy sources (for Lion support), it might be worth a try to *not* include the third section advertising the ‘Time Machine’ share in /etc/avahi/services/afpd.service.

    Netatalk 2.2 has avahi support already built in, and for me the setup only started to work after removing that section.

  • Apr 4th 201201:04
    by Jens Krämer

    When afpd from netatalk 2.2 won’t to start with ‘PAM DHX2: libgcrypt versions mismatch. Need: 1.5.0’ in /var/log/daemon.log,

    apt-get -t wheezy install libgcrypt11

    will help.

    Also after installing netatalk from wheezy, the passwd command refused to work (‘passwd: Module is unknown’) – turns out it didn’t find the pam_cracklib module.
    Running ‘ln -s /lib/x86_64-linux-gnu/security/pam_cracklib.so /lib/security’ fixed this.

    All in all, I think next time I’ll backport netatalk 2.2 to squeeze again – I didn’t have any of the above problems when I manually compiled netatalk 2.2 for squeeze on the previous box I set up this way.

  • Jan 20th 201409:01
    by am

    hi there – did you find the n36 noisy? could it sit in the living room for example? thx

  • Apr 4th 201614:04
    by Jan Schulz-Hofen

    It makes some noise. I wouldn’t put in the living room.

  • Leave a Reply