outcoldman
outcoldman Denis Gladkikh

Ubuntu as a home server. Part 5. Reliability.

Ubuntu, RAID, CrashPlan, Dropbox, and OneDrive
  1. Dynamic DNS.
  2. OpenVPN.
  3. AFP Server (for OS X).
  4. SMB server (for Windows and Linux).
  5. Reliability (Backups).

Reliability (Backups)

You probably know that it is important to backup data from your computers. This is how I backup my data

You already know how to configure TimeServer on Ubuntu, so let’s talk about RAID, Back in Time and CrashPlan.

RAID

In my case I have two RAID volumes, both of them RAID-0

RAID-0 does not give you better redundancy. It just simplifies your life by combining multiple disks in one. When one of them fails - you loose entire RAID-0 volume. This is not a problem for me, because I backup most important data using CrashPlan to the cloud. But if you will decide to go only with local backups you can configure RAID-1 (or some other RAID levels) volume, which mirrors data between two drives. This means that if one of them dies - you can restore data from another one. If both of them die - you loose everything.

Local backups are better than not having backups at all, but having external backups (physically external) is better. In case of flood or fire you can loose all your data if you keep everything in your house.

To configure RAID on Ubuntu at first you need to install mdadm (tool for managing RAID volumes)

$ sudo apt-get install mdadm

After that take a look on all drives you have

$ sudo lsblk -o NAME,FSTYPE,SIZE,MOUNTPOINT,LABEL

Find drives which you want to use in RAID and invoke next command

$ sudo mdadm --create /dev/md0 --level=0 --raid-devices=2 /dev/sdf /dev/sdd

This command will create RAID-0 with two drives /dev/sdf and /dev/sdd. If you want to build RAID-1 (or any other RAID) just change --level, if you have more disks just change --raid-devices and add more of them in the list.

After that verify that they are created

$ cat /proc/mdstat

If you are trying to configure RAID-0 it will be available almost right after you created it. If this is another level - it will take some time to synchronize blocks between drives, you can watch for the progress

$ watch cat /proc/mdstat

After that you need to modify /etc/mdadm/mdadm.conf to include information about this RAID volume to make it available on startup

# Tell that RAID depends on these two disks
DEVICE /dev/sdf /dev/sdd
# RAID configuration
ARRAY /dev/md0 devices=/dev/sdd,/dev/sdf level=1 num-devices=2 auto=yes

After that using standard tool you can create new partition on this volume and mount it.

Back In Time

As I said above I use Back In Time on Ubuntu instead of standard backup tool, as I like it more. To install it do

$ sudo apt-get install backintime-common backintime-gnome gksu

gksu is required to launch Back In Time in root mode (if you want to backup not only user folders). Without this tool you can still launch Back In Time (root) but standard sudo command will not set right home directory which Back in Time expects, so you will find out that Scheduler will use different configurations than you will expect. So just install this tool and launch Back In Time (root) from Applications menu.

After that open Back In Time (root) and change some settings

.gvfs
.cache*
[Cc]ache*
.thumbnails*
[Tt]rash*
*.backup*
*~
.dropbox*
/proc/*
/sys/*
/dev/*
/run/*
/mnt
/media
lost+found
/var
/proc
/run
/home
/export

As you see I exclude /home because I backup it with CrashPlan and I exclude /mnt because I have mounted here only two partitions: one is the SSD drive with Virtual Machines, and another one is a partition where I store backups.

Using GUI you cannot specify more than one Scheduler for Back In Time, I used crontab editor to do that

$ sudo -s
$ crontab -e

Where I included two lines

0 2 * * 1-6 /usr/bin/nice -n 19 /usr/bin/ionice -c2 -n7 /usr/bin/backintime --backup-job >/dev/null 2>&1
0 2 * * 0 /usr/bin/nice -n 19 /usr/bin/ionice -c2 -n7 /usr/bin/backintime --checksum --backup-job >/dev/null 2>&1

As you see I scheduled two backups

CrashPlan

I love CrashPlan. I pay for Individual account as I backup only one Computer which is my Ubuntu server.

Actually my first experience with it was terrible

No sure that I understand what is the point to use #CrashPlan? to backup 10 Gigs/month? Better to use #DropBox pic.twitter.com/LGRA7EVVBx

— Denis Gladkikh (@outcoldman) October 14, 2014

After googling I found that a lot of users were complaining about the same issue. CrashPlan was way to slow for them. That it can take around 1 year to backup 1Tb of data. I thought that it is expected, so I gave up and started to look for alternatives.

But to my luck CrashPlan answered on my complain which gave me some hope that I could do something to fix this issue

@outcoldman That's way too slow, yes! Did our Customer Champions investigate?

— CrashPlan (@crashplan) October 15, 2014

After googling one more time I found manual on CrashPlan support website Speeding Up Your Backup (why people don’t read manuals?) and so

Using this manual http://t.co/CFUGQ80nXw I could speed up #CrashPlan backup process. Nice, like it! /cc @crashplan pic.twitter.com/DMWnci8zSa

— Denis Gladkikh (@outcoldman) October 16, 2014

I had few other issues with CrashPlan after that, but I could easily fix them by using support pages on CrashPlan website or by contacting CrashPlan support.

I uploaded 0.7Tb in just one week. At current moment I have over 1.1Tb data in CrashPlan, my backup is encrypted using Custom Key (which means that CrashPlan don’t have access to my data). This is what I upload

You can sign up for the 30 days trial to see how CrashPlan works.

Cloud alternatives

Because my first experience with CrashPlan was terrible I also tried some other alternatives, I could not find any alternative services to CrashPlan with native Linux client. So I tried to use cloud storage providers instead

With them I just tried to backup most important data from my home folder directly without trying to backup anything else. This works too.

Have feedback or questions? Looking for consultation?

My expertise: MongoDB, ElasticSearch, Splunk, and other databases. Docker, Kubernetes. Logging, Metrics. Performance, memory leaks.

Send me an email to public@denis.gladkikh.email.

The content on this site represents my own personal opinions and thoughts at the time of posting.

Content licensed under the Creative Commons CC BY 4.0.