RaspberryPi and RaspBMC : Set System to boot on a USB Key/HD

On ce I finally got my hand on a [url=, I began with the obvious by setting some SD card with [url=, Debian and some other fun stuff[/url]. Nevertheless, I knew from the beginning my first Rpi would be used as a media center replacement from my [url=//bambou.viens.la/wiki/doku.php?id=reseau:transformer_sa_wii_en_serveur_web]hacked Wii[/url], which, albeit working well for most of the files, just doesn't allow for the same kind of control you can get of a box with SSH access.

[url= is a project by Sam Nazarko to port last version of Xbmc to the Pi. It is based on a Debian distro, with added scripts and optimizations. To get RaspBMC, [url= should follow these steps.[/url].

Rpi being that low cost, The SD card player might not be the fastest model available as today, and solid statte memory tends to wear out fast, because they just weren't designed for a frequent 24/7 r/w cycle. You may also have noticed that access on a flash memory is slooooow... and expensive !

That're a lot of reasons why you could want to use a USB hard drive as a host for your filesystems.

It should be mentionned that all steps bellow are Gnu/linux universal, so you could pretty much apply this tut to other available distributions.

[size=medium][b]So first, some explanations :[/b] [/size] Once you've imaged your SD card with the Rpi installer, you end up with 3 partitions :

[list] []One vfat sized 63 MB []One swap partiton sized 122MB [*]One Ext4 partition that fits the remaining space of your SD card. [/list]

The vfat (or FAT32 ) partition contains the bootstrap files, firmware, kernel and boot arguments. It HAS to stay on the SD card. This is due to the hardware of the Pi and the fact that we're bootstrapping from a GPU. This one stays where it is.

The Swap space is used by *nix like systems to cache unused RAM data when this last one gets full. It is recommended to be twice the size of you ram when less than 1GB. I'm not sure wether it's accurate or not, but since we're moving to a large HD, we can enlarge the original 122 MB one to reach 256MB :) This one will move to HD.

The Ext4 partition contains the debian system files that serve as a software base for xbmc to run. This one is moving too.

[size=medium][b]Separate Home folder[/b][/size]

Raspbmc being at alpha stage, it means that you might have to go through the process of re-imaging your SD card, and recreating your filesystems a lot.

We could set up our filesystem so that when you'll upgrade you system files, all your media files, but more important, all your settings would survive an update or re-imaging.

For this, we'll have to create another partition that will old your personal 'Home' folder.

[b]Some stuff you need to know[/b]

The raspbmc image default id infos are :

[code]login : pi pasword: raspberry[/code]

These commands should be run from your default user location. To make sure you're there :

[code]pi@raspbmc: ~$ cd ~[/code]

Most of the commands described here need the 'sudo' prefix in order to work ! If you get a 'permission denied' or 'command not found' error message, it is likely that you forgot the 'sudo' prefix.

[b][size=medium]Let's go[/size][/b]

What you need :

[list] []A Raspberry Pi (duh) []A SD Card with Raspbmc set up [*]A USB Hard drive with an external PU ( or USB key ) [/list]

[color=##FF0000][size=medium]IMPORTANT : The steps described here will DESTROY the data that resides on your hard drive ! Make sure to backup everything important. I will refuse to be held responsible for any file loss. You've been warned. If you agree to this disclaimer and to take your responsibilities, just breath once.[/size][/color]


An important step is to disable udevd [b]before[/b] connecting your USB drive. Udev is the daemon that's in charge of detecting new devices. It will prevent us from going to a lot of hassle with it trying to mount our new filesystems :

[code]pi@raspbmc:~$ sudo service udev stop[/code]

We can also stop xbmc :

[code]pi@raspbmc:~$ sudo initctl stop xbmc[/code]

[size=medium][b]Step 1 : Setup your hard-drive[/b][/size]

First, we need to set up your hard drive with the right file system. To achieve this, we're going to use some nice and powerful *nix programs. Fear not of the CLI, for it is a mighty full tool !

Unfortunately, you'll have to use another computer to SSH into your Rpi, since there is no option to boot on CLI, without XBMC launching for the moment ( Could be a good idea ? )

Once you ssh'd into you Rpi, you can check your hard-drive is there by listing present devices :

[code]pi@raspbmc:~$ ls /dev/sda[/code]

It should issue a line saying /dev/sda. If otherwise, you might want to check your HD is spinning, USB connections, etc. You can also try to increment the last letter in 'sda', in the alphabetic order - ie 'sdb', 'sdc', etc.

Once you made sure /dev/sda is your HD alright, we're going to partition it this way :

Let's unmount all partitions on sda :

[code]pi@raspbmc:~$ sudo umount /dev/sda*[/code]

As much people won't feel confortable without a GUI, I'm going to use the 'cfdisk' utility :

[code]pi@raspbmc:~$ sudo cfdisk -z /dev/sda[/code]

You should now see a nice text user interface that lists what's on your disk right now, with a menu on the bottom of the screen.

You should now write down what partitions the different device files refers to, just in case.

It should be :

[list] []/dev/sda1 == SWAP (256MB) [] /dev/sda2 == / (8 GB) [*]/dev/sda3 == /home (remainder of the disk) [/list]

Once done, you MUST select 'Write' in the bottom menu and confirm by answering 'yes' to the question that you'll make sure to read and check.

You're back at the cli. Now, we have defined our partitions, but the filesystems (fs) were not created yet. That's what we're going to do with the mkfs utilies. The swap uses a different utility.

Build the SWAP fs : [code]pi@raspbmc:~$ sudo mkswap /dev/sda1[/code]

Build the Ext4 fs :

[code]pi@raspbmc:~$ sudo mkfs.ext4 /dev/sda2[/code] [code]pi@raspbmc:~$ sudo mkfs.ext4 /dev/sda3[/code]

If you get an error message telling you '/dev/sdX' doesn't exist, you can unplug, then replug your USB drive. Then check that 'dev/sdX' is accessible by entering :

[code]pi@raspbmc:~$ ls /dev/sdX[/code]

It should be noted that you do not want to see any /dev/sdaX in the ouptut of

[code]pi@raspbmc:~$ df -h[/code]

yet. If you do, you have to unmount then by issuing :

[code]pi@raspbmc:~$ sudo umount /dev/sda*[/code]

( The wildcard is literal ! )

That's it for this step. We now have a nice setup hard drive. To check for this, we can issue the command :

[code]pi@raspbmc:~$ sudo fdisk -l /dev/sda[/code]

that should print something like :

[code]pi@raspbmc:~$ sudo fdisk -l /dev/sda

Disk /dev/sda: 200.0 GB, 200049647616 bytes 255 heads, 63 sectors/track, 24321 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x000baa4f

Device Boot Start End Blocks Id System

/dev/sda1 1 12814 524288 82 Linux swap / Solaris /dev/sda2 12814 13834 8192000 83 Linux /dev/sda3 13834 24322 84243456 83 Linux [/code]

[size=medium][b]Step 2 : Copy the system files[/b][/size]

In order to copy the files from the SD to the USB HD, we'll have to do some mounting . We create the directories that will hold our mounts , then mount the corresponding partitions :

[code]pi@raspbmc:~$ sudo mkdir /tmp/sys && sudo mount /dev/sda2 /tmp/sys[/code]

[code]pi@raspbmc:~$ sudo mkdir /tmp/home && sudo mount /dev/sda3 /tmp/home[/code]

Beware ! These directories will disappear upon reboot, as they are place in the '/tmp' folder.

[b]Should I use 'dd' or 'rsync' ? [/b] I've seen several answers that mentioned the use of 'dd' commands to copy your system files. I personaly don't think it's a good idea, nor the good tool.

I'll just copy what I said on another thread :

[i]Unless there is a good reason for using 'dd', I would recommend using rsync. 'dd' copies a disk bit-by-bit, thus it's very slow, plus it can be very dangerous to play with when you don't know what you're doing.[/i]

I'll add that if you want to duplicate a mounted filesystem you're working off, you have to use the rsync method.

Let's install that sweet utility :

[code]pi@raspbmc:~$ sudo apt-get install rsync[/code]

[b]A. Personal folder and user settings[/b]

We'll begin easy, and copy the user settings on our home, reimaging-proof partition, with rsync first :

[code]pi@raspbmc:~$ sudo rsync -avh /home/pi/ /tmp/home/pi/ [/code]

We use 3 options here :

-a : archive mode : it will preserve owner,group,permissions,links and devices information, and perform recursively. This is REQUIRED. -v : Because we want more output of the progress -h : Because we want human readable output !

Let's get rid of this one for now :

[code]pi@raspbmc:~$ sudo umount /tmp/home[/code]

[b]B. System files[/b]

We'll continue a little harder now . Since the filesystem we want to copy the files from is running ( your Rpi is on right ?), we'll have to remount it elsewhere in order to rsync it without falling into a deadly recursive loop.

Plus, we don't want to rsync what's in the '/home/pi' folder sinc we just rsynced it :

EDIT : And obviously, we don't want to rsync the '/tmp' folder, since we're working there :/ Thanks to Baldjedi for bringing that up ! We also can exclude the '/boot' folder, as it won't be of any use on the HD.

We remount the FS :

[code]pi@raspbmc:~$ sudo mkdir /tmp/sd_sys && sudo mount --bind / /tmp/sd_sys [/code]

We rsync, without '/home' , '/tmp', '/boot' contents :

[code]pi@raspbmc:~$ sudo rsync -avh --exclude 'home/pi' --exclude 'tmp/' --exclude 'boot/' /tmp/sd_sys/ /tmp/sys/ [/code]

[i]This is where you could optionally use the '-u' option, so that when you upgrade the system files from the SD card next time, only files that were modified are updated. [/i]

We yet have to recreate the folders we excluded during rsync :

[code]pi@raspbmc:~$ sudo mkdir /tmp/sys/boot /tmp/sys/tmp[/code]

You can check all went well by issuing :

[code]pi@raspbmc:~$ df -h /dev/sda2 [/code]

It should show a use of at least a few hundreds MB.

You can also :

[code]pi@raspbmc:~$ ls -lah /tmp/sys [/code]

which should answer something like:

[code] total 89K drwxr-xr-x 23 root root 4.0K Jun 7 09:15 . drwxr-xr-x 23 root root 4.0K Jun 7 09:15 .. drwxr-xr-x 2 root root 4.0K Jun 7 21:38 bin drwxr-xr-x 2 root root 1.0K Jan 1 1970 boot drwxr-xr-x 12 root root 3.2K Jan 1 1970 dev drwxr-xr-x 76 root root 4.0K Jun 8 15:18 etc drwxr-xr-x 3 root root 4.0K Jun 7 00:45 home drwxr-xr-x 12 root root 4.0K May 30 03:25 include drwxr-xr-x 11 root root 4.0K Jun 7 23:18 lib drwx------ 2 root root 16K Jun 7 08:51 lost+found drwxr-xr-x 10 root root 4.0K Jun 7 00:46 media drwxr-xr-x 2 root root 4.0K May 7 15:28 mnt drwxr-xr-x 4 root root 4.0K Jun 7 09:21 opt dr-xr-xr-x 70 root root 0 Jan 1 1970 proc drwx------ 4 root root 4.0K Jun 7 22:20 root drwxr-xr-x 2 root root 4.0K Jun 7 00:49 sbin drwxr-xr-x 4 root root 4.0K Jun 7 00:49 scripts drwxr-xr-x 2 root root 4.0K Jul 21 2010 selinux drwxr-xr-x 3 root root 4.0K Jun 7 00:45 srv drwxr-xr-x 12 root root 0 Jan 1 1970 sys drwxrwxrwt 6 root root 4.0K Jun 8 15:17 tmp drwxr-xr-x 10 root root 4.0K Jun 7 00:39 usr drwxr-xr-x 13 root root 4.0K Jun 7 00:39 var [/code]

Yay, we're done with that !

[size=medium][b]Step 3 : Setup the new Swap file and System files[/b][/size]

In order for Gnu/Linux to find our newly created Swap file and system files, we have to tell him where it is.

For that, we have to declare it in the file '/tmp/sys/etc/fstab'.

[code]pi@raspbmc:~$ sudo nano /tmp/sys/etc/fstab [/code]

You should have something that looks like :

[code] proc /proc proc defaults 0 0 /dev/mmcblk0p1 /boot vfat defaults 0 0 /dev/mmcblk0p2 none swap sw 0 0 /dev/mmcblk0p3 / ext4 defaults,noatime 0 0 [/code]

Edit the three last lines so that they look like :

[code] proc /proc proc defaults 0 0 /dev/mmcblk0p1 /boot vfat defaults 0 0 ## SD card bootstrap files /dev/sda1 none swap sw 0 0 ## New swap file /dev/sda2 / ext4 defaults,noatime 0 0 ## New System files /dev/sda3 /home ext4 defaults,noatime 0 0 ## New User personal folder [/code]

Once this done, save by hitting 'Ctrl'+'O', then 'Return'. You can then quit by hitting 'Ctrl'+'X'.

You can unmount all these now :

[code]pi@raspbmc:~$ sudo umount /tmp/sys /tmp/home /tmp/sd_sys [/code]

You might see an error related to /tmp/home not being mounted. This is of no importance here. We just want to make sure everything's unmounted.

[size=medium][b]Step 4 : Tel the Rpi where to find what[/b][/size]

In order for the Rpi to boot on the right partition, we have to tell it in the '/boot/cmdline.txt' file :

[code]pi@raspbmc:~$ sudo nano /boot/cmdline.txt [/code]

Edit :

[code]dwc_otg.lpm_enable=0 root=/dev/mmcblk0p3 rootfstype=ext4 rootwait quiet[/code]

To look like this:

[code]dwc_otg.lpm_enable=0 root=/dev/sda2 rootfstype=ext4 rootwait quiet[/code]

It should be noted here that we've modified the 'root=' argument, so that it looks for our file system on '/dev/sda2'. That's the partition we set up in Step 1. We could also use another technic to refer to our partitions by their UUID. Let me know if you'd like to see me describe this here.

[size=medium][b]Step 5 : Reboot ![/b][/size]

Hopefully, if you followed all these steps carefully, after issuing :

[code]pi@raspbmc:~$ sudo reboot[/code]

Your Rpi should boot fine, and you should notice some improvements in terms of reactivity.

A new issue of :

[code] pi@raspbmc:~$ df -h[/code]

should echo something like :

[code] Filesystem Size Used Avail Use% Mounted on /dev/sda2 7.7G 826M 6.5G 12% / tmpfs 62M 0 62M 0% /lib/init/rw udev 10M 168K 9.9M 2% /dev tmpfs 62M 4.0K 62M 1% /dev/shm /dev/sda3 80G 264M 75G 1% /home /dev/mmcblk0p1 63M 6.0M 58M 10% /boot [/code]

You can check that the swap file in use is our new one by issuing

[code] pi@raspbmc:~$ sudo swapon -s [/code]

It should answer with :

[code] Filename Type Size Used Priority /dev/sda1 partition 249948 0 -1 [/code]

[b][size=medium]What if I want to upgrade raspbmc ?[/size][/b]

If you were in the need to upgrade raspbmc, you'd simply reimage the sd card, that would boot on the system files that are on the SD card. You would then have to reproduce Step 2.A, Step 3 and Step 4.

[b]It should be mentionned that you do not need to do this if you upgrade with a nightly build, as it is my understanding that it will only replace the xbmc binaries in /opt.[/b]

You could even use the '-u' option on rsync the first time you copy system files. That way, only the files that were modified since last rsync gets updated.

I've been careful not to let any typo or mistake slip in but you always forget some. Be sure to let me know if you find one, or have any suggestions/remarks. Should also mention english is not my native language, hence the numerous mistakes and bad grammar.

Hope it's useful !

[size=medium][b]Thanks to BaldJedi and r8rae for their corrections, suggestions and commitment ! [/b][/size]

EDIT : Corrected some typos.

After some time running the RC2 with this setup, here are my observations :

I should have mentioned that I use a 4g class 10 SDHC card, that is supposed to offer à 10MB/S min rate.

[size=large][b]Sam's Note: Remember, kernel-vfat-latest.tar.gz overwrites cmdline.txt, where you edited -- so be sure to chattr +i it -- cheers [/b][/size]