Saturday, November 16, 2013

Using debootstrap and chroot to install Debian on a NAS (DS210+)

If you have a Linux based NAS with a powerful enough CPU, you may get an inkling to use it for more than what it was designed for, such as a general purpose server.  Well ... you can!!! ... and this blog tells you how to do it right!

I have a Synology DS210+, so this blog will be tailored around it, but the same approach should work for any NAS.

If you are thinking to use ipkg as outlined in the Synology Mod web-page,  DON'T DO IT!

You will quickly run into dependency conflict hell, especially as new DSM versions come out.  A much more elegant way is to use debootstrap and chroot to set up a parallel  Debian environment.  This has the benefit to compartmentalize your changes so that they do not interfere with the normal functionality of the original NAS OS + it opens the door to use any Debian package rather than the restricted few available under ipkg.

The DS210+ uses a Freescale CPU with an e500 PowerPC core.  In the Debian world, this CPU is not officially supported, but fortunately it has been accepted as an unofficial port under the powerpcspe architecture.  A big thank you to all the folks that made this happen ... the official page is here:

As mentioned on that page, the powerpcspe port makes use of packages in both unstable and unreleased depositories.  Debootsrap only supports fetching packages from one depository, so a special depository with the packages combined is provided.  This got me, and I spent significant time chasing phantom dependency problems due to not all packages being present in unstable ... including in some cases not having perl installed for properly running debootstrap on the target.

If someone has a pointer to a complete and detailed blog/tutorial on how to use multistrap (which is similar to debootstrap but allows multiple depositories) to generate and then unpack an installation archive, please add the link to the comments.  I searched and tried to put together the disjointed pieces for the powerpcspe architecture, but could not get it to work.

Enough talk, let's get going on this.

  1. First you need to enable SSH access on your NAS.  I suggest you also back up any critical data (even though chances are low that data will be impacted).  You assume total responsibility for your actions when you follow this guide.
  2. On a home PC, install VirtualBox with a current version of Debian.  There are plenty of tutorials out there, so I will not go into this.  A very easy one to follow is:
  3. In the Debian VirtualBox, open a Root Terminal and generate the Debian installation archive that will be installed on the target NAS:
    1. Install debootstrap if it is not already installed:

      apt-get install binutils debootstrap

    2. Go to your user directory and make the directory where you will generate the Debian environment (debian for this tutorial)

      cd ~
      mkdir debian
      cd debian

    3. Run debootstrap as below to get all the packages from a Debian depository.  Note that we are using the --foreign option, which tells debootstrap to just get and extract the packages, but it does not install them (this will be done on the target NAS device).  We are also using the --no-check-gpg option which tells deboostrap not to bother checking the packages against a Keyring.  The --arch option tells debootstrap that the target uses a CPU with a powerpcspe architecture.  Note that the depository used is the special /antcom.de one which combines packages from unstable and unreleased!

      debootstrap --foreign --no-check-gpg --arch powerpcspe unstable . http://antcom.de/powerpcspe/

    4. It should just run. If you run into any problems you will probably end up in dependency hell, so it is best to find an alternate distribution (check the debian page mentioned above to see if there are any updates). I ran it on the evening of Nov 12, 2013 and had no problems.
    5. Archive it all up now for transfer to the target:

      tar czf ../debian-stage2.tgz .

  4. Now, let's get the NAS ready to accept and finalize the Debian installation.  Log into your NAS through the SSH shell and do the following:
    1. Select the location where you would like to install the Debain installation.  In this case I will be installing it in /volume1/@debian/wheezy/ (wheezy is the name of the current Debian distribution):

      cd /volume1
      sudo mkdir /@debian
      cd @debian
      sudo mkdir wheezy

    2. Copy the debian-stage2.tgz file into the above wheezy directory.
    3. Extract the archive, chroot to the extracted archive and run the second stage of debootstrap, which installs all the packages on the target.  Then exit the shell for some other housekeeping:

      cd /volume1/@debian/wheezy/
      sudo tar xzf debian-stage2.tgz
      sudo chroot /volume1/@debian/wheezy /bin/bash
      sudo /debootstrap/debootstrap --second-stage
      exit

  5. Ok, now you have Debian installed on your box in parallel with the original OS; albeit this distribution is a bit dated.  No worries, we will bring it up to date next.  For this, you have to add current depositories to the sources.list file:
    1. Update /volume1/@debian/wheezy/etc/apt/sources.list with your favorite text editor with pointers to the debian-ports depositories (ie sudo vi /volume1/@debian/wheezy/etc/apt/sources.list). Add the following lines to the file (it should be blank to begin with):

      deb http://ftp.de.debian.org/debian-ports unstable main
      deb http://ftp.de.debian.org/debian-ports unreleased main
      deb http://ftp.de.debian.org/debian-ports experimental main

      (Note: I did not put in any deb-source entries, maybe someone can comment if they would be worthwhile to add)
    2. After saving the file, go to your home directory:

      cd ~

    3. Make two script files that will come in handy to run Debian. The first file is called initWheezy.sh and will be used to set up the environment every time after a power up.  It should have the following content:

      mount -o bind /volume1/@debian /opt/debian
      mount -o bind /dev /opt/debian/wheezy/dev
      mount -o bind /proc /opt/debian/wheezy/proc
      mount -o bind /sys /opt/debian/wheezy/sys
      cp /etc/resolv.conf /opt/debian/wheezy/etc/
      cp /etc/hosts /opt/debian/wheezy/etc/
      chroot /opt/debian/wheezy /bin/bash

    4. The second file is called wheezy.sh, it will be used to just get into the debian installation after exiting it (but not rebooting the NAS):

      chroot /opt/debian/wheezy /bin/bash

    5. Now make the scripts executable and run the first script to set up the chroot environment:

      sudo chmod 777 initWheezy.sh
      sudo chmod 777 wheezy.sh
      sudo ./initWheezy.sh

    6. We are now running within the Debian environment.  Before proceeding with upgrading the packages, we must upgrade the debian keyring so that apt may verify the packages being installed.  First you need to try and get the updated package list (first line below); this should fail and give you an error with a missing public key.  You take the key provided and plug it into the following commands replacing the put_key_here placeholder with the key number:

      apt-get update
      # should get signature here with missing PUBKEY
      apt-get install debian-keyring
      gpg --keyserver pgp.mit.edu --recv-keys put_key_here
      gpg --armor --export put_key_here | apt-key add -

    7. Now we are ready to perform the Debian update. Execute the following commands:

      apt-get update
      apt-cache gencaches
      apt-get upgrade
      apt-get dist-upgrade

      As before, everything should just run. You may want to run the update, upgrade, and dist-upgrade triplet a second time to make sure all packages that could be upgraded where. You may have some packages that were held back due to dependency reasons and not upgraded, this should not be an issue.
  6. Now you should have a fully functional Debian distribution running in parallel with your NAS OS. Before going all out, you may want to do some minor clean-up and archive your installation in case you screw up during a problematic package download.  If this happens, then you can just delete the wheezy directory and re-install it in a couple of minutes. I suggest you do this every time you get to a point where everything is stable and you have installed the  desired packages.
    1. Set up a user, so that once you run chroot you can "login user_name" before running headless X11 VNC servers or anything else, so that you are not running them from root (but still give it sudo powers):

      adduser user_name
      adduser user_name sudo

    2. Now do a quick clean-up, including cleaning up the downloaded archives:

      rm debian-stage2.tgz
      apt-get clean
      exit

    3. Now you need to power cycle the box. After that, archive the current installation by doing the following:

      cd /volume1/@debian/wheezy
      sudo tar czf ../wheezy_date.tgz .

  7. Now you are ready to use it as you please. Re-run the script to initialize the chroot environment (you need to run it after every power cycle):

    sudo ./initWheezy.sh

    use the other script (sudo ./wheezy) after you exit or logout and need to get back into it.

At this point you can install any available Debian package, only limited by what is in the distribution and the resources of your NAS. The DS210+ has a 1GHz e500 single core CPU with 512MB of RAM, which is not much by todays standards; but I was pretty amazed at how smoothly everything ran. I added a headless VNC servers so that I could give it a desktop feel. Using the above distribution, GNOME was not available, so I ended up using vnc4server with Xfce4 desktop instead. I also installed Jetty and Tomcat Java servers (you must use a different port than is being used by your NAS OS, though).

Use Aptitude instead of apt or dpkg, as it works really well in identifying and resolving any dependency conflicts.

This really opens up the doors of what you can use these NAS boxes for ... just awesome!

Below are a couple of other blogs that provided the inspiration and helped me to put this blog together:

Also, as far as finding stable Debian distributions to upgrade to, look at the following locations as potential alternatives (the one used in the blog is a mirror of http://ftp.debian-ports.org/debian/):

--- Good Luck! ---

5 comments:

  1. I had to re-image my NAS as I was upgrading my drives, so I tar'ed up the wheezy directory to port it to my new install and noticed that I missed a couple of steps in the above description:
    1. Unless you install the ipkg bootstrap from the Synology page, sudo is not available. To get around this just log in as 'root', and drop all the 'sudo' keywords from the instructions.
    2. Before performing step 5.E, you actually have to make the /opt/debian directory: mkdir -p /opt/debian
    That's it ... back in business! Glad I wrote this ... worked great as a reference!

    ReplyDelete
  2. Hi Steve. Hope you can help: I'm at step 4.C sudo chroot /volume1/@debian/wheezy /bin/bash but got error chroot cant execute /bin/bash: exec format error.

    I'm not sure what to do next.

    ReplyDelete
    Replies
    1. If chroot cant execute bash, it is likely that debootstrap was run for the wrong architecture (ie 32 bit vs 64 bit, or wrong CPU target altogether). If the NAS does not have a similar PPC CPU as the one in my unit (Freescale PowerQUICC III MPC8533E PPC), you have to select the correct architecture and depositories in step 3.C ... if you are lucky, it will be supported by the base Debian distribution instead of the ports project.

      Delete
  3. Kernel must be upgraded |
    | |
    | This version of the GNU libc requires kernel version 3.2 or later. |
    | Please upgrade your kernel before installing glibc.

    ReplyDelete
    Replies
    1. Yes, the kernel on these NAS is getting pretty old now, so it is no longer compatible with the packages in the newer distributions. You can only upgrade to an older distribution that is still compatible with the installed kernel. Find older distribution snapshots at: http://snapshot.debian.org/archive/debian-ports/

      I am not sure when the breakage took place, so you may need to do some trial and error, and there was a period that the powerpcspe port was poorly maintained (unfortunately the https://buildd.debian.org/stats/ graph no longer reports status prior to Feb 2016 . . . maybe that's when the kernel incompatibility creeped in). Your best bet would be to start to synch it to a Feb 22, 2015 snapshot; back it up; and then upgrade to newer version (or just stay there if has everything you need).

      Delete