Updating my WD My Book Live (part 2)

In my previous post, I had successfully installed the Debian Jessie image and was I able to SSH onto the box. The remaining work was to clean up as suggested by Ewald and also to update to the latest available kernel, 4.19. Having done this I found out that there were a few things that needed fixing before I could use the My Book Live (MBL) for my backups.

Basic configuration

If you remember, I still had the WD MBL in pieces on my desk, so I plugged the MBL HDD back into my PC. Then I followed the instructions provided by Ewald in his instructions: https://github.com/ewaldc/My-Book-Live/tree/master/debian/debian%208%20(Jessie)#how-to-install-with-ext4-

Warning: the commands could be break your PC if you forget to cd to mount point first! For example:

cd /mount/andy/<UID for HDD>

For the rest of this section, the commands work if you have changed to the root of the mounted WD MBL HDD.

First, I set up a static address for the server by editing the the etc/network/interfaces file to look like this:

# interfaces(5) file used by ifup(8) and ifdown(8)
#allow-hotplug lo
iface lo inet loopback 

#iface eth0 inet dhcp 
iface eth0 inet static 
  address 192.168.1.203 
  netmask 255.255.255.0 
  gateway 192.168.1.1                                                 
allow-hotplug eth0 
#auto eth0 

Then I changed the etc/exports file to use the correct sub-net.

To change the host name, added the new host name ‘mbl’ to /etc/hosts so it looks like this:

127.0.0.1       localhost mbl

Then I replaced the host name in etc/hostname.

Debian Jessie is no longer supported so set apt to use the archive. Comment out all lines in etc/apt/sources.list except for the line:

deb http://archive.debian.org/debian jessie main contrib non-free

Finally, the users ewald,tea and share need to be removed. Delete all user files

sudo rm -rf home/ewald

Edit etc/passwd to remove the users by deleting the lines starting with ewald,tea and share. Edit etc/group to remove the line starting with share and then remove ewald and tea from all other groups.

That was all that I could do from the convenience of my host PC so it was time to test all my changes and then complete the setup of my WD MBL.

Setting up

Having logged in as root using SSH, the first thing to do was to replace the root SSH key:

rm /etc/ssh/ssh_host*
dpkg-reconfigure openssh-server

Then I added me as a sudo user.

adduser andy
adduser andy sudo

Having done that, I rebooted, SSH’d in as me and then started setting up the WD MBL the way I wanted it.

DNS problems

The first thing I did was to try to install vim as it is my preferred command line editor. However, the apt install command didn’t work with what looked like a network problem. The network connection was OK as I was using SSH, so I tried the command:

$ ping google.com
ping: unknown host google.com

This showed that the DNS server not being found. Verified using:

nslookup google.com
;; connection timed out; no servers could be reached

There are several ways to add DNS servers and, after some trial and error, I found that changing /etc/resolv.conf to look like this worked.

nameserver 8.8.8.8 
nameserver 8.8.4.4

You can use other DNS servers but these values are great for testing as they are easy to remember. Re-testing with nslookup succeeded so I then changed the DNS servers to those recommended by my ISP.

Finally, I was able to install vim so I could then move on to the next problem.

Date and time

While I was sorting out the DNS problems, I found out that the system time and date was reset after each boot to some time in 2018. This was easy to temporarily fix using the command date but I wanted this server to be able to reboot without manual intervention so I needed to get the time and date to update automatically.

My immediate thought was to use the ntpd daemon. I checked to see if it was running using:

$ sudo systemctl status ntp
 ● ntp.service
    Loaded: not-found (Reason: No such file or directory)
    Active: inactive (dead)

As ntpd was not running, I googled for how to start it up and this is when I found out about systemd-timesyncd. This is basically a cut down version of the full ntpd that is integrated into systemd that is only used to synchronise the local system time with the time from NTP servers. So I used the following commands:

$ sudo systemctl start systemd-timesyncd
$ sudo systemctl enable systemd-timesyncd 
Created symlink from /etc/systemd/system/sysinit.target.wants/systemd-timesyncd.service to /lib/systemd/system/systemd-timesyncd.service.

The status of the daemon can be found using:

$ sudo systemctl status systemd-timesyncd     
 ● systemd-timesyncd.service - Network Time Synchronization
    Loaded: loaded (/lib/systemd/system/systemd-timesyncd.service; disabled)
    Active: active (running) since Fri 2019-07-19 10:34:23 UTC; 6s ago
      Docs: man:systemd-timesyncd.service(8)
  Main PID: 1487 (systemd-timesyn)
    Status: "Idle."
    CGroup: /system.slice/systemd-timesyncd.service
            └─1487 /lib/systemd/systemd-timesyncd

And the time and date can be checked using:

$ timedatectl
      Local time: Fri 2019-07-19 15:25:13 BST 
  Universal time: Fri 2019-07-19 14:25:13 UTC 
        RTC time: n/a 
       Time zone: Etc/UTC (UTC, +0000)
     NTP enabled: yes 
NTP synchronized: yes 
 RTC in local TZ: no 
      DST active: n/a

As you can see, the time zone was set incorrectly to Etc/UTC, so I had to find out how to set it. The man pages proved useful: https://manpages.debian.org/jessie/systemd/timedatectl.1.en.html

First, I listed the time zones:

$ timedatectl list-timezones

From the huge list that you get, I selected my time zone Europe/London and set the time zone like this:

$ sudo timedatectl set-timezone Europe/London

Then I checked the date and time again and everything was working:

$ timedatectl
       Local time: Fri 2019-07-19 15:48:13 BST   
   Universal time: Fri 2019-07-19 14:48:13 UTC
         RTC time: n/a
        Time zone: Europe/London (BST, +0100)
      NTP enabled: yes
 NTP synchronized: yes
  RTC in local TZ: no
       DST active: yes
  Last DST change: DST began at
                   Sun 2019-03-31 00:59:59 GMT
                   Sun 2019-03-31 02:00:00 BST
  Next DST change: DST ends (the clock jumps one hour backwards) at
                   Sun 2019-10-27 01:59:59 BST
                   Sun 2019-10-27 01:00:00 GMT

Upgrading the kernel

With the benefit of hindsight, I should probably have upgraded the kernel soon after installing the Debian Jessie image. However, it worked first time so all was well.

Download the new kernel files from: https://github.com/ewaldc/My-Book-Live/blob/master/kernel/precompiled/linux-4.19.33.tgz. On my main PC, I mounted the NFS share from my WD MBL and copied the file linux-4.19.33.tgz to the NFS share (located at /nfs).

Using SSH on to the WD MBL, copy all the files needed for the existing kernel into a new directory just in case it all goes horribly wrong.

cd boot
sudo mkdir 4.9.99
cd 4.9.99
sudo cp ../apollo3g.dtb .
sudo cp ../boot.scr .
sudo cp ../uImage .

Now unpack the archive file into a new directory.

cd /boot
sudo mkdir 4.19.33
sudo tar xf /nfs/linux-4.19.33.tgz
sudo mv boot/* .
sudo rm boot
ls
apollo3g.dtb  uImage_4.19.33+x

All that needs to be done now is to overwrite the old kernel image with the new one.

cd /boot
sudo cp 4.19.33/uImage_4.19.33+x .

I then rebooted and it all just worked. Great!

One of the main reasons for updating the kernel is to ensure that the server is reasonably secure. Ewald has also patched this kernel to improve throughput on the hard disk and the improvement is noticeable when transferring large amounts of data. Well worth doing.

Enable mDNS

The final thing I decided to do was allow SSH access using mDNS. This would allow me to access the MBL this command:

ssh andy@mbl.local

This is “a good thing” as I can still access the server if I decide to change the IP address and I also wanted to try this out having never set it up before.

I installed the following:

sudo apt-get install avahi-daemon avahi-discover libnss-mdns

Then created the file /etc/avahi/services/ssh.service with the contents:

<?xml version="1.0" standalone='no'?><!--*-nxml-*--> 
<!DOCTYPE service-group SYSTEM "avahi-service.dtd"> 
<service-group> 
  <name replace-wildcards="yes">%h</name> 
  <service> 
     <type>_ssh._tcp</type> 
     <port>22</port> 
  </service> 
</service-group>

I tried the SSH command and it just worked.

My WD MBL has been working as my backup server for about a month now and, hopefully, I shouldn’t have to touch it again for a few years.

Updating my WD My Book Live

I have a Western Digital My Book Live, WD MBL, with a single 2TB disk that I wanted to update as the kernel and OS were over 5 years old. My aim was to use this a backup for my music collection that is currently hosted on my other NAS server, a ZyXel NAS520 with 2 1TB HDDs using Raid 1. I also wanted to transcode my music collection into several different formats and host them on the MBL instead of just the FLAC that is on my NAS.

After some googling, I found the OpenWrt project https://openwrt.org/toh/wd/mybooklive and decided to out their image. The first problem was that I had to take the HDD out of the MBL case. This is not easy but this video really helped:

Once I had the HDD out, I hooked it up to my PC using a HDD docking station, much easier than opening up my desktop and wiring it up directly. I installed the latest OpenWrt installation image, 18.06.4, following the excellent instructions on the OpenWrt site. I then took the HDD out of the docking station and reconnected it to the MBL circuit board and powered it up. The image worked just fine but I had forgotten that OpenWrt sets everything up like a router and I didn’t want that. So I tried to work out how to configure OpenWrt to not be a router and after a bit reading around, I realised that OpenWrt is not very standard so I could waste a lot of time working out how to set up the NAS the way I wanted.

After some more googling, I found this thread on the WD community site https://community.wd.com/t/any-interests-in-kernel-4-0-on-my-book-live/60483/. Debbie Jessie is a lot like Kubuntu that I use on most of my PCs so there was a minimal learning curve. A lot of the thread was about how to install Debian Jessie from scratch but later on, there was a series of posts from Ewald who had created his own distro and made it available to download and install. I think to myself, ‘Great, I’ll be done in a couple of hours!’ Famous last words! Many days later, I had something working and the following is a summary of what I learnt.

The boot process

There are a few key things to understand about the boot process on the WD MBL.

1. U-Boot is installed in flash and cannot be changed without a lot of effort. This version of U-Boot can only boot onto ext2 or ext3 partitions and always uses /dev/sda1.
2. U-Boot will use a script called boot.scr on the root of /dev/sda1 if it is present. This script can run any U-Boot commands but in this case it is told which kernel image to use, the kernel device tree file and where the root partition is.
3. Once the kernel is running, it uses the /etc/fstab file to set up the partitions.

This is all basic stuff but you have to get everything right or nothing happens.

As I had previously installed OpenWrt, I had to re-partition the HDD to match what was expected by Ewald’s Debian Jessie distribution. Initially, I tried following the instructions here: https://github.com/ewaldc/My-Book-Live/tree/master/debian/debian%208%20(Jessie)#how-to-install-with-ext4-. If this had worked, I would have had a minimal boot partition using ext3 and the rest of the rootfs install on a much larger ext4 partition. However, I wasted an awful lot of time trying to make this work with no success.

I eventually opted for the ext3 only method described here: https://github.com/ewaldc/My-Book-Live/tree/master/debian/debian%208%20(Jessie)#how-to-install-a-combined-boot-and–with-ext2ext3. These instructions are fine but they rely on the disk being partitioned correctly which is not explained clearly. The following instructions show how I eventually made it work.

Installing the distribution

Once you know what you are doing, the installation is actually fairly straightforward and goes like this:

1. Remove the HHD from the WD MBL.
2. Attach the HDD to your PC.
3. Partition the HDD.
4. Create the file systems.
5. Unpack the Debian Jessie image.
6. Change the boot.scr file.
7. Modify the /etc/fstab file.
8. Attach the HDD to the WD MBL circuit board and verify that it boots and that you can connect using SSH.
9. Re-assemble the WD MBL.
10. Do all the basic tidying up that Ewald suggests.
11. Customise as required.

Partition the HDD

When I tried de-bricking the WD MBL just I case I messed up, I successfully recovered the MBL. I then tried to find out what partitions were created by the factory image. There were several problems with trying to read the partition info and fdisk couldn’t do it. I found out that gdisk worked and after fixing the errors that it found, I was able to read the partition info (see below).

 Partition Table: 
 gptNumber  Start   End     Size    File system     Name     Flags
 3          15.7MB  528MB   513MB   linux-swap(v1)  primary 
 1          528MB   2576MB  2048MB  ext3            primary  raid 
 2          2576MB  4624MB  2048MB  ext3            primary  raid 
 4          4624MB  1000GB  996GB   ext4            primary

This is what is expected by the default /etc/fstab file but I wanted a bigger root partition so that I wouldn’t run out of space. I used gdisk to create the following partition table.

Number  Start (sector)    End (sector)  Size       Code  Name
    1            2048        20973567   10.0 GiB    8300  OS
    2        20973568      3904931982   1.8 TiB     8300  Data
    3      3904933888      3907029134   1023.1 MiB  8200  Linux swap

Set up the file systems

This is simply:

sudo mkfs.ext3 /dev/sdh1
sudo mkfs.ext4 /dev/sdh2

(/dev/sdh was where the disk appeared on my system). Once the file systems were created, they were automatically mounted on /media/andy.

Unpacking the tarball

Again this was pretty simple. Just make sure that you unpack the tarball on the correct partition. I used df -h work out which partition to use as this shows the mount points.

cd /media/andy/d8710e6e-34e8-4e59-bb19-74de7238a97d/
sudo tar xf /home/andy/software/WDMyBookLive/DebianJessie8.11.tgz

Set the right boot.scr file

Replace the /boot/boot.scr file with this one: https://github.com/ewaldc/My-Book-Live/blob/master/uboot/boot_ext3_sda1/boot.scr. It contains the following U-Boot commands:

setenv md0_args 'setenv bootargs root=/dev/sda1 rw rootfstype=ext3 rootflags=data=ordered ipv6.disable=1'
setenv load_sata 'sata init; ext2load sata 1:1 ${kernel_addr_r} /boot/uImage; ext2load sata 1:1 ${fdt_addr_r} /boot/apollo3g.dtb'
setenv boot_sata 'run load_sata; run md0_args addtty; bootm ${kernel_addr_r} - ${fdt_addr_r}'
echo ==== Loading Linux kernel, Device tree, Root filesystem ====
run boot_sata

This file tells U-Boot which kernel to use, /boot/uImage, and passes a bunch of commands to the kernel, the most important of which is the path to the root FS, /dev/sda1.

Amend the /etc/fstab file

The /etc/fstab file in Ewalds Debian Jessie “distribution” looks like this:

/dev/sda1       /       ext3    defaults,noatime,nodiratime,barrier=1   0 0
/dev/sda3       swap    swap    defaults        0 0
/dev/sda4       /DataVolume     ext4    exec,rw,noatime,nodiratime,data=writeback,barrier=0,errors=remount-ro,suid      0 1
/DataVolume/cache       /CacheVolume    none defaults,bind 0 0
/DataVolume/shares      /shares         none defaults,bind 0 0
/DataVolume/shares      /nfs            none defaults,bind 0 0

The problem with this is that the swap and data partitions are not where the /etc/fstab file expects them to be but it was simple to fix. I simply edited the /etc/fstab file to change /dev/sda4 to /dev/sda2.

After that, I plugged the HDD into the WD MBL circuit board and tested the new system. It booted and I was able to SSH into the newly updated MBL. There was still a lot of work to do to clean up and get everything working and that will be in my next post.

Links

The following links might be useful.

• How to connect the MBL UART: http://mybookworld.wikidot.com/wd-mybook-live-uart
• Setup the MBL with a fresh Debian image: https://aaronredding.com/mybooklive-fresh-installation-debian-7/
• How to de-brick your WD MBL: https://community.wd.com/t/guide-how-to-unbrick-a-totally-dead-mbl/56658/545