USB-boot on bubba 3

[dturpin]

Hi,

looking into buying a NAS and came across bubba. However one of my requirement is to be able to boot from flash (usb/sd(compact flash/flash-chip is ok) since I'm very (very) picky about noise. This way the harddrive(s) can be off when not using the NAS, making it completely silent.

Is it possible to boot your sw (our vanilla debian is also ok) on bubba 3 from USB? I know that you build on the same architecture as the sheevaplug for example so as I see it, it should be technically possible anyway. Do one have access to some serial port so I can configure uboot? From what I have seen you don't have a SD-reader as the sheevaplug, right?

[Ubi]

I'd say the only way of fixing that is by plugging the HD in through an external USB hub and buying a bubba with an SSD.

[dturpin]

Ok that sounds like overkill (or at least overbudget =) ) for my interest. Is there possible to configure uboot in any way on the bubba? Could a serial port be modded onto the device? I don't mind some soldering

[pa]

If you don't mind a bit of soldering then use this howto:
http://wiki.excito.org/wiki/index.php/S ... cess_on_B3

/PA

[dturpin]

cool, thanks! I'll try that

[dturpin]

Ok took a bit longer than waited, here is whats happened anyway:

What I wanted was a totally quieted (I'm very picky when coming to noise) file server, at least when not in use. Still wanted plenty of space without paying a ton of money (ie no ssd) and I didn't want to add an external drive either. So my plan was to bootup a b3 on usb instead of mechanical hdd.


1. Step one was to disassemble a b3 and trying to figure out how it booted.


After checking some datasheets for the marvell CPU
http://www.marvell.com/embedded-processors/kirkwood/assets/FS_88F6180_9x_6281_OpenSource.pdf
http://www.marvell.com/embedded-processors/kirkwood/assets/HW_88F6281_OpenSource.pdf
and checking pins sampled during startup I could see that b3 boots up on a serial flash on the board. Not suprising since I already spotted the flash

2. To be sure not to destroy anything the next step was to dump the SPI-flash. I know I could have done that in Linux, but where would the fun be with that, so I used a buspirate. Also, by dumping the flash by external hardware I knew I could restore it the same way. If I did something wrong Linux would definitely not bootup.

I lifted Vcc from its pad and soldered a couple of wires to the chip so I could attach the buspirate. Some pins I could access with the small clips on my testwire directly.

3. The first block contains the bootheader, so I did a small program (see end of post for code) to decode that and check the CRC. To see that I could trust the dump.

Code: Select all

Decoding file...
Boot from: SPI(0x5A)
ECC algorithm (valid for nand only):default, based on pagesize
NAND page size (valid for nand only): 0
block size to load into RAM: 210104(205kB)
image offset: 512
dest address in DDR: 134217728
execution address on DDR or SPI (incase of direct boot): 134217728
An extra header of 480bytes is appended directly after the main header
checksum correct


4. Ok so the dumped looked ok and I should be ok if I messed anything up. Next step was to solder an UART connection on the board. As you can see above post you can find the UART on a couple of testpoints on the board, I also found it in a couple of pinheader holes and the port on the back (for production test maybe??). I used a pinheader so I could deattach the cable if I wanted.


5. Ok ready to go. Booting into uboot and changing the configuration to boot on usb instead of SATA. I kept the SATA boot as backup. These was the environment variables that I changed:
setenv bootalt2 'run sataboot || reset'
setenv bootalt3 'run usbinstall || run usbflash || run sataboot || reset'
setenv bootalt1 'run usbroot || reset'
setenv setusbargs 'setenv bootargs root=/dev/sdb1 console=$console,$baudrate serial=${serial#} key=$key button=$button rootdelay=5'
setenv usbroot 'usb start; run setusbargs; ext2load $usbinstalldev $loadaddr /boot/$bootfile; bootm'
setenv bootcmd

The bootalt3 will never be used (without recompiling uboot), its just the old bootalt2 kept as a backup. For some reason I got very strange results with this. I had to save it twice, with a restart in between. Otherwise bootcmd was invalid (had traces of other variables in it). Also note the rootdelay to allow linux to found the usb memory.

6. Ok I need a usb stick with correct rfs and kernel. I kept it simpel and just copied my current RFS/kernel and changing root in fstab. Also played with the time options to allow spindown. Found a physically small usb-stick of 16Gb (to allow some wear levelling).


7. Configure the HDD to spin down when not used with hddparm and relax

Code: Select all

root@bubba:/home/thommy# mount
/dev/sdb1 on / type ext3 (rw,noatime)
tmpfs on /lib/init/rw type tmpfs (rw,nosuid,mode=0755)
proc on /proc type proc (rw,noexec,nosuid,nodev)
sysfs on /sys type sysfs (rw,noexec,nosuid,nodev)
udev on /dev type tmpfs (rw,mode=0755)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=620)
/dev/mapper/bubba-storage on /home type ext3 (rw,relatime)
usbfs on /proc/bus/usb type usbfs (rw)
root@bubba:/home/thommy#
root@bubba:/home/thommy# hdparm -C /dev/sda

/dev/sda:
 drive state is:  standby


Code for checking bootblock in flash:

Code: Select all

#include <stdio.h>




int main(int argc, char **argv)
{
   FILE *f = NULL;
   char unsigned buffer[32];
   unsigned short *shortp;
   unsigned long *longp;
   unsigned char calc_checksum=0;
   int i;

   if(argc<2){
     printf("usage: %s image\r\n",argv[0]);
     return 0;
   }

   f = fopen(argv[1],"r");

   if(!f){
     perror("unable to open file: ");
     return 0;
   }


   if(32 != fread(buffer,1,32,f)){
     perror("unable to read from file: ");
     return 0;
   }


   printf("Decoding file...\r\n");

   printf("Boot from: ");
   switch (buffer[0])
   {
     case 0x5A:
       printf("SPI(0x5A)\r\n");
       break;
     case 0x8B:
       printf("NAND(0x8B)\r\n");
       break;
     case 0x78:
       printf("SATA(0x78)\r\n");
       break;
     case 0x9C:
       printf("PEX(0x9C)\r\n");
       break;
     case 0x69:
       printf("UART0(0x69)\r\n");
       break;
     default:
       printf("unknown %u\r\n",buffer[0]);
       break;
   }

   printf("ECC algorithm (valid for nand only):");
   switch (buffer[1])
   {
      case 0x0:
        printf("default, based on pagesize\r\n");
        break;
      case 0x1:
        printf("force hamming\r\n");
        break;
      case 0x2:
        printf("force RS\r\n");
        break;
      case 0x3:
       printf("disabled\r\n");
       break;
      default:
       printf("unknown\r\n");
       break;
   }
 
  shortp = (short*)&buffer[2];
  printf("NAND page size (valid for nand only): %d\r\n",*shortp);

  longp = (long*)&buffer[4];
  printf("block size to load into RAM: %lu(%lukB)\r\n",*longp,*longp/1024);
 
  if(buffer[0x8]!=0 || buffer[0x9]!=0 || buffer[0xa]!=0 || buffer[0xb]!=0)
    printf("warning, reserved fields not zero\r\n");

  longp = (long*)&buffer[0xC];
  printf("image offset: %lu\r\n",*longp);

  longp = (long*)&buffer[0x10];
  printf("dest address in DDR: %lu\r\n",*longp);
  if(*longp == 0xFFFFFFFF){
    printf("dest address is 0xFFFFFFFF, image booted directly from media\r\n");
  }

  longp = (long*)&buffer[0x14];
  printf("execution address on DDR or SPI (incase of direct boot): %lu\r\n",*longp);
 
  if(buffer[0x18]!=0 || buffer[0x19]!=0 || buffer[0x1a]!=0 || buffer[0x1b]!=0
   || buffer[0x1c]!=0 || buffer[0x1d]!=0 )
    printf("warning, reserved fields not zero\r\n");

  switch(buffer[0x1E])
  {
    case 0x0:
     printf("No extra header\r\n");
     break;
    case 0x1:
     printf("An extra header of 480bytes is appended directly after the main header\r\n");
     break;
    default:
     printf("unknown value in header extension field\r\n");
     break;
  }

  for(i=0;i<0x1F;i++){
    calc_checksum += buffer[i];
  }
 
  if(calc_checksum == buffer[0x1F]){
    printf("checksum correct\r\n");
  }else{
    printf("warning: checksum seems to be incorrect. Expected 0x%02X was 0x%02X\r\n",buffer[0x1F],calc_checksum);
  }
}


[Ubi]

This is outstandingly cool. How much wattage does the machine draw when you take the HD out?

[dturpin]

I havent any good power meter that's reliable when dealing with low power numbers. But measuring the current after the ACDC-converter with my multimeter I get:
hdd on 0.8A
hdd in standby 0.5A
Both the measurements where done at an idling system. So that times 12 and with some efficiency factor of the converter would equal the power.

I also tested performance. The system takes noticeable longer to boot, but that doesn't matter for me since I never turn it off. Here are the seq read times from hddparm

Code: Select all

hdparm -Tt /dev/sda
/dev/sda:
 Timing cached reads:   416 MB in  2.01 seconds = 207.35 MB/sec
 Timing buffered disk reads: 320 MB in  3.01 seconds = 106.39 MB/sec

hdparm -Tt /dev/sdb
/dev/sdb:
 Timing cached reads:   398 MB in  2.00 seconds = 198.58 MB/sec
 Timing buffered disk reads:  46 MB in  3.06 seconds =  15.02 MB/sec


As you can see the normal reads are quite a bit slower (usb has ~14% of hdd speed) but the cached reads are almost the same. On the other hand trying random access USB beats the HDD. Using http://www.linuxinsight.com/how_fast_is_your_disk.html to test speed I get:

Code: Select all

./seeker /dev/sda
Seeker v2.0, 2007-01-15, http://www.linuxinsight.com/how_fast_is_your_disk.html
Benchmarking /dev/sda [1907729MB], wait 30 seconds..............................
Results: 56 seeks/second, 17.55 ms random access time

./seeker /dev/sdb
Seeker v2.0, 2007-01-15, http://www.linuxinsight.com/how_fast_is_your_disk.html
Benchmarking /dev/sdb [15267MB], wait 30 seconds..............................
Results: 805 seeks/second, 1.24 ms random access time


So USB is around 14times quicker when it comes to random access times. What you loose at non-cached seq. read you gain at random access times.

In practice I haven't noticed any change in time/speed for my normal services, except startup time then.


If one wants to try this its quite easy to change the uboot enviroment from within Linux, but I would still recommend to count on adding a UART. Its very easy to screw something up the first time (I know I did ) and if you changed the uboot variables from within Linux you might not be able to bootup

[johannes]

Really impressive!

[dturpin]

Really impressive!

Thanks

[nielsek]

This is awsome
I just copied the rfs and kernel too and mounted the lvm on /home/storage, so the disk doesn't spin up when bash history is updated and stuff like that.

The silence is great, thank you dturpin!

[matze]

Hello dturpin,
thats, what i am looking for....
But my problem is that i do not know too much about the B3.
Is it possible to send me an image or a zip containing the stuff i have to copy to an USB-Stick, so the HDD is spinning down. I use the B3 with the squeezebox-server to hear music. but not the hole day. I hate it, if the HDD is spinning the most of time i do not use it.
Could you send me such a zip-file?
Thanks very much.

[dturpin]

Hi matze,

The files that are put on the usb-stick are just the same files that are on your b3-harddrive, except that I changed the fstab. I could probably zip that up (using the default files, so all your settings would be gone) for you, but your bubba wouldn't start on the stick anyway. For that to happen you would need to change the uboot parameters, that has to be done in uboot like I did or it should also be possible in Linux. But since you say that you dont know that much about these things I would recommend you to read up on it first. There is a big risk of bricking your device if you don't know what you are doing. Thats the same reason for why I dont feel that it is a good idea that I provide a script for this, there is too big of a risk of bricking your device.

But it isnt any rocket science, there is plent of documentation of it on the internet. You can for example have a look at the sheevaplug examples, it is pretty much the same hardware