Recently in howto Category

For a last month or so I used Barnes & Noble Nook Color as Android tablet. This post will try to explain what is wrong with Android tablets and now to fix this.

For a start, original Android distribution on Nook doesn't make it really Android tablet (no market). But there is easy solution since this device is supported by CynanogenMod 7 for Nook Color to get proper Android distribution. So, let's get started with hints for easier life with your new tablet...

adb doesn't see Nook Color

For a start, you won't see Nook in adb devices listing. This is easily fixed with proper device id in ~/.android/adb_usb.ini like this:

dpavlin@t61p:~/.android$ echo 0x2080 > ~/.android/adb_usb.ini && adb kill-server && adb devices
* daemon not running. starting it now on port 5037 *
* daemon started successfully *
List of devices attached 
2011120012345678        device

Nook Color specific application

There are lot of application which will look quite broken on your Nook when running Android 2.3. Some of them require audio input, GPS or camera and they generally won't work. There is also problem with application which just don't know about wired 1024*600 screen size and produce strange small screen centered on big display.

However, this is just a list of few extremely useful apps which you should install on Nook with CM7:

• Nook Color Tweaks - change speed of CPU, turn USB host more and more
• Nook Screen Recalibrate - having problems with screen sensitivity?
• Nook Key Editor allows you to map back and menu keys which are very useful on Android 2.3 to volume up/down which makes tablet much more useful.

Once you install all of this, you will be painfully aware that Android 2.3 just isn't OS which works well on tables. So, let's try to see what Google has in source for tablets...

Ice Cream Sandwich

To get most out of Android on tablet, you have to hop over to CM9/ICS Nightly Builds and install unstable builds on ICS on Nook Color. After initial install reboot into recovery, flash Google Applications from gapps-ics-4.0.3-sam-noinit.zip, turn off signature verification and install fix-bootanimation.zip and telephony-permission-fix.zip. Unfortunately browser application doesn't work for me well, and usually segfaults with:

F/libc    ( 2878): Fatal signal 11 (SIGSEGV) at 0x0000001f (code=1)
I/DEBUG   ( 1072): *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
I/DEBUG   ( 1072): Build fingerprint: 'bn/bn_encore/encore:2.3.4/IML74K/228551:user/release-keys'
I/DEBUG   ( 1072): pid: 2878, tid: 2895  >>> com.android.browser <<<

ICS on Nook currently doesn't have hardware acceleration (but older Androids also doesn't have it) and it freezes from time to time (so you will get quite used to holding power button for 8 seconds or more to turn it off), but ICS is really so much better experience on tablet that I would really recommend it.

Finally a note about hardware performance: Nook Color has 512Mb of RAM and Ti OMAP 3621 @ 800 MHz which makes it probably a bit slow for ICS. As o consequence I wouldn't really recommend any Android laptops which are slower than this for ICS.

As you might know by now, I started an informal podcast in Croatian about Free and Open Source software called Razgovori o slobodnom softveru. This blog post will try to summarize my experiences and serve as information for podcast guests on how to get the best possible audio quality.

Every podcast currently starts as google wave. While this isn't ideal, it does work for now. I would love some other service which would allow me to see information color-coded by users. Those notes are later used to write blog post with all links and information about episode.

First, let's get Skype out of way. Only reason why we aren't using free software for audio conversation is simple fact that everybody has Skype and there is great skype-call-recorder which allows us to get uncompressed audio recording with local audio on one channel and remove one on another which enables easy matching of audio levels for local and remote speakers.

While we are at audio levels, let me share a little secret: right after recording audio from Skype, I use Levelator to adjust audio levels. There is Linux version of it, but it doesn't seem to work for me on recent Debian. But, effect is so valuable to good quality podcast that I just use Windows version under wine.

As far as guest audio is concerned I have found following notes really important for my guests:

• use any microphone other than one on your laptop. Built in microphones catch typing sounds and all huss and buzz from machine itself. Even cheapest possible microphone will generate better results, just compare Ivan's audio from episode 0 (audio via built-in laptop microphone) versus episode 1 (which is cheap plastic external microphone)
• if you have headset with microphone, use it. Adjust your microphone level to be above your nose. That will prevent recording of air that you breath.
• When you need to take a pause while speaking, just do it! I edit whole podcast to remove bloopers and breading anyway, but it helps if you repeat last word of full sentence when you start over so it will fit in context (and intonation) of previous material.

If at all possible, it's great if guests can record local audio (only microphone sound, not full mix from conversation) so you can replace Skype's remote channel with better quality audio.

Much more useful information can be found in Skype for interviews by Doug Kaye and Paul Figgiani (guys behind Conversations Network which also implemented Levelator mentioned above), so if you need to record on Windows or Mac take a look there.

For editing audio I use audacity which is great audio editor, but I still have to learn all keyboard shortcuts because using mouse to edit audio is too slow and cumbersome if you want real flow of montage (and I did video montage back in 1990 on local TV station, so I know unnecessary mouse clicks when I see them). It would be nice if labels moved when I do cut of audio, because editing of audio leaves labels at old positions which isn't useful since audio moved. To give you some idea of how much effort is involved in editing audio (mostly removing silences, bloopers and breading): for last 35 minute podcast, it took 420 edits to get it into final state. And most of single day. It's about the same as creating good presentation: I often need 6 times more than presentation length to create it. For audio editing, you can count on 10-15 times ratio for editing.

Finally, I hop over to some creative commons audio site (to be honest, it was first one in google search for CC music), pick one of top tracks and extract intro and outro music from it to put it at beginning of podcast and end. Finally, audio introduction and web address information for end are recorded and put on top of music.

I have two screens, and 9 virtual dekstops on my machines. Which sometimes makes it tricy to remember on which virtual desktop is xterm connected to klin. Fortunatly, I have that information in window title, so something as simple as:

xdotool search --name klin windowactivate

Will do the right thing. However, my mouse pointer is not always over xterm on local machine, so keyboard shortcut in awesome window manager for this simple xdotool usage would be helpful. So, here it is:

-- XXX dpavlin: search windows and active them
awful.key({ modkey }, "F12",
  function () 
    awful.prompt.run({ prompt = "window: " },
      mypromptbox[mouse.screen].widget,
      function (find)
        awful.util.spawn("xdotool search --name "..find.." windowactivate", false)
      end)
  end),

I would love to be able to insert this snippet somehow dynamically instead of editing rc.lua by hand, so suggestions are welcomed.

I know that title is mouthful. But, I occasionally use this blog as place to dump interesting configuration settings and it helps me remember configuration which helps me to remember it and might be useful to lone surfers who stumbles upon this page.

This weekend our nice network admin upgraded VLAN setup to enables 1Gb/s network across whole private segment. That is clearly visible in green arrows on picture, especially if you compare it with last one. We still have a few slower links (100 Mb/s) which pass through NAT to Internet (red arrows), but we now have private segment on every server in library and system room in main building and each server has IP address in new 1 Gb/s segment (you can notice that by blue arrows which are loopback interface on same machine).

All is not well, however. I had to reconfigure my OpenVZ containers from point-to-point ip configuration using venet devices to ethernet bridge setup because multiple hops on OpenVZ machine produced havoc with connections from our private network to containers. It was very wired, I saw TCP retransmission requests, but first packet somehow managed to pass through. ICMP traffic worked (thanks to small packet sizes), but I can't honestly say why our new VLAN setup and/or OpenVZ had problem with it. Up to that point, I just had private IP addresses assigned to OpenVZ container using vzctl set --ipadd 10.60.0.12 and ping -R did show multiple public and private addresses, but it all somehow worked.

First, I had to make network bridge which will be new VLAN 60

root@koha-hw:~# cat /etc/network/interfaces

auto br60
iface br60 inet static
        bridge_ports eth1 veth212226
        bridge_fd 0
        address 10.60.0.10
        netmask 255.255.254.0
        gateway 10.60.0.1

Then, I removed old private IP addresses:

root@koha-hw:~# vzctl set 212226 --ipdel 10.60.0.12 --save

and add new veth device (my version of vzctl requires MAC addresses, so beware!):

root@koha-hw:~# vzctl set 212226 --netif_add veth1,00:12:34:56:78:9A,veth212226,00:12:34:56:78:9B,br60 --save

This create veth1 device inside container to which I can assign IP address:

koha:~# ifconfig veth1 10.60.0.12 netmask 255.255.254.0 up

I decided to put that in /etc/rc.local because /etc/network/interfaces is managed by OpenVZ.

Next, I needed to reconfigure my ssh VPN which was using point-to-point tunneling with tap into real ethernet tunnel:

root@brr:~# tail -10 /etc/network/interfaces 

# vpn lib -> home
auto tap0
iface tap0 inet static
        pre-up ssh -M -S /var/run/tap0 -v -f -i /root/.ssh/koha-hw-tun0 -w 0:0 -o Tunnel=ethernet -o ServerAliveInterval=5 -o ServerAliveCountMax=2 koha-hw.ffzg.hr true
        address 10.60.0.81
        netmask 255.255.0.0
        up iptables -t nat -A POSTROUTING -s 192.168.1.0/24 -o tap0 -j MASQUERADE
        post-down ssh -S /var/run/tap0 -O exit koha-hw.ffzg.hr

On remote side, force tunnel number and add new device to bridge:

root@koha-hw:~# head -1 /root/.ssh/authorized_keys 
tunnel="0",command="ifconfig tap0 up ; brctl addif br60 tap0" ssh-rsa AAAAB3Nza...

Finally, little cron job to start (and restart) VPN:

root@brr:~# cat /etc/cron.d/tap0 
SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
*/1     * * * * root    fping -q 10.60.0.10 || ( ifdown tap0 ; ifup tap0 )

Back in 2007 I wrote draft of blog post about semi-working solution for remote console on Dell's servers. I never finished it, but now I got a few more Dell servers and I'm not impressed. Linux support is even worse. Drek is shit in Croatian, btw.

First, update your RAC to latest firmware. It's a challenging task on Dell's web (I did it three times allready and still I can't remember procedure, so I'm documenting it here): search for drac 4 firmware (DRAC 4/I) and select newest version available (due to ranking sorting this can be hard task, select date sort). If you have DRAC 5, searching over Dell's site isn't much help. But, you can hop directly to ftp site and select file with name f_drac5*.exe.

Now download hard-drive format (it's self-extracting zip) and extract firmware:

dpavlin@t61p:~/Downloads/dell/2800$ unzip f_0603v17en_d4.exe 
Archive:  f_0603v17en_d4.exe
  inflating: firmimg.dm1             

dpavlin@t61p:~/Downloads/dell/2900$ unzip f_drac5v145_A00.exe 
Archive:  f_drac5v145_A00.exe
  inflating: firmimg.d5              

Upload firmimg.* via web interface (if you have really old version of RAC, you might need IE to do this, but recent versions from last year or two work also with Firefox. Firmware update doesn't work with Google Chrome). This is just the beginning of Internet Explorer dependencies, so get used to it.

Good idea is to ping your DRAC while uploading firmware. While flash is programming, you will get interruption in ICMP echo reply, but that's not a reason for worry. It will come back.

After all that, you still won't get Linux support. At least not with recent Firefox. Lousy work, Dell! How are you accessing your Dell servers from Linux?

If you have any objection your X11 setup (especially if you are not running Gnome or KDE) it's probably: Firefox menu font size is too big! However, that's really not right complaint. If you create your own userChrome.css and install it in correct location, you will notice that rest of applications still won't look good. But, Firefox was biggest problem, so you move on...

But, solution is really simple:

dpavlin@t61p:~$ cat .gtkrc-2.0 
gtk-font-name = "Verdana 7"

This will change all GTK application's (including FIrefox, OpenOffice.org and others) to sane font size for huge monitors with high DPI settings. It's a single line fix to all your problems!, take a look:

Your web site is slow. What do you do? Quick grep over access.log shows a flood of various crawling bots:

koha:~# grep bot /var/log/apache2/other_vhosts_access.log | tail
koha.ffzg.hr:80 71.181.32.73 - - [28/Oct/2009:14:16:04 +0100] "GET /cgi-bin/koha/opac-search.pl?q=su:in%C5%BEenjerstvo HTTP/1.1" 200 6579 "-" "Mozilla/5.0 (compatible; MJ12bot/v1.2.5; http://www.majestic12.co.uk/bot.php?+)"
koha.ffzg.hr:80 66.249.65.156 - - [28/Oct/2009:14:16:09 +0100] "GET /cgi-bin/koha/opac-detailprint.pl?biblionumber=127887 HTTP/1.1" 200 849 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
koha.ffzg.hr:80 66.249.65.156 - - [28/Oct/2009:14:16:14 +0100] "GET /cgi-bin/koha/opac-detail.pl?biblionumber=81565 HTTP/1.1" 200 4240 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
koha.ffzg.hr:80 66.249.65.156 - - [28/Oct/2009:14:16:19 +0100] "GET /cgi-bin/koha/opac-showmarc.pl?id=164101 HTTP/1.1" 200 1675 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
koha.ffzg.hr:80 66.249.65.156 - - [28/Oct/2009:14:16:24 +0100] "GET /cgi-bin/koha/opac-detail.pl?biblionumber=38700 HTTP/1.1" 200 4343 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
koha.ffzg.hr:80 71.181.32.73 - - [28/Oct/2009:14:16:28 +0100] "GET /cgi-bin/koha/opac-search.pl?q=su:iracionalno HTTP/1.1" 200 5140 "-" "Mozilla/5.0 (compatible; MJ12bot/v1.2.5; http://www.majestic12.co.uk/bot.php?+)"
koha.ffzg.hr:80 66.249.65.156 - - [28/Oct/2009:14:16:30 +0100] "GET /cgi-bin/koha/opac-ISBDdetail.pl?biblionumber=73120 HTTP/1.1" 200 3255 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
koha.ffzg.hr:80 66.249.65.156 - - [28/Oct/2009:14:16:35 +0100] "GET /cgi-bin/koha/opac-detail.pl?biblionumber=53576 HTTP/1.1" 200 4466 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
koha.ffzg.hr:80 66.249.65.156 - - [28/Oct/2009:14:16:40 +0100] "GET /cgi-bin/koha/opac-showmarc.pl?id=85958 HTTP/1.1" 200 1691 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
koha.ffzg.hr:80 66.249.65.156 - - [28/Oct/2009:14:16:45 +0100] "GET /cgi-bin/koha/opac-detail.pl?biblionumber=91684 HTTP/1.1" 200 4477 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

Every 5 seconds? This impacts our performance, especially during working hours. So, for a start let's teach bots which URLs are really interesting. Create robots.txt which all good crawlers should support:

koha:~# cat /var/www/robots.txt 
User-Agent: *
Crawl-Delay: 300
Disallow: /cgi-bin/koha/opac-search.pl
Disallow: /cgi-bin/koha/opac-showmarc.pl
Disallow: /cgi-bin/koha/opac-detailprint.pl
Disallow: /cgi-bin/koha/opac-ISBDdetail.pl

User-Agent: MJ12bot
Crawl-Delay: 300

...and configure it using rule like this (if you web root isn't /var/www):

koha:~# cat /etc/apache2/conf.d/robots.conf 
Alias /robots.txt /var/www/robots.txt

This would work, but Google bot doesn't use Crawl-Delay (sigh!). So we have to hop over to Google's webmaster tools - Site config - Settings and ask Google bot nicely to slow down:

However, that's not enough, because it takes a while for Google bot to update his crawl speed, and our site is heavily loaded right now! Let's try to limit bandwidth available to all crawlers (identified by bot in User-Agent header) using bandwidth limiting.

First, install mod-bw:

koha:/# apt-get install libapache2-mod-bw

koha:/# a2enmod bw
Enabling module bw.
Run '/etc/init.d/apache2 restart' to activate new configuration!

Edit your configuration to have something like following (very draconian) policy:

        BandwidthModule On
        ForceBandWidthModule On

        BandWidth "u:bot" 8192
        MaxConnection "u:bot" 1

        BandWidth "u:wget" 8192
        MaxConnection "u:wget" 1

It seems that you have to set BandWidth to value which is larger than BandWidthPacket to have effect (which is 8192 by default).
This will slow down download speed to all crawlers (including wget for testing) to 8 k/s. So, does this work?

koha:~# cat /var/log/apache2/other_vhosts_access.log | grep ' 503 ' | cut -d\" -f6 | sort | uniq -c
    107 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
     61 Mozilla/5.0 (compatible; MJ12bot/v1.2.5; http://www.majestic12.co.uk/bot.php?+)
     54 Mozilla/5.0 (compatible; MJ12bot/v1.3.1; http://www.majestic12.co.uk/bot.php?+)

This is just for first two hours of operation, so it does help.

I already stated several times that I really love ZFS file-system. Since I deployed it in production for backup and recovery of virtual machines, I had intuition how it works, but recent conversation on fuse mailing list turned out to have hidden gem in it: Keynote speech from Jeff Bonwick and Bill Moore at Kernel Conference Australia 2009. It's Solaris kernel conference (and has wrong title in program: Deduplication in ZFS), so I would never watch videos from it, but this one is well worth your time if you are interested in ZFS.

If you are wondering about ZFS performance, read VMWare + FreeBSD + ZFS soft-raid with SATA drives - performance. Yes, it's on FreeBSD which is great because, now I can compare performance (in terms of orders of magnitude) on different systems.

I have been using compression on my storage server for a while, and saw dramatic reduction in disk space usage, but didn't do any performance comparisons. On other hand, we have some idea about zfs-fuse performance on dial SSD so I will probably have to do some update to latest zfs-fuse from Ricardo Correia which recently moved zfs-fuse development to git.

I have been following Tokyo Cabinet for a while, and I was especially keen to try full-text indexes which where added recently. I'm actually so obsessed with it that I had Google alert set on words "Tokyo Cabinet" and apart from occasional political event in Tokyo's political cabinet, it was useful in finding interesting information when I got link to this blog post how to create simple intranet search by Mikio Hirabayashi which include easy to follow instructions to make local intranet search, complete with web crawler (in ruby).

Unfortunatly, Google translate isn't really kind to it and creates something which is not really usable. But, I managed to condense it to following script:

#!/bin/sh -x

url=http://blog.rot13.org/

test -f intra.tsv || ruby wgettsv -allow "$url.*html" -deny cgi -max 10000 $url > intra.tsv
tctmgr inform tctsearch.tct
tctmgr importtsv tctsearch.tct intra.tsv
tctmgr setindex -it qgram tctsearch.tct title
tctmgr setindex -it qgram tctsearch.tct body

...which is really awesome if you ask me. All the good things of Tokyo Cabinet with a little bit on qgram index on the top.

I will try to summarize blog post here to provide English speaking web public an opportunity to find out more if they can read Japanise Concept is really simple:

...and very nicely split into components. If you still didn't give Tokio Cabinet a try! If you want, you can take a look at my Tokyo Cabinet scripts as a starting point. I really need to make proper Debian packages for recent versions, so watch this space...

It seems that I wasn't the first one to have idea of sharing MySQL installation between OpenVZ containers. However, simple hardlink didn't work for me:

root@koha-hw:~# ln /vz/root/212052/var/run/mysqld/mysqld.sock \
     /vz/root/212056/var/run/mysqld/
ln: creating hard link `/vz/root/212056/var/run/mysqld/mysqld.sock' to
    `/vz/root/212052/var/run/mysqld/mysqld.sock': Invalid cross-device link

So I tried something a little bit different:

root@koha-hw:~# mount --bind /vz/root/212052/var/run/mysqld/ \
     /vz/root/212056/var/run/mysqld/

Which worked for me on old etch Debian kernel:

root@koha-hw:~# uname -a
Linux koha-hw 2.6.18-openvz-18-53.5d1-amd64 #1 SMP Sat Mar 22 23:58:13 MSK 2008 x86_64 GNU/Linux

I have wrong permissions, but socket is world-writable anyway... There is also related bug for kernels 2.6.24 and 2.6.26.