Dobrica Pavlinušić's Weblog / Blog

For a last month or so I used Barnes & Noble Nook Color as Android tablet. This post will try to explain what is wrong with Android tablets and now to fix this.

For a start, original Android distribution on Nook doesn't make it really Android tablet (no market). But there is easy solution since this device is supported by CynanogenMod 7 for Nook Color to get proper Android distribution. So, let's get started with hints for easier life with your new tablet...

adb doesn't see Nook Color

For a start, you won't see Nook in adb devices listing. This is easily fixed with proper device id in ~/.android/adb_usb.ini like this:

dpavlin@t61p:~/.android$ echo 0x2080 > ~/.android/adb_usb.ini && adb kill-server && adb devices
* daemon not running. starting it now on port 5037 *
* daemon started successfully *
List of devices attached 
2011120012345678        device

Nook Color specific application

There are lot of application which will look quite broken on your Nook when running Android 2.3. Some of them require audio input, GPS or camera and they generally won't work. There is also problem with application which just don't know about wired 1024*600 screen size and produce strange small screen centered on big display.

However, this is just a list of few extremely useful apps which you should install on Nook with CM7:

• Nook Color Tweaks - change speed of CPU, turn USB host more and more
• Nook Screen Recalibrate - having problems with screen sensitivity?
• Nook Key Editor allows you to map back and menu keys which are very useful on Android 2.3 to volume up/down which makes tablet much more useful.

Ice Cream Sandwich

To get most out of Android on tablet, you have to hop over to CM9/ICS Nightly Builds and install unstable builds on ICS on Nook Color. After initial install reboot into recovery, flash Google Applications from gapps-ics-4.0.3-sam-noinit.zip, turn off signature verification and install fix-bootanimation.zip and telephony-permission-fix.zip. Unfortunately browser application doesn't work for me well, and usually segfaults with:

F/libc    ( 2878): Fatal signal 11 (SIGSEGV) at 0x0000001f (code=1)
I/DEBUG   ( 1072): *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
I/DEBUG   ( 1072): Build fingerprint: 'bn/bn_encore/encore:2.3.4/IML74K/228551:user/release-keys'
I/DEBUG   ( 1072): pid: 2878, tid: 2895  >>> com.android.browser <<<

ICS on Nook currently doesn't have hardware acceleration (but older Androids also doesn't have it) and it freezes from time to time (so you will get quite used to holding power button for 8 seconds or more to turn it off), but ICS is really so much better experience on tablet that I would really recommend it.

Finally a note about hardware performance: Nook Color has 512Mb of RAM and Ti OMAP 3621 @ 800 MHz which makes it probably a bit slow for ICS. As o consequence I wouldn't really recommend any Android laptops which are slower than this for ICS.

Each year, we get a treat right before end of the year in form of 28C3. This year, I can recommend following lectures as a must-see:

• r0ket++ video
  Very nice ARM Cortex M3 embedded device with Nokia LCD, battery, RF interface and extension headers.
• 802.11 Packets in Packets video
  Drop wifi packet on unsuspecting listeners without access to network segment!
• Defending mobile phones video
  Visit GSMmap.org to see if your GSM networks are protecting you or contribute data!
• Reverse Engineering USB Devices video
  Very similar to my approach to USB examinations.
• How governments have tried to block Tor video
  It's arms race all over again, this time on Internet.
• Reverse-engineering a Qualcomm baseband video
  I can't wait to see release of this tools on code.google.com Available at http://code.google.com/p/qcombbdbg/
• KinectFusion video
  I would love to see something similar in open source code.
• Cellular protocol stacks for Internet video
  Very good overview of cellular stack as it is today
• Implementation of MITM Attack on HDCP-Secured Links video
  NeTv is very interesting ARM/FPGA/Linux device which overlays webkit content over HDMI signal on your TV using chroma key.
• Introducing Osmo-GMR video
  Osmocom got some support for satellite phones
• bup: Git for backups video
  Nice backup system with de-duplication implemented on top of git's pack file

We had another gathering called Nothing will happen in Moravice, and this time we wanted to be prepared for it. So we got 2 WRT and 2 TP-Link devices with good faith that we will have stable Internet there. However, our telecom provider decided to screw up our order to increase bandwidth from 4Mbit/s to 10Mbit/s for event and instead decided to downgrade our access to 512Kbit/s. So we opted to create following simple network architecture which involved multiple hops to alternative 4Mbit/s upstream and batman-adv mesh setup.

batman-adv is quite cool layer-2 mash network which operates in ad-hoc mode and allows adaptive routing over mash and multiple upstream providers (at DHCP request or renew time, so we made our DHCP lease time down to 5 minutes). Joining network is quite simple:

iwconfig wlan0 mode ad-hoc essid nsnd-batman
ifconfig wlan0 mtu 1528
modprobe batman_adv
batctl if add wlan0

ifconfig wlan0 up
ifconfig bat0 up
dhclient bat0

My own experience with this experiment is very positive, mostly because we had around 5 clients in mash at any time, compared with overloaded WRTs which handled rest of 40 wifi enabled devices. But, I hope that during next event more people will upgrade to 2.6.33 or newer kernels, so we can have even better mash connectivity.

As you might know by now, I'm very happy user of Kindle 3 (keyboard) and Kindle DX graphite. One of reasons why I choose those devices was community around Kindle Developer's Corner at mobileread forum. With new generation of Amazon's Kindle device on the way, let me stress that for me older Kindle 3 devices are more interesting since we still don't have the way to run home-brew software on newer Kindle 4.

One of the good things about next generation of Kindles is that older models get sold second-hand at reasonable price, and let me re-iterate: for around 100€ older Kindle 3 with keyboard are great devices. But what would you do with one when you get your hands on? Start a k3libre project to develop and document free libre command-line tools for Kindle development. It's very much work in progress, with useful example scripts which describe framebuffer format, but also a handy step-by-step instruction on how to liberate your Kindle by installing jailbreak and usbnetwork to get root ssh access over usb cable. Next step is launchpad which listens on /dev/input/input? and allows you to bind execution of programs on key-presses. We are doing all this to install to nice full-screen Kindle terminal. You can do most of this work in under an hour, so there is no excuse not to read man pages on your Kindle!

So now that you can run your own software on Kindle, with which would you like to start? I would suggest kindlepdfviewer - a PDF viewer made for e-ink framebuffer devices, using muPDF. It's implemented in lua, so adding new features are rather easy, and latest development include serialization of state into SQLite database and drawing characters on screen! OK, it's in early stage of developments, but already useful on real Kindle, and if you don't like something you can always just edit lua code a bit :-)

Sometimes, you need to connect two networks in some way. My usual motivation is ability to access machines behind multiple NATs for easy system administration. So far, I used combination of OpenVPN and DynamicForward in ssh with clever use of ProxyCommand and nc with a sprinkle of proxy.pac for Firefox to make everything seemingly work. However, I never successfully managed to tunnel various JavaWebStart based remote consoles which want to connect directly from your machine to remote IP using sockets (for which you have to disable all proxy settings using jcontrol and selecting direct connection).

So this got me thinking. I could configure another OpenVPN for this, but it has many steps and I was lazy. Wouldn't it be great if there is some kind of P2P network like Skype or Hamachi for Linux? Something like this:

n2n: a Layer Two Peer-to-Peer VPN is exactly what I was looking for. It allows you to construct IP network over nodes behind NAT. But is it really easier to configure for specific example of accessing private network on another LAN behind NAT? Let's find out.

Steps are simple:

• Install n2n (you will have to do this on supernode and two nodes)

  all$ sudo apt-get install n2n
  

• Start super node on public address with DNS name super.example.com

  internet$ supernode -l 1234
  

• Start first client

  local# edge -c community -d community  -k secret \
    -l super.example.com:1234 -a 10.1.2.1
  

• Start remote end-point somewhere within LAN

  remote# edge -c community -d community  -k secret \
    -l super.example.com:1234 -a 10.1.2.2 -r
  

  Note changed IP address and -r flag which will allow us to route over this node.

  remote# sysctl -w net.ipv4.ip_forward=1
  remote# iptables -t nat -A POSTROUTING -s 10.1.2.1 -o tun0 -j MASQUERADE
  

  This will turn forwarding and NAT for our packets coming from community tap interface and going out through tun0 to LAN. We also need to setup route on local side for remote LAN network:

  local# ip route add 172.18.0.0/16 via 10.1.2.2
  

Last few weeks, I was configuring huge ZFS pool of 50 disks over three machines. Aside from benchmarking, I wanted to setup monitoring of this disk pool. smartctl comes as natural candidate for getting smart data, but where should I keep it? I recently learned of git log -p output format which shows nicely changes in your source files, so natural question was can I use git to track smart disk statistics?

As it turns out, getting overview of disk layout is really easy under Linux if you know where to look. /proc/partitions first comes to mind, but it lacks one really important peace of information: disk serial number. It's only peace of information which won't change between reboots when you have to spin up 30+ disks, so you really want to use it as identification for disks, instead of device name for example (which I tried on first try and learned that disks move around).

Good naming of dump files is as important as always. In the end, I opted to use smart.id where id part is from /dev/disk/by-id/scsi-something. Paths in /dev/disk/by-id/ are essential useful when creating storage pools because they also don't change between reboots.

Now that we know where to look for disk identification and serial number, we are ready to start collecting smart data. However, this data is much more useful if coupled with info from controllers, so final version of smart-dump.sh script also supports dumping of controller status for LSI Logic / Symbios Logic and 3ware controllers. Have in mind that collecting smart info from disks does interrupt data transfers, so if you have huge pool you might want to spread those requests (or even issue them in parallel if you want one huge interruption as opposed to several smaller ones).

So was all this worth an effort? In fact, it was! In our sample of 50 3T disks, one disk reported errors after just 192 hours of lifetime. It would probably report it earlier, but this was second time that I run smartctl -t long on it. On the other side, it passed long check on first test which was 8 hours of LifeTime. Even if you read Failure Trends in a Large Disk Drive Population paper from Google, and concluded that smart is lying to you and you could ignore it, please monitor your drives!

I have been using ZFS on Linux for some time to provide backup appliance using zfs-fuse. Since then, we got native ZFS implementation on Linux, so I decided to move by backup pool from zfs-fuse to in-kernel ZFS.

Additional reason to move pool over to new machine was to change pool's RAID level. In current ZFS implementation(s) you can't change mirror to RAIDZ1 without re-creating pool and then transfering data over using zfs send and zfs receive. However, when you are creating snapshots for years, and expiring them using script you will have hundreds of snapshots which you need to transfer.

This is where zfs-pool-replicate.pl script comes handy. It uses Net::OpenSSH to connect to two machines (source and destination), list all snapshots on source and transfer them to destination. If you have filesystem without snapshots it will create one @send snapshot which will be transferred. It will also optionally use compression for transfer of snapshot over the network. I am using LZO which is fast compression which nicely transfers 150Mb/s or more over normal 1Gbit/s network without much CPU overheard (and we all have multi-core machines anyway, right?). Current implementation allows you to re-run replication script to transfer only new snapshots creating handy disaster recovery solution.

Current implementation is designed to run from third (management) machine, so I can envision central storage administration tool which will also allow you to transfer LVM snapshots into ZFS snapshots. For now, I'm using shell script for that, but rewriting it in perl would improve error recovery and reporting.

By default, MySQL installation on Debian comes without innodb_file_per_table option which spread tables in individual InnoDB files. Based on your usage patterns or backup strategies this might be better filesystem organization than one big /var/lib/mysql/ibdata1 file. I first heard about it in OurSQL Episode 36: It's Not Our (De)fault!. It's great podcast, but to be honest with each new episode I wish to have only PostgreSQL servers to maintain...

To enable this option you will need to create configuration file and restart MySQL server:

koha:/etc/mysql/conf.d# cat > file-per-table.cnf 
[mysqld]
innodb_file_per_table
CTRL+D
koha:/etc/mysql/conf.d# /etc/init.d/mysql restart

This won't change anything, because only new tables will be created in separate files. But, we can use ALTER TABLE table ENGINE=InnoDB on each table to force InnoDB to re-read tables and create separate files:

mysqlshow koha --status | grep InnoDB | cut -d'|' -f2 | sed -e 's/^/alter table/' -e 's/$/ engine=InnoDB;/' | mysql -v koha

If you replace grep InnoDB with grep MyISAM you might use same snippet to convert MyISAM tables into InnoDB (if you still have them or don't use fulltext search).

I think that system administration is like gardening. I don't know anything about gardening, but it seems to involve a lot of care here and there, seemingly without much pattern. In that sense, it's similar to wiki editing, you start somewhere and you really don't know where it lead you to.

You have to start reading by singing lady Ga-Ga with words: S, s, s, ss... SAML, SMAL2! It will help, really.

SAML 2 is latest in long line of different SSO implementation you will have to do sooner or later if you want to be part of larger web. Google and others seems to be using it, so it must be good, right?

It has two end-points: identity provider (IdP) which has user accounts and Service Provider (SP) which is usually your application. But of course, it's more complicated than that. For a start, you will need https on your host. I will assume that you already have domain, and you can get free SSL certificates at StartSSL so hop over there if you need one.

First, install SimpleSAMLphp. It's pimpliest possible way to get working SAML2 implementation of IdP and SP. You will want to follow first simpleSAMLphp Installation and Configuration and then SimpleSAMLphp Identity Provider QuickStart to configure simple IdP with static accounts so you can test your application against it. You will need both IdP and SP under your control to do development. It will also help if your remote IdP (identity provider which you intend to use) is also simpleSAMLphp (as AAI@EduHr is).

Installation is rather easy:

dpavlin@lib:/srv$ sudo apt-get install install memcached php5-memcache

dpavlin@lib:/srv$ wget http://simplesamlphp.googlecode.com/files/simplesamlphp-1.8.0.tar.gz

dpavlin@lib:/srv$ tar xf simplesamlphp-1.8.0.tar.gz
dpavlin@lib:/srv$ cd simplesamlphp-1.8.0/
dpavlin@lib:/srv/simplesamlphp-1.8.0$ cp config-templates/* config/
dpavlin@lib:/srv/simplesamlphp-1.8.0$ vi config/config.php

• auth.adminpassword
• secretsalt
• enable.authmemcookie

dpavlin@lib:/srv/simplesamlphp-1.8.0$ php5 -l config/config.php 
No syntax errors detected in config/config.php

To configure Apache side, you need Auth MemCookie but it isn't available in Debian package, so I opted for Apache::Auth::AuthMemCookie so I can flexibly modify IdP response before passing it on as environment variables.

dpavlin@lib:~$ cat /etc/apache2/conf.d/authmemcookie.conf 
Alias /simplesaml /srv/simplesamlphp-1.8.0/www
perlModule Apache::Auth::AuthMemCookie
<Location /cgi-bin>
        # get redirected here when not authorised
        ErrorDocument 401 "/simplesaml/authmemcookie.php"
        PerlAuthenHandler Apache::Auth::AuthMemCookie::authen_handler
        PerlSetVar AuthMemCookie "AuthMemCookie"
        PerlSetVar AuthMemServers "127.0.0.1:11211"
        PerlSetVar AuthMemDebug 1
        PerlSetVar AuthMemAttrsInHeaders 0
        AuthType Cookie
        AuthName "Koha SAML"
        Require valid-user
</Location>

To test it, easiest method is to create account in Feide OpenIdP and test against it. After all, it easiest to start with same implementation of SAML2 on both sides just to prevent following scenario:

On perl side I first tried Net::SAML2 and found out that it doesn't handle IdP without adding HTTP-Artifact support to the IdP. However, even after that I wasn't managed to make it work with simpleSAMLphp IdP implementation mostly because of my inpatience with SSL configuration of it.

On the bright side, for first test I didn't need to modify Koha (Library management software which I'm configuration SAML2 for) at all because it already has support for HTTP authorization.

As some of you already know, almost a moth ago we placed order for our first Kindle (3G with wifi). Week after it arrived, we also ordered bigger DXG to complement our reading habits. So why do we have two Kindles and do we have regrets with them? Well, no! This post will try to summarize what I learned since...

For a start, both devices have ARMv6, but K3G has 256 Mb of RAM while DXG has only 128 Mb. Both have 4Gb of flash of which about 3.5Gb is available for user content. Both have working 3G connectivity in Croatia. wpa_supplicant on Kindle won't allow connections to "enterprise" networks (like EduRoam). Smaller K3G is just too small for reading pdf files. Bigger DXG has e-paper in A5 size, so it's much more useful, but comes with older 2.5.8 firmware which has terrible browser (Netfront as opposed to WebKit in 3.1) and pdf reading without contrast settings (which is very, very useful).

But, I was fortunate enough to find mobileread forum post on migration of 3.1 software from K3G to DXG by Yifan Lu to whom I'm eternally grateful for much needed software upgrade.

However, even if you don't want to upgrade Amazon's own firmware, there is Duokan alternative reader software for Kindle. It has superb support for two column pdf files which allows you to just press next page key and read column on the left (from top to bottom) and then on the right (from top to bottom). It has also various other layouts for reading, and it makes smaller Kindle almost useful for pdf reading (and reading on bigger one a joy).

But, sooner or later, you will come across pdf which would really benefit from manual cropping. Take a look at briss which is cross-platform application for cropping PDF files.

While we are at software, you should really look at mobileforum's Kindle hacks thread which documents latest jailbreak (which will give you root permissions on Kindle), usbnetwork (to connect over USB cable) and various fonts and screensaver hacks.

Development for kindle can be done using Java2ME (remember that from cell phones?). However, Amazon doesn't actually seem to give out it's Kindle development kit but don't despair: Andrew de Quincey figured out a way to develop kindlets without the KDK. With KDK API from Amazon and JSR217 specification for Java2ME you can be in your marry Java development in no time.

I would like to see something like OpenInkport running on Kindle. This would allow free software developers to take full power from this nice e-paper device. If only I didn't have so much books to read...