Few days ago I decided to run ipkg update ; ipkg upgrade on my moko, and this didn't end up quite well. For a start, I managed to fill root filesystem, and while I tried to remove almost everything that I didn't use, I ended up with non-working unit.

Since I'm following qemu-neo1973 tree and testing images in emulator, I decided to re-flash with latest image. This proved to be very bad since latest image doesn't have gsm drivers, so moko dies on splash screen making device unusable (ssh isn't started at that point so I couldn't connect to it -- If I only had development board...)

Most interesting (and scary) thing were error messages from jffs2 which led me to beleve that my device has developed additional bad blocks. So, I started reading wiki and dumped my bad-blocks table (dump from u-boot prompt):

GTA01Bv4 # nand bad

Device 0 bad blocks:

  00070000

  00ab0000

  00f00000

  03ff0000

  03ff4000

  03ff8000

  03ffc000

Last four entries are bad-block table itself, but my device really has bad block. I wasn't able to access additional fields which would tell me if they are factory bad of developed in use...

Since I suspected that my device is completely broken (selecting factory default option in u-boot didn't help) I decided to try re-flash with different image. I went to wiki, selected another u-boot/boot/root images and flash them using following commands described in wiki:

dfu-util -a u-boot -R -D u-boot-gta01bv4-r12_0_2632_0.bin
dfu-util -a kernel -R -D uImage-2.6.22.5-moko11+svnr3238-r8-neo1973.bin
dfu-util -a rootfs -R -D OpenMoko-scaredycat-openmoko-devel-image-glibc-ipk-P1-Snapshot-20071118-fic-gta01.rootfs.jffs2

After a looong wait (it's a USB 1.1 device) I must report that I'm actually amazed by amount of progress. New home screen, working browser and media player!

However, I still can't make any calls (this will take a bit more fiddling) and file dialogs are just unusable (too big and with too much options) and I still can't type on any screen orientation.

Google's announcement of Android platform provoked me to think about leaving OpenMoko on the shelf with other toys for which I don't have time any more, but when I saw new software upgrade I have to re-consider this.

1. OpenMoko is here: I have the device, it exists in hardware
2. qemu-neo1973 just got very good GSM emulator (you can select emulated GSM network, send messages and calls to emulated phone) which will probably bring GSM and SMS functionality to good shape real-soon-now(tm)

To replace my aging Nokia I need following:

1. working phone (voice calls) - almost there, but not quite :-)
2. SMS messaging - there is beginning of it in GUI, but no real functionality
3. keyboard on which I can type with my big fingers

This brings me back to my plans: since I can't develop anything for GPS until there is binary (sigh!) driver available I would probably want to write almost full-screen 3*4 keyboard just like the one on regular cell phones. This would allow me to actually use device with my fingers.

I probably won't have time to scratch this itch in current year, so if there is anything similar (T9 implementation or even something similar to xstroke which worked well for me on Zaurus) I would love to hear about it.

After nine months of playing with Thompson ADSL modems I have two projects which I wrote in perl, both of which is, as far as I know, first Open Source GPL implementation of those protocols.

MDAP

MDAP is protocol used by Thompson CPE devices to issue commands to CPEs (called ants) using multicast address 224.0.0.103 and port 3235 registered by IANA.

It's very cool idea, since you can connect as many devices as you have network ports or bandwidth, and allthough they all will boot with same IP address (and this create conflicts on IP network), you can still sand commands to each individual device using multicast.

Originally I developed it to flash multiple modems at once, but since I also added simple rules to change IP addresses and issue commands to devices. This essentially enables you to flash devices to some version of firmware and then change configuration a bit and have you test lab ready in few moments.

This project doesn't have a real project page yet, but you can take a look at source if you are interested...

This project also include (but doesn't use yet) simple perl BOOTP and TFTP servers, so in the end it will probably have perl-only solution for MDAP. If you just use included scripts and documentation for setup, it will use binary bootp and tftp server since this configuration was in use for at least half of the year, and I consider it stable.

CWMP

Perl CWMP server is essentially a low-level support for broken idea to communicate with devices using persistent connection SOAP with invalid XML (name spaces in some responses are just invalid) known as TR-069.

This is work-in-progress, and right now it's stable enough to work with multiple devices at once. In essence, it's protocol-violating SOAP server implementing persistent connection handling as described in TR-069 documentation (empty post, even without headers as first request, ehhh....).

Idea is to enable you, the user, to write perl rules against CPE devices.

It's half-way there: It does have disk-based command queue for each device (which is also NFS save, which is nice if you want to have multiple servers) and persistent storage for each CPE internal data tree implemented using DBM::Deep or YAML. When used with YAML, it great way to understand protocol. However, not all methods are implemented, but I hope to have full implementation by end of January 2008.

Several years after I wrote about software RAID1 and RAID5 I must say that I'm happily surprised how much software and hardware support for software RAID under Linux improved. My articles are more or less obsolete now, but this is short story of my RAID5 array for home.

I have Compaq brand-name desktop PC (refurbished, good buy) with one 160Gb SATA disk. I wanted to add three more 500Gb disks in RAID5 array to create ~1TB storage with redundancy.

Motherboard has just two SATA channels and form talking with a lot of people, it seems to me that SATA devices can't be chained (please correct me if I'm wrong). I somehow assumed that I can connect more than one disk on one SATA channel, partly from SCSI world, partly from old IDE (err, PATA) master/slave relationship. So, additional controller was in order, and only one which was available (with two SATA ports and in PCI-X variant) was no-name Sil 3132 based one:

20:00.0 RAID bus controller: Silicon Image, Inc. SiI 3132 Serial ATA Raid II Controller (rev 01)

I googled a bit and found some horror stories (including binary driver on Silicon Image's site for RHEL kernels), but latest Debian kernel 2.6.23-1-686 just worked. I wanted to use it just as SATA controller (with RAID part), so I was all set. It doesn't have RAID5 anyway.

Now, let's take a short de-tour and explain why do I want to have RAID5? Isn't RAID1 better/faster/right stuff for system?

I know only one use for RAID1: booting. And that's if you don't have hardware RAID5 which shows you RAID arrays as one disk anyway. If you want to boot Linux from software RAID5 you are out of luck.

But there is simply no reason to install system on hardware RAID1 as opposed to hardware RAID5. If you have enough disks (five for example) worst configuration is 2*RAID1 + 3*RAID5 since you get maximum of 2 disks accumulated performace. Much better performace (accumulated 4 disks) is just 5*RAID5.

Having said this rand for RAID5, I also didn't want to boot from it (since I have system on 160Gb disk), and I just need reliable and fast disk. Since I could fit just 4 disks in case, I will get 2 disks of accumulated performance and reliability (third disk).

Why not simple striping over three disks (hack, I would get 500Gb of storage more)?

I don't really believe in desktop-grade disks, (even SATA). Those disks are designed to have 8 hours/day life cycle and will probably die in two years. And I like my data...

Let's see some performance, first single disk:

root@brr:~# hdparm -tT /dev/sda
/dev/sda:
 Timing cached reads:   2066 MB in  2.00 seconds = 1033.49 MB/sec
 Timing buffered disk reads:  232 MB in  3.00 seconds =  77.22 MB/sec

and RAID5 array:

root@brr:~# hdparm -tT /dev/md0
/dev/md0:
 Timing cached reads:   2030 MB in  2.00 seconds = 1014.76 MB/sec
 Timing buffered disk reads:  444 MB in  3.01 seconds = 147.49 MB/sec

After several hours of uptime, they are not very hot (order is same as in case):

root@brr:~# uptime
 21:53:24 up  5:57,  1 user,  load average: 0.00, 0.00, 0.00
root@brr:~# echo -e 'c\nd\na\nb'  | xargs -i hddtemp /dev/sd{}
/dev/sdc: WDC WD1600JS-60MHB1: 46°C
/dev/sdd: WDC WD5000AAKS-00YGA0: 45°C
/dev/sda: WDC WD5000AAKS-00YGA0: 48°C
/dev/sdb: WDC WD5000AAKS-00YGA0: 44°C

But they do go up to 55°C under heavy load (specially unhappy sda which is always a bit warmer than other disks).

As you can also see, it moved my old 160Gb disk to sdc. Hack, I'm quite sure that I could force bios into re-ordering this (or turn off RAID BIOS on controller), but I just don't have time.

And lastly how does it look?

root@brr:~# vgdisplay 
  --- Volume group ---
  VG Name               raid5
  System ID             
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  5
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                1
  Open LV               1
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               931.52 GB
  PE Size               4.00 MB
  Total PE              238468
  Alloc PE / Size       76800 / 300.00 GB
  Free  PE / Size       161668 / 631.52 GB
  VG UUID               onzKEw-TaRF-JBe1-9YWN-D6aJ-FIuY-1VRmLo

I didn't wrote exact commands to perform creation of array, but madm --create --help was basically enough. And that I waited for two hours for RAID array to sync...

There is one more useful tit-bit: if you have ext3 filesystem formated with resize_inode option you can do on-line resize of ext3 filesystem with resize2fs. Debian already does this by default (since etch, I beleve), but you can check with:

root@brr:~# tune2fs -l /dev/raid5/rest | grep resize_inode
Filesystem features:      has_journal resize_inode dir_index filetype needs_recovery sparse_super large_file

So, if you installed LVM, you can on-line resize your logical volume while coping data to it. I did that while coping from old PATA and USB disks at same time and it works well... It does take much longer (and does more disk I/O) than resize_reiserfs, however.

And don't forget noatime for ext3!

I wanted to edit some video for my local LUG. However, this doesn't seem to be simple task under Linux.

I have some background since I worked on local TV station from 1991-1995 among other things in video editing. So I do know semi-professional and professional video equipment from those days (hack, I was earning good buck with it!) and my judgment is somewhat biased towards ease of use and practical problems which I encountered during editing of professionally shot video material (I don't nearly have so steady hand, it seems).

I have video materials recorded with JVC Everio GZ-MG575E (There doesn't seem to be any good official JVC page about it. Go figure that!) Reposted by lsusb as:

Bus 005 Device 004: ID 04f1:0008 Victor Company of Japan, Ltd GZ-MG30AA/MC500E Digital Video Camera

Files from internal read-only vfat 40Gb disk are available when camera is mounted as USB device on Linux as sd_video/prg001/mov*.mod. File on one of files report MPEG sequence, v2, program multiplex.

Let's look at available options:

1. Kino - Safe choice, right? Debian package, version 1.1.1.
   It can't import avi in original format. When I tried to convert it some other .avi format using ffmpeg it segfaulted on loading of that avi.
   It has somewhat home video look which I don't like.
2. cinelera - GUI hurtes my eyes and distracts too much from video footage. I tried. I really tried. It didn't work for me.
3. The Open Movie Editor can't be compiled from upstream repository. Looked somewhat promising up to that point...
4. Avidemux - to be honest, I found this one while Googling for video editing on FreeBSD software section video. Good catch, I had Debian package version 2.3.0 available from www.debian-multimedia.org.
   Very nice tool, it fits my brain rather well. Well, I would have to remap keys because it's just half-usable on notebook keyboard (no numeric keymap, allthough I might buy USB one just for this application :-)
5. pitivi able to import avi, but blocked after a few clicks and stopped refreshing screen. It's interesting to note that clips which where loaded into avidemux had little picture from movie, while the ones which weren't loaded didn't.
6. LiVES - last time I tried to find video editing software, LiVES turned out as only software which worked at all. Still, strange intraface, in part Gtk based, and in part just ugly doesn't sit well with me.
7. Kdenlive is another project which is well worth a look. Clean QT interface, all the options I would like, want or need.
   Just two problems: I don't hear any audio. I do get blink sound from KDE, but not from video sound. Another problem is related to by attempts to solve previous one: upstream Subversion repository source code doesn't pass configure on my machine. Yikes.
   Update: My problems with audio are solved after I insalled sdl-alsa variant (instead of old alsa-oss, my fault). It's quite capable software, my only remaining gripe is that every action has a click or two indirection until you can have it done. Other than that I have problems with audio-video sync when producing any format other than mpeg (which is same as my source format), but I don't care much about that, because I intend to do recoding as final step into multiple formats anyway...