Let me try to explain my mail system: I have multiple mailboxes on IMAP server (filtered by procmail) which are synced to my laptop using offlineimap on which I read them using mutt.
This configuration has served very well, in fact so well that I have been using it for last 5 years every 45 minutes while my laptop was on. My python-fu skills are very low, I just can't read language, sorry... I tried to learn it several times, and it's just somehow not readable to me without much effort. I don't really know why. Maybe my brain was forever influenced by begin and end in Pascal (which later turned out to be curly braces in other languages), but indentation is not working for me. But, offlineimap worked beside the fact that I promised myself to get rid of all critical python code after every mailman upgrade :-)
It had few hiccups, mostly from full disk, but cleaning some space always turned out to be solution. It worked so well that I recommended it to one very unsatisfied Evolution+IMAP user... Until today, that is...
Today I run out of disk space. I think. Then I cleaned some space. And got disk full message again. Strange? In fact no. I have configured news servers sometime in last century and I still do remember i-nodes. I use ext3 on this partition by the way.
Quick df -i reviled that I have, indeed, run out of i-nodes (sorry, Jakov, you weren't quick enough to see df output with 0 i-nodes :-)
Did you know that make clean in compiled Linux kernel tree can give you 26000+ i-nodes? I didn't. Now I know. However, couple of moments later I discovered that I have again run out of i-nodes again.
Nobody around (and I was surround by at least 5 very knowledgeable Linux users -- which is a benefit of being at Linux track of conference while running out of i-nodes :-) didn't know how to find a huge pile of small files which where eating my i-nodes.
Lucy guess revealed that there is something strange with my INBOX/tmp directory. ls just hung in D state (blocked on I/O) and find . (which should display files because it doesn't try to read them all in as ls does to have them sorted) just blocked in D state also.
So, I decided to do rm -Rf tmp. It has been running for more than two hours now. I got 700000+ of i-nodes back (probably even more by the time I finish this post).
Here are few lines from offlineimap log:
OSError: [Errno 28] No space left on device: '/home/dpavlin/Mail/INBOX/tmp/1192813332_0.6409.llin'Last 50 debug messages logged for Copy message 319050 from INBOX prior to exception:
maildir: savemessage: using temporary name 1192813326_9.6409.llin
maildir: savemessage: using temporary name 1192813326_10.6409.llin
I don't really know from which run this log is. From one which failed, for sure. My maildir knowledge disappeared also sometime in last century when i replaced last qmail installation with postfix.
So, lazywab, my question is: What is tmp directory in maildir? Is it safe to just re-create it and let offlineimap re-sync my folders?
My INBOX has 6778 mails and I wouldn't want to loose them. I will make backup on server, but re-syncing them would be (slow, ADSL) pain which I would like to avoid. offlineimap has two-way sync, so my worries are not without reason.
Update: The correct procedure would be to erase correct directory in ~/.offlineimap and re-sync mailbox meta-data (which again took allmost 24 hours).
I still can't quite explain where did 2500000+ i-nodes went, but in the process new version of offlineimap went into Debian unstable so this might have fixed problem also. It working now, so life is good again...