dropping duplicate email in evolution

I recently had to reload all my mail from gmail again (I have an always-on computer that runs postfix and stores the email in maildir format, fetchmail gets the email from gmail and delivers locally, I then check my mail at that box regularly).

I had to reload because I decided to run a script that I have that removes duplicate emails from mboxes (evolution uses mbox, unfortunately). Due to a brain fart and Ctrl-C, and because the script isn't very smart, I lost some mboxes. It was simplest to just reload everything from gmail, even if it takes three days or so to get everything from gmail.

My scripts used formail and wrote temporary files which, once complete, were written back to the main mbox, with duplicates removed.

I'm switching to a safer alternative. In Evolution, I've created a filter thus:

Filter action is Pipe to Program
The program is /usr/bin/formail -D 9999999 /home/tiger/.evolution/dup-idcache
Action is move to some folder (e.g., ZZZZDups)

I've tested it with a mailbox that was horrendously bloated due to duplicate emails, and it found the dups, kept one email in the original mailbox and moved the duplicates to the designated dup folder). I double checked the emails (well, sampled, since there were too many) and ever dup found really was a dup (same email was in the original mailbox).

This is a much safer way to find dups, although I'll need to Ctrl-E the Inbox regularly, it's not that efficient (runs formail for every incoming email) and evolution doesn't much like the program specification (when I open filters, it shows an error with the filter with spaces converted to %20, etc., so URL-encoded). It's very convenient though and I'll use this until I can figure out a better way (probably hack up a script that hides the parameters and returns what formail returns).

The same technique works pretty well with bogofilter. I'd wondered how to integrate bogofilter into evolution. I'm OK with just finding spam and running bogofilter -s on it directly, so I don't need any fancy interface. For now (well, for my first test run earlier), bogofilter successfully found spam and moved them to a designated folder. I'll see if it catches more spam in the future.

I don't really get a lot of spam anymore. Gmail catches most of what goes there, and the mailserver at work has reasonably good spam filters. But sometimes some spam does get through. I'll see if bogofilter does a good job with those.

Wrong place, and how to transfer

Alright, The link to bacon wrapped inedible crepe was supposed to be on my other me. I'm going to waste some time trying to see if there's a quick way to move it over without having to copy and paste. And if there isn't, well, it may or may not move over, depending on how lazy I feel just then :-).

No Google Browser Sync

I used to use Google Browser Sync. I resisted it for a while, although Ian Sison at Q Software Research kept telling me how great it was. I resisted because I didn't think it would be useful (in the same way I didn't buy a celphone for years, and when I'd got one, I wouldn't send SMS messages because all the replies I got were illiterate). I also resisted using it because I was concerned about security. But BrowserSync won't sync passwords if you don't want it to.

When I finally installed it, it was just as with SMS. Eventually I was using it quite a lot (I also deal with fewer illiterates these days). I've given up on it though. I've removed it from all the firefox profiles I use. The main reason is that browser sync goes insane if the computer goes off without the browser being shut down nicely. Now, that doesn't happen a lot with me since I use linux, but I do occasionally use a computer that isn't on an AVR and when the power trips (gonna have a lot more of those for the next 3-4 months, it's summer now in the Philippines), BrowserSync loses its memory and tries to sync everything from the network when firefox comes back up. That takes too damn long, even on broadband. I don't want to wait 5 minutes or more for the browser to become usable again after a power outage. Once I went to sleep and in the morning it was still trying to sync. Something probably went wrogn with the bandwidth or the connection, but still...

In any case, i found myself cancelling the sync whenever this would happen. I was doing this enough that I finally realized there's no benefit to BrowserSync anymore. In any case, the only reason I really liked BrowserSync was, I could open a whole lot of windows in one browser, close it, go home (or to work, or to QSR, or wherever) start a browser, and have BrowserSync open the same windows/URLs. I was using it as a multi-client session synchronizer. After thinking about it though, I concluded that using it that way is anti-productive. Most of the time, when I'm reading articles from digg or reddit (or my feeds at bloglines) I'm just wasting time anyway. There's far too much to read, I spend too much time on it and not enough time thinking or coding.

Dropping BrowserSync increased my productivity by a bit because I didn't have to wait for syncs after power outages, and I didn't have to read non-work related articles at work. Of course I still read a lot at work. But it's more focused, it's more work related, and it's not synchronized all over the place :-).

bacon-wrapped, cheese-filled, battered and fried hot dogs!!!

If syphilis were a product this is what it would look like

But you'll have to follow the link and read the article. Or, hmmm, click here for the image. I would post an IMG link there, but again, that'd be rude.

Gmail Loader

Wow, I see that Mark Lyon has developed a Gmail Loader, which can take mail in standalone mail clients and load them into gmail. That's cool. I'd use it, but I've been using gmail so long that I don't even remember some of my old email addresses :-). I certainly don't have those mboxes around anymore.

It'll be very useful though, for when I convince everyone around me to upload their mail to gmail for archiving/backup purposes :-).

I should test it though to see what happens with executable attachments. Currently Gmail blocks executable attachments and even looks inside zip and .tar/.tar.gz files to see if they contain executables, so likely the same happens with uploaded email.

SQL Injection cheat sheet

SQL Injection cheatsheet is over there

Looks good, so far. Haven't read it all yet. I'm learning a lot though. There are a lot of tricks there that I might have seen before but didn't pay enough attention to.

Apple Not for me

A friend of mine bought himself a MacBook. That's a lovely computer and it's pretty rugged too. It survived having beer fizz up and explode all over the keyboard. Another friend is considering getting the same (MacBook, not the beer).

I've never been one to buy brands, so, while they'll certainly be very happy with their purchase(s), it's not a choice I would have made. At the price the MacBook is going for (close to $2000 in the Philippines), I could get two reasonably fast (but not dual-core) Windows/Linux capable laptops. And since I can get Winbook laptops (or Thinkpad, or HP) from my brother in the U.S., for even less than that, I could get three previously but lighly owned laptops for the price of one MacBook :-). Since I use Linux (it does all I need a computer to do) and didn't much like the Mac interface when I tested it out, there's nothing there to tempt me to go Mac. I might be slowly seduced by a Mac's smooth design (hardware and software) if I had one at work, or if I could borrow one and use it exclusively for a month :-). I doubt, though, if that seductive smoothness would get me to overlook the price difference though. I would enjoy the MacBook, but then forget it and switch back to Linux since $1000 is (to me) much more valuable and useful than whatever psychological benefits I would get from using a MacBook.

On the other hand, I'll work very hard to justify the cost, if I get one for free :-)

in-place upgrade successful but bumpy

I did the in-place upgrade to Edgy and it was mostly successful. I got stuck on one thing though. It seems to be a common bug with upgrade-in-place from Dapper to Edgy. A quick google search finds this:

https://launchpad.net/ubuntu/+source/apt/+bug/64615
http://www.nabble.com/Removing-a-Broken-Package-(courier-authdaemon)-t3331317.html

I only just found those links after I'd fixed the thing though. The fixes suggested there are too scary for me since I'm not an experienced debian user. Instead, I followed the hack that mhulboj suggested at this blog post

He noticed that the courier-authdaemon un-install script was trying to first stop the daemon and that it was getting stuck on the fact that /usr/sbin/courierlogger didn't exist anymore (probably a package dependency thing, something else removed courierlogger, could be fixed in the package by first checking if courierlogger even exists before removing it).

Anyway, the fix was:

sudo touch /usr/sbin/courierlogger
sudo chmod a+rx /usr/sbin/courierlogger

and continue with

dpkg --configure -a
apt-get remove courier-authdaemon

which successfully removed the package. I'm now continuing with the update in place (re-downloading a lot of packages since I also misunderstood something and did an aptitude autoclean :-).

Feeling Daring, in place upgrade to Edgy

I'm feeling daring and am going to do the in place upgrade to Edgy from Dapper.

Oh, there's a better link
here and running update-manager with -c and -d brings up the help page on upgrading to edgy

ten coolest numbers

There's a page that discusses ten cool numbers. Naturally, some of that judgement is subjective. I think the page itself is pretty cool.

I would write about e to the i pi and -1, except, I have no idea how to formulate that in HTML :-) and I'm too proud to read the HTML of CoolNumbers :-)

I like a lot of those numbers. I'm going to read all of that and understand perhaps 20% of it :-). hahahaha. I'm not enough of a number theory geek to know the math. I just know enough to enjoy what I see. Including the parts I just don't grok.

Downloading FreeBSD

I'm downloading FreeBSD tonight. It'll all be here by tomorrow. I've got a big spare partition because I was finally able to migrate my root directory to a smaller partition, although I had to switch back to lilo to do it.

I'll install FreeBSD tomorrow on that spare 40G partition. I'll then see if I like it. I'll install all my favorite tools, see if I can do everything I need to do in it, and I'll decide whether to switch to it from Linux. I'll probably keep both, actually. Although I'll work in Linux more than on FreeBSD.

I'm told that FreeBSD doesn't quite have the SMP support that Linux does, among other things. That doesn't really matter to me. I don't have two CPUs (or even hyperthreading) on this laptop, although the Toshiba laptop has that. I may install FreeBSD there too, just to see which OS performs better with hyperthreading.

I've got a postgresql performance test case ready to go ;-).

So it's going to be a fun weekend. I'm very happy with Ubuntu just now. If I don't want to keep testing FreeBSD, I'll drop it for Ubuntu Feisty. Or I'll just split that partition and have Feisty and FreeBSD plus my workhorse ubuntu Edgy on the same box. If I can get a copy of Mandriva 2007, I may install that too. Either that or Mandriva 2006 had svk support built in already. I don't use svk a lot, but I definitely need it. So I may split the drive yet again :-).

This whole thing approaches lunacy. Fortunately, I don't actually have the time to try out all of that. I'll certainly test out FreeBSD. I'll almost certainly test out Feisty. Probably won't try Mandriva or NetBSD though. Although, I'm always ready to be surprised by myself.