Erlang concurrency discussion

Joe Armstrong has a good and interesting discussion of concurrency in Erlang. One point being, if the multiple/concurrent processes don't share memory but instead only send messages to each other, concurrency is easy and robust. Certainly, it's race conditions and incorrect locking of shared resources that makes concurrency difficult in languages which share memory.

Recently too, Joel had a discussion of MapReduce which, apparently, is something google uses to parallelize massively. Which is related to the Armstrong's article since the concurrent processes don't share memory either, they just work independently allowing massive concurrency scaling pretty much linearly as the resources.

I saw a blog post that said that Google doesn't work that way (and that discussed how map/reduce actually works in terms of the, I think, C++ API). I've lost that link though. I don't know if I ever bookmarked it, and if I did, well, I work on around 5 computers. So it's somewhere, but I don't know where it is. I could use Google Browser sync I guess. But I'm not convinced. I'm a bit paranoid about it (mainly about keeping passwords in there). I'm told that that's optional (a feature I'd expect from such an excellent company as google, certainly. But I'm still thinking about it. Oh, I'm sure I'll capitulate, eventually. But there's going to be at least one tulog before I do so.

Alright. and that's quite enough randomness already.

Libraries, books, Wow

I don't quite care for the title, but yeah, those *are* Lovely pictures of books and libraries and stacks. I would love to visit each of those at least once. For a year each :-).

per-site color preferences in mozilla would be nice

I was surfing over to phpPatterns and I couldn't read it. It was in some sort of white over green font that was hard for me to read. At least it wasn't white (or light yellow) on black. But it's still not pleasant to read.

On the other hand, I *really* want to be visiting there regularly. I didn't want to have a separate firefox profile just to view that one site (or those sites which are hard to read due to their color schemes). I might still do that, but I'd rather avoid it if possible. So I thought I'd just go into Preferences|Content|Colors and force all pages to display in black on white.

That works OK. It messes with the tabbed style on, for instance, blogger, so that the line between the editor form and the Edit Html and Compose tabs is not there. I'm sure it messes with lots of other stylish pages too. I think I'm liking this though. I'll stick with it. If I start missing the styles on other pages I may go back, and then I guess I'll read phpPatterns on My RSS reader instead. I don't think that works right though, since I'd need to follow links, and those links wouldn't be in bloglines but would be straight to phpPatterns, and the problem re-emerges.

I would be nice if site color preferences in firefox were configurable per-page. So that when someone's pages goes unreadable, I can force the colors, but just for that page.

Re-engineering: Assertions versus Unit Tests

I'm working on re-engineering a whole lot of badly designed code at work. The code has been mostly working for 4 to 5 years (some of it seems to have been working since 2000 even, if I can go by the author:date comment at the top). Part of the problem is that many things have changed in the business since (any particular/the) code was written. Another part is that the database design was optimized for uploading (easy to identify all the new records for the day to be uploaded), but it is incredibly anti-optimized for querying. There are around 24 basic transactional tables (it could be whittled down to around 12-15, probably, that's another issue, related things should be together in one table) and every day, an additional 24 are created with the date (in mmddyyyy format, not even yyyymmdd) tacked on.

So creating a multi-day report involves running the same query over multiple tables and processing the data on the client side, or it involves a union over many tables.

The first part of the re-engineering involves creating suitable abstractions over this mess (and no, it's not fixing the table structures to be more sane, that comes later, there's too much other code that depends on the insanity). The abstractions will allow us to provide a more reasonable view of the DATA (hiding the database details) to higher (reporting) levels. This is at some cost in CPU processing though. We use PHP, and it's just not very efficient when shoveling a lot of data around while transforming it.

I'm working on basic classes that will interact with the database (everything else interacts with these classes, no higher level code will touch the database directly). I'm finding a problem though. The code aggressively checks parameters and (for anything complex), invariants inside the methods. Checking is done through assertions. At the same time though, I developed a UnitTest class (a simpler replacement for PHPUnit and similar). And when I develop unit tests, I find myself wanting to write tests that make sure the assertions are working. But the assertions can't be tested if assert.bail=1, and the assertions would get much longer if I wanted them to return failure codes to the unit testing framework (because assertions should be simple, just make the program fail immediately).

I would also like to know *which* assertion failed (my parameter checking is pretty paranoid, so there are many checks upon method entry, even if there's only one paranoid, it might even be that there are checks for object validity even if there are no parameters).

ifdef-ing (well, if(defined(....)-ing), the assertions when unit testing doesn't seem to be the right thing to do either. But I haven't really thought that through yet. Maybe I'll try that and see what falls out.

I wonder how others have resolved assert versus unit test (actually, assert versus the goal of complete test coverage, or close to it). Hmmm, I'll google that over the weekend, maybe.

Jerry Pournelle's Byte column -- moved

It's too bad that Jerry Pournelle's contract with byte was not renewed. I've always enjoyed his columns (well, back when I could read them in dead-tree byte, and for a while, on free online byte). It's good to see that he's keeping the format though, and maintaining the site for himself at Chaos Manor Reviews.

I'd like to be able to subscribe, but that's not possible (the subscription is not inconsequential in the third world, even at my pretty good IT salary). I don't have a credit card, and even if I did, I'm not sure the transaction would be honored. Philippine credit card transactions are tricky.

I do hope that he keeps enough subscribers to make maintaining the whole chaos manor system worth his while. There are always good discussions there. I don't go there as often as I used to (due to Digg and Reddit, but it's always educational to swing by every few weeks to see what's being discussed.

ubuntu wifi

I'm at Bo's coffee club in the Robinson's Galleria mall. I would put links there but frankly, I can't find the official websites with some very half-hearted googling. I'm running ubuntu on a toshiba laptop with an Atheros AR5212 802.11abg wifi card and ubuntu detected the wifi card flawlessly.

I also installed ubuntu on a winbook laptop with an Intel wifi chipset (it was a 1.5Ghz centrino system) and that was directly detected too. I'm not sure how ubuntu did that (with the intel system), since the intel firmware is required for that. I wasn't paying very much attention, just installing stuff and downloading packages from the network. Maybe it auto-downloaded the firmware upon detection. Or maybe it came with EasyUbuntu.

That probably cements my transition to ubuntu from mandriva. The main factor in the transition is the rate of development. Ubuntu has working svk packages (I never could get svk working correctly with Mandriva, although the packages are there) long before Mandriva does. And I never could get wifi working with Mandriva (although, to be fair, that was me being lazy, if I were younger and had more time and patience, I'd have figured it out long ago).

I never have been a linux fanboy. I'm a pragmatist. Whatever works better, that's what I use. Windows doesn't work for me since it's impossible to keep secure and, frankly, I can't afford the MS-Office license (windows itself I could afford, barely). Now, I *do* get a valid and legal MS-Office license with the laptops I purchase from the U.S., but then they have that pesky "legal only in the U.S.A." clause, so even if they're legal there, I can't use them in the rest of the world. At one point RedHat was the distribution to use (and before that, slackware). But now RedHat is moving in the direction of enterprise systems and I don't want to be in Fedora (where I would need to be if I wanted svk, etc).

I moved to Mandrake/Mandriva because of the superior package management, but now the speed of development isn't quite right for me. Ubuntu feels right. Although I wouldn't be surprised if I were to move on to something a bit faster later on.

Ah, but this post was not supposed to be about reminiscence. Instead, I was going to celebrate finally getting wifi working. I'm sitting in Bo's coffee club and for the price of a Cafe Latte I'm able to connect to Robinson's free wifi. The DNS is very slow (one reason why I don't have links above, the other being mentioned above), but surfing to this or that site, once it's been resolved, is pretty good. The DHCP server provides two IPs (the first of which does not reply to ping). That's a bit incompetent. But removing it from the list doesn't speed up DNS queries much (or at all, as far as I can tell). Once DNS resolves though, things are pretty good.

Connection to my office VPN is pretty fast (faster than at home), so I guess they use either meridian or PLDT (my home connection via Destiny Cable Internet takes a roundtrip overseas, so the VPN connection is reasonable, but not this fast).

OK. nmap -v says that the first DNS server *is* up, just not replying to ping.

I'm very happy to get wifi up and running, finally. Previously I didn't care very much. I still don't, really, since fast internet is available at work and at home. But it's a great convenience to be able to connect to wifi networks.

Next thing to learn, I guess, is how to connect to closed networks. It's been 4 years since I last had anything to do with that and it was a pain then (even with windows). I'm sure it's much easier now. Haven't done it at all though, yet. So there's going to be some learning involved.

switching svn URLs with rsync

I've got an svn working copy of close to 300MB. I'm across the internet from the source and I didn't want to check the whole thing out again. I had originally checked it out with svn+ssh:// but I wanted to switch to svn://. I needed to switch because svn+ssh takes longer than just svn://. The ssh part needs to reconnect with every svn * command, and I didn't want to pay the 2-3 seconds of that ssh negotiation. There might be a way to use a persistent ssh session, but that would look dumb, providing no beneft.

I thought maybe svn switch might do the trick, but after some trial and error, and actually reading The free svn online book show that switch doesn't work the way I thought. While trying stuff out I also looked at --relocate. There's an example of svn switch --relocate here, but that doesn't do what I need either. I'm not moving anything to a branch or elsewhere in the repository, I actually want to change the protocol used to communicate with the repository.

Finally, I fell back on rsync. I use rsync a LOT. It solves a lot of problems that I run across at work (and in the occasional consulting job). For this task, I first did an svn co svn:// checkout *at* the same computer as the repository. This was just so I'd have a working copy with svn://, normally, I use file:/// there.

Once I'd got a checked out copy using svn://, I rsynced that with my remote copy (which used to be svn+ssh://). There was probably something slightly wrong with what I did. It correctly synced the subdirectory .svn directories to use the new protocol, but the root didn't get changed. So I rsynced that .svn directory separately.

Total bandwidth cost of the rsync was around 4MB received and 2MB sent. Since the whole checked out working copy is 327MB, that's quite a savings. It actually took less time to do the rsync than my previous attempt, which was to tar and compress the remotely checked out svn:// working copy. I was then going to scp or rsync the archive over. When the archive blew up to 60MB though (and still going), I gave up on that. I certainly didn't want to copy over 60MB if there was some other way.

Switching to ubuntu -- but with icewm

My desktop computer (inherited from my brother-in-law) always had problems with Mandriva 2006 installed on it. These were mainly graphics problems, e.g., on drakconf, rpmdrake, gaim (some KDE programs too, so it wasn't necessarily just GTK), icons in windows would display to the left of where they were supposed to show. This wasn't a big deal at all, but it was irritating. And there was the svk problem (which I solved as below, by installing from CPAN, but it was another thing to worry about).

I thought I'd try some other distribution. I'd installed Ubuntu and Kubuntu at work. There were problems (mainly because we didn't understand what we were doing). Ubuntu and Kubuntu apparently come in one CD only. I'm not sure if they come in DVD or multiple CDs. In any case, at work both installed great looking desktop environments but then we spent some time figuring out how to get postgresql-8, php-5, devel tools, servers etc on there. Now, after looking at the Ubuntu website, it looks like there are different install CDs for desktop and home use.

At work, I didn't really spend enough time to figure out how to do all of that, so we just ignored Ubuntu and we now use debian (with some old versions of software, see above, installed from source or from debian-unstable) for the server and Mandriva or Novell Linux for the desktop.

On my home desktop, though (I've got Mandriva 2006 on DVD, but the computer doesn't have a DVD reader), I first installed debian (via network install, just for kicks). That worked out pretty well, but I decided not to stay with debian-stable because the versions of software on there were just too old for me. I tried debian-unstable, but I gave up on that since I got confused by some things. some things were also, well, unstable but those were just me not knowing what I was doing and getting confused).

I had upgraded to debian-unstable by just changing the apt sources and then upgrading everything automatically. That worked well enough (better than the similar technique would have in Mandriva), but I didn't like something about the whole thing, so I upgraded once more (same technique) to ubuntu. That worked pretty well and, while I had packages from three different versions of debian, I didn't really have any problems. The icons problem went away and the whole thing was much more responsive than it had been under Mandriva.

For the future, I'll be leaning toward ubuntu for my desktop computers. I haven't made up my mind yet about servers.

I don't like Gnome or KDE (preferring, instead, icewm), so the first thing I do after I've gotten a good install (well, I've only done this twice so far, but I'll be doing it a lot more, I bet) is to change the apt sources and install icewm, rxvt, etc, and remove as much of gnome and KDE as possible. I keep individual gnome or KDE applications though (quickly coming to mind are evolution, kuickshow and k3b).

I doubt if I'll go to debian. The distro moves too slowly (postgres-7.4 instead of postgres-8, for instance), and I'll never get anyone to agree to install debian-unstable, just because of that name :-). Ubuntu, though, seems to have just the right amount of development/upgrading velocity. I liked the Mandrake velocity, I don't think I'd be comfortable with gentoo, I *know* I don't want to be using Fedora.

Long ago I switched to mandrake because of urpmi (and later, rpmdrake), and also because of the rate of development. I didn't really need everything that the Mandrake *drake software gave me, but urpmi's automatic dependency handling was great, and the rate of development felt just right.

All of that is still there, so I don't *really* need to switch distributions. The kicker though is that I use svk for working with version-controlled source code when I'm not connected to the internet. And I haven't yet gotten svk to work from the Mandriva packages that are available. Not sure why that is, it's probably just pilot error, but, on the other hand, debian and ubuntu make it dead easy to install svk.

On Mandriva, I usually install svk from CPAN, which is a major pain (that's another rant right there). CPAN has some major problems finding the right protocols and servers to use for installing things, last time I did that I had to figure out what it was trying to do and do them manually, putting the downloaded files where it expected to find them, again, probably pilot error or just not enough deep knowledge of how to configure CPAN installations, but if ubuntu makes something dead easy, I'm switching as long as nothing else I absolutely need is missing.

byte.com -- accessible

So I still think that Byte.com still has a dumb login/logout interface but at least now it's letting me in. Previously, I would type in my login/pass (the one they had emailed me) and that wouldn't work. That was after the point when the ddj page would say that I needed to login but wouldn't give me the form to log into.

Today (about a week after I first encountered the problem), the site finally showed me a login screen so I could login with the login/pass that had been emailed to me. I think it might have shown me a page before and I couldn't use it because the password was wrong. Anyway, there was a link to have it email me another password, so I clicked on that sometime last week, logged in today, and that works now.

I would still be annoyed, except I've almost forgotten the problem now. I'll be annoyed next time there's some non-deterministic behavior on that site that makes me unable to login again, or do something similarly obvious, but perhaps implemented in a non-obvious way over there.

Maybe things got fixed earlier than a week ago, but I've been to Banaue with my wife, mom, sister and her entire family. That was a great trip. I wish I could afford to do that (timewise, since it's really not that expensive a trip) more than once a year or so.

byte.com's incredibly bad interface

So I pointed out that byte.com is now free. That's great. A few days after that post though, I'm seeing the downside. That web site has an incredibly bad interface, at least with respect to logins and logouts.

1. There is no logout button or link. or even any indication, as far as I can tell
that one *is* logged in or logged out.

2. There is no login form either. There's a form to subscribe to the free
newsletter, but no login form.

So I guess the only way to login is to click on a protected article (any article, really, and hope that it's protected, since i'm able to view some articles but not others and I sure can't tell why there's a distinction except maybe age of the article) and if it brings me to a page that says it's protected and I should login, then to login.

That's what happened when I first tried it out for free a few days ago. It didn't even ask for registration details, just wanted an email address and then it let me in. Then it emailed me with my login and (random) password.

That's all great so far. Except now, since it won't let me logout, I keep getting a page that says I should register. Except I've *already* registered. I think maybe the authentication credentials are wrong (after all, it just let me in that first time without me having to first check my email, maybe it didn't insert my randomly generated password in my session yet). But now I can't find out where to logout so I can login properly.

I *could* throw away all my cookies and saved passwords, but I'm not about to do that for one free site that has bugs in its authentication system.

Of course, though, I might be unfair here. Maybe the 1 year old article I'm trying to read is just protected and I need to pay for it (similar to how the NYTimes has things setup, and why I never go to the NYTimes site anymore, nor even link to it), but the page it brings me to sure doesn't seem to say that. It won't even let me login, instead asking me to register again! But then remembering who I am and pre-filling the form with information I already posted before (but strangely enough, with the wrong country, I remember registering as being from the Philippines but it thinks I'm in Anguilla, which might be a firefox bug since I often register as being from Anguilla to other sites, e.g., the Washington Post).

So, OK, maybe I'm just confused. But if that's the case, they still need to fix their site. If they can get me confused, I've got to wonder what their paying customers think of the ease of use of that site.

evolution outbound filters and mangling the Sender

Related to my previous post, I'd always thought that it would be great if the Gnome Evolution email client were capable of mangling the Sender in the email based on user-specified conditions. If that were possible, then I'd be able to setup an outgoing filter so that if at least one of the recipients were those mailing lists then it would change the Sender to my mailing list address.

There would still be corner cases where that wouldn't work (but where an elaboration of it might work, e.g., instead of mangling the sender directly, it might create a new email with the sender being corrected, but leaving the original email intact).

I could probably do something like that with the "Pipe to Program" action, with some sed magic thrown in. But, no. It *is* a useful inconvenience and I'm too lazy to figure it out. Hehe, long ago when I was wishing for that feature, I already knew I'd be able to do it with Pipe to Program and sed. I was too lazy then too :-).

A useful inconvenience

I have several email addresses and one of them I use for subscribing to mailing lists. I never use gmail for reading mailing lists (instead i download it via fetchmail and use evolution to check the local POP3 server). This is mainly because I just don't have the time to monitor mailing lists anymore.

Evolution uses my default email address for sending mail out (unless I change the From address explicitly). That isn't the same as my mailing lists address. That's inconvenient, since often I'll send out a reply to a list post and it'll bounce back because I forgot to change the From address.

I find though, that the inconvenience is useful. It's an additional brake on emails that I might not actually want to send out yet (they need editing, the tone is too strong, the tone is too weak, the point is missing, whatever). It's too bad though. Sometimes I'll write a long, informative, interesting (maybe) post, and then I'll delete it when it bounces back because it doesn't pass muster. But that's the nature of useless posts. They're a waste of time, created in the heat of the moment, starting as a rant and ending as a flame, or the other way around, moving from to whimper, or just wandering around, pointless and dumb. If they're a waste anyway, better not to pollute lists (and the impressionable) with them, or feeding the trolls.

byte.com is free!

I read Jerry Pournelle's Chaos Manor proto-blog regularly. Not as much as I used to, I've been busy, but regularly. This week he says that Byte.com is now free. It used to be free and for many years I read it online. Before that, I was in the U.S. and i actually had a subscription (along with subscriptions to Scientific American, Dr Dobbs and various C/C++ magazines.

Byte became payware for though, so I stopped reading Jerry's column there, instead switching to the main blog. Now that it's free again, I'm going to have to read several years of articles :-). Jerry is always fun, so I'll read all of that. The articles though, I'll read only if they're relevant and not too dated.

It's too bad that Moshe Bar's column ended, I enjoyed those, although they were quite a bit over my head (probably still are, since I'm not as hard core as I'd like to be).

devshed rss

I use Bloglines as my RSS reader (too lazy to find standalone RSS readers for Linux). I had DevShed in there, but I find that it doesn't seem to be a good RSS citizen, or maybe I'm just doing something wrong.

With other feeds, as soon as I even click on the folder in bloglines, the articles are marked read and when I go to bloglines again, I don't see those articles as unread anymore. Devshed articles aren't like that. The same articles keep coming up as new even though I've read them already. Not sure what that's about. I've already removed it from my list of feeds and I don't care to add them again to debug the problem. My guess is, though, that they allow comments to update the feed. Most other sites (e.g., digg) don't.

It's really no big deal, but even the little effort required to look there and notice that I've read the articles already is too much for me, I'd rather not see anything there anymore than have to go through tomorrow seeing the same articles about how to optimize mysql or how to implement a socket server in php.

Creative Zen mp3 player and gnomad2

We got a Creative Zen Xtra MP3 Player as a gift from my brother. It was to be just my birthday gift, but it's so perfect for just now (sol is pregnant, for a bit she was on bed rest, very bored), and Sol's birthday is near enough to mine (which is still months away) that we thought it was the perfect gift for both of us for the rest of the year :-).

Unfortunately, the Zen doesn't look like just a USB external storage device, it uses some other protocol. I didn't want to be dual-booting into Windows (at some point I'm going to remove Windows from this laptop completely, I kept it around for the Zen) everytime I need to move files onto the MP3 player or to modify playlists and such.

Fortunately, GNomad2 implements the MTP protocol and allows file copying and manipulation of playlists and such. It didn't work for me the first few times I tried it, but after a week of looking at it off-and-on, I finally got it working (devfs was the charm). It might be possible to run it without devfs, but the RPM I installed it with didn't seem to have the nomad.usermap file on it, so I just boot into a devfs capable kernel when I need to copy files. And since I don't want to dualboot, well, I just always boot into a devfs capable kernel these days :-).

Things to remember, and the point of this post: DEVFS.
Also: less laziness and maybe building from source and actually reading the documentation might make DEVFS unnecessary. Unfortunately, the laziness is systemic.

Frameworks and new development

At work the IT team is re-implementing/improving the software system for managing the business. The head office software and database schemas are going to be replaced entirely, and the remote site software is going to be partly re-implemented, but we're keeping the remote database schema as is for now. Fixing *that* monster is probably for next year.

It's exciting to be rebuilding the system since the current system is so much of a mess.

After a week and a half of checking out various php MVC frameworks we've settled on Symfony. We looked at a lot of frameworks (including
one that was developed by the contractor we're working with). Many were cut because it didn't look like they could serve our needs, they had one or two vital features missing. Even symfony was almost cut, except we figured out a way to get around its assumption that primary keys are serial (so, never editable, in its generated CRUD scaffolding). CakePHP was cut because it doesn't seem to support compound primary keys. I really like Prado but it seems to have a sufficiently high learning curve that I'd have trouble training new developers to work with it. I'll use it on some personal project at some point, but we won't use it at work.

Some other frameworks (Seagull, QCodo, etc) we tested for a bit, but when we encountered bugs or couldn't make them work with our system (postgresql, pretty complex database structure) we gave up on them.

The framework will be a help, but I'm more excited about the great work we'll be doing with architecting the business logic and database layers so that we will be able to more easily fix/modify the database without needing to modify higher layers. Or if the change propagates that high, that we'll be able to easily determine which database service providing classes/methods need to be changed and which reports need to be modified because of that.

Doxygen (or PHPDocumentor) will be a big help with all this too, and subversion/svk, of course.

We're also architecting how much of the business logic should sit inside the database. I'm aware that some people (a lot in the Oracle camp, and in MS SQL Server too) like to put all their business logic in the database (some to the point of not allowing access to tables directly, instead allowing access only to functions which are the ones that access the tables). I'm not on that side of the issue.

The old database went too far the other way, having pretty much no primary keys, no check constraints, no referential integrity constraints or (in many cases) even not null constraints.

I'm leaning toward something in the middle, where the database has enough business logic (constraints, primary keys, referential integrity and some triggers for complex constraints) to protect itself from some program bugs and invalid inputs), rejecting transactions which fail those constraints. The user level programs will check their inputs and reject invalid inputs, etc, but the database will reject anything that gets through the user level programs.

This will involve some cost due to doing some of the work in both places (the programs and the database constraints need to be in sync about what is or is not invalid, I'm not too concerned about the execution cost since good database design and insightful use of indexes will mitigate those), but I think it's worthwhile. Otherwise, the database is at the mercy of buggy programs.

Slow computers

My brother-in-law, my wife's younger brother, is blind. He's a good programmer though, great brains in there. He uses MS-Windows with JAWS (screen reader software). What he can do with windows is amazing, considering that it's so graphical.

He used to use a 400Mhz PIII with 192MB of RAM. I've inherited that PC since I found a used PC that is much faster (an AMD 1.7Ghz box) and gave that for him to use.

I run linux on the slow box, with icewm for the window manager (I use icewm everywhere, even on the fastest computers because I can't stand the slowness of Gnome and KDE). It runs everything reasonably well except Firefox. It's great to use Firefox on fast boxes, but it's dog slow on a slow box. I've got to look at Opera for that box.

I don't remember running openoffice on that box. Maybe I've done it, I don't use very complex files though, so I wouldn't have been stressing openoffice anyway, even if I ran it. I'm sure it's slow, but it's probably usable. I tend to use gnumeric for spreadsheets and I just don't use a wordprocessor that much though. So on a slow, memory-constrained computer, careful choice of office software can make the computer usable.

The PIII box used to run windows 2000. It was reasonably usable, but I'm not sure how well it compared with linux. It was probably about the same speed, or maybe a little slower (because of the antivirus reading everything before handing data off to applications reading the disk). On the other hand, it was *far* less capable than my current setup (apache, php, C, C++, Java, ruby, scheme, guile, postgres all running on it, plus all the client tools I use).

I've got to find an alternative to firefox though. Or wait until someone comes up with a comprehensive howto on how to minimize the memory and CPU that firefox uses.

philvbdev

I subscribed to the philvbdev yahoogroup to see if I could hire VB developers off there. The company I work for is hiring. After I posted though, I started getting all these emails from Indian freshers. There must be a network (or a bot) where newbie South Indian software developers get email addresses and the email addresses spread like wildfire. It's bad enough that if anyone sends me anything with "fresher" in the subject, I'm not going to see that email at all, it'll be automatically deleted [not even spending any time in the purgatory that is my spam mbox or trash].

It's a waste of time for them to be emailing me because we need the developers for local employment and we're not going to be paying anyone to come over from a foreign country to work here (with the attendant problems with visas and such).

I joined the group because I thought it was a forum for philippine VB developers. It looks like it's been abandoned by filipinos though. Very few posts are about VB problems or solutions. Mostly what I see (what I saw, I've since unsubscribed) are spam posts about certifications, certification questions and training, etc.

laptop computer speed

My cousin-in-law asked me last week what I'd choose between a computer (laptop, actually), with a 1.xGhz CPU and 512MB of RAM and a 2.xGhz CPU with 256MB of RAM. I said I'd choose the first.

I've been monitoring CPU use on the laptop I currently use (icewm makes that really easy, with CPU and network monitors on the taskbar), and I just don't see a lot of CPU peaking. There are occasional peaks when a scheduled rsync job runs (copying data over a vpn from the office to my laptop, cheap remote backup), and when i run xboard+gnuchess. When I test one or another program I'm developing I may see some peaking, and sometimes when I run vmware (not a lot of that though).

Mainly, though, my CPU usage stays pretty low.

I recently finally built a kernel that works well with my mobile AMD CPU and now I'm able to set the CPU maximum speed to a large range of speeds (.66Ghz to 1.7Ghz, plus 6 other speeds in between). I've also got the cpufreq/scaling_governor set to ondemand. The computer pretty much ALWAYS stays at the slowest speed all the time except when running xboard+gnuchess or very occasionally, when something CPU intensive runs. When an application needs more CPU, the CPU speed ramps up immediately, and when it's no longer needed, the CPU slows down to .66Ghz.

From the fact that the scaling_max_freq pretty much stays at the slowest speed except for very short periods, my opinion on which laptop to buy is reinforced. The icewm monitor also displays related information (e.g., how much IO is occurring) and I find that the laptop slows down more due to waiting for IO (well, laptop drives are slow) than due to needing more CPU. So more RAM will help with the IO requirements (the OS can cache more of the disk), speeding up the general experience, more than higher CPU speed.

Actually, his original question had to do with a celeron (or some other internal-cache-challenged CPU) versus a full CPU at the same or a higher frequency. I think that RAM is still more important, generally, than CPU speed.

Of course, if the main use of the computer is going to be for photoshop or corel draw effects or CAD and 3D rendering, then I'd go with all the CPU power I could get PLUS all the RAM I could get. but for general use, even for a software developer like me, any 1.3Ghz box with more memory is probably going to be the better deal than a higher frequency box with half the memory.

mozilla extensions

I've been wary/skeptical for a long time about mozilla/firefox extensions. Firefox is a memory hog as it is, and there are some hard to isolate weird CPU hogging things in there too. So, while I love the browser, I go out of my way to NOT make it any more memory/cpu hungry. And in general, for complex things that have weird behaviors, I prefer to keep things as simple as possible so that problem isolation is easier, or at least that the chances of tickling a bug are lower.

Late though, and strangely enough, because I saw a link to an article that discussed how to make firefox eat less memory and use less CPU, I followed another link and decided to install a few very conservatively chosen extensions. I'll add links at some point, for now, just discussion.

NoScript is great. I have it block all javascript by default. And then, for sites that need scripts, I turn javascript on either temporarily (current session) or permanently. Very nice since most of the time I don't want javascript and it just slows things down anyway.

AdBlock works pretty well. Some ads get through and it's not convenient to point them out to AdBlock, but all in all, it's great. I wish I could *unblock* google ads. They're not intrusive, no moving graphics and crap like that, and they're generally relevant, although I rarely click. I'll figure out how to do that at some point.

Tab Mix Plus is great. There are lots of features there, most of which I don't know how to use. The main thing for me is that it has tab memory like Opera has. That is, it's possible to configure it so that when the browser starts, it will auto-load the tabs that were there when the browser was last closed. It also saves tab status while browsing, so that firefox and crashes (rare, but not completely eradicated) don't completely wipe me out. If I've downloaded 15 tabs to view, then tab memory will bring them all back up when I restart firefox. It seems to work better than Opera's memory, but I haven't used Opera in 2 years. I'm sure Opera has made their tab memory much more robust by now, although it was already a great and wonderful feature when I used it back then.

PDF download is convenient since I've never been able to stand acrobat reader displaying itself in the browser and then taking up so long to start. Not that I have acrobat reader on this computer anymore, or any computer I use, but the extra control over PDF downloads is nice. This is a relatively low profit item for me though, I may drop it.

Work hard, crash

I've been working very hard for about six months now, since I took over management of the development team at work. Previously I'd been a consultant, project based. That was fun. Lots of work, but no management responsibilities.

The company insisted on hiring me on though and after some resistance, I accepted. Things are still fun, particularly when i get to code, but there's too damn much coordination and meeting. All very necessary, but it sure reduces time available for programming.

After six months of this though, reality catches up. I got a very bad sore throat some other stress related infections and just general weakness (due to the combined infections, probably). This happens to me, once in a while. After a sustained period of late nights, not enough sleep, stress, my body crashes for a short period. If I don't get enough of a break, my mind snaps too and I quit ;-). Not going to happen with this job, but it's sure happened before.

Five days of rest have made for an improvement. Not enough though, still going to be weak tomorrow. Damn. I'll rest again on Wednesday.

php+postgresql+cursors

I ran into a problem recently because PHP was dying when processing a query that returned some 20 million rows. PHP was running out of memory.

After some googling and running the program several more times (with some runs involving increased memory limit for PHP), it became clear that PHP's postgres support code is loading whole result sets into RAM (and feeding it to PHP one by one, when the PHP code calls pg_fetch_row or pg_fetch_array) rather than using cursors so that it can load one row at a time. I don't know if ADODB or PEAR do better. If they're still using the underlying postgres specific functions though, then they're still going to have the same problem.

The solution is to use cursors explicitly. There are some caveats.
1. Using a cursor requires using a transaction. This isn't onerous, just
something to remember.

2. It's necessary to check after the FETCH to see if there are no more rows.
There's probably a prettier way, but I just go with an infinite while loop
that breaks when there are no more rows.

// assume that $conn is the db connection and that $q is the query, the ff is typed
// straight into blogger, might contain minor errors.


pg_query($conn,"BEGIN WORK");

$cres=pg_query($conn,"DECLARE my_cursor CURSOR FOR $q FOR READ ONLY");
if(!$cres)
{
die(pg_last_error($conn);
}

// query succeeded, fetch the rows from the cursor
while(true)
{
$res=pg_query($conn,"FETCH my_cursor");
if(!$res)
die(pg_last_error($conn));

// quit loop if no more rows in cursor
if(pg_num_rows($res)==0)
break;

// there's a row, get it and process it.
$row=pg_fetch_array($res);
process_row($row);

pg_free_result($res);
}

pg_query($conn,"CLOSE my_cursor");
pg_free_result($cres);

pg_query($conn,"END WORK");


fetching rows one at a time, above, is for illustration only. of course,
for efficiency, one would fetch many rows at a time and loop through them.

Title Wrong - ridiculously

So in my previous post the title says that java studio is ridiculously slow, and then I forgot to mention that fact in the rest of article. On Saturdays I lose my brains.

Anyway, when my colleague was showing me the class diagrams that he developed in Java Studio, I couldn't stand the fact that scrolling across the diagram space, zooming in, and dragging items from one place to another was slow. It wasn't really unusably slow, but it was irritating to anyone used to how fast dragging things in Konqueror (itself slow enough that I never use it, even on a fast box, but a good comparison because, despite it being slow, Java Studio is slower) is.

Maybe things will get better on my own computer though. I'm still waiting for the sun Developer Network site to let me in though, before I can download and test Java Studio Creator. Maybe under strace, in case that can show what's making it slow (I doubt if it'll help, but it's a weekend, I can spend the time looking at strace :-).

Java Studio - slow - ridiculously

A colleague at work was using Sun's Java Studio Creator for some prototyping. He was running it on the LTSP server and I noticed that it would sometimes eat 100% of the CPU (well, 50% of the CPU since it's a dual-CPU box, the culprit to be mentioned further on doesn't seem to be a multi-thread issue, at least) continuously, while it was doing nothing (everyone had left for home, no one was working in Java Studio anymore).

That seems like a stupidity, and I thought I'd download it to my own computer so I could see if the behavior would manifest there also.

I've subscribed to the Sun Developer's network though, and I can't log in to where I can download Java Studio. They haven't sent me a confirmation email yet, I can't log in. Seems like some slowness over there at Sun. probably due to java. Damn, I'm getting disillusioned with working in Java.

First rule of debugging

My wife and I came home from Tagaytay and there was no internet. The cable modem had the receive light blinking and the send and online lights off. OK, I had to get to work and didn't care enough to fix it right then (mistake, Sol wanted to work at home, but it's a short walk to work, so no huge deal). When I got home, the cable modem was still down. I texted the destiny cable help line and went to bed. In the morning, same thing.

I texted destiny again and they called me (but I didn't answer since my phone was on silent+vibrate).

And then I looked at the cable modem and realized that the cable wasn't connected. In removing the laptop power cable for the trip to Tagaytay, I had moved the cable enough that it disconnected from the modem.

I had violated the first rule of debugging, it's *MY* bug. Almost all bugs are my bug. First thing to do is check if I did something wrogn before reporting to anyone else that they have a problem with their system.

No big deal, but useful to be reminded.

cpufreq on an AMD 2400+ - strange

I rebuilt my 2.6.12-15mdk kernel, with a Pentium III/Celeron/Coppermine ... CPU processor family because when I built it with either AMD K6 or K7, it was just not showing the right CPU frequencies to choose from in cpufreq.

Or maybe it was, but I didn't like the choices as they all seemed too high.

cat /proc/cpuinfo says (among other, snipped, things):

vendor_id : AuthenticAMD
cpu family : 6
model : 10
model name : mobile AMD Athlon(tm) XP-M 2400+
stepping : 0

Maybe there's a problem with the detection of the CPU frequencies. Or maybe the frequencies it shows are right (unlikely, since the lowest frequence is around 1.3Ghz or so, and the highest is much higher than 2.4Ghz, even though the CPU only goes to around 1.7Ghz in actual clocks.

Anyway, no big deal. I've got a working kernel, it does swsusp, it does cpufreq, it does most of what I need. The only thing I'd really like now would be if I could get the damn ALSA sound working. When I load the ALSA sound driver for this VIA82cxxx, I get no sound, and when I load the OSS driver I can't change the volume in kmix. I can get around that by using headphones with volume control, but I just want the ALSA sound driver to work right :-).

I *did* get it working at some point, or maybe that was with the Mandriva 2005 kernel, 2.6.12 something.

Hmmm, might be time to build a generic kernel now.

Kernel rebuilding week

I upgraded my Mandriva installation and some things are not working right, so it's kernel rebuilding week.

I use cpufreq to slow down the CPU when I'm not doing anything CPU intensive. So I needed that to work (/sys/devices/system/cpu/cpu0/cpufreq/scaling_available_frequencies or similar). With the default distro, I get an incorrect frequency list. So I need to build the kernel and tell it what CPU i've got. it looks like the kernel tries to use the powernow K7 module for cpufreq, but /proc/cpuinfo tells me that this CPU is AMD family 6, so I should probably be using the K6 powernow module).

I thought I could use the default distro kernel, but when I do that, the USB keyboard and mouse don't work (very strange). So my first kernel rebuild was just to get usbkbd and usbmouse working correctly. I built a kernel that was pure i386 (for vmplayer, going to test openmosix) and tested with AMD K7, now building one for AMD K6.

There is also a problem with sound. If I use the ALSA driver there is no sound, but if I use the OSS driver, there is sound but I can't change the volume. Not sure what that's about. OSS is fine with me, I don't care very much. There's probably something simple I'm missing there. I just need to figure that out and I'll have sound plus volume control.

Not that that matters very much. the sound on this laptop is ridiculously low in volume, so I'd need to get speakers anyway, and I could control the volume from there :-). But for now, it's a small challenge to either get ALSA working or to get OSS working with volume control. I *did* have ALSA working for a bit there, but I don't remember exactly what I was doing. I might have been playing with a 2.4 kernel for openmosix (but 2.4 doesn't have cpufreq built in and I was too lazy to apply that patch and then fight with openmosix' patch incompatibilities).

One of these days it'll all get done.

Spam mail policies at ISPs

I had some trouble with my internet provider, destiny cable internet last week. Emails being sent out their their mail were not getting sent. I use a local postfix server with the local server sending mail out through the ISP's official smtp server using relayhost. This, because, like most large internet providers, destiny blocks outgoing port 25 (as they should).

I was getting error messages like this:

Jan 24 22:31:25 my-laptop postfix/smtp[4704]: CA13C270A9:
to=, relay=smtp.mydestiny.net[202.8.224.8],
delay=4, status=bounced (host smtp.mydestiny.net[202.8.224.8] said:
554 : Client host rejected: Access denied (in
reply to RCPT TO command))

After getting in contact with destiny, I was told that their server though that my IP was spamming and so had decided to block everything from me.

I think this is a mistake. It's a convenient thing to do, notice a few emails outbound that seem spammy and block the IP completely, but it's too prone to error. And users are going to be confused. I'm lucky I have an inside pipeline to the destiny technical people. users who don't have that access would have trouble even figuring out what was going wrong, let alone explaining it so that it could be correctly diagnosed and fixed.

The right thing to do is to check every email going out, and if it's spammy, deny it with a useful error message (e.g., Destiny Internet has determined that this email is spam, please contact technical support if you think it isn't spam [plus some verbiage on how the ISP was protecting the user from spam and apologizing for inconvenience if the spam classifier got it wrong]). That way, emails that are clearly not spam don't get blocked.

Don't block IPs, block individual emails.

I suppose it's possible that that's what their spam classifier was doing. It had seen too much spam from my IP (when the IP was still used by someone else) and the bayesian learning system in there had locked on to the IP so that anything coming from that IP would be spam even if the rest of the email was innocuous. I don't think that's the case though. and it's something they should fix.

To be sure, blocking an IP after a whole bunch of spams have gone out is a cheap way to minimize load on the spamassassin cluster (or whatever they use), but it's still a mistake. Add servers to the cluster if necessary, but don't block an IP (which might have been spamified by someone else and now has been inherited by some innocent subscriber) just because of a history of spamming.

openvpn and stalling scp/sftp/rsync

I had a problem with openvpn for a while. rsync, scp would work, they would show progress of uploads/downloads, but then they'd stall at the end, when they should have completed and ended.

They just wouldn't complete the file copy.

A bit of googling (which I could have avoided, since the document was just the man page :-) said to try --mssfix [smaller_number] to tell TCP sessions what packet sizes to use (for the case where MTU discovery doesn't work correctly).

That worked. I tried *far* too small a number, 256. But it works for me, I'm running something over the vpn link and I don't want to cut it off yet since it's very long running. When the processing completes, I'll try larger values for --mssfix. Or maybe not, I don't care about efficiency on that line too much, it's not like I'm downloading or uploading mp3s. Mostly it's just ssh, the occasional svn commit/update, and rsync of my email from work to home :-). That last could take advantage of the efficiency of larger MTU, but it's not important that it run faster, so I don't care much.

Might play with larger values later just for fun though.

dar - for multi-volume tar-like backups

I just found DAR, Disk ARchive a backup program for linux (and windows and other OSs too, it looks like) that can do slices (i.e., produce multiple files which can fit on whatever removeable media you use). There have been other, similar programs, but this is the first one that I cared enough to actually try to get working :-). Apart from old programs that I wrote myself (mostly for fun, I never actually used them for much) that cut files apart and pasted them back together.

dar -c w2k3 -vv -s 795M -R /[path_to_directory_to_backup]/

there are options for including and excluding files and patterns, etc.

It does differential backups too. Huh, it's time to buy that external DBD-Writer :-). Or just a whole bunch of CD-RWs. I never did buy a DBD-Writer because of the expense (need to use for laptop, and external drives are expensive) and because I don't download movies. Now that I'm playing with lots of vmplayer images though, and also now that my digicam archives are getting ridiculously large, it's time to seriously think about removeable backup (and not just backup to the 40G USB HD).

Filipino pirates?

I was reading over at jerry pournelle's site when I came across a link to an article on seven who pirated Revenge of the Sith and put it online and went to jail for it.

I wouldn't normally take notice. This stuff is so common (particularly in the Philippines) that even I get upset with all the pr0n that's right out on the street with no one minding. And I don't even have children yet.

But then I read the article and I find that a lot of the names seem to be filipino.

Marc Hoaglin probably isn't, nor is Michael Fousse. And maybe Albert Valente and Ramon G Valdez are hispanic, not necessarily filipino.

But then there are Jessie Lumada, Dwight Wayne Sityar, Joel de Sagun Dimaano, and maybe Stephani Gima.

clickthecity - damn slow

It's a lazy saturday and I decided to surf over to Click The City to see what movies are showing this week.

The front page loads *very* slowly. Some of that is "Waiting for spots.clickthecity.com", which is their advertising site, I suppose. they need to make that server either much faster, or (more likely), optimize the programs that choose the ads to show. I clicked on a link and now it's been waiting for spots for 2 minutes. Not only are they losing the customer because he closes the window, they're losing me forever because I won't deal with a site that makes me wait that long.

When I got to the movies page, finally, I decided I'd right click on a film and open in another tab so that the information about the film would load slowly in another tab and I could continue looking at the current page. But they probably have some sort of javascript (too lazy to look at the source) that disables right click. They want you to surf their slow site in only one window. Of course that doesn't stop me, I just copy the URL, open a new tab and paste the URL there. but that's just pissing me off, giving me more reason never to visit their slow, stupid and fascist site ever again.

Huh, still waiting for spots.clickthecity.com. That's it, 5-8 minutes waiting for ads is just too much. I'll fix my DNS so that clickthecity.com never ever resolves. Idiots.

Upgrading blues

auto-upgrading in Mandrake/Mandriva with versions 10.1 to higher versions doesn't seem to be dangerous. I've not had trouble with 10.1, 10.2, 2005, 2006 (unless 10.2 *was* 2005, I forget).

so I decided to upgrade in place from 2005 to 2006. Most things worked pretty well. Certainly, there isn't the wholesale breakage that comes with earlier versions where urpmi and rpm themselves break and can't continue at all without some tedious surgery (copying, rpm and all necessary things from a working installation, or installing a fresh copy of the original revision from CD).

Some things need tweaking. This seems almost always to be the case. There was some flakiness in apache (Mandriva uses 2.0 for it's default apache version now, version 1.3 is now apache1), and the php sub-packages are now much finer-grained, so an automatic upgrade didn't upgrade php-postgres support (although I thought it should have). There was more bogosity with apache2 and php 4 versus apache2 and php 5, I was getting some sort of segmentation fault when calling php programs. Since that was probably a throwback from some old apache or php packages that did not get upgraded, I uninstalled everything apache and php related and then reinstalled them (with php 5 now, although in production I use php4, it's a good time to review php5 on my laptop, for future upgrading of production).

As always, the new kernel broke things (sound stopped working in ALSA, although in OSS it usually works (but not always, and I don't know why that is), but then I can't change the volume so I really need ALSA to work, the usb keyboard and mouse don't work if I use the kernel from Mandriva 2005, so I can't stick with that unless I'm on the road). SWSusp doesn't work with this laptop and the original Mandriva 2006 kernel (2.6.12-14mdk), the laptop suspends to disk, but then it won't wake up correctly, instead, freezing. I built a kernel from the 2.6.12-14mdk source that loads from suspend correctly but the sound is still flaky, so I'll probably keep working on this until it works or I give up, download a linus kernel and build that.

Still dangerous, urpmi update with old Mandriva

So I Tried to upgrade an old (10.0) Mandrake server to 10.2) and it worked for a bit. And then some part of it failed. I did the 10.0 to 10.1, then 10.1 to 10.2 shuffle, but somewhere in the second stage urpmi got trashed and now I can't urpmi (or, in fact, rpm, there are dependencies to URPM.pm and URPM.pm can't be found).

I can recover pretty easily but it's a pain. Of course, I wasn't totally upgrading everything though. That was probably the bug. The last time I tried a complete update using that technique it worked fine. For now I'd say that it's probably possible to totally upgrade a Mandrake 9.x->10.0 and upwards using the multi-stage technique mentioned in the link, but selective updates (particularly selective updates from two generations ago) are a bad idea.

What I did was:

upgrade rpm and urpmi from the versions in 10.0 to 10.1, then upgrade them again to 10.2, but without upgrading everything else. Well, it seems that some packages (e.g., perl-URPM) need to be upgraded but the dependency trees might not be complete, or, at any rate, something goes wrong in the process and some things got lost (URPM.pm, from perl-URPM). they were removed but then installing the new copies didn't work, so now urpmi and even rpm don't work anymore.

Heh, that's the last time I'm doing that :-). Partly, because I don't have any more old servers to maintain, but also because (while it's possible, and not so difficult, to fix it), it's a pain and not worth the time. Just backup, reinstall everything, and restore, is what I say. I couldn't follow that advice with this latest server (won't boot from any newer media, something wrogn in the bios or similar), but I'll do that with everything else.

Of course, at work we're using SuSE and I'm getting ready to upgrade in place with that. Hay, never learn :-).

Upgrading old Mandrake versions upward with urpmi

Long ago (I can't find the posts on Jijo's archives of the Plug mailing list) I saw a post by Marvin Pascual on upgrading a Mandrake based linux system to the next version by using urpmi. This was great because it wasn't necessary to reinstall the OS,

just pop in the distribution media (or put it on the network somewhere),
urpmi.removemedia -a
urpmi.addmedia [name] [url/filesystem_location] [with hdlist_location] (or similar)

and wait.

upgrades usually too *longer* than a fresh install, and some things always broke, but it was convenient.

when i tried that though, it didn't work for me. in fact it broke the system enough that I had to reinstall anyway (fortunately, since /home was separate, without having to backup and restore my data, although i backed up anyway, so actually i just saved the restore part :-).

I never did figure out why that failed (it was sometime around the 8.x to 9.x upgrade). I tried it again when 10.0 came out. that failed too, as did 10.0 to 10.1. finally though, 10.1 to (i think) 10.2 or Mandriva 2005 worked without error. But I was still stuck with some old servers (from old consulting clients) which were 10.0.

Recently (sometime in 2005) I decided to try something easier, I first downloaded the new RPM and URPMI packages (and all their dependencies) from the newer version, installed those with rpm -i and then, when rpm, urpmi, urpmi.removemedia and urpmi.addmedia had installed correctly, continued with the auto-upgrade from fresh media. That worked. So I had found a way out of the point where the process had blocked. Apparently, there was something wrong with how it was downloading one or another of those files (and their dependencies, might just have been some order of installation issue, or perl installation order errors too, I don't remember).

So now it's possible to upgrade 8.x to 9.x, 9.x to 10.0 and 10.0 to 10.1 (10.1 upwards are fine without this workaround). I thought I wouldn't need this information anymore, except a year and a half ago or so I completed installation on a server that, for some reason, would not boot from the 10.1 DVD. So I had to install 10.0 from CDs. I need to upgrade some packages on that box though, and install some other packages that weren't available (or were seriously deficient) in 10.0.

So I followed the procedure I've outlined:

urpmi.removemedia -a
urpmi.addmedia [the new media, actually, i just used web sources for the
packages since the server is for an ISP and their bandwidth is huge]

urpmi rpm (say yes to the dependencies)
urpmi urpmi (again, say yes)

then urpmi anything else necessary. i haven't upgraded *everything*, but if something breaks, well, i'll urpmi anything that might need it.

Kind of useless knowledge now (can't think of any other servers that i've installed that are older than mandrake 10.1), but there it is anyhow. maybe an old client will contact me about this. or maybe it'll help someone.

Suse (even opensuse) on Vmware

I use Mandriva as my preferred distribution. The *drak* stuff isn't that big a deal for me, I prefer the command line, usually, and it still isn't easy to add an ethernet interface alias via drakconf (actually, I don't think I see how to do it at all, although you can do it with webmin). I'm *really* in love with the urpmi stuff though (and mainly the command line urpmi, although rpmdrake is very good too).

I thought I'd try suse though, either the evaluation version, or opensuse. But I can't afford to overwrite my laptop since I don't want to have to reconfigure everything (openvpn to two different networks, etc). So I thought I'd try it with vmware. When I downloaded the DVD ISO, it wouldn't work in vmware. It kept giving some sort of boot error. I thought maybe the DVD download was corrupted so I downloaded again, same problem. Next I downloaded the CDs. Same problem. It looks like opensuse (or the suse evaluation version) just won't install in vmware. That's a shame, since I'm certainly not going to switch to it if I can't test drive it, and I just don't have a spare computer to test drive it on. Ah well.

Bloglines and Google Reader

I looked at bloglines last night, partly because of a post here, or on a mailing list about how it's so much easier to use than google reader. too lazy to put links there.

I think I *will* be switching to bloglines. It does seem much faster than reader (although that might just be because I've got digg in reader and there are a heck of a lot of entries in there for reader to load). It's also got subfolders, feeds are subfolders on the left (useful for ignoring digg until i've got enough time to waste, on reader everything is in the same place and digg overshadows everything), and it's got "Mark all read" (which I sometimes need to resort to if I've been away for a few days and there's no hope of reading everything that digg has found).

There's probably some way to filter things in digg (which is still too useful to drop, despite the inconvenience of having so much there), but I don't see how to do it in reader yet. I thought maybe digg would have a feature where the RSS feed is identified per user, so that my preferences in digg (what I want to see, which is not everything) would be what I'd see in my RSS reader, but even if there were a per user RSS feed, there doesn't seem to be any way to have preferences in digg similar to preferences in slashdot, where I'd only see articles in categories I'm interested in.

So it's bloglines I'll be using for now. When reader catches up and gets faster, I'll look at it again.

Dictionary of Algorithms and Data Structures

i saw the Dictionary of Algorithms and Data Structures long ago, posted on a mailing list. Saw it again on digg.com just today.

I didn't graduate CS, so a lot of this is new to me. I wish there were a more structured way to browse it though. alphabetical is doable, but doesn't link concepts very well together. alphabetical plus following related links (see also, etc) works a bit better, but that has its own problems. Heh, I'm being characteristically lazy and I want a tutorial that covers everything in there :-). I doubt if even Knuth covers everything in there and if he were to try, he'd need the immortality pill from The Myth.

visayan on mailing lists (KLUG)

I see that sacha is looking into learning bisaya. I don't know why that would be, it's not like she's not got enough on her plate already. It's certainly a good thing, learning languages is good.

I hope, though, that it's not so that she can decipher things like this. And that she doesn't try to learn things from that and similar posts on some mailing lists. Almost all visayan posts on that list are like textingese in tagalog or english, essentially illiterate. That's not to insult the poster, that's just students communicating lazily in whatever language they use. It's not their fault, it's cultural. There are wonderful, literate visayan posts somewhere on the Net, i'm sure, but they're not on mailing lists populated by students who don't care enough about their posts to try to spell correctly.

Not that spelling is an easy thing in visayan. There's no given orthography, and the examples in sacha's post feel sort of archaic, although that's better than the illiteracy that passes for communication these days). Sounding archaic isn't a big deal though. Visayan varies from place to place, people will just assume she learned visayan with people who spoke classical visayan.

National Geographic Pics

Huh, in the preceding link to Google Video I wasn't quite sure whether it was a *good* way to waste bandwidth. So there's no qualifier there. On the other hand:

Pics from National Geographics and
ScubaDiving.com cannot possibly be considered a waste.

Lots of other good photo sites out there. for instance, APOD, Astronomy Photo of the Day.

All those sites better than webshots or screensaver software that installs adware and spyware on the computers of the gullible. I think so anyway (wouldn't really know, don't use windows anymore, maybe there are webshots or screensaver freeware that aren't evil).

Google Video

Huh, I see that Google Video is up. What a way to waste bandwidth:

Asian Backstreet Boys
How men screw up romance

lots of useless stuff there too. i wouldn't be surprised if there's some pr0n there though, somewhat like that smart telecom's post your phonecam pictures here site (too lazy to find the link).

tiger

Try ruby not working

I saw a link to Try Ruby! (in your browser) and thought, why not.

It's not working for me though. After the first few things to try, I get to the "Jimmy".reverse example and I get:

>> "Jimmy".reverse

<?xml version="1.0" encoding="iso-8859-1"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<title>500 - Internal Server Error</title>
</head>
<body>
<h1>500 - Internal Server Error</h1>
</body>
</html>


PHP already has most of what I need and I'm interested in trying Win Binder, a native Windows binding for PHP. I don't know what that's like yet, but if it's reasonable, then I may use that to write some quick windows programs (anything to get away from the horror that is Visual Basic).

Hay, so much to learn, so little time. I need to delegate.

swsusp very useful

I was on a long bus trip recently and I decided to see if the new laptop supported swsusp. I had given up on swsusp on a previous laptop (also a Winbook, but a slightly older one, since sold) because it didn't work there. It would try to save and then would fail on trying to halt mysql. Or if i stopped myself myself, it would try to save, and then on restore it would try to save again (instead of restoring), and then give up and reboot (or something, it was confusing, which is why the account above is a mess).

I'm very happy that it now works on the new laptop though. I've got three batteries for the laptop. They only last around 45 minutes at the full 1.7Ghz AMD speed (something like 2300-2400+ intel equivalent, i think), but in combination with cpufreq and setting the speed to the slowest CPU frequency available (around 600Mhz AMD, so maybe around 800+ intel equivalent?), the individual batteries last around 1.5-1.7 hours.

But I didn't want to shutdown and swap batteries since starting up takes some time and all of that time is at 1.7Ghz, eating battery life. Swsusp is a lifesaver since, when battery gets low and the laptop starts beeping, i just run a script that tells the laptop to suspend. It writes its status to swap. The laptop turns off. I swap batteries. And when I turn the laptop on it restores from swap, switches to 600+Mhz automatically and gets back to X in far less time than it takes to boot.

I think I'm going to swsusp all the time now, even when not trying to save battery, just because it starts up so much faster when restoring.

A constant state of insecurity

Another link from schneier has:

As a security professional, my friend often attends security conferences and teaches security classes. She noted that the number of passwords she collected in these venues was higher on average than in non-security locations. The very people who are supposed to know more about security than anyone appeared to have a higher-than-normal level of remote access back to their companies, but weren’t using any type of password protection.

A Constant State of Insecurity

Good God.

On the other hand, I admit that I do use several levels of insecure passwords for free but insecure web based services. Any service that requires registration for free access will get one of my fake (I lied about everything, including what country I'm supposedly from [I don't know where Anguilla is, but it sounds pretty]) identities with my lowest level password (I don't mind giving it away to anyone, although I don't actually post it on a blog or anything either :-).

Maybe some of those people were doing something like that. On the other hand, well, if companies are exposing FTP or POP3 on the internet (or maybe even ssh with their users having the same ssh password as their POP3 password), well, this is depressing my donkey. I wonder if anyone learns anything at those security seminars. Maybe they're Windows security seminars and the lessons have to do with which buttons to click to turn the firewall on. A lesson that is instantly forgotten because the listeners are Windows security professionals.

Schneier this week

Schneier in the current installment of Crypto-Gram has some interesting links. This month's installment seems very good. More interesting than previous months. Although they're all worth reading.

He gets something wrong though about the manila times and profiling of terrorists (male, between certain ages, cellphone, uneasy). Maybe that's just because he needed a short tagline. The article attributes the profile to the NCR Police Office (from which stupidities like this are expected).

The article certainly doesn't sound as stupid as schneier's comment makes it sound. That there is some controversy in the police organizations about the usefulness of the stereotype would normally be a good thing. Except that might just be normal intra-governmental infighting with no good reason behind it.

In the same issue of Crypto-Gram though are a whole bunch of links related to the Sony DRM rootkit (heh, it can be used to avoid WoW spyware (what's that doing, maybe just checking for piracy and such?), a trojan uses the Sony rootkit/trojan to hide, Sony has given up on it and Microsoft will detect it and remove it. Heh.

And There's an interesting weblog on malware from F-Secure, found it linked to from Schneier. I may look at it every day now. Or maybe add it to an RSS aggregator, if i can figure out how to do that :-) Huh, madali lang pala, using reader.google.com. Too lazy to try other RSS readers.

Hmmm, now to figure out how to get a feed from TechScene (kind of bass ackwards, since I could just get feeds direct from the people, but I'm lazy or too busy at work to remember too many feeds or URLs, or both).

There's also a link to an article discussing the prank article on aluminum foil hats previously discussed here. It looks like some heavy handed humor back and forth, but I'm not so sure about that zapatopi link. Some people are nuts about this stuff, and he might be one of them. He *does* have a book on practical mind control protection with aluminum foil beanies but I haven't read that, so I don't know if it's just a relatively long lived prank feeding on the paranoids and taking their money :-).

Lots more articles, all of it interesting. Every 15th of the month (well, 16th in the Philippines) is a huge time sink (but only about 33% a waste of time) because of this monthly email :-)

Aluminum helmets *help* the government spy on You!

although on average all helmets attenuate invasive radio frequencies in either directions (either emanating from an outside source, or emanating from the cranium of the subject), certain frequencies are in fact greatly amplified. These amplified frequencies coincide with radio bands reserved for government use according to the Federal Communication Commission (FCC). Statistical evidence suggests the use of helmets may in fact enhance the government's invasive abilities. We theorize that the government may in fact have started the helmet craze for this reason.

On the Effectiveness of Aluminium Foil Helmets:
An Empirical Study

IVRS must die

I see that Edong (he also has a "Ka", but I'm not on that side of the political spectrum) has a post on IVRS (interactive voice response system.

IVRS was all the rage in the US when I was there. It saved labor cost because the operator could be partly replaced by a machine voice. Instead of hiring 2 or 3 operators, a company could keep one receptionist/operator and fire the rest.

I can't stand IVRS though. Whenever I hear automated voice prompts I immediately hang up.

It's not the impersonality of it. I work with computers, I sort of like impersonal. What I can't stand about IVRS is the inefficiency of it, and the fact that, if it's to be used efficiently (memorize the call tree and type the digits in directly instead of waiting to hear the prompts), *I*, the customer, have to expend extra effort to remember. I have enough trouble memorizing my name, some days. I don't need to buy from a company that forces me to either waste my time listening to a ridiculously long call tree. And even if I wanted to memorize the direct route to where I want to go, no company I'm buying from is important enough to spend that much memory real estate on. Even if I had that real estate to spare. Any company that tries to communicate with me through IVRS is (1) going to lose a sail, (2) going to earn a rant, either in person, when I get to talk to someone in management, and/or a blog entry, for being more concerned with cost than customer experience. If they want to save cost, let them relocate their call center to the Visayas or Mindanao, salaries there are 1/2 to 1/3 what they are in Metro Manila and it'll help the economy and decongest the capital.

Which reminds me, is it still Metro Manila or did that go out with the Marcoses? No one mentions the Metro much anymore. Although there's still that extra M in MMDA.

Large copies to USB 1.0

I've got a laptop that only has USB 1.0. This is great for my USB keyboard and mouse (there are no PS/2 ports at all, so external input devices *have* to be USB, I've never heard about PCMCIA keyboards or mice :-) and also for my USB bluetooth and memory sticks, but it's terrible for a USB hard drive.

Recently I was copying multi gigabyte files to the USB drive and I couldn't stand that it was taking so long and that the laptop would pause and block for tens of seconds. I thought what was happening was that the data was being buffered and then, when the data was actually written down to the hard drive the kernel was so busy handling interrupts (or polling, or whatever) that the computer blocked.

I came up with two workarounds. The first workaround was to set the buffer flushing period to very short, so that the laptop wouldn't block completely for tens of seconds but instead would just get slow every few seconds.

update -3 32 -f 1 -s 1

The second workaround was to limit the bandwidth going down the USB pipe. Rsync is a great tool. This time I couldn't use it to do incremental syncing since in the configuration given checksumming would be far slower than just copying the data file over directly. I didn't want to fill the pipe though since I thought I'd noticed the laptop choking a bit even with the update trick. So what I did was use rsync as a slow cp. Rsync has a --bwlimit option for app level bandwidth limiting which works even within the same computer. So:

rsync -a --bwlimit=384 [srcfile] [dstfile]

limits the amount of data transferred to 384 kilobytes per second. The link can handle up to around 1 megabit per second, but limiting the copy bandwidth keeps the laptop usable and then I just hid the rsync window so that I could continue using the laptop.

rsync --bwlimit wouldn't be enough here. Without the update trick (making buffer flushes shorter and smaller) the buffers would still get filled with hundreds of megabytes of data which would then take around one second per megabyte to go down the wire. With update and no --bwlimit, I was finding slowness and inconvenience. Together though, they made the laptop usable while it was copying large files in the background.

Eventually though, I'll just have to use a laptop that's got USB 2.0 :-).

tiger

600+ kBps!

I downloaded OpenOffice.org 2.0 just now and I was shocked at the download speed. I downloaded through bittorrent and I was getting 600+ kilobytes per second. that's one advantage of going with an ISP that doesn't do bandwidth capping. I use destiny cable internet and while I've never before seen 600+ kBps, i regularly see 100-200 kBps.

I guess someone else in the destiny network was seeding the torrent. Or maybe there's just a whole lot of bandwidth available on monday morning and I got all the benefit of it.

I downloaded the SUSE 10 eval DVD ISO a week ago and that came down at around 60kBps on average. There's something to be said about downloading an ISO via the official torrent too. When SUSE 10 was very new I tried to download with the official torrent and that was so slow (2-3kilo*bits* per second) i decided to download it over the edonkey network instead. That download completed but the ISO was bad, so it was quite a waste of time. When i downloaded from the official torrent again, it slowly went up to around 60 kiloBytes per second. So official torrent+wait+a+few+days is the right thing to do I guess.

Possibly, too, downloading other (more popular)distributions would be better via the official torrents. I've done my bit at misinforming people about SUSE licensing (old information from when it was still SUSE and not novell). Probably people stay away from SUSE because they still have the impression (as I did, until I was gently corrected by a SUSE/novell representative) that SUSE is not yet freely distributable. Well, the eval edition *is* now freely distributable.

Anyway, I'm waiting for Mandriva 2006 to become available to see what the download rate on that will be :-). I'm the second worst type of ISP client, really, downloading things sometimes only to see what the ISP's performance is like. The worst type, of course, are those guys with huge hard disks who download movies, games and mp3s as if there were no tomorrow.

tiger

rsync algorithm talk by tridge

Andrew Tridgell has a great talk on The Rsync Algorithm

rsync and --partial

rsync --partial doesn't do what I thought it would do. In fact, for large files (my current project involves rsync of half gig files over relatively slow networks, and often networks which are relatively unstable) it can actually make the destination file lose so much information if the rsync dies early that it would become necessary to re-download practically the whole file.

I was under the impression that --partial did something smart, e.g., maybe it would be truly incremental. That is, if a large file was to be downloaded (from source to dest) but the download died somewhere in the middle of the file, that all the *good* data from the source half that was correctly downloaded would be saved AND THEN the other half (on the dest) would be *APPENDED* to the partly downloaded half. This way, if the download were to be continued, then the already good half would all hash correctly and the download would start at the first wrong block.

Or maybe --partial would keep all of the good downloaded file in those temporary .[real_filename].[random_letters_and_digits] files and the temporary would not be removed so that the next download would use the same temporary file.

Unfortunately, --partial doesn't do either of those. It's pretty good if you're downloading a completely new file (the destination didn't exist before or it's *much* smaller than the original). In that case, the file is partially downloaded and when the rsync dies, the partially downloaded file is kept. If you tried to do this with a large file already downloaded, and the new download died before it got to the size of the already downloaded file, you'd *lose* data since the short newly downloaded file would *replace* the old, larger, already downloaded partial file.

So --partial is very useful when downloading files from scratch, but it's not useful (in fact, may be very harmful) when syncing files which you've already partly downloaded before. For that, just use the regular rsync parameters and don't use --partial.

As good as rsync is, I think there's quite a bit of improvement yet to be done for the case where the bandwidth is slow/unstable and the files to be transferred are very large. And either of the suggested mechanisms above might be a good thing to implement.

Resumes with no email address

I received a resume a few weeks ago and the candidate is very interesting. It's in the U.S. style, two pages with only relevant information in there. the philippine style, with parent's names, religion, and picture, is useful for discriminating against people based on surface characteristics. I discriminate against people based on stupidity, and while that can sometimes be seen in resumes, more often I have to wait until the interview for that.

Many of his skills are just what we need, and he's interested in learning other skills that we need that he doesn't quite have yet. That he studied at Mindanao State University and Ateneo de Davao and graduated from the University of the Philippines at Diliman doesn't hurt either.

But his resume doesn't have an email address. Or a celphone number either. Maybe HR got this from JobsDB or similar and they need to pay extra money to get his contact information. Whatever the case, I dropped the resume and the positions have since been filled.

Resumes with no convenient contact information is just not going to be seen. Well, by me, anyway. He's got a snailmail address there too. But it's in Los Angeles, CA, for some reason. Now that's another filter. If he's in the U.S., he's either immigrating (legally or otherwise) or he's just visiting. If the first, why submit a resume to a philippine based company? If the second, he should put down his philippine snail address since a U.S. address is instantly off-putting. Why waste time considering someone who is probably already making more money in the U.S. as a busboy than he would make in the Philippines?

Of course there are exceptions. There are people who go to the U.S. for a few months, work there as computer consultants or software developers for very small (t<6 months) projects and come home. But there should still always be an email address or at least a chikka-able number, so that the employer can check whether any such exceptions apply.

Finally, bluetooth on Linux

I've finally got Bluetooth working in linux. For a while I was just looking at bluez and I couldn't figure out how I was actually supposed to transfer files. I could connect, the pin was right, but I didn't know how to send or receive files.

I installed gnome-bluetooth-manager and I still couldn't figure it out. All that did was see the phone. I couldn't do anything to the phone connection or find a way to transfer files.

Finally though the kdebluetooth RPMs finally made it to my urpmi server and after i did urpmi kdebluetooth I'm now able to transfer files. It took a while to figure out which programs to use (didn't realize that they'd be in the icewm menus too, I'm a moron). But finally found kbtobexclient and was able to transfer a demo java game (J2ME with the wireless toolkit makes development easier, not easy [yet, but i haven't worked on it much yet except to look at the demos], but easier) and actually run it.

Moving Tech blog entries here

My main blog is at Bounding Overwatch. I've created this new blog as my tech blog though (struggling very hard to avoid using some amusing yet faux latin for the blog name) and I've copied most of the tech related blog entries from the old blog to here.

The entries are still there too though. Didn't want links to break if people are linking to me there or search through google.

I wonder what the result of all this moving will be for jijo though. His Filipino Tech blog aggregator picks up posts via RSS. But if I posted all of these entries this morning (except I used the original datetimes), will his aggregator be suddenly inundated with all my entries? Or will only this first entry make it?

It's probably the second, but it'll be interesting if it's the first. Hahaha.

Dynamic DNS in Linux

That title is a bit over the top, of course. I only looked at one solution, and when that worked for me, I stopped looking.

I've got a sort of static IP. My internet provider, Destiny Cable Internet, as of this writing, uses the ISC DHCP Server. That server tries to allocate the same IP to the same requesting MAC if the IP is still available. So the IP is pretty stable. However, there is no promise that the IP will never change, so it's semi-stable.

I thought I'd be OK with just assuming that it was stable, but I changed my mind. It *might* change and when it does, then I'll have to go to everywhere that assumes it's stable and fix it. And then it might change again.

Instead I decided to use a dynamic DNS server. For the client software, I first looked at EZ IP-Update and, since there's a list of dynamic dns servers at that site, I chose DynDNS. Both choices were pretty random, but I'm happy with both of them.

ez-ipupdate took around 20 minutes to set up (there was a minor compile error, something wrong in the ./configure script's deciding how to use errno, so I had to figure that out and fix it, it's a quick and dirty hack though, not good enough to propagate upward, but I may email the ez-ipupdate maintainer so that he can look at his configure configuration) and DynDns took 2 minutes (plus the less than a minute to wait for the email to arrive).

So I've got dynamic dns working now. bopolissimus.homelinux.net

Port forwarding with ssh

I find forwarding ports (and, actually, also reverse forwarding ports) over ssh very useful. I'll probably find it less useful now since I've found OpenVPN so easy to setup and use, but it's still a very useful technique, particularly where I don't have root access.

I always use -v when doing this because -v will tell me if the link is slow or the link is down (while i'm still connecting, less wasted time). It also shows a message everytime a request is forwarded down the ssh tunnel, useful for debugging to make sure I'm doing the right thing.

With the setups below, surfing to http://localhost:8080 will forward requests to the internal or external server specified. Note: by default, port forwarding binds only to 127.0.0.1, so from the box itself you can connect to the forwarded service, but you can't from another box. Use GatewayPorts for exposing the service (warning, think about that, it might open security holes).

1. I'm on my laptop and I want to surf the website on remote.com.ph as if I were physically there. So what I want to do is forward my port 8080 to remote.com.ph:80 (it's an internal website and is inside a firewall, so I can't surf to that site directly from the open internet).

ssh -v -L 8080:localhost:80 remote.com.ph

what that does is, connections on my local port 8080 will be forwarded to remote.com.ph. From there, it will be forwarded to remote.com.ph's localhost:80.

2. Same as #1, I can ssh to remote.com.ph but the web server isn't on remote.com.ph, it's on another internal box inside the firewall, 192.168.80.80:80.

ssh -v -L 8080:192.168.80.80:80 remote.com.ph

What that does is, connections on my local port 8080 will be forwarded to remote.com.ph over the ssh tunnel. Remote.com.ph will then forward them onward to 192.168.80.80:80

3. Same as #2, but I'm trying to surf to some external web site that doesn't let me in if I surf from the Philippines, or if I surf there from the Philippines something bad will happen. For instance, PayPal will block paypal accounts if they're used from IPs that it identifies as being in the Philippines. This has to do with very high fraud rates. But maybe I'm not a fraudster, I just want to use my paypal account, but I can't because I'm physically in the Philippines. If I have ssh access to a host in the USA which paypal won't be suspicious about, I can do port forwarding through that server, e.g.,

ssh -v -L 8080:www.paypal.com:80 my_us_server.com

There is also reverse port forwarding. I do that when the server I need to connect to (usually not for http, but for some internal server) is inside a firewall and I can't get to it directly and the firewall won't port forward to the internal server. In that case, what I do is ssh to the gateway, and from there ssh to the internal server. Then I do reverse port forwarding. What that does is, it will ssh back to *me*, and open a localport on *me* which will be forwarded down that second ssh link back to it. This, of course, only works if the internal server has a route out to the internet, if it doesn't, then some other solution will have to be found.

# first ssh to the gateway
ssh gateway.remote.com.ph

# at the gateway, ssh to the internal box
ssh my-internal

# at my-internal, open the reverse tunnel back to me, i am, client.com.ph

ssh -v -R 8022:localhost:22 client.com.ph

What that does is get my-internal to ssh to client.com.ph (my box, outside the firewall). Once it gets there, it will set up port forwarding so that port 8022 at client.com.ph so that when I (at client.com.ph) connect to port 8022, the request will be forwarded to port 22 at the my-internal computer. I use that much less often than -L, but when -L doesn't work, -R is often a lifesaver.

Comparisons of different SQL implementations

Online there's a pretty long (not sure how comprehensive, but informative and useful) Comparison of different SQL Implementations

Haven't read it all yet (busy), but I'll need to get to that within 1-2 days.

Destiny Cable Internet SMS number

To get Destiny Cable Internet to call me (in case of technical problems),SMS to 09175771111 MD [account name] [account phone number]

Very convenient since I have a cell phone and cell phone charges being exorbitant in the Philippines, I'd rather not have to call them. Besides, I don't know what the tech support number is and I'm too lazy to look :-).

It's weird that one has to put one's phone number in there, but I guess that's for people who have landlines. They can put their landline number there. I don't have a landline so it seems a waste to type the number in since they should just call me on my cel and they've already got that when they receive my text.

It's nice to be able to add (after the phone number) some text about the particular problem. So they can look at things on their side (e.g., signal level, problems in my area, etc) before even calling me. Saves me time explaining things, and saves them some telco costs.

Windows Solutions

Long ago I saw a list of "Windows Solutions". I had it in my sigs and now I can't find them. Fortunately, I keep my old sig file around even though I don't use it anymore (I haven't seen a tool yet that will modify the gmail sig via cron).

Anyway, here they are, so I don't forget them, hopefully google will cache them and google can find them with "Windows Solution bopolissimus" :

1. Don't use Windows
2. Stop the program and start it again
3. Ctrl-Alt-Del and kill the program in Task Manager
4. Reboot
5. Uninstall the program and reinstall it.
6. Reinstall Windows and Everything else
7. Backup, format the disk, install windows, reinstall everything else, restore your data.

to be fair, you sometimes need to do all of these (or the transliteration) for other operating systems, and #2, #3, #4 aren't so common anymore in XP. But they're still much more common there than in Linux. All that spaghetti inside is turning into much and Windows is turning more and more into a big pile of mud (if I may be allowed to mix metaphors)