Spam mail policies at ISPs

I had some trouble with my internet provider, destiny cable internet last week. Emails being sent out their their mail were not getting sent. I use a local postfix server with the local server sending mail out through the ISP's official smtp server using relayhost. This, because, like most large internet providers, destiny blocks outgoing port 25 (as they should).

I was getting error messages like this:

Jan 24 22:31:25 my-laptop postfix/smtp[4704]: CA13C270A9:
to=, relay=smtp.mydestiny.net[202.8.224.8],
delay=4, status=bounced (host smtp.mydestiny.net[202.8.224.8] said:
554 : Client host rejected: Access denied (in
reply to RCPT TO command))

After getting in contact with destiny, I was told that their server though that my IP was spamming and so had decided to block everything from me.

I think this is a mistake. It's a convenient thing to do, notice a few emails outbound that seem spammy and block the IP completely, but it's too prone to error. And users are going to be confused. I'm lucky I have an inside pipeline to the destiny technical people. users who don't have that access would have trouble even figuring out what was going wrong, let alone explaining it so that it could be correctly diagnosed and fixed.

The right thing to do is to check every email going out, and if it's spammy, deny it with a useful error message (e.g., Destiny Internet has determined that this email is spam, please contact technical support if you think it isn't spam [plus some verbiage on how the ISP was protecting the user from spam and apologizing for inconvenience if the spam classifier got it wrong]). That way, emails that are clearly not spam don't get blocked.

Don't block IPs, block individual emails.

I suppose it's possible that that's what their spam classifier was doing. It had seen too much spam from my IP (when the IP was still used by someone else) and the bayesian learning system in there had locked on to the IP so that anything coming from that IP would be spam even if the rest of the email was innocuous. I don't think that's the case though. and it's something they should fix.

To be sure, blocking an IP after a whole bunch of spams have gone out is a cheap way to minimize load on the spamassassin cluster (or whatever they use), but it's still a mistake. Add servers to the cluster if necessary, but don't block an IP (which might have been spamified by someone else and now has been inherited by some innocent subscriber) just because of a history of spamming.

openvpn and stalling scp/sftp/rsync

I had a problem with openvpn for a while. rsync, scp would work, they would show progress of uploads/downloads, but then they'd stall at the end, when they should have completed and ended.

They just wouldn't complete the file copy.

A bit of googling (which I could have avoided, since the document was just the man page :-) said to try --mssfix [smaller_number] to tell TCP sessions what packet sizes to use (for the case where MTU discovery doesn't work correctly).

That worked. I tried *far* too small a number, 256. But it works for me, I'm running something over the vpn link and I don't want to cut it off yet since it's very long running. When the processing completes, I'll try larger values for --mssfix. Or maybe not, I don't care about efficiency on that line too much, it's not like I'm downloading or uploading mp3s. Mostly it's just ssh, the occasional svn commit/update, and rsync of my email from work to home :-). That last could take advantage of the efficiency of larger MTU, but it's not important that it run faster, so I don't care much.

Might play with larger values later just for fun though.

dar - for multi-volume tar-like backups

I just found DAR, Disk ARchive a backup program for linux (and windows and other OSs too, it looks like) that can do slices (i.e., produce multiple files which can fit on whatever removeable media you use). There have been other, similar programs, but this is the first one that I cared enough to actually try to get working :-). Apart from old programs that I wrote myself (mostly for fun, I never actually used them for much) that cut files apart and pasted them back together.

dar -c w2k3 -vv -s 795M -R /[path_to_directory_to_backup]/

there are options for including and excluding files and patterns, etc.

It does differential backups too. Huh, it's time to buy that external DBD-Writer :-). Or just a whole bunch of CD-RWs. I never did buy a DBD-Writer because of the expense (need to use for laptop, and external drives are expensive) and because I don't download movies. Now that I'm playing with lots of vmplayer images though, and also now that my digicam archives are getting ridiculously large, it's time to seriously think about removeable backup (and not just backup to the 40G USB HD).

Filipino pirates?

I was reading over at jerry pournelle's site when I came across a link to an article on seven who pirated Revenge of the Sith and put it online and went to jail for it.

I wouldn't normally take notice. This stuff is so common (particularly in the Philippines) that even I get upset with all the pr0n that's right out on the street with no one minding. And I don't even have children yet.

But then I read the article and I find that a lot of the names seem to be filipino.

Marc Hoaglin probably isn't, nor is Michael Fousse. And maybe Albert Valente and Ramon G Valdez are hispanic, not necessarily filipino.

But then there are Jessie Lumada, Dwight Wayne Sityar, Joel de Sagun Dimaano, and maybe Stephani Gima.

clickthecity - damn slow

It's a lazy saturday and I decided to surf over to Click The City to see what movies are showing this week.

The front page loads *very* slowly. Some of that is "Waiting for spots.clickthecity.com", which is their advertising site, I suppose. they need to make that server either much faster, or (more likely), optimize the programs that choose the ads to show. I clicked on a link and now it's been waiting for spots for 2 minutes. Not only are they losing the customer because he closes the window, they're losing me forever because I won't deal with a site that makes me wait that long.

When I got to the movies page, finally, I decided I'd right click on a film and open in another tab so that the information about the film would load slowly in another tab and I could continue looking at the current page. But they probably have some sort of javascript (too lazy to look at the source) that disables right click. They want you to surf their slow site in only one window. Of course that doesn't stop me, I just copy the URL, open a new tab and paste the URL there. but that's just pissing me off, giving me more reason never to visit their slow, stupid and fascist site ever again.

Huh, still waiting for spots.clickthecity.com. That's it, 5-8 minutes waiting for ads is just too much. I'll fix my DNS so that clickthecity.com never ever resolves. Idiots.

Upgrading blues

auto-upgrading in Mandrake/Mandriva with versions 10.1 to higher versions doesn't seem to be dangerous. I've not had trouble with 10.1, 10.2, 2005, 2006 (unless 10.2 *was* 2005, I forget).

so I decided to upgrade in place from 2005 to 2006. Most things worked pretty well. Certainly, there isn't the wholesale breakage that comes with earlier versions where urpmi and rpm themselves break and can't continue at all without some tedious surgery (copying, rpm and all necessary things from a working installation, or installing a fresh copy of the original revision from CD).

Some things need tweaking. This seems almost always to be the case. There was some flakiness in apache (Mandriva uses 2.0 for it's default apache version now, version 1.3 is now apache1), and the php sub-packages are now much finer-grained, so an automatic upgrade didn't upgrade php-postgres support (although I thought it should have). There was more bogosity with apache2 and php 4 versus apache2 and php 5, I was getting some sort of segmentation fault when calling php programs. Since that was probably a throwback from some old apache or php packages that did not get upgraded, I uninstalled everything apache and php related and then reinstalled them (with php 5 now, although in production I use php4, it's a good time to review php5 on my laptop, for future upgrading of production).

As always, the new kernel broke things (sound stopped working in ALSA, although in OSS it usually works (but not always, and I don't know why that is), but then I can't change the volume so I really need ALSA to work, the usb keyboard and mouse don't work if I use the kernel from Mandriva 2005, so I can't stick with that unless I'm on the road). SWSusp doesn't work with this laptop and the original Mandriva 2006 kernel (2.6.12-14mdk), the laptop suspends to disk, but then it won't wake up correctly, instead, freezing. I built a kernel from the 2.6.12-14mdk source that loads from suspend correctly but the sound is still flaky, so I'll probably keep working on this until it works or I give up, download a linus kernel and build that.

Still dangerous, urpmi update with old Mandriva

So I Tried to upgrade an old (10.0) Mandrake server to 10.2) and it worked for a bit. And then some part of it failed. I did the 10.0 to 10.1, then 10.1 to 10.2 shuffle, but somewhere in the second stage urpmi got trashed and now I can't urpmi (or, in fact, rpm, there are dependencies to URPM.pm and URPM.pm can't be found).

I can recover pretty easily but it's a pain. Of course, I wasn't totally upgrading everything though. That was probably the bug. The last time I tried a complete update using that technique it worked fine. For now I'd say that it's probably possible to totally upgrade a Mandrake 9.x->10.0 and upwards using the multi-stage technique mentioned in the link, but selective updates (particularly selective updates from two generations ago) are a bad idea.

What I did was:

upgrade rpm and urpmi from the versions in 10.0 to 10.1, then upgrade them again to 10.2, but without upgrading everything else. Well, it seems that some packages (e.g., perl-URPM) need to be upgraded but the dependency trees might not be complete, or, at any rate, something goes wrong in the process and some things got lost (URPM.pm, from perl-URPM). they were removed but then installing the new copies didn't work, so now urpmi and even rpm don't work anymore.

Heh, that's the last time I'm doing that :-). Partly, because I don't have any more old servers to maintain, but also because (while it's possible, and not so difficult, to fix it), it's a pain and not worth the time. Just backup, reinstall everything, and restore, is what I say. I couldn't follow that advice with this latest server (won't boot from any newer media, something wrogn in the bios or similar), but I'll do that with everything else.

Of course, at work we're using SuSE and I'm getting ready to upgrade in place with that. Hay, never learn :-).

Upgrading old Mandrake versions upward with urpmi

Long ago (I can't find the posts on Jijo's archives of the Plug mailing list) I saw a post by Marvin Pascual on upgrading a Mandrake based linux system to the next version by using urpmi. This was great because it wasn't necessary to reinstall the OS,

just pop in the distribution media (or put it on the network somewhere),
urpmi.removemedia -a
urpmi.addmedia [name] [url/filesystem_location] [with hdlist_location] (or similar)

and wait.

upgrades usually too *longer* than a fresh install, and some things always broke, but it was convenient.

when i tried that though, it didn't work for me. in fact it broke the system enough that I had to reinstall anyway (fortunately, since /home was separate, without having to backup and restore my data, although i backed up anyway, so actually i just saved the restore part :-).

I never did figure out why that failed (it was sometime around the 8.x to 9.x upgrade). I tried it again when 10.0 came out. that failed too, as did 10.0 to 10.1. finally though, 10.1 to (i think) 10.2 or Mandriva 2005 worked without error. But I was still stuck with some old servers (from old consulting clients) which were 10.0.

Recently (sometime in 2005) I decided to try something easier, I first downloaded the new RPM and URPMI packages (and all their dependencies) from the newer version, installed those with rpm -i and then, when rpm, urpmi, urpmi.removemedia and urpmi.addmedia had installed correctly, continued with the auto-upgrade from fresh media. That worked. So I had found a way out of the point where the process had blocked. Apparently, there was something wrong with how it was downloading one or another of those files (and their dependencies, might just have been some order of installation issue, or perl installation order errors too, I don't remember).

So now it's possible to upgrade 8.x to 9.x, 9.x to 10.0 and 10.0 to 10.1 (10.1 upwards are fine without this workaround). I thought I wouldn't need this information anymore, except a year and a half ago or so I completed installation on a server that, for some reason, would not boot from the 10.1 DVD. So I had to install 10.0 from CDs. I need to upgrade some packages on that box though, and install some other packages that weren't available (or were seriously deficient) in 10.0.

So I followed the procedure I've outlined:

urpmi.removemedia -a
urpmi.addmedia [the new media, actually, i just used web sources for the
packages since the server is for an ISP and their bandwidth is huge]

urpmi rpm (say yes to the dependencies)
urpmi urpmi (again, say yes)

then urpmi anything else necessary. i haven't upgraded *everything*, but if something breaks, well, i'll urpmi anything that might need it.

Kind of useless knowledge now (can't think of any other servers that i've installed that are older than mandrake 10.1), but there it is anyhow. maybe an old client will contact me about this. or maybe it'll help someone.

Suse (even opensuse) on Vmware

I use Mandriva as my preferred distribution. The *drak* stuff isn't that big a deal for me, I prefer the command line, usually, and it still isn't easy to add an ethernet interface alias via drakconf (actually, I don't think I see how to do it at all, although you can do it with webmin). I'm *really* in love with the urpmi stuff though (and mainly the command line urpmi, although rpmdrake is very good too).

I thought I'd try suse though, either the evaluation version, or opensuse. But I can't afford to overwrite my laptop since I don't want to have to reconfigure everything (openvpn to two different networks, etc). So I thought I'd try it with vmware. When I downloaded the DVD ISO, it wouldn't work in vmware. It kept giving some sort of boot error. I thought maybe the DVD download was corrupted so I downloaded again, same problem. Next I downloaded the CDs. Same problem. It looks like opensuse (or the suse evaluation version) just won't install in vmware. That's a shame, since I'm certainly not going to switch to it if I can't test drive it, and I just don't have a spare computer to test drive it on. Ah well.

Bloglines and Google Reader

I looked at bloglines last night, partly because of a post here, or on a mailing list about how it's so much easier to use than google reader. too lazy to put links there.

I think I *will* be switching to bloglines. It does seem much faster than reader (although that might just be because I've got digg in reader and there are a heck of a lot of entries in there for reader to load). It's also got subfolders, feeds are subfolders on the left (useful for ignoring digg until i've got enough time to waste, on reader everything is in the same place and digg overshadows everything), and it's got "Mark all read" (which I sometimes need to resort to if I've been away for a few days and there's no hope of reading everything that digg has found).

There's probably some way to filter things in digg (which is still too useful to drop, despite the inconvenience of having so much there), but I don't see how to do it in reader yet. I thought maybe digg would have a feature where the RSS feed is identified per user, so that my preferences in digg (what I want to see, which is not everything) would be what I'd see in my RSS reader, but even if there were a per user RSS feed, there doesn't seem to be any way to have preferences in digg similar to preferences in slashdot, where I'd only see articles in categories I'm interested in.

So it's bloglines I'll be using for now. When reader catches up and gets faster, I'll look at it again.

Dictionary of Algorithms and Data Structures

i saw the Dictionary of Algorithms and Data Structures long ago, posted on a mailing list. Saw it again on digg.com just today.

I didn't graduate CS, so a lot of this is new to me. I wish there were a more structured way to browse it though. alphabetical is doable, but doesn't link concepts very well together. alphabetical plus following related links (see also, etc) works a bit better, but that has its own problems. Heh, I'm being characteristically lazy and I want a tutorial that covers everything in there :-). I doubt if even Knuth covers everything in there and if he were to try, he'd need the immortality pill from The Myth.

visayan on mailing lists (KLUG)

I see that sacha is looking into learning bisaya. I don't know why that would be, it's not like she's not got enough on her plate already. It's certainly a good thing, learning languages is good.

I hope, though, that it's not so that she can decipher things like this. And that she doesn't try to learn things from that and similar posts on some mailing lists. Almost all visayan posts on that list are like textingese in tagalog or english, essentially illiterate. That's not to insult the poster, that's just students communicating lazily in whatever language they use. It's not their fault, it's cultural. There are wonderful, literate visayan posts somewhere on the Net, i'm sure, but they're not on mailing lists populated by students who don't care enough about their posts to try to spell correctly.

Not that spelling is an easy thing in visayan. There's no given orthography, and the examples in sacha's post feel sort of archaic, although that's better than the illiteracy that passes for communication these days). Sounding archaic isn't a big deal though. Visayan varies from place to place, people will just assume she learned visayan with people who spoke classical visayan.

National Geographic Pics

Huh, in the preceding link to Google Video I wasn't quite sure whether it was a *good* way to waste bandwidth. So there's no qualifier there. On the other hand:

Pics from National Geographics and
ScubaDiving.com cannot possibly be considered a waste.

Lots of other good photo sites out there. for instance, APOD, Astronomy Photo of the Day.

All those sites better than webshots or screensaver software that installs adware and spyware on the computers of the gullible. I think so anyway (wouldn't really know, don't use windows anymore, maybe there are webshots or screensaver freeware that aren't evil).

Google Video

Huh, I see that Google Video is up. What a way to waste bandwidth:

Asian Backstreet Boys
How men screw up romance

lots of useless stuff there too. i wouldn't be surprised if there's some pr0n there though, somewhat like that smart telecom's post your phonecam pictures here site (too lazy to find the link).

tiger

Try ruby not working

I saw a link to Try Ruby! (in your browser) and thought, why not.

It's not working for me though. After the first few things to try, I get to the "Jimmy".reverse example and I get:

>> "Jimmy".reverse

<?xml version="1.0" encoding="iso-8859-1"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<title>500 - Internal Server Error</title>
</head>
<body>
<h1>500 - Internal Server Error</h1>
</body>
</html>


PHP already has most of what I need and I'm interested in trying Win Binder, a native Windows binding for PHP. I don't know what that's like yet, but if it's reasonable, then I may use that to write some quick windows programs (anything to get away from the horror that is Visual Basic).

Hay, so much to learn, so little time. I need to delegate.

swsusp very useful

I was on a long bus trip recently and I decided to see if the new laptop supported swsusp. I had given up on swsusp on a previous laptop (also a Winbook, but a slightly older one, since sold) because it didn't work there. It would try to save and then would fail on trying to halt mysql. Or if i stopped myself myself, it would try to save, and then on restore it would try to save again (instead of restoring), and then give up and reboot (or something, it was confusing, which is why the account above is a mess).

I'm very happy that it now works on the new laptop though. I've got three batteries for the laptop. They only last around 45 minutes at the full 1.7Ghz AMD speed (something like 2300-2400+ intel equivalent, i think), but in combination with cpufreq and setting the speed to the slowest CPU frequency available (around 600Mhz AMD, so maybe around 800+ intel equivalent?), the individual batteries last around 1.5-1.7 hours.

But I didn't want to shutdown and swap batteries since starting up takes some time and all of that time is at 1.7Ghz, eating battery life. Swsusp is a lifesaver since, when battery gets low and the laptop starts beeping, i just run a script that tells the laptop to suspend. It writes its status to swap. The laptop turns off. I swap batteries. And when I turn the laptop on it restores from swap, switches to 600+Mhz automatically and gets back to X in far less time than it takes to boot.

I think I'm going to swsusp all the time now, even when not trying to save battery, just because it starts up so much faster when restoring.

A constant state of insecurity

Another link from schneier has:

As a security professional, my friend often attends security conferences and teaches security classes. She noted that the number of passwords she collected in these venues was higher on average than in non-security locations. The very people who are supposed to know more about security than anyone appeared to have a higher-than-normal level of remote access back to their companies, but weren’t using any type of password protection.

A Constant State of Insecurity

Good God.

On the other hand, I admit that I do use several levels of insecure passwords for free but insecure web based services. Any service that requires registration for free access will get one of my fake (I lied about everything, including what country I'm supposedly from [I don't know where Anguilla is, but it sounds pretty]) identities with my lowest level password (I don't mind giving it away to anyone, although I don't actually post it on a blog or anything either :-).

Maybe some of those people were doing something like that. On the other hand, well, if companies are exposing FTP or POP3 on the internet (or maybe even ssh with their users having the same ssh password as their POP3 password), well, this is depressing my donkey. I wonder if anyone learns anything at those security seminars. Maybe they're Windows security seminars and the lessons have to do with which buttons to click to turn the firewall on. A lesson that is instantly forgotten because the listeners are Windows security professionals.

Schneier this week

Schneier in the current installment of Crypto-Gram has some interesting links. This month's installment seems very good. More interesting than previous months. Although they're all worth reading.

He gets something wrong though about the manila times and profiling of terrorists (male, between certain ages, cellphone, uneasy). Maybe that's just because he needed a short tagline. The article attributes the profile to the NCR Police Office (from which stupidities like this are expected).

The article certainly doesn't sound as stupid as schneier's comment makes it sound. That there is some controversy in the police organizations about the usefulness of the stereotype would normally be a good thing. Except that might just be normal intra-governmental infighting with no good reason behind it.

In the same issue of Crypto-Gram though are a whole bunch of links related to the Sony DRM rootkit (heh, it can be used to avoid WoW spyware (what's that doing, maybe just checking for piracy and such?), a trojan uses the Sony rootkit/trojan to hide, Sony has given up on it and Microsoft will detect it and remove it. Heh.

And There's an interesting weblog on malware from F-Secure, found it linked to from Schneier. I may look at it every day now. Or maybe add it to an RSS aggregator, if i can figure out how to do that :-) Huh, madali lang pala, using reader.google.com. Too lazy to try other RSS readers.

Hmmm, now to figure out how to get a feed from TechScene (kind of bass ackwards, since I could just get feeds direct from the people, but I'm lazy or too busy at work to remember too many feeds or URLs, or both).

There's also a link to an article discussing the prank article on aluminum foil hats previously discussed here. It looks like some heavy handed humor back and forth, but I'm not so sure about that zapatopi link. Some people are nuts about this stuff, and he might be one of them. He *does* have a book on practical mind control protection with aluminum foil beanies but I haven't read that, so I don't know if it's just a relatively long lived prank feeding on the paranoids and taking their money :-).

Lots more articles, all of it interesting. Every 15th of the month (well, 16th in the Philippines) is a huge time sink (but only about 33% a waste of time) because of this monthly email :-)

Aluminum helmets *help* the government spy on You!

although on average all helmets attenuate invasive radio frequencies in either directions (either emanating from an outside source, or emanating from the cranium of the subject), certain frequencies are in fact greatly amplified. These amplified frequencies coincide with radio bands reserved for government use according to the Federal Communication Commission (FCC). Statistical evidence suggests the use of helmets may in fact enhance the government's invasive abilities. We theorize that the government may in fact have started the helmet craze for this reason.

On the Effectiveness of Aluminium Foil Helmets:
An Empirical Study

IVRS must die

I see that Edong (he also has a "Ka", but I'm not on that side of the political spectrum) has a post on IVRS (interactive voice response system.

IVRS was all the rage in the US when I was there. It saved labor cost because the operator could be partly replaced by a machine voice. Instead of hiring 2 or 3 operators, a company could keep one receptionist/operator and fire the rest.

I can't stand IVRS though. Whenever I hear automated voice prompts I immediately hang up.

It's not the impersonality of it. I work with computers, I sort of like impersonal. What I can't stand about IVRS is the inefficiency of it, and the fact that, if it's to be used efficiently (memorize the call tree and type the digits in directly instead of waiting to hear the prompts), *I*, the customer, have to expend extra effort to remember. I have enough trouble memorizing my name, some days. I don't need to buy from a company that forces me to either waste my time listening to a ridiculously long call tree. And even if I wanted to memorize the direct route to where I want to go, no company I'm buying from is important enough to spend that much memory real estate on. Even if I had that real estate to spare. Any company that tries to communicate with me through IVRS is (1) going to lose a sail, (2) going to earn a rant, either in person, when I get to talk to someone in management, and/or a blog entry, for being more concerned with cost than customer experience. If they want to save cost, let them relocate their call center to the Visayas or Mindanao, salaries there are 1/2 to 1/3 what they are in Metro Manila and it'll help the economy and decongest the capital.

Which reminds me, is it still Metro Manila or did that go out with the Marcoses? No one mentions the Metro much anymore. Although there's still that extra M in MMDA.