While reviewing some code today, a principle of software design somehow distilled itself to clarity in my head.

When designing your system, think of every major system¹ upon which your own system directly depends² as a bug.

By "think of it as a bug," I mean that sooner or later, you are going to come to truly hate this dependency. It won't do what you want, or it will turn old and crufty, or it will get outdated, or your system will outgrow it. Perhaps it already stinks. And therefore, think about what you are going to have to do to take it out and replace it with something better, and possibly not even having similar API's.
Yes, you should have your code sufficiently factored and modular that such a replacement will be minimally invasive. But more importantly: if that replacement requires any change in the API's³ by which the outside world is using your system, then there is something wrong with your design. Stop and fix that immediately.

¹Both external dependencies and major subsystems of your own code. Both will suck in time, I promise you.
²If the systems upon which you directly depend have done this properly, you don't need to worry about your indirect dependencies. If they haven't, then you should consider replacing them now, because you are obviously dealing with the work of madmen.
³Or UI's, if your software is at the top of its software stack. UI's are just API's for communicating efficiently with humans. (Or perhaps API's are just UI's for communicating with computers?)

(FYI: The following entry is going to be much more technical than most of what I post. Anyone who doesn't care about code or data serialization can pretty much hit "next" right now.)

A few days ago, Google open sourced one of its key data serialization formats, protocol buffers. There's already been some chat on how they're similar to or different than other wire formats, but I thought it would be useful to post some useful tips I've come across over the years about how to make them do useful things.

Don't expect any deep insights into computer science here, just a few notes about working with these libraries.
( Read more...Collapse )

My former boss just published a great little article about things you need to know when building a search engine. It's chock-full of some really excellent advice for anyone building any large-scale computer system. For example:

Ah, but SCSIs are hot-swappable, you say. Get over it. Remember, no colo. You cannot afford it and you don't want it. So if you're worried about disk failures since you picked your disks out of a Dumpster, then my advice is don't screw the covers onto your machines and don't use four screws per disk. This makes IDEs pretty easy to repair, but certainly not hot-swappable.

I do sometimes miss working with Anna.

Media companies are getting antsy about Web companies, as you've probably heard. At a recent conference, various representatives of the media talked about this. The quote that caught my eye was:

"The Googles of the world, they are the Custer of the modern world. We are the Sioux nation," Time Warner Inc. Chief Executive Richard Parsons said, referring to the Civil War American general George Custer who was defeated by Native Americans in a battle dubbed "Custer's Last Stand".

"They will lose this war if they go to war," Parsons added, "The notion that the new kids on the block have taken over is a false notion."

I wonder if Parsons is aware of how that war ultimately turned out for the Sioux?

I've spent a lot of time in the past few years reading resumés and interviewing people. A really surprising fraction of those resumés have been really lousy; either uninformative, or full of meaningless junk, or just plain illegible. I tend to throw those out. On the other hand, good interviews with competent people leave me in a better mood all day... so in the interest of getting more of those in the future, I'm going to post a couple of notes on good versus bad resumés. (OK, to be honest: In the interest of not having to slog through any more piles of really bad resumés. It makes me feel like my brains are going to leak out of my ears.)

( Read more...Collapse )
Disclaimer: This is not the official position of my employer, or anyone else except me. I offer no guarantee whatsoever that if you follow this, you will get interviewed, hired, even noticed by a potential or current employer, or even not be shot at by them. Nor is any warranty, express or inferred, offered in conjunction with the reliability or usefulness of this advice.

"We're engineers. we don't always need to know why black magic works, we just sacrifice the chickens and hope for the best."

Yup, one of those nights.

Sometime in the past month, there seems to have been a shift of sentiment in our government, and it's finally become allowable to admit certain things, like the fact that the war in Iraq was an ill-conceived operation, based on entirely unrealistic ideas, and has ended in almost total failure. Of course, this doesn't quite reach to the highest levels - Bush still makes speeches saying things like "Iraqis are taking control of their country, building a free nation that can govern itself, sustain itself and defend itself. And we're helping Iraqis succeed" - but there's a remarkable willingness of officials involved to finally admit that we're leaving in the foreseeable future, having achieved little or nothing of the lofty goals which were touted at the start of the war. This article from the Washington Post is a good example; lots of quotes from various people, mostly speaking on background but unusually candid.

At the same time, there's no conjoint movement to cut back on the encroachments into American civil rights over the past few years, nor do I expect there to be one so long as this administration continues; people have come to expect continuous surveillance, background/credit/affiliations being easily investigated, in cases tagged as "relevant to national security" things like the detention of people without any of the protections we would normally expect in civil society. Recent more subtle changes seem to assist that; for instance, US cell phones are now required to transmit GPS locations (for emergency services, of course...), and Nokia's latest mobile services API makes it possible to write server-side applications that use that location information - without any software being installed on the cell phone. (Nokia isn't the only one, of course; that's just the first example I found to hand) I'll let the technically minded among you imagine a couple of uses for that; it's sort of a fun exercise, if you think about it as though you were a police officer, a private investigator, a marketer, or simply an armed robber.

I'm remembering why I deeply loathe having Windows boxen in the house.

OK, question for those of you familiar with this sort of thing: I've got a box that was, until recently, in use as a DVD player, media station, etc. Apparently in the very recent past, nVidia, as a result of pressure from Macrovision, pushed a new version of the video driver which completely breaks DVD output, causing any attempt to play a DVD to fail with a copy-protection error. Searching a bit on the net found this out - they didn't bother to call anyone's attention to the fact, and they just pushed the drivers as part of their automatic updates. It simply requires that one roll back to the 40.72 version.

Unfortunately, doing this seems to have failed utterly; despite the installer claiming to have succeeded, Windows does not believe that there is any driver at all, and the DVD problem is persisting.

Does anyone know a solution to this short of nuke & pave? Or alternately, would anyone like to buy a nice little Shuttle box with a 2GHz Athlon and a gig of RAM, so I can replace it with a Mac mini?

Can anyone point me at a reliable source of information about the activities of frienditto, and if at all possible to their terms of service? (Their site itself is down right now, answering 403, and I'm not certain if this is because they took the site down or if it's a response to a DDoS) Please respond by e-mail.