Geoff Arnold

Bringing up ZFS on my Ferrari

As I mentioned, I wanted to check out ZFS now that it's finally available in the latest Solaris build. My plan was simple: to upgrade my Ferrari to Nevada B27 and then "blow away the Ubuntu partition and create a couple of 10GB partitions" to test ZFS. Well, it wasn't quite that simple.

On Monday I borrowed a B27 DVD from a colleague and upgraded my Solaris partition. This went just fine, although I did run into a fiddly little xscreensaver bug that meant I had to snarf the B28a version of the Xorg bits. Never mind: I was now ready to repurpose that 20GB Ubuntu partition. But how? Solaris format/fdisk wouldn't touch it. I booted up a Ubuntu LiveCD and used Linux fdisk: this let me change the type code to 0xbf, which is Solaris2, but Solaris still wouldn't see it.

It turns out that Solaris only recognizes one primary Solaris partition on a drive; you can't have more. So on Tuesday I rebooted the Ubuntu LiveCD and used fdisk to delete both the Solaris and Linux partitions (leaving WinXP untouched). I then created a new partition, and reinstalled Solaris from scratch; I sliced up the partition as 20GB root, 1GB swap, two 10GB slices for ZFS, and the rest in /export/home. Of course I now had to customize the system the way I like it, so I downloaded a ton of stuff, went home, and got things working during the commercial breaks while watching House.

Finally this morning I was ready to test ZFS:

zpool create -f test c1d0s5 [the -f flag because the Solaris installation had put a UFS filesystem on the slice]
zfs create test/tfs
cd /test/tfs

Verdict: if you want to experiment with ZFS, it's a lot easier on a desktop machine where you can simply plug in another disk. You can use a laptop, but the chances that your disk layout will be appropriate are pretty slim; you should be prepared to repartition your disk and reinstall. Once you do, it all just works - kudos to Jeff and the team.

OK, next step is to try mirroring:

zpool create mtest mirror c1d0s5 c1d0s6
zfs create mtest/tfs
cd /mtest/tfs

Assuming I can free up a few hours on Monday...

Like most of us in Sun, I've been waiting for ZFS to arrive. Now it has. So on Monday I plan to update my Acer Ferrari 3400 laptop to Nevada build 27. Right now it's set up to triple-boot Nevada (50GB), Ubuntu Linux (20GB), and WinXP (10GB). I'm going to blow away the Ubuntu partition and create a couple of 10GB partitions which I hope will be sufficient to let ZFS show its paces. (However if anyone has a tried-and-tested laptop configuration for demoing ZFS, I'd welcome a link. No point in reinventing things.)

Essential reading on the Sony/BMG Rootkit fiasco

Here's a detailed account of the incompetence of Sony/BMG and First 4 Internet, the cowboys who wrote the brain-dead rootkit masquerading as DRM (digital rights management). From Mark's Sysinternals Blog, the bottom line: "Instead of admitting fault for installing a rootkit and installing it without proper disclosure, both Sony and First 4 Internet claim innocence. By not coming clean they are making clear to any potential customers that they are not only technically incompetent, but also dishonest."

And yes, they try the same trick on Macs too. Scumbags!

Starting conversations

Today we had our "geek-to-geek" meeting at StorageTek's Louisville, CO campus. Originally I'd planned to hold a brief and informal get-together for a few senior engineers; instead it evolved into a full-blown day-long colloquium involving more than 50 engineers (and two lawyers). Not all are in the picture; a few were slow getting back from lunch, and one was balanced on an SUV taking this photo! Of those attending, about a third were from Sun and two-thirds from StorageTek.

The schedule was tight: an hour and a half for a series of brief introductory presentations; a break followed by a lively Q&A session; lunch (of course); then break-outs from 1pm to 5pm. There were ten hour-long break-out sessions on topics ranging from product processes to storage virtualization. Amazingly, we stayed on schedule, for which kudos to the speakers. Obviously the presentations and break-out sessions weren't long enough to dive really deep into specific technical and business issues, but that was never the intent. The point of this meeting was simply to make connections and start conversations, and in this I believe we succeeded. The next step is to broaden the participation and link the discussions to the organizational and product planning processes. The work is just beginning....

Anyway, my thanks to all those who participated, especially to those who made particular travel-related sacrifices to attend, and to my colleague Richard for handling the facilities. I owe you guys.

P.S. Over dinner after the meeting, I was talking for a long-time StorageTek employee, and I mentioned that I was planning to blog about the events of the day. We discussed the fact that StorageTek, like many (?most) companies, had a tradition of secrecy, even over minor matters. For some, Sun's open style - "living life in public" - is likely to be a culture shock. So this evening after I'd drafted this blog entry I applied my usual test with more care than usual: Should I be concerned that a malevolent marketing type from HP might read my blog and use the contents to disparage Sun to our customers or anyone else? I don't think so.

My first laptop...

I've just bought my first laptop. No, of course it isn't the first laptop that I've used: over the last 13 years* I've worked with countless systems from various vendors (IBM, Toshiba, Fujitsu, Sharp, Apple, and Acer) in a variety of form factors (from desktop replacements to pocket-sized subnotebooks). But all of them - even the little Toshiba Libretto - were bought by Sun for me to use, and their hard disks have been filled with the documents, software, tools, and other materials that I work with on a daily basis.

As I spend a lot of my time travelling, I find that my laptop does double duty. Yes, it's a business tool - but it's also where I transfer digital photographs when my camera is full; where I store the music that I listen to on the road; and how I watch DVDs in the airport or in my hotel room. This is simply a practical matter: I'm hardly going to carry two laptops with me, one for work and one for personal use. But as a result I've found recently that more and more of the hard disk space on my laptop was being occupied by personal materials - music, videos, photographs, DVD projects - that have nothing to do with Sun.

So what was I to do? I'm committed to helping my colleagues to make Solaris 10 an excellent laptop OS (for which I use my Acer Ferrari), but I have to recognize that my personal multimedia data is tied to Apple's iLife application suite. So I started to think about getting myself a laptop for both work and personal use. This principled approach was nudged along by practical considerations: my existing PowerBook (a 12" 867MHz G4 with 640MB/60GB) was starting to feel really slow: start-up time for some of the big apps like NeoOffice/J was getting painful.

So this evening I visited the Apple store up the road and paid in cold, hard plastic for the first laptop I've owned: a 15" PowerBook (1.67GHz G4, 1GB/80GB). The migration tool worked perfectly: I strung a Firewire cable between the old Powerbook ("medieval") and the new ("silk"), rebooted medieval in target disk mode, and it sucked everything over - user info, documents, applications, network settings.

And yes, I did buy AppleCare extended warranty. Nobody's perfect.

--
* I think that the first was an IBM Thinkpad 700, back in 1992.

A heavy hand, or bad engineering?

Tom Yager of Infoworld recently reviewed Microsoft's first attempt at a 64-bit operating system. While most of the review was unexceptionable, a couple of comments really irked me:

"Windows Server 2003 Enterprise x64 Edition debuted with an anemic shelf of 64-bit apps. Skeptics will rejoice to learn that 64-bit Windows isn't load-and-run compatible with many, if not most, 32-bit Windows applications.... Windows x64 runs 32-bit applications stably or not at all; it won't allow an incompatible app to install or load. This is neither Microsoft's heavy hand nor bad engineering. It is genuinely impossible to run a great many 32-bit applications directly on AMD64 and its Intel derivative in pure 64-bit mode." [My emphasis.]

Now this last point is simply false. As I replied to Tom:

"I'm typing this email into Mozilla Thunderbird on an Acer Ferrari (AMD Athlon 64) laptop, running Solaris 10 in 64-bit mode. The Thunderbird executable that I'm running is 32-bit:
/usr/local/lib/thunderbird-1.0.2/thunderbird-bin: ELF 32-bit LSB executable 80386 Version 1, dynamically linked, stripped
(It's the same binary that runs quite happily if I boot into 32 bit mode.) However the "ls" command is 64-bit:
/bin/amd64/ls: ELF 64-bit LSB executable AMD64 Version 1, dynamically linked, stripped
So far, I personally haven't encountered a single 32-bit Solaris application that won't run under 64-bit Solaris."

So how come Solaris 10 can mix and match 32-bit and 64-bit processes* while Microsoft can't? As my colleague James Carlson said, it's:

"... mostly because of a great deal of effort that happened almost a decade ago when we originally switched to 64-bit kernels, and continuing work since then to make sure everything just works right.... I'm sure that Windows users hope that one day MS approaches that level of maturity."

--
* OK, it's true (as James pointed out) that Wine doesn't work (yet) on Solaris 10. However I don't really think of Wine as an "application". I really can't think of any 32-bit Solaris x86 application that won't run in 64-bit mode - but if there is, I'm sure the blogosphere will know.

Political shellscripting

Read Craig's blog (including comments). What a kick-ass community creation! (Even better if the entire CVS history was available, Craig....) Now buy the t-shirt. W00t!

How far we've come

Earlier today, I posted a little piece about a classic children's book on computing. When I got home, I found in my inbox an announcement of the latest edition of The Edge. It begins with a simple and thought-provoking assertion:

One aspect of our culture that is no longer open to question is that the most significant developments in the sciences today (i.e. those that affect the lives of everybody on the planet) are about, informed by, or implemented through advances in software and computation.

The piece that follows is a conversation with J. Craig Venter, Ray Kurzweil and Rodney Brooks on biocomputation. It's fascinating as always. The Edge has become essential reading; I highly recommend it. In the meantime, I've just re-read the Ladybird book on computers from 34 years ago. Hmmm.

How It Works...The Computer

Someone has scanned in all the pages of the 1971 and 1979 editions of the Ladybird book How It Works...The Computer. This is wonderful stuff. I remember using the 1971 edition to explain to relatives (elderly, young, and just plain confused) what it was that I did for a living; I also bought the 1979 edition for my son, Chris, who was five at the time (and a voracious reader). Both pictures and text are priceless.

(Via Boing-Boing.)

Somewhere, an Apple engineering manager is committing seppuku....

I just installed the latest version of Apple's iTunes on my PowerBook, along with an updater for the software on my iPod. Perhaps I should have read my horoscope first, or consulted the i ching. Whatever the reason, when I started iTunes it showed me an empty library, rather than the 18GB of music that I have, and it also warned me that my iPod was associated with a different library, and did I want to erase it and copy my new (empty) library? Aargh! No, cancel, quit, unplug iPod, back away from the keyboard really slowly.....

From browsing the Discussions section of Apple's support web, it seems that a number of us have had this problem with iTunes 4.9. (See, for instance, the thread entitled "iTunes 4.9 - lost library - please help".) The only remedy seems to be to back up the old Library files (before iTunes can mess with them), let iTunes build a new library, and then import all of the existing music. This requires that you have enough space for two copies of all your data, so I'll have to wait until I can get home to use a FireWire drive. (I only have 7GB of free space on my 60GB PowerBook.) And you'll also wind up losing all of your playlists, ratings, and "last played" information, which is a huge pain.

UPDATE: A Mac user called Dave Garrett just posted the following workaround:
1. Open your Music folder/iTunes folder/Previous iTunes Libraries folder
2. Re-name the file named iTunes 4 Music Library to iTunes Library
3. Drag your newly named file iTunes Library into your iTunes folder, replacing the iTunes Library that the new iTunes had created.
4. Voila.

Seems to work for me.

On the guilty pleasure of reading a really bad book

This is going to be long - skip it if you're in a hurry.

Today I was at Sun's Santa Clara campus for an all day meeting of the DEs. We finished up on time, just after 5pm. The last session had left me feeling exhausted: a 20 minute presentation stretched to a relentless 40 minutes, followed by a complicated debate. I felt like a drink and some food (my body is still pretending that it's on East Coast time), but 5:30 seemed a bit early to eat. I therefore decided to drive over to the nearby Micro Center store and do what geeks do: ogle hardware and software. There's a passable Mexican restaurant in the same plaza (the Mexicali Grill), and I thought I might find a book or magazine to read over dinner.

The store was very quiet, and the few customers seemed to be lowering their voices as if they were in a library. I found nothing of interest in the Mac section, or the PDA accessories, or the magazines, or even the discount DVDs. (I wonder who buys those boxed collections of 20 horror movies from the 1950s, not to mention The Neverending Story Volume 2.) And so I made my way to the book section.

It just so happens that I've been discussing the possibility of doing some work with Sun's Network Storage Division, the group that sells such products as the StorEdge 9990 array and the QFS file system software. I'm quite familiar with our products, and I used to work on distributed file system software such as PC-NFS, but there are parts of the storage business that I know little about. So when I came across a large book about storage systems, I started browsing it. The table of contents looked promising. I checked the price: $5.99, reduced from around $50. I put this down to overstocking, bought a copy, and went off to have dinner and a bit of a read.

By the time I'd finished my salad, and a Silver Bullet margarita, I realized that I had acquired a Really Bad Book. It was weird: the organization was plausible, and by speed-reading I could sustain the illusion that it more or less flowed and made sense. But if I slowed down and looked carefully at individual sentences, they were gibberish: ungrammatical, rambling, cliché-ridden, and full of non-sequiturs. At first it was annoying, but by the time I reached the end of the first chapter it had become simply hilarious. Some examples, with original punctuation:

"The corollary, or trade-off to this condition, is the economics of speed and capacity to price."

"Within the SAN, these operations become more logical and have to coexist with other servers that share the fabric network and devices connected."

"Finally, as the sophistication of the centralized mainframe computers was downsized, the capability to house larger and larger databases demanded the deployment of the database server."

It goes on and on like that. Verb agreement is a matter of happenstance; dereferencing a pronoun should only be attempted by trained professionals. At times we seem to enter an Alice in Wonderland world of topsy turvy relationships:

"The most critical element of performance for a business application is its availability to its own data."

And sometimes a sentence seems to have been assembled by a surrealist playing with magnetic fridge poetry pieces; here's a final, glorious example.

"Unless the hardware and firmware release levels are inventoried and tracked in conjunction with the network, the NAS systems become unassociated storage servers unbound to the confines of the network in which they operate."

I cannot shake off the image of a row of NFS servers growing large, colourful wings and fluttering away like butterflies towards the setting sun - unbound, free of the confines of the network!! Excelsior!!!

[I've done my best not to identify this book or its author. If you figure it out, please keep quiet. There's no point in stirring the pot.]

Open everything!

Congratulations to my colleagues who were involved in today's OpenSolaris launch. The biggest single OSS release in history! From Sun!! (A tip o' the hat to Rob Gingell, wherever he might be.) But surely Microsoft, IBM and HP aren't going to take this lying down; they're not going to give in without a fight - are they? C'mon, you guys: I want to see OSS releases of Windows XP, z/OS*, and VMS! And... oh heck, why not throw in OS/2 Warp as well - just for old time's sake? (But don't bother with AIX or HP/UX, because... well, I'm sure I don't need to spell it out.)

And why stop at operating systems? Earth to Larry (probably in his jet somewhere): it's time to open source Oracle before IBM gets around to opening up DB2. You know it makes sense! In fact, I bet there's more lines of code in that sucker than everything else put together!!

--
* I want to try running z/OS on my laptop. A quad-boot setup with Solaris, Linux, z/OS, and WinXP: that's a configuration to really get a geek's pulse racing....

On not knowing whether to laugh, cry, shrug and walk away, or gaze in fascination....

Dion Hinchcliffe is embarking on a project which I think I'm going to watch with morbid fascination: Taking Stock of Web Service Description. Specifically, he's going to put up a simple order entry web service, and publish a description of the service in a number of different candidate service description languages (SDLs). The mind-boggling part is the list of candidate SDLs:

"The list of SDLs to try to use is: WSDL 1.1, WSDL 2, NSDL, SSDL, WRDL, RSWS, WADL, Resedel, SMEX-D, RDF, RDF Forms, OWL/OWL-S, WSML, and WDL."

That's fourteen different languages. Plus he's going to explore how to use these with three different programming languages: Java, C#, and Ruby.

Back in the Craig McMurtry blog entry that I cited recently, he wrote: "One must grant though, that a primary and very good idea behind Jini lives on in Indigo. That idea is that there should be an excellent, simple programming model usable across any kind of networking infrastructure. In the years since Jini, though, we have learned a lot about how NOT to design those programming models, and those lessons suffuse Indigo." With all due respect, Dion's experiment demonstrates pretty clearly that there is no consensus whatsoever on the "how not to" question. It also seems to confirm what I said: that one size will not fit all, and that we're going to need a variety of technical solutions ranging from Jini to WRDL and beyond.

Jini, Indigo and the "Highlander Fallacy"

In Craig McMurty's blog entry that stirred up discussion of Indigo and Jini, he mentions a piece by Don Box, in which "the four fundamental tenets" of service-oriented development are laid down. Craig further asserted that 'The "services" that one can construct with Jini do not conform to any of those four tenets'. Really? Well, given how badly Craig seems to have misunderstood Jini, I decided to go back to the source and see what the good Mr. Box has to say for himself.

So what exactly are Don's four fundamental tenets?

1. Boundaries are explicit

2. Services are autonomous

3. Services share schema and contract, not class

4. Service compatibility is determined based on policy

Plausible? Well, the first two are unexceptional, and describe Jini services quite nicely. I suspect the devil may be in the details. The third seems unduly coupled to a particular technology and language; if we replace "schema" with "interface" and "class" with "implementation", it feels OK - and describes Jini very well. And the fourth? I have no idea what it means, but on the surface it seems to conflict with the second (autonomy) and to overlap with the third. Perhaps we can resolve these conflicts later on. Let's dig into each of these ideas.

1. Boundaries are explicit: Back in the early days of distributed computing, it was considered a good idea to hide the boundaries, to make remote resources look just like local ones. This is what gave us remote file systems and early RPC schemes; in extremis it gave us things like Locus. Over time the wisdom of folks like Peter Deutch and Jim Waldo cured us of this. These days we all believe in explicit boundaries.

   Box, however, introduces what seems to me like a non-sequitur: "Because each cross-boundary communication is potentially costly, service-orientation is based on a model of explicit message passing rather than implicit method invocation." Now this is bizarre. He's saying that because an interaction may be expensive, we have to describe it explicitly at the lowest level. Even if one is performing a higher-level pattern, such as a method invocation, one is required to unpack it by describing the individual message patterns and how they are correlated. While sometimes this may be appropriate, it hardly seems fundamental. For a start, it conflicts with the notion of autonomy. Why should a client and service, mutually consenting adults, be required to describe their interaction in such detail in advance? Why could they not negotiate a mechanism on the fly, using available resources?

   Box goes on to say, "the fact that a given interaction may be implemented as a method call is a private implementation detail that is not visible outside the service boundary". But this is disingenuous. How detailed, how explicit is the message description? Either we give up on type (or schema) based notions of compatibility, or every interface requires a detailed description of request and response messages which will make the "private implementation detail" instantly public (and will significantly constrains the ways client and service can interact).

   A final thought on boundaries: while they may be explicit, they are not necessarily static. Refactoring is a fact of life. In practice this means that software components will be constructed in such a way that potential boundaries are declared as such (using, for instance, the remote interface in Java), and services will be assembled and composed in terms of such interfaces. Describing a service in terms of a low-level message exchange pattern then becomes one technique among many for defining service boundaries, and one that is generally effected by a design tool of some kind. It's an implementation artefact, not a design principle.

2. Services are autonomous: Box's account of service autonomy is excellent. He discusses version skew, the dynamic nature of the aggregations that comprise a system, the effects of unintended usage, partial failure, and security. (He could also have said something about dynamic discovery, self-healing, the use of patterns such as broker and facade, and the use of techniques such as introspection to perform dynamic adaptation.) There is nothing here that a Jini enthusiast would disagree with.

3. Services share schema and contract, not class: Earlier I wondered if this referred simply to a separation between interface and implementation. This understates Box's position. He writes, "services interact based solely on schemas (for structures) and contracts (for behaviors). Every service advertises a contract that describes the structure of messages it can send and/or receive as well as some degree of ordering constraints over those messages." And he also says that "the contract and schema used in service-oriented designs tend to have more flexibility than traditional object-oriented interfaces. It is common for services to use features such as XML element wildcards (like xsd:any) and optional SOAP header blocks to evolve a service in ways that do not break already deployed code". So what we are dealing with here are relatively simple interfaces, evolving slowly, with weak consistency and ad hoc extensions.

   Now there are plenty of situations where this kind of approach is just fine. As Box notes earlier, it's the kind of thing that made Amazon.com's services the success that they are today. But for other applications, this kind of thing is hopelessly inefficient or inflexible. For the vast majority of intranet services, for instance, the most common way to invoke a service isn't going to be by parsing its description and constructing a new client stub ab initio: it's going to be by invoking a copy of the stub code generated when the service was created. And if you're going to do that, why not take advantage of it - not just at compile time, but dynamically, so that the interactions with the service can reflect the characteristics of the deployed service? As for the suggestion that one needs more flexibility than "traditional object-oriented interfaces", mechanisms like introspection allow for powerful and robust run-time flexibility, without sacrificing type safety.

   It seems to me that the fundamental principle involved here is the one I alluded to earlier: establishing a clear separation between interface and implementation. How we do that seems to depend on the application and context; there is no reason to believe that one size fits all.

4. Service compatibility is determined based on policy: It turns out that what Box is talking about here is the distinction between structural compatibility and semantic compatibility. The latter is familiar and unremarkable: XML schemas, class signatures, on-the-wire encodings, that kind of thing. But what Box is interested in is "[s]emantic compatibility... based on explicit statements of capabilities and requirements in the form of policy". He gives a few hints about what he's thinking of, but at this stage I think it's safe to say that he raises more questions than answers. Suffice it to say that this whole area is in its infancy, and it's not an immediately important issue. By the time we figure out exactly how to express such things (OWL-S?), how and when services get to " apply simple propositional logic to determine service compatibility", and exactly what customers want to use this for, our initial expectations are likely to change significantly. There's certainly nothing here that is incompatible with Jini.

So what's the verdict? Was Craig correct to claim that Jini doesn't conform to Box's "tenets"? Unambiguously not: Jini absolutely conforms to every aspect of Box's "autonomy" tenet. How about the others? Well, it all depends on whether you read the interface or the implementation: whether you concentrate on the broad principle, or how Box chooses to implement it. Jini is entirely compatible with the broad principles, but not surprisingly it's not going to conform to Box's Procrustean Bed of message-based service patterns. But why should it? Box gives us no reason to believe that this technology is the only way to realize the principles. It's just the way Microsoft chose.

I've already discussed why Microsoft took this path, but it's important to recognize that it's not the only - or even the best - approach. There are more powerful, elegant, and efficient models for realizing the principles of service-oriented distributed computing, and Java and Jini represent such a model. For Box and McMurty to argue that only one technology fulfills these principles is self-serving, not to mention ahistorical. The wide range of problems to be solved, the variety of contexts in which the solutions must operate, and the kinds of constraints and requirements that must be met, are such that multiple technologies will clearly be needed. As Jim Waldo pointed out, the "Highlander Principle" ("there can be 'only one'") is a fallacy.

"Cluster" is so yesterday: try "flock". Literally

So this is what they're getting up to at my alma mater, as reported in Linuxdevices: "Researchers at the University of Essex are using Linux and tiny embedded computer modules to build fleets of unmanned aircraft that fly in flocking formations like birds, while performing parallel, distributed computing tasks using Bluetooth-connected Linux clustering software." Check out the project website for pictures, video, etc. And where are they going to present their first conference paper? Why, at the IEEE Swarm Intelligence Symposium.... [groan]

(Via Boing Boing.)

My next laptop for Solaris?

Tulip have announced a rather unusual laptop. It's based on the new AMD Turion CPU, the successor to the Athlon 64, so it should run Solaris 10 in 64 bit mode quite nicely. However what caught my eye was the limited edition Tulip E-Go Diamond version, which has some "unusual" styling touches:

"Tulip E-Go notebook inlaid with solid palladium white gold plates in which thousands of brilliant cut diamonds have been set. The quality is V.V.S. top-Wesselton and the total weight is 80.00 Crt. The brilliant cut diamonds are microscopic and pave set with surgical precision. This magnificent end result is possible thanks to the use of brilliant cut diamonds with a large variety of diameters. A unique square cut ruby has been set in both Tulip logos. For the Tulip E-Go diamond project, Marcel van Galen Design worked closely together with Design Department product engineering and Laurent de Beer Master Jewelry Designer. Consumer price € 283,000"

I can see it now - PC Magazine comparison shopping tables listing CPU speed, RAM, screen size, weight, battery life, and carats....

Open source, closed repositories, and snake oil

Over in the Register, Andrew Orlowski has a fascinating article entitled Torvalds knifes Tridgell about another bizarre outburst by Linus Torvalds. This time it's all about BitKeeper, the source code repository system. "Torvalds uses the pay-for proprietary software to manage the Linux source code (obliging other kernel developers to follow suit), but last week its owner, Bitkeeper CEO Larry McVoy, yanked the license, pushing Torvalds to look for an alternative. He's now going to write his own. For this inconvenience, he blames [Andrew] Tridgell", the genius behind SAMBA (the technology which finally killed my old PC-NFS product).

And what was Tridgell's crime? He wanted to reverse-engineer the BitKeeper protocols so that Linux developers could browse the repository metadata. This sounds innocuous enough - after all, BitKeeper's own website says that "Read-only users (people browsing the source, tracking progress, doing builds, etc.) still need a license but there is no charge for that license.", so it's not a question of money. Clearly there is something big at stake - something so important that McVoy is prepared to forego the prestige of hosting the Linux kernel repositories. According to Andrew Orlowski, "McVoy was adamant: 'sorry, we're not in the business of helping you develop a competing product.'" So that's it? The key intellectual property is in the protocols? That seems odd.

I had two reactions to this piece. First, why on earth is the acknowledged flagship product of the FOSS world relying on a proprietary, closed source repository - particularly one run by a guy who clearly has no sympathy with FOSS, nor any understanding of the related business models? I would (naïvely) have thought that BitKeeper would want to hang on to the data and proliferate clients like crazy. (A famous LBJ quotation comes to mind.) And second, what is it that makes BitKeeper so wonderful? Let's check out their web site. Truth in advertising? You be the judge:

Hardware costs: BitKeeper does not have this problem [of scale] because of its distributed model.... This model means that the hardware costs can be spread over a set of inexpensive PCs rather than a $300,000 SMP machine. BitMover hosts the Linux kernel repositories for thousands of users on a single inexpensive PC.

Human costs: An administrator is the person who makes sure that the hardware and the software is working, the repositories are backed up, etc. The distributed nature of BitKeeper removes the need for such a person.

Wow. Thousands of users on a single PC. No administrators. How cool. No wonder Linus was impressed. [That's sarcasm, in case you didn't notice.] I think that in the long term we'll see that Andrew Tridgell has done the FOSS community a service, by provoking Linus and Larry into falling out. Hopefully the community can create a better - and truly open source - repository. However I wouldn't rely on Linus to create it - he doesn't seem to believe in open source any more....

The dirty little secret of the computer biz

I spent several hours on Saturday replacing the CPU fan on my wife's computer. The old one had started making a noise like a vacuum cleaner that you could hear all over the house. It's an middle-of-the-road PC, a bland eMachines box with a ~900MHz Celeron. We talked about replacing it with a Mac Mini, but there's plenty of life in the old system and it seemed wasteful to replace it unnecessarily.

While I was disassembling the innards to get at the CPU, I took the opportunity to clean out the dust from the power supply fan and replace the video adapter with something a little more functional. When it was all back together, I ran some tests and spent a few minutes upgrading her copies of Firefox and Thunderbird to the latest releases. Nothing earth-shattering: the parts cost about $60 at CompUSA. The biggest challenge was bending the spring clip on the fan to fit more securely onto the tabs on the CPU's ZIF socket.

The point is, there's no way she could have done all of this stuff herself: it's just too complicated. A nice piece on the BBC website makes the point: "But all the people who called me had one thing in common: they were at their wits' end because they had bought computers after being seduced by advertising into thinking that they would be easy to use and fun, but had found them to be much more complicated than they had expected. And most importantly, none of them knew what to do or where to turn for help."

I've decided that in future I'll recommend that people get laptops. Not because they need the mobility, not because it's cheaper (it isn't) or more comfortable (most laptop keyboards suck), but because if when things go wrong, they can simply fold up the computer and carry it to a human being, to get help.

Tim, Rio, Jini, Rob

Over in ongoing · Java, the Grid, and Rio, Tim was "thinking about how you’d run a big distributed Java system as a service across a whole lot of networked computers". Dan Templeton pointed him at Rio, and I followed up with a link to the papers from last December's Jini Community Meeting in London. And I remembered a comment by Rob Gingell, adapting Santayana: "Those who do not use Jini are doomed to reinvent it."

AOL responds

AOL reacted pretty quickly to all the negative publicity about their AIM Terms of Service. The new language is much better. The power of the web, eh?

The new text (with my emphases):"You or the owner of the Content retain ownership of all right, title and interest in Content that you post to public areas of any AIM Product. However, by submitting or posting Content to public areas of AIM Products (for example, posting a message on a message board or submitting your picture for the 'Rate-A-Buddy' feature), you grant AOL, its parent, affiliates, subsidiaries, assigns, agents and licensees the irrevocable, perpetual, worldwide right to reproduce, display, perform, distribute, adapt and promote this Content in any medium. Once you submit or post Content to any public area on an AIM Product, AOL does not need to give you any further right to inspect or approve uses of such Content or to compensate you for any such uses. AOL owns all right, title and interest in any compilation, collective work or other derivative work created by AOL using or incorporating Content posted to public areas of AIM Products."

"Oh! I have slipped the surly bonds..."

"Oh! I have slipped the surly bonds of" RJ-45 connectors and CAT-5 wiring. Thanks to the folks at Sun's Beijing office, the internal WiFi on my Ferrari is now working. Only in 32 bit mode at this point, but I'll take it. Now for suspend/resume (he said hopefully).

[Since Broadcom doesn't release specs or source code for its devices, we're using the "ndiswrapper" technique, in which a Windows-style NDIS driver is wrapped in a little bit of magic to make it work like a Solaris driver. Wonderful what they can do nowadays, eh?!]

[UPDATED: Curses... foiled again. The drivers worked fine at the office earlier today, but when I tried to boot up just now to use my home network, I was unable to plumb the bcmndis0 interface; some kind of binding error. The only obvious difference was that I was running on batteries, but that shouldn't affect things. Oh well, more testing....]

[UPDATED: It turns out that it was inadvertent operator error: where the instructions said 43XX, I was supposed to use 4320 or 4324, depending on configuration. I have no idea how it could have worked yesterday. Anyway, 32 bit mode is working fine; I've tried the 64 bit drivers, but there are a number of issues to be resolved there.]

CEC - pulling it all together

I tried real-time blogging at last weekend's CEC (Customer Engineering Conference), using my Treo 650, but without a decent blog tool it wasn't really practical. I found myself wrestling with the web interface to MovableType rather than listening to the speakers - bad idea.

So here are my collected notes from CEC (slightly edited, definitely selective), followed by a few closing thoughts.

SATURDAY MORNING

One of the great traditions of CEC is the collection of video clips produced by various geo and functional orgs. It would be invidious to pick one as best, but the French piece - a Ken doll scaling the heights of a server to fix it, and earning the fulsome thanks of Barbie and her friends - got most applause. (But NZ had the best lip-sync.) And US PTS nailed the "piggybacking" joke perfectly. Best music (including alpenhorn) from Switzerland.

Next, Jim Baty & Hal Stern. Moving to utility model, refactoring business. Feels like it's 1995 - tectonic shift again. Key messages: Technology is cultural. Addressing the PE (principal engineer) role - align with DE model. Community is key - blogging, BOFs. At CEC: Engage - act - share. When you go home: Communicate - train - improve.

Bob MacRitchie - EVP GSO: Described evolution of sales model. Review progress of Project Genesis [reorg of sales, professional services, and field engineering launched 12 months ago]. Simplify, flatten, empower org. (I'd missed that the US sales headquarters is moving to Boston - most of our US sales are east of the Mississippi.)

Marissa Peterson - services: services revenue & gross margin are improving significantly

Jonathan Schwartz - who never uses sports metaphors - appears in a Dallas Mavericks shirt.

How do we grow? Sell more to existing customers, or steal other people's customers.

What's changed over 3 years?

• Sparc vs. Itanium
• Solaris
• Storage revival - 6920
• x86 - now #1 Opteron seller, more coming
• JES
• Utility computing & grid (compare with IBM OnDemand fiasco)
• Java on devices

In response to Q&A:

• How come the OSS community is never satisfied with Sun, while IBM can do no wrong? The GPL people will never be happy, but they're a minority. Second, we're going to open source everything - JES, N1, etcetera - and if IBM won't opensource WebSphere or Tivoli, they're going to be left behind.
• OSS grows revenue. The key is developers, and they have no money. (Jonathan has only one button on his blog - Download NetBeans)
• What about N1? The technology in N1 was right on; our mistake was in overestimating the readiness of existing data centers to buy in. They don't call them "brownfield sites" for nothing. On the other hand, the N1 technology is going to be critical to successfully creating a utility computing business

Greg Papadopoulos, CTO (via video): Computing becomes a commodity, but (network scale) computer systems aren't. Consequences: operational concerns dominate, scale matters.

Robert Youngjohns - utility grid: What we've done, where we're going. Great presentation - more material at the Sun Grid page.

SATURDAY AFTERNOON

SOA and Jini - Tom Barratt & Larry Mitchell: Nothing unfamiliar, just wanted to see how people were presenting - and reacting to - SOA and Jini. Basic background, ray-tracing demo. Excellent discussion, good questions, lot of interest.

N1 SPS/SJS App Server - David Ogren: Talking about AppServer 8.1 + N1SPS 5.0. We got what David called the "Fire and Brimstone" to "Nirvana" presentation... Plus a nice demo.

InstallFest and Demo Room: Lots of cool stuff in the demo room. Re-installed Solaris 10 on my Ferrari from the latest flash archive.

SUNDAY MORNING

John Loicano, SW EVP: Big emphasis on Solaris, tools, restructuring JES Suites (especially Identity Management with Identity Auditor) Tag-team with Juan Soto (SW CTO & MktDev) for a deep dive on leading with SW for opening new customers. Emphasized importance of Netbeans vs. Eclipse. (Netbeans nailed all the recent tools awards.) Impressive performance numbers on the new TCP stack. Great demos of Solaris 10 Predictive Self-Healing and Identity Auditor.

Mark Canepa - network storage: Data management is more than storage.... Industry survey, strategy, product overview. Nice discussion of synergy between Solaris 10 zones and 6920 virtualization. Head-to-head comparison against EMC. Plea for help in improving remote monitoring connectivity. Java Storage System - not a technology, but a JES-style busness model. [The idea is interesting; I'm not crazy about the name.]

John Fowler & Andy Bechtolsheim: network systems: John summarized NSG history & progress. Stunning benchmark numbers, unveiled Galaxy: 8 socket (16-way, with dual cores) in new 4U packaging. BIG fans. The dual cores are coming very soon - well ahead of Intel. Full product line from 1U 2 sockets up to 4U 8 socket. Also blades - but no compromise in performance. Blades will support virtualized SAN port sharing, will save huge dollars. (Low cost, low speed blades aren't cost-effective because of software licensing costs.) Will mix-and-match AMD and SPARC blades. Box design is dramatically future-proofed. Also mgmt sw and Nauticus (N2000) switches.Many early sales have been driven by customer solution and Blueprint sales. Seed units work well. Challenge: every sale is an audition. We can sell the boxes, need to make sure service will be able to meet the challenge.

David Yen, scalable systems EVP: What's the difference between NSG and SSG? Ultimately, competence in system packaging vs. competence in silicon. SPARC roadmaps. Lots of interesting stuff: I wish he'd skipped the umpteenth repetition of "how chip-level multithreading works" to spend more time on the new material. Oh, well.

SUNDAY AFTERNOON

Real World Cluster Grids - Tony Kay: Disambiguating "grid", all the way back to Foster & Kesselman. Important to match language with customer expectation - we say grid, they (may) say cluster, for example. Detailed discussions of HPC grids, especially oil & gas biz. Importance (or not) of various technologies: OGSA, Globus; cluster, MPI libs, network fabrics, file systems, specialized protocol stacks. Had a chance to talk to my former SunLabs colleague Bruce Daniels who's now in PS.

ZFS - Nolen Hayden: (Jeff Bonwick was sick: his director subbed for him.) Interesting to hear the issues that were uppermost in the minds of the customer-facing engineers.

Grids for Financial Services - Alec Muffett: Intensely, relentlessly, and amusingly pragmatic and iconoclastic. But you knew that.

FINAL THOUGHTS

• An excellent conference - kudos to Hal Stern and Jim Baty, and their team.
• I really regret that I had to miss the Monday morning session, especially Scott's talk.
• While I understand the traditional focus on the field engineering organization, I really think that CEC has turned into something that speaks to all of engineering. How we could achieve this while preserving the value to the field I have no idea....
• The openness of the whole event was remarkable. At lunch on Saturday, I asked Hal whether it was all bloggable. "Absolutely!" he said.

All monocultures are dangerous

In Internetnews.com, Dan Ravicher, executive director of the Public Patent Foundation (PUBPAT) is quoted as saying: "Open source is not about having five different operating systems, it's about everyone working together to create one rock-solid operating system."

Wrong. The last "one rock solid operating system" was OS/360. Dan is suffering from a grievous lack of imagination. This is like Pamela at Groklaw, saying "The FOSS community needs to face the world with a united face", and earlier "when [Sun] say 'the Open Source community'... they don't mean Linux. When I say 'Open Source community,' I do."

Open source is about collaboration. It's about groups (plural) coming together to work on stuff, and sharing the results. It's not a cult, not a political movement, not a utopian (i.e. unrealistic) dream. Above all, it cannot be about monoculture: one technology, one group with one leader, one license, one goal. All monocultures are dangerous: Microsoft Windows, Monsanto's 'Terminator' seeds, and influenza vaccine - even Linux if "true believers" have their way. I want more OSS operating systems, not fewer. I wish Palm would release BeOS to the world. (Yes, I know about OpenBeOS. I want the original.) I wish HP would post the full source of VMS for all to use. Competition is good. It's good for engineering. It's good for customers.

(For better coverage of these issues, check out Simon's blog.)

Quis Custodiet Ipsos Custodes?

El Reg just reported a major cross-platform flaw in 30 of Symantec's security products, including Norton AntiVirus 2004, corporate anti-virus apps and Brightmail spam filters. Of course the root cause is a system architecture which is so broken that it requires the use of antivirus software that is so tightly integrated that it becomes a potential source of compromise.

I've always thought that I understood the history - or at least the mythology - of how this came about. Cutler and crew knew (from their VMS days) how to make NT secure, but chip support, backward compatibility and performance "optimizations" did them in. They could have used Win31/DOS VMs to cope with the legacy crud, but it wouldn't have been fast enough. We're all living with the results today (even if we don't run Windows.)

I wonder how close this mythology is to reality....

On Apple, BMW, and Minis

Like many Mac users, I've dismissed talk of Apple's miniscule share of the personal computer market by (a) pointing out that many of those PCs are just glorified 3270s/VT100s/Wang word processors/cash registers, and (b) invoking the "BMW argument": what market share does BMW have - and does that stop them from being a really important, cool, desirable brand? So now Apple goes and releases a couple of down-market products, and various people are asking, understandably, "is Apple blowing its BMW model?". Frank Steele has a nice response: "Perhaps BMW could create (or purchase) a second brand that sold cars that were not quite so expensive. Maybe comparable in price to other cars, but maybe a little smaller, and fun. [...] But what could BMW possibly call such a company?"

(Via Oren.)

More Java cores than you know what to do with

After publishing a skeptical and rather petulant piece about Azul last October, El Reg decided to give Azul's CMO, Shahin Khan, his own soapbox this week. He certainly waxed lyrical "If you could count CPUs the same way that you count memory, some problems would simply become uninteresting and others would transform in a qualitative way. And completely new possibilities would emerge. [...] No need to plan capacity for each individual application. Let all of your users share a huge compute pool and plan capacity across many applications."

Well, maybe. Remember that Azul is planning to ship up to 1,200 cores in a single rack, but these core will be specialized Java™ engines. Now I'd love to see Java take over the world and remove the need for any other kind of operating environment, but for the next few years, while we're waiting for this brave new world, systems like Azul's are going to have to coexist with mundane Solaris and Linux boxes. In other words, it's a co-processor, an "applications accelerator". And ever since the days of "intelligent Ethernet cards" (anyone remember the 3C505?) I've observed that such co-processors are doomed to be overtaken by general-purpose processors. The only obvious exception is in the area of graphics. Not only are the specialized processors not that much faster than their general-purpose brethren; the cost and complexity of the software needed to manage the co-processor usually eats up all of the savings. In the case of the 3C505, I remember that the host driver to manage the on-board TCP/IP stack was roughly as complex as a TCP/IP stack!

Don't get me wrong - I think that multiple core are absolutely the way to go. Various companies - Sun, IBM, even Intel - are realizing that the best way forward is to simplify their pipelines to reduce the size and complexity of their cores so that they can stuff more cores on a chip. Designing around Java byte-codes rather than RISC ops doesn't save all that much.

Will Azul prove me wrong? I'm not holding my breath....

A blast from the past: CaveBear (a.k.a. Karl Auerbach)

Back in the late 80s and early 90s, when I was working on NFS, Windows Sockets, and other TCP/IP related stuff, I would often run into Karl Auerbach, network tools wizard and latterly ICANN member-at-large. Many of our encounters took place in the NOC during that strange, timeless period the night before the opening of each Networld+Interop show. The first Interop took place in San Jose, but as it grew, and merged with Networld, it moved to the Moscone Center in San Francisco and eventually headed off into the desert at Las Vegas. Over time it went the way of all trade shows, and puffed itself up into a content-free carnival, whereupon I stopped attending and lost touch with Karl.

Today a serendipitous blog chain led me to the following gem, reproduced in full:

CaveBear Blog: Sartre meets ICANN

I notice that ICANN issued a press release with the title:

ICANN successfully concludes Cape Town Meetings

Which makes me wonder: What would an unsucessful conclusion be?  Would the ICANN board and staff have to be trapped forever in the meeting room like the characters in Sartre's play No Exit?

It's an attractive proposition, isn't it?

An embarrassment of riches...

One of the joys problems with all of this cool stuff that we have at Sun is figuring out how it all fits together... or doesn't. Case in point: I was reading John Clingan's piece about Zones on an E25K, and I started to think about how one might use such a beast. Suppose one was running a horizontally-scaled load-balanced Sun Java System Application Server Enterprise Edition 7 2004Q2 (surely there must be a simpler name) configuration on a cluster of V880s. Can I rehost this in a collection of zones on an E25K? What works? What breaks? How much of my administrative model carries over, and how much has to change? (Everybody talks about ABI compatibility, but compatibility of administrative models is just as important. It's one of the major issues with Linux today, and it's bound to affect how we run Linux apps in Solaris x86.)

And that got me thinking about clustered data bases (we use the Clustra technology to support App Server failover), and from that to storage and file systems. (I'm an old NFS guy.) One of Sun's hidden gems is QFS (OK lawyers, Sun StorEdge QFS software), a massively scalable high performance file system. Although designed for (and mostly used in) high performance technical computing, it's getting a lot of attention in other applications, due in part to the symbiosis with SAM-FS (Sun StorEdge SAM-FS software), a policy-based archiving system. (Think SarbOx. Think Infinite Mailbox.) Do QFS and SAM-FS work in zones? I turn to the on-line documentation: Solaris Containers-Resource Management and Solaris Zones: "Mounting File Systems in Zones: Options for mounting file systems in non-global zones are described in the following table. Procedures for these mounting alternatives are provided in Configuring, Verifying, and Committing a Zone and Mounting File Systems in Running Non-Global Zones." Followed by a long table, which doesn't include SAM-FS or QFS. Hmmm. Can't tell from this. More reading required, I guess. And so it goes.

There's nothing wrong with this. It's just an inevitable combinatorial explosion, exacerbated by our commitment to preserve backward compatibility. (In other words, you can never take a feature out of Solaris.) The challenge is in managing unrealistic expectations. (It isn't all going to work together seamlessly from day one; in fact some combinations may never work together. It all depends on the business case.) The upside lies in the opportunities for serendipitous synergy. (Or should that be synergistic serendipity?)

At the Jini Community Meeting

Live at the 8th Jini Community Meeting at The Brewery in London; listening to Jan Newmarch of Monash University talking about a variety of Jini based projects at Monash.

(Sign of the times: 90% of the laptops here are Macs....)

Bizarre stuff: hearing references to Geoff Arnold that resolve not to me but to the other Geoff Arnold (who's not here).

Bob Scheifler is now presenting the changes and new features for the next Porter release of Jini. Cool stuff.

Update: After the coffee break, Dennis Reedy is talking about Rio, the policy-based service provisioning framework based on Jini.

Searching for the perfect Linux laptop

Quite a few of my friends and colleagues are running Linux on their laptops, but it seems that each of them reports that something doesn't work quite right - WiFi, or sleep mode, or power management. (And the Web seems to be filled with horror stories, hacks, and half-baked solutions.) I'm curious if this is a universal truth, or whether someone has managed to achieve The Perfect Linux Laptop configuration. I'm thinking of things like:

• sleep to RAM works
• everything works correctly after waking from sleep (even if you've unplugged a USB or FireWire device while sleeping)
• WiFi automatically connects to known and public networks, and reconnects after sleep
• power settings (screen brightness, CPU speed) automatically adjust when you unplug from the mains
• able to play, read and write CDs and DVDs
• automatically switch to mirrored or multiple screens if an external monitor or projector is plugged in
• etc.

Tedium is...

Tedium is installing Windows XP SP2 over a dial-up link, on a machine that's not up-to-date with security patches. Updating the Software Update libraries took an hour; downloading SP2 took five hours (overnight). Having 6 Mb/s cable modem service at home has spoiled me....

Musings on standards

I've been involved in standards work since the late 1970s, and I've always viewed the primary objective as interoperability. Interoperability demands unambiguous specifications (as much as humanly possible) and verifiable conformance - preferably machine-verifiable. A standards body creates a spec and defines conformance criteria; people implement that spec and test their implementations for conformance. That's it. (I like to think in OO terms: a standard is an object with one method, conforms(), which takes an implementation and returns or false.) When someone proposes that a standards body address a particular issue, I always ask myself "how does this affect the spec?" and "what are the conformance criteria?"

About a year ago, this came up in a certain web services group: there was a great flurry of activity to try to develop glossary entries for the terms synchronous and asynchronous, even though the terms were not used in the standard. Everybody had their own pet definition, usually in terms of some (irrelevant) implementation behaviour. I tried to apply my usual thinking to the issue, and I got stuck. I generally find that this is a good reason not to act. (Of course such self-restraint is hard for a standards group: like fishes, lack of forward movement usually presages death....)

Musings on standards

I've been involved in standards work since the late 1970s, and I've always viewed the primary objective as interoperability. Interoperability demands unambiguous specifications (as much as humanly possible) and verifiable conformance - preferably machine-verifiable. A standards body creates a spec and defines conformance criteria; people implement that spec and test their implementations for conformance. That's it. (I like to think in OO terms: a standard is an object with one method, conforms(), which takes an implementation and returns or false.) When someone proposes that a standards body address a particular issue, I always ask myself "how does this affect the spec?" and "what are the conformance criteria?"

About a year ago, this came up in a certain web services group: there was a great flurry of activity to try to develop glossary entries for the terms synchronous and asynchronous, even though the terms were not used in the standard. Everybody had their own pet definition, usually in terms of some (irrelevant) implementation behaviour. I tried to apply my usual thinking to the issue, and I got stuck. I generally find that this is a good reason not to act. (Of course such self-restraint is hard for a standards group: like fishes, lack of forward movement usually presages death....)

Travel plans (slightly updated)

In a couple of weeks I'll be heading back to my birthplace for a Jini Community meeting. It should be a lot of fun....

Nit-pickers will notice that although the graphic shows West End tube stations, the the earlier, misleading graphic has been updated. The original version is still to be found on Jini.org. Graphics notwithstanding, Jini Community Meeting itself will be at The Brewery in the City of London, near Moorgate and the Barbican.

Humpty-Dumpty on IT

Herewith a collection of the most ill-defined terms in the computer business today. I've chosen them because in the last few months I've encountered at least two WILDLY incompatible uses of each one of them - often many more!

• policy
• virtualization
• agent
• edge
• web service
• solution
• service
• architecture
• SOA

Anyone like to suggest a few more? I'll roll them into the top-level blog item as updates. By the way, I'm not suggesting that we stop using these terms, but I would like to see more judicious qualification and less universalization...

New blog on e-voting

There's an interesting new blog on E-Voting News and Analysis, from the Experts. Sample issue under discussion: "Suppose, hypothetically, that I knew of a vulnerability that would allow someone to corrupt vote counts or interfere with voting on some e-voting system being used in tomorrow’s election. And suppose further that it was too late to get the vulnerability fixed. What should I do?"

(I'll be watching the RSS feed.)

It's a small world: The Votemaster

For some months now one of my "must read" web sites has been Electoral-Vote.COM. (Me and over half a million others every day!) I've watched the maps showing the aggregated results of recent opinion polls, I've read "the Votemaster's" pained accounts of wrestling with different algorithms for aggregating, averaging, aging, and presenting the data. I've even contributed a few bucks to support the effort. Through all this, I had no idea who "the Votemaster" might be, until today. In the Votemaster FAQ , all is revealed: "My name is Andrew Tanenbaum. I am one of the 7 million U.S. citizens living abroad. I am a professor of computer science at the Vrije Universiteit in Amsterdam, The Netherlands. [...] I can write fairly complex software. I wrote MINIX, the precursor to Linux, for example."

Ah, that Andy Tanenbaum! "Mister Minix." Now where did I put my battered copy of Operating Systems: Design and Implementation?

Let's talk RSS (on the Mac, of course)

There's an interesting discussion of RSS by the authors of five RSS clients for Mac OS X. Quite apart from the subject itself, the format is really nice - I'd like to see more like this.

Charging, charging...

It was my own fault. I put down the power adapter for my PowerBook in the cafeteria, and when I returned a little while later it had gone. I blame Apple's sexy styling: it's simply irresistible. Whatever: I was chargeless. I kept working on battery power until the system went to sleep to save me from myself. That was yesterday evening.

This afternoon, I was driving back from Sun's Santa Clara campus to Menlo Park, and I decided that I had just enough time to stop in at Micro Center to pick up a new charger. They didn't have the model I needed (scratch that store in future) but they did have a third-party "universal" charger. Reluctantly, I bought it.

Now I've always thought that electricity is electricity, right? The charger supplies juice, the battery charges, that's it. Simple. Well, maybe not. Take a look at the X-Charge graph above. It shows my PowerBook charging up from zero to full over the course of 5 hours. There's a 2 hour gap (leaving work, getting food, doing a conference call), and then after a weird curve (sure looks like it's approaching an an asymptote to me!) the system decides that it's not 85% charged, it's 100%. Finished. Complete. Instantanteously. Weird. And iBatt confirms that it's fully charged, at 4.151 Ah.

I never got that kind of curve with a standard Apple charger. And besides, this "universal" thing is running awfully hot. I think I may have to stop by the Apple company store tomorrow and pick up a kosher unit. I wonder if I'll bump into Steve Jobs, as I did on Monday....

Debating WS-*

Tim Bray continues to discuss the relevance of the so-called WS-* stack: the collection of specifications related to XML-based web services. I'm not going to dive into the technology or business issues here; however Tim referred to a piece by Dare Obasanjo which argues that WS-* Specs are like JSRs. I tried to add a comment to this, but Dare's blog engine collapsed in a mess of XML, so I'll just post it here. Hopefully you'll be able to get back to read the original piece if you're interested. [Update: It looks as if my comment made it into Dare's blog after all.]

Just out of curiosity... if WS-* are like JSRs, what's the equivalent of the JCP? Where's the process documented, and what's the governance model? The statement "A JSR is basically a way for various Java vendors to standardize on a mechanism for solving a particular customer problem" ignores the fact that it's not just any old "way"; it's a particular "way" that has been publically codified, ratified by the community, and evolved to meet the needs of participants.

And then Mike [Champion] writes, "One difference of course is that Microsoft exerts a lot more architectural influence over the WS-* stuff than Sun attempts over JSRs. I think that's generally a positive thing". Hmmm: does this mean that he and his employer were actively engaged in JSR-171, JSR-215 and so forth, arguing in favour of stronger architectural influence by Sun?

Update: Over in LooselyCoupled, there's a response to Tim et al which essentially takes the position, "let a thousand flowers bloom, let mutually consenting parties decide what kind of daisy-chains to weave". My response is that this is fine until cookie-cutter garland tools start stamping out the same bloody flower patterns everywhere....

Did I say that?

One curious feature of search engines is that they remember stuff better than you do. I bumped into this truth earlier this morning, while testing the new a9.com search engine from Amazon. I naturally(?) began by searching for my name. After getting the usual hits, I found a bunch of stuff I'd never seen before - things like email exchanges about the NFS implementation in the BSD/386 distribution, and JXTA discovery. And then I came across this article by James Odell from 2002: Objects and Agents Compared. This post-dated my active involvement in the autonomous agents community, but nonetheless James quoted me twice - and both seem relevant to my recent pieces on software engineering, such as this.

On synchronous vs. asynchronous interactions:
According to Geoff Arnold of Sun Microsystems, "Just as the object paradigm forced us to rethink our ideas about the proper forms of interaction (access methods vs. direct manipulation, introspection, etc.), so agents force us to confront the temporal implications of interaction (messages rather than RMI, for instance)".

On typing:
Geoff Arnold has considered the question of third party interactions which are very hard for strongly typed object systems to handle. Here, two patterns come to mind. The first involves a broker that accepts a request and delegates it to a particular service provider based on some algorithm that is independent of the type of service interface (e.g., cost, reachability). The second involves an anonymizer that hides the identity of a requester from a service provider. Models based on strong typing, such as CORBA, RMI, and Jini, cannot easily support these patterns.

Hmmm. I wonder what else I said. In the meantime, I'll happily cite myself as these issues unfold....

The computer is the network is the computer

In a recent blog posting, Masood Mortazavi waxed lyrical on the importance of the fundamental statement of Sun's vision: The network is the computer. He wrote:

In my mind, there's no more revolutionary concept in computing, networking and information technology than the motto which Sun coined in many of its corporate PR campaigns: The Network is the Computer. [...] Many others, including Tim O'Reilly, have opined on the motto.[...] To me, it has an almost esoteric meaning.

I added a comment that I then decided to reproduce here:

Masood: I agree absolutely. As a 19 year veteran, I have found that "the network is the computer" has always been at the core of what Sun means to me. Occasionally we get distracted: we focus on the components and lose sight of the vision; but we always come back to it. You've reminded me to do something that I've wanted to do for a while, now: grab the domain name thecomputeristhenetwork.com. I want to use it to talk about some of my thoughts on the future of computing. Much of what I do revolves around the question that Rob Gingell asked a few years ago: "If the network is the computer, what is the computer that is the network?" It sounds Zen-like, but there's a profound issue here. Hint: it's NOT a Von Neumann machine. And no, it's not isomorphic to a Turing machine. Turing machines are fundamentally synchronous. The network is fundamentally asynchronous.

The future of software - postscript

Jim Waldo expressed an understandable skepticism about my inclusion of autonomous agents in my piece on the future of software; I thought I'd share my reply to him:

I think that autonomous agents may be an ever-receding category, a bit like AI. Remember how every advance in AI would provoke the retort, "But that's just X - that's not A.I." Every time we add a bit more autonomous capability to a software component (advertisement, peer group discovery, self-monitoring for SLO, negotiating over a shared security context,....) people will say, "Oh, yes, I see how you can do that, but that's not REAL agent behaviour." Some people will insist that we expose the inner workings, and if there isn't an obviously "AI-like" mechanism like multi-level planning or BDI they'll deny that it's an agent. Whatever. [shrug] I just want software systems that are a bit more robust, a bit more tolerant of version skew, a bit better about proactive resource management and self-diagnosis, a bit more flexible about how they organize themselves with their peers.

The future of software

Many of my colleagues at Sun are busy blogging about the software technologies that they work on - Solaris 10 features like zones (sorry, N1 Grid Containers), ZFS and dtrace; Java platforms (J2*E) and applications like speech synthesis and phone games; even Perl. And all of this is important - as Bruce emphasized, it's important to articulate who we are and what we do.

I spend much of my time thinking about the future: about what cutting-edge systems and software will look like three or five years out. Now I know that this isn't fashionable: Jonathan's priorities for the year, immortalized here, emphasize the tactical rather than the strategic, and that's just fine with me. And I also know that for most people the answer will be "same as it ever was", because innovation in computing tends to be additive: augmenting rather than replacing. But the beauty of the future is that it's already happening. Check out two articles in last week's Computerworld if you don't believe me.

Have Jini, will travel describes how Orbitz built their travel reservation system using Java and Jini. To quote from the article:

Under the Orbitz architecture, a customer request to book an airline ticket passes to a Java servlet container -- the Jini client -- running on BEA Systems Inc.'s WebLogic application server. The Jini client uses the Jini discovery protocol to find a Jini lookup service, which sends a proxy back to the Jini client. Through the proxy, the Jini client uses the lookup service to find a Jini service that can do what it needs. The lookup service then delivers the Jini service proxy back to the Jini client, and the client uses the proxy to communicate directly with the Jini service.

Orbitz registers its 1,332 Jini services on multiple instances of lookup services for redundancy, so there's never a single point of failure in the event of a power supply or hard drive failure. It also builds redundancy into the servlet containers and the services themselves, scaling horizontally through Intel-based dual-processor PCs running Linux. The PCs act as servers in the Orbitz environment. "It's not just redundancy; it's also capacity," says Hoffman. "If you need today 10 boxes to service a particular request and your traffic doubles, we can just add 10 more, and it's not only handling twice the capacity but it's now twice as redundant as it was before."

While analysts are starting to label every kind of new software methodology as "Service-Oriented Architecture", Orbitz is running a system consisting of thousands of services, with a high degree of self-configuration and self-healing. It's a compelling vision. And it's happening right now. (Shameless plug: I used Orbitz to book last week's vacation trip to England.)

So what will all of these thousands of application services be doing in the future? Another article in the same paper, Agents of change, looks at the current developments of autonomous agent software. (This is an area that I worked on some years ago; some of our thoughts are summarized here.) Agents represent a paradigm shift as fundamental as object orientation: from building software as a bunch of reactive services and wiring them up according to a fixed pattern, to creating a community of self-interested goal-driven agents that can negotiate with their peers to solve complex problems. (The anthropomorphism is inevitable.) What I find interesting is that the ideas apply at so many levels - to application business logic, as described in the article, but also to service lifecycle and systems management.

What links these two articles is that today the technologies are widely regarded as rocket science, or even science fiction. It's tempting to hand-wave and assert that a little syntactic sugar and tool-building will allow us to transform these powerful techniques into Lego components for the journeyman programmer. Well, maybe. But we should also consider how they might be used to solve intrinsically complex problems - the kind of thing that today we don't even try to tackle. One size definitely won't fit all.

(The other thing that links them is that they describe how our customers are building the future today. And that's a tactical business opportunity - right, Jonathan?!)

Interaction: Typing

[This won't make much sense unless you've read the original piece first....]

There are at least three issues packed up in this topic:

• An implementation issue, about whether the middleware even permits the sending or receiving of messages that do not conform precisely to a particular encoding and type/schema.
• The relationship between the type(s) of messages that can be exchanged and the description of the service.
• The extent to which message types are accessible, manipulable, and negotiable. Another way of thinking about it is how early or late message types are bound.

[DIGRESSION: For different distributed computing frameworks, we need to distinguish between feasibility (is it possible?) and idiom (is it a natural way of working within the framework?). For example, I can use XML to encode a rigid, unchanging message structure based on a well-known DTD; I can also use Java RMI to exchange opaque blobs that I interpret using some private mechanism; neither of these reflects the natural, idiomatic use of the technology.]

When we talk about "untyped", we must recognize that it's a relative term. Eventually there has to be a semantic match: the request must be expressed in a form that the service can interpret, and the response must be comprehensible to the requester. In that sense, the client and the server must structure their messages using mutually compatible formats, schemas, types, whatever.

Some mechanisms (e.g. CORBA, RMI or COM) bind the message type at design time. They depend on the use of software tools which typically generate client and server stub code that is, literally, incapable of handling messages that do not conform to this type. The client cannot generate invalid messages, and the service application logic never even sees invalid messages; they are rejected at a lower level. There are XML-based mechanisms that support this kind of model, using XML simply as an object serialization format.

In some cases it may be desirable to defer type binding to run time. This is particularly true if a service is identified by a persistent identifier (such as a URI) with no, or weak, type information. The most common example is a simple XML/HTTP web service: the client sends an XML message via an HTTP POST, and the service parses the message to determine whether it corresponds to a request that it can handle. In simple cases, the service only understands one type, and if the request doesn't conform it will be rejected. Semantically this is similar to a classic RPC; however it is likely to be less efficient, and the failure modes are different. However we are not restricted to such simple cases. A service may, for example, delegate the request to another, more capable service; or it may invoke a translator to map the request into a form that it can understand. Such examples highlight an assumption that is not found in the RPC world: that type matching is not always a simple black-or-white, true-or-false proposition. This in turn requires [or is that too strong?] that the message be expressed in a language that supports some form of composition.

[DIGRESSION: I was trying to imagine how one might do this in Java. Given a blob that represented a serialized Java object, one could deserialize it into something like a Java bean that supported introspection so that for each property you could obtain both accessor methods and type information that could be used by a classloader. This feels convoluted, but maybe someone has done it.]

The interface type is also involved in service descriptions. In some cases, such as Jini, a service description is based on the annotated interface type: it makes no sense to talk about discovering a service independent of its type. At the other extreme, a service is simply a network addressable end-point, with no type information specified or even available. And then there's WSDL, the web services definition language which is rich enough (or perhaps over-engineered enough) to describe a spectrum of service types. Although the specifications suggest that it can be dynamically interpreted at runtime, the complexity and (relative) rigidity of WSDL seems best-suited to design-time use. (Curiously, there is no standard way of retrieving the WSDL corresponding to a web service URI, although there are some common practices.)

So why is all this important? Well, as distributed systems scale in various ways - number of services, number of replicated component services, number of cross-domain service dependencies, lifetimes of services, and so forth - there is increasing interest in services that are relatively loosely coupled, with more flexible, less brittle, often asynchronous interactions. (Most of this is additive, not alternative: existing RPC-style services have advantages that are still important.) Maybe it's a throwback to the Internet mantra of "be liberal in what you accept", maybe it's influenced by agent-style anthropomorphism, maybe it's the result of overloading the simple HTTP protocol to do distributed computing, maybe it's just a recognition of a world in which version skew is a way of life. In any case, one way in which we decouple these components is by deferring type binding, from design time to run time, and by making the type of a message accessible to the application rather than being hidden below the marshalling and serialization infrastructure.

Is this a dichotomy or a spectrum? I can certainly identify a number of styles which I can order in various ways, so it feels like more than just an XOR. But that's all for now.

More on Interrelated Aspects of Interaction

I got a number of interesting comments on this piece. I think I'm going to follow up in a series of individual posts rather than one big chunk. Jim felt that most of the ideas were XORs rather than continuums (continua?). I suspect that this is because I was trying to pack too many ideas into each axis. Let's try again.

Interrelated aspects of interaction

For the last few weeks I've been trying to figure out how to write up an idea that's been bugging me. I think it's an interesting idea, but until I can actually write it up I won't be sure. So today I just decided to blurt it out into the blog, relatively unformed, certainly unfinished, definitely half-baked.

There seems to be an expectation today that distributed computing means web services - or that it will do, just as soon as the various standard bodies and corsortia get their acts together. I'm unconvinced. I see three ways - three axes, if you like - in which there are strong arguments for widely differing and incompatible positions; when you combine these, you get a 3D design space which is much bigger than what web services aficionados would claim. (And just to be even-handed, it's also way beyond what we can do naturally with things like Jini and RMI.)

I'm just going to describe these three axes, without attempting much justification. Zealots can argue all they want....

• The first axis is interface typing. At one extreme, a service can only accept a message of a single type, encoded in one way; if the request doesn't match exactly, no interaction is possible. (Think CORBA or RMI.) At the other extreme, a service will accept any type of message, although of course it may have to reply "Sorry, I don't understand".
• The second axis is control. At one extreme, the behaviour of the client and the service is rigidly specified, and the binding between the two is explicitly controlled and predictable. (Choreography languages, workflows, message exchange patterns, protocol FSMs come to mind.) At the other extreme, components are substantially autonomous; a client can search for a service at run time, a service can choose to delegate a request to a third party, and parties can renegotiate their interaction patterns on the fly.
• The third axis is synchronicity. At one extreme is the traditional synchronous request-response style, which is directly derived from the procedure call paradigm. The service is essentially passive; indeed we usually draw this interaction with a single thread of control running from client to server. There are other synchronous interaction patterns, typically described by state machines. At the other end of the spectrum, messages may be sent between the parties without temporal constraints. Partners may repeat messages, "interrupt" each other, or even "change the subject". (I don't want to get into the definition of [a]synchronous right now - that's a whole blog piece on its own. Let me just say that it's about the pattern, not the implementation technique; I'm not interested in dependency on a transport connection or blocking i/o.)

To summarise, we have:
Interface typing, from strongly typed to untyped
Control, from choreographed to autonomous
Synchronicity, from synchronous to asynchronous

Now strongly typed, choreographed, synchronous is very familiar: it's the classic RPC style. Untyped, choreographed, synchronous is what we find in the WWW today. At the other extreme, untyped, autonomous, asynchronous corresponds to the Holy Grail of the agents and semantic web community; it's also a pretty good description of human beings, which should come as no surprise.

What I find interesting is that I can think of really compelling use cases for both ends (and some middles) of each of these axes. One size will certainly not fit all. And in some cases we want to have our cake and eat it too:

• On control: I want the elements in my distributed system to be substantially autonomous and self-organizing, to improve scalability and avoid single points of failure. On the other hand, I want to be able to define policies that will influence the self-organization, and to have ways of observing and measuring what's going on.
• On typing: I want the safety and efficiencies that arise from working with predefined types with efficient encodings. On the other hand, I'd like to be able to refactor my distributed system by introducing generic facades for brokering without having to teach my broker about every service type that it might have to support.

Anyway, that's what I've been thinking about. Now to see if it makes any sense in the cold clear light of blog....

Community action on viruses

A number of my colleagues at Sun have been blogging about the number of MyDoom-generated emails in their inboxes today. Compared with most companies, Sun is barely touched by these infestations, but in a company which generates tens of thousands of emails a day, even "barely touched" means a lot of effluent. The incidence should be even lower, because (with a few exceptions) running Microsoft Outlook within Sun is a violation of IT policies, and the penalties can be severe. (In fact, even running Windows is prohibited unless the system has been "neutered" in various ways.) But people still do it.

My approach, which I recommend, is social pressure. In my email client, I've set up a rule that says "if this message originated from within Sun, and if it was created using Outlook or Outlook Express, flag it in red". Then whenever I see a red message in my inbox, I drop a brief note to the sender asking if they realize that they're violating IT policy.

If enough of us do this....

[Updated: In response to Dan's comment: I have never had a false positive. My filter checks for "sun.com" in both the "From:" and the "Message-id:", and for "Microsoft Outlook" (and variants) in "X-Mailer:"; it also checks that the message was NOT processed by any of Sun's external gateways.]

[Updated: Here's a representative Mail.app rule. You will have to add "X-Mailer", "Received" and "Message-id" to the list of headers that Mail.app understands. I actually use a number of rules to let me identify particular versions of Outlook, but I don't expect others to be so obsessive-compulsive about it....

Hippocratic oaths for software designers

ALL AUTO-CORRECTION SHOULD BE OFF BY DEFAULT!!!!

This is a rant. If I'm going to be responsible, I guess the rant should eventually get turned into a bug report. But I'll start with the rant.

I hate autocorrection. Automatic hyphenation, automated spellchecking, automated URL completion, automated correction of capitalization.... I'm not saying that I don't make mistakes that need correcting: what I find is that autocorrection gets in my way, interrupts my thinking, and - overall - makes me less productive.

Now I'm sure that some people, somewhere, must like it (although I hear more complaints than compliments for it), and I'm not opposed to making these features available. What makes me furious is that they are turned on by default. Every time I encounter a new spreadsheet or word processor, I get bitten by the "feature" and have to take the time to turn it off. And how should I turn it off? That's another source of frustration. In OpenOffice, things like automated capitalization and hyphenation are specified in Options->Language Settings->Writing Aids, while things like word completion are in Tools->Autocorrect, and configuring spellchecking requires you to start a spellcheck via Tools->Spellcheck->Check and then use Options to change its behavior. (Then remember to cancel the spellcheck!) (OpenOffice is not unique in this: it follows Microsoft's lead slavishly in this respect.)

OK, you're saying, but it's just a one-time frustration. It's not a big deal. Wrong. Consider the situation that the Register reported a couple of days ago in a piece entitled Excel ate my DNA. Scientists imported genetic data into Excel, Excel "autocorrected" it, and the result was unrecoverable data corruption:

The errors are introduced because some genetic identifiers look very like dates to Excel. If the spreadsheet is not properly set up, it will convert an identifier, such as SEPT2 to a date: 2-Sep. The conversion, the researchers say, is irreversible: once the error has been introduced, the original data is gone.

I'm sure that these researchers installed MS Office in good faith. I'm sure that the installation program never asked them, "Shall I enable a bunch of options that may lead to silent data corruption?" And now they're screwed.

Software designers need the equivalent of a Hippocratic Oath, combined with Asimov's Laws of Robotics. I do not expect my word processor or spreadsheet to change my data in any way without first asking me if I want it to do so. Out of the box, all automated corrections should be turned off. No exceptions. First, do no harm. (And one day someone will test the legal disclaimers in the software licenses, and the whole "repudiation of liability" nonsense will be exposed. But that's another discussion.)

So which Bugtraq category do I file this under?

Better, Faster, Lighter Java

I assume that you know that feeling when you're in a bookshop and a book title just grabs you, and you instantly know that you have to read this book - you hope that it's good, but even if it's crap, you need to understand that too. Well, that particular experience hit me in my local B&N this afternoon. The book in question is Better, Faster, Lighter Java by Bruce A. Tate and Justin Gehtland. I'll let you know how it turns out. In the meantime, it's sent me hunting through the usual chains of blogrefs, in the course of which I happened upon Bruce Tate's Don't make me eat the elephant again. Curiouser and curiouser....

Timing is everything

On my Mac PowerBook I have a folder called Downloads where my web browser (Safari) dumps everything that it downloads. Every RealAudio stream, every PDF, every software installer.... Right now there's about 2.2GB of stuff in there.

This morning, I thought about cleaning up my disk, opened Downloads, and typed Cmd-A to select all of the items, so that I'd get a count at the bottom of the window. 314 items. Wow. Too much to tidy right now, I'll do it later... oh look, there's that PDF I was reviewing... let's take another look..."

What I intended to do was click the PDF, thereby deselecting everything else, and then double-click it to launch Acrobat. What I actually did was to double-click it. And, obediently, Finder started opening all of the selected items - all 314 of them. Finder windows popped up, disk images started mounting, audio streams started playing, all sorts of applications started running. And of course this left no cycles to respond to UI events. So I did the only reasonable thing. I power-cycled.

Maybe the Finder needs to be a little less literal-minded.

Computing needs a Grand Challenge

Tony Hoare's for a "Grand Challenge" for computing is discussed here in the Register. The draft proposals are discussed here.

It looks like a nice mix of topics, but inevitably they are at such a high level that it's hard to be sure how they might be transformed into concrete research and engineering activities. And of course it's important to avoid the Fifth Generation effect....

On the difficulty of keeping on topic....

Like a lot of people in the computer business, I am intrigued by the impact of RFID tags and other sensor technology on IT. But my interest is fairly narrow: I'm curious about what kind of workloads these technologies will impose on corporate data centres. To understand this, I want to get a handle on the numbers: the sensor event rates (both the flow rate and the burstiness), what kind of intermediate aggregation and filtering will be performed, and what the resulting datacenter workload will look like.

Sounds straightforward, doesn't it? Not that it's a simple problem, but we can construct some scenarios, assign some numbers, plug them into a queueing model, see how it looks. (Capacity Planning for Web Services: Metrics, Models and Methods includes some simple models.) The problem that I've found (repeatedly) is keeping people on topic whenever I try to discuss it.

"But we can't ignore privacy issues." "Centralized data centers are passe - let's project the data centre to the edge." (I love that - what on earth does it mean?) "Data centres - not data centre! Federation!!" Or we replace inventory control tags with hospital patient tags, in which case discussions of domain-specific issues rapidly crowd out everything else.

The general problem, which I've observed in various contexts, is that it's increasingly difficult to keep people focussed on simple problems. Of course all of the issues that people raise are real, but in most cases they are either irrelevant or simply complicate the problem in incalculable ways. We need to focus on the simplified versions of the problems in order to use them as tools to analyze alternative architectures.

My dream is that one day someone will listen to my scenario and immediately propose a simplification, in order to make it more computationally tractable. Most of the systems that we've dreamed up over the last twenty years are far too complicated, and the analysis of the whole becomes even more problematic if we load even more complex application patterns on top of them.