Piling On: More Kibitzing about the iPad

What do I think about the new iPad? Glad you asked. What's that? You didn't, really – or not many of you did. But that's OK, really; at least two other writers argue that we're going from an era of "nearly universal literacy" to "nearly universal authorship", so here's my two rupiah worth.

John Gruber, the blogger behind Daring Fireball, is a justly respected voice on a variety of topics, notably all things Apple. Two of his posts on the iPad, The iPad Big Picture and Various and Assorted Thoughts... are a good starting point if you've somehow just managed to crawl out of a bubble that protected you from hearing anything about it for the past, oh, six months or so, or more crucially the last three days. I also enjoyed reading Stephen Fry's thoughts on the device.

But one bit I would recommend above all to anybody who is thinking about "what does this iPad thing/phenomenon really mean?", or anyone who just cares about free expression, open societies, and all the other progress that humanity has made these past few centuries, should seriously ponder Alex Payne's thoughts On the iPad. Al3x makes some very good, if deeply disturbing, points.

As many people have pointed out, the iPad differs in one critical respect from every personal computing device as computing device that Apple have ever built, from the Apple I on up to the iMac that I am typing this on. The iPad is a device, first, foremost and specifically, for the consumption of digital "content". As James Kendrick says, Apple just want us to push the 'Buy' button, in an endless, mindless Pavlovian dystopia.

"Hang on," you say, "it can't really be as bad as all that. You're just pushing histrionics!" I truly hope so. But consider: Apple has sold every Mac, as well as all their earlier models, on the basis of a small group of generally positive, empowering ideas:

• "It just works better."
• "The power to be your best."
• And, of course, "Think different".

One of the (many) things that make the Mac special is the fact that every single Mac sold comes with a copy of the full development toolset for Mac OS X, right in the box. Anybody who wants to invest the time and effort can become a Mac developer. (The money involved, of course, is in buying a Mac in the first place, but you were going to do that anyway, right?) You. The smart-aleck kid down the street. Your Aunt Tillie. Anyone. That has been one of the core strengths of the Apple Mac platform and product; the ability for anyone, without genuflecting before any sort of gatekeeper, to write anything they can conceive of, using some pretty great tools. And thousands, dozens of thousands have. You don't need to be a big corporation. You don't even need to ask Apple's permission, or use Apple's site to market your creation. All you need is an idea, some persistence, and the willingness to learn.

Apple made a radically different statement with the iPhone. If you want to write a "real" app for the iPhone, you need to submit your bits to the App Store, a process that is the seeming antithesis of open, collaborative or fair. As a practical matter, you need to join the iPhone Developer Program, and pay a fee. Doing so will subject you to various license agreements, limitations of what can be developed, and so on.

Apple justified this by saying "hey, the iPhone is a phone, an appliance. We have to keep some control over things, to make sure that our (non-technical) customers have the best possible experience — and incidentally to ensure that we continue to honor agreements we've made with carriers like AT&T." And the developer community grumbled and moaned, but large numbers went along. And the App Store, by almost any measure, is a raging success in the aggregate — even though most individual developers aren't making much at all.

Now comes the iPad, which has been widely, often dismissively, described as a "super-sized iPod Touch." Which, in several senses, it is. But whereas the "iTouch" is an accessory in one's life, being a combination PDA, music player, and (ostensibly) simple application platform, the iPad promises to be all that and more. Specifically, it's being pitched as this "magical" piece of technology which you'll "always" have at hand. Why? Well, you can listen to music on it, or watch videos, or run the same apps you ran on your iTouch or iPhone along with a new generation of "super-sized" apps. Oh, and Apple did announce that their office-software packages would be available on the iPad — so you can use it for "work stuff."

What about the kind of creative outlets that have been highlighted at each Mac introduction since the 1980s? Well, um, good luck with that. Because, even though this is a "real" computer, it's "really not"; it's an up-sized iPod Touch, which is (officially, mostly) a passive device.

The iPad is being pitched as a "magical", but inherently passive, device that consumers will use to buy (or, usually, rent) "content". There'll be "social media" like Facebook, Twitter and so on; the iPhone already supports those. Anything that can be done entirely over the Web, without the use of plugins like Java or Flash (like blogger.com, is kosher, too. But anything truly creative, "revolutionary", "game-changing", is going to have to survive the App Store gauntlet — which means that it's going to have to be consistent with Apple's view of how the iPad "should" be used. As a passive device which consumers use to buy access to content.

Think "57 Channels and Nothing On," a million times over. Perfect for the top-down, don't-make-me-think society. (I'm sure it will be very popular here in Singapore, for precisely that reason.)

Apple, you can do better. We know, because you've mostly done much better before, and encouraged us, developers and users, to do great things with what you've built for us. The iPad isn't so much a step in the wrong direction, it's a leap of faith worthy of Wile E. Coyote — but we know how far off the edge of the cliff we've gone. And the only way to go from here, is down.

What's wrong with this picture?

Apple chose the only name that ever made sense for their new change-the-world device: the iPad.

So what's not to like about it? Well, that depends on who you are and what you really wanted this to be. No surprise there.

My biggest problem with it was summed up well by James Kendrick in his post, Thoughts on the iPad — Just Push the Buy Button, says Apple: it really is primarily about media consumption - i.e., paying big corporations for temporary, passive access to data. Something to switch your creative intellect off, not on.

And those of us who are often called the "Apple faithful" were hoping, praying if you will, for something so much more. Which in fact this could still be. But for the first time in Apple history, a product with serious computing power is being positioned in the marketplace such that creative, collaborative expression by individuals is almost a subversive act.

We've fallen a long, long way from "Think Different". And it's now much, much harder to argue that Apple is as much a computer company dedicated to creativity and the pursuit of excellence, than to argue that Apple has become "just another" corporate big-media company, helping to turn the Internet from what was once called "the greatest opportunity for expressive, collaborative democratic action in human history" into just another television set. Active creation becomes passive "consumption." All hail the primacy of content, and don't even think about upsetting the applecart. That would be Thinking (too) Differently.

ANFSD: Newman and Redford, Not Streisand and Farrow

The leads in the classic 1973 movie, "The Sting", of course. Both leads were male, playing roles that were prototypically male in our social order. Why is that?

Clay Shirkey has some ideas about that, which he wrote up as "A Rant About Women". Not "A Rant Against Women", mind you. Some did not see that as a difference that makes a difference, notably Zo at this post on her (misnamed) humorlessbitch.com blog. This post, beyond this explanatory header, is my comment to her blog-post-as-comment, the 190th comment in 30 hours. I have slightly reformatted it, as the original was only able to use CAPS for emphasis.

A tip of the hat to Venessa Miemis (@venessamiemis on Twitter) for tweeting about this.

---

Maybe this is because I have the ability to look at this as a Shirkeyesque male who “only told lies I could live up to, and I knew when to stop.” The sorry fact that he’s pointing out – too obliquely – is that many/most arenas of professional and creative endeavour have been created, led and/or taken over by men who fit the portrait that Shirkey is painting. Risk-taking, self-promotion and so on are seen as predominantly “male” character aspects/flaws, in large part because society has been and is fundamentally sexist – and is likely to remain so unless and until women can function within the existing framework successfully enough to alter it.

That’s what I get from reading Shirkey’s piece; not a snarky bit of braggadocio that “men are on top because we know how to game the system,” so much as “until increasing numbers of women in a variety of fields – professional, academic, artistic, and so on – can work within this aspect of the system, it won’t be changed.” Not because change wouldn’t benefit men as well as women. It would. But rather, because those “gatekeepers” who control the system, having got to where they are now by gaming the system, lack the will if not the means to change it.

We are at a turning point in human history, of a kind that hasn’t been seen in well over a thousand years. Institutions and conventions are changing all around us, and, not knowing how to change and survive, they change and die. Others watch the process attentively, determined against reality to avoid the same fate. But if we do NOT come up with a way to fundamentally change our society, to make it more equitable, transparent and open, we risk a new Dark Ages that will compare to the European mediaeval one as a broken fingernail compares to a petrol bomb.

I believe that is what Shirkey fears, as do I.

---

Comments, please.

Fixing A Tool That's Supposed To Help Me Fix My Code; or, Shaving a Small Yak

Well, that's a couple of hours I'd really like to have back.

One of the oldest, simplest, most generally effective debugging tools is some form of logging. Writing output to a file, to a database, or to the console gives the developer a window into the workings of code, as it's being executed.

The long-time "gold standard" for such tools has been the Apache Foundation's log4j package, which supports Java. As with many tools (e.g., PHPUnit), a good Java tool was ported to, or reimplemented in, PHP. log4php is uses the same configuration files and (as much as possible) the same APIs as log4j. However, as PHP and Java are (significantly) different languages, liberties had to be taken in various places. Add to this the fact that PHP has been undergoing significant change in the last couple of years (moving from version 5.2 to the significantly different 5.3 as we wait for the overdue, even more different, 6.0), and a famous warning comes to mind when attempting to use the tool.

Here be dragons.

The devil, the saying goes, is in the details, and several details of log4php are broken in various ways. Of course, not all the breakage gives immediately useful information on how to repair it.

Take, as an example, the helper class method LoggerPatternConverter::spacePad(), reproduced here:

    /**
     * Fast space padding method.
     *
     * @param string    $sbuf      string buffer
     * @param integer   $length    pad length
     *
     * @todo reimplement using PHP string functions
     */
    public function spacePad($sbuf, $length) {
        while($length >= 32) {
          $sbuf .= $GLOBALS['log4php.LoggerPatternConverter.spaces'][5];
          $length -= 32;
        }
        
        for($i = 4; $i >= 0; $i--) {    
            if(($length & (1<<$i)) != 0) {
                $sbuf .= $GLOBALS['log4php.LoggerPatternConverter.spaces'][$i];
            }
        }

        // $sbuf = str_pad($sbuf, $length);
    }

Several serious issues are obvious here, the most egregious of which is acknowledged in the @todo note: "reimplement using PHP string functions." The $GLOBALS item being referenced is initialized at the top of the source file:

$GLOBALS['log4php.LoggerPatternConverter.spaces'] = array(
    " ", // 1 space
    "  ", // 2 spaces
    "    ", // 4 spaces
    "        ", // 8 spaces
    "                ", // 16 spaces
    "                                " ); // 32 spaces

If you feel yourself wanting to vomit upon and/or punch out some spectacularly clueless coder, you have my sympathies.

The crux of the problem is that the function contains seriously invalid code, at least as of PHP 5.3. Worse, the error messages that are given when the bug is exercised are extremely opaque, and a Google search produces absolutely no useful information.

The key, as usual, is in the stack trace emitted after PHP hits a fatal error.

To make a long story short(er), the global can be completely eliminated (it's no longer legal anyway), and the code can be refactored so:

    public function spacePad(&$sbuf, $length) {
        $sbuf .= str_repeat( " ", $length );
    }

Of course, this makes the entire method redundant; the built-in str_repeat() function should be called wherever the method is called.

I'll make that change and submit a patch upstream tomorrow... errr, later today; it's coming up on 1 AM here now.

But at least now I can use log4php to help me trace through the code I was trying to fix when this whole thing blew up in my face.

At least it's open source under a free license.

Virtual Real Work

(As posted in a comment on a very detailed Ars Technica review:

After giving PD5 and Fusion 3 full evaluations, I'm going back to Fusion for another cycle.

A lot of what I do is testing software in odd developer builds of various BSDs, Linuxes and OpenSolaris; Fusion gives me the least trouble when venturing away from the Microsoft megalith. I have XP and Win7 VMs too; I just don't fire up either more than once or twice a month. I typically have 2-3 other VMs up and on my network at any given time.

I also did take a good look at OpenBox; it's the best of the bunch for running OpenSolaris (wonder why?), but for anything else, particularly the oddball Linuxes, it gave me as much trouble as PD 5. Where Parallels would crash like a drunken teenager in a Maserati, OpenBox just would not install several important systems. So much for that.

VMWare isn't as slick in some areas as PD5 (LOVE that one-click 'exclude from TM' feature, and the VM list is nice), but it DOES have one feature that's absolutely non-negotiable: It tends to work better/more often. That's worth the extra US$10 or so for the upgrade. (with an additional new license for my new MBP).

I've put about 20 hours over the past 2 months into formal evaluation of the hypervisors mentioned (along with a couple of hundred hours of just using them to run VMs with which I was working, and not trying to think "is this better than...?"). The pain level with Parallels has been fairly consistent; VMWare only surprised me a couple of times. Parallels also gives you only half the review time that VMWare does before you have to pony up. That tells me that they aren't as confident in their product (and rightfully so); there are problems that merely suggest themselves in 15 calendar days of varied use that you could really get a handle on in 30. Not having confidence borne of experience in whether I can solve subtle issues as they arise wasn't the only reason for rejecting Parallels Desktop 5, but it certainly didn't help.

The impression I get from using both products, from reading what has been written and from chatting with other users, is very consistent. VMWare portray themselves as a company that tries, and often succeeds, to do things well - starting with a truly usable, stable, versatile virtualisation platform. Parallels, on the other hand... by bringing out releases well before they're really ready, and with a shorter evaluation time, give the impression that they're desperate for your money. Give it to the company that actually earn it.

Address Book him, Danno!

This is a follow-up to a tweet I left yesterday, where I was praising a great little app called ABMenu. This little guy just sits in the system menu bar and gives you easy, two-click access to any entry in your Address Book. I'd installed it earlier on my main iMac. This morning, I noticed that it wasn't installed on the new, two-week-old MacBook Pro, and fixed that.

Then of course I noticed that the new system didn't have all the entries that the old one did. The customary way of syncing data like this between Macs is to use Apple's MobileMe service; the Mac can also use LDAP or other enterprise-style directory services. "Fine, " I thought, "this definitely sells a lot of MobileMe subscriptions." (In fact, getting one is on my New Year's resolution list; the reasons for the delay have been Complicated™.) Continuing the thought process, " There's gotta be a dedicated app out there that will let me sync just Address Book."

A couple of minutes browsing VersionTracker brought me to address-o-sync, which does just what it says on the tin. Obviously reasonable but slightly irritating irritation: the app has to be installed and running on both Macs in order to sync; address-o-sync can't reach across your local net and pull "raw" Address Book data. (It compensates nicely, with a couple of features that would be impractical otherwise.)

So, I've updated my "how to bring up a new Mac" checklist and starred both ABMenu and address-o-sync on my installed-app lists. Take a look, and see if you agree.

(Now if I could only get the Hawaii Five-O theme song out of my head.)

The Audacity of Hype

This is the content of a comment which I posted to a TechCrunch article on The World Doesn't Need Someone Telling Us What We Don't Need In Tech. For what it's worth, I hope the iSlate/iPad/iTablet/iWhatever is real this time; if we have another Charlie Brown, Lucy and the Football moment, I think the market will see this as one of Apple's rare blown opportunities - to the tune of maybe $50 off the share price.

I'd be interested in your thoughts.

---

I also disagree with the "comfort zone" limitation idea; cf. Shaw and the Unreasonable Man. Apple's successes have all been revolutionary innovations; the few times that a "me too" product has come out of Cupertino (Apple III, Lisa), it's sunk at high warp speeds.

If the iPad is even halfway affordable (by Apple standards, anyway), they'll sell like iPods on steroids; I'd certainly buy one. Even having a big, beautiful iMac and a trusty MacBook Pro, I can see several useful areas for an iPad. More-convenient-than-laptop media browsing and ebook reading is the one everybody latches on to. But I could take an iPad onto the train every morning and get stuff done - standing on a lurching train filled with people from cultures that don't know how to move in crowds efficiently. I could take it into meeting-room meetings and hallway meetings more easily and effectively than a full-clamshell laptop. And it would probably be easier to gather a couple of people and run through a quick presentation with an iPad than a laptop, too. (Think "elevator pitch.")

Sure, it would fill a niche and scratch a few itches that I already know about. But for it to be truly revolutionary, it will need to find uses that I/we can't really predict yet, because we've never really had something capable of filling them - we've just sort of worked around limitations we weren't really aware of. (See 'iPhone' for previous example.) I've got more faith in Apple to pull something that revolutionary out of their hats than I do in any other tech company today — or in the 30 years I've been in this industry.

We ain't seen nothin' yet.

ANFSD: The Present Isn't Always Perfect

If anybody has had doubts about the decline and fall of the English language over the past two or three decades, current media (print and online) should clear that up very nicely.

If I see another reviewer who starts off with "I've had the (Product X) for a week and I am loving it," it will be very difficult to restrain myself from throwing a brick through the display. There's this widespread tendency to use the present perfect in place of the simple present.

Part of this, I understand, is simple cultural differences. Many of these writers either come from or work in an environment heavily influenced by South Asian variations of British English. Somebody in a position of educational or institutional power, somewhere along the line, decided that if the "present perfect" was indeed "perfect", then it was better to use that in any conceivable sentence.

This influence is relatively new in the larger English-literate world. I don't remember seeing this style when I started writing seriously 30 years ago, or even during the time of my first real multi-national distributed development team back in the early 1990s. People had varying levels of language ability, just as we all had varying abilities in other areas. One of the great things about those early teams was that each participant clearly understood that they had something to learn; that other teammates had skills and abilities that they could learn from. Since that time, the offshoring/outsourcing industry has devolved into what too often seem a series of sealed bubbles bumping against each other, but that's a topic for another post. Just let me say that, as has happened so many times before throughout history, language patterns mingled (or 'mangled', depending on your views). Absent widespread, effective English language instruction in American schools over the last 30 years (yet another topic unto itself), many younger American technical folks adopted the "new", "cool", "exotic" English usage borrowed from their peers. And since the software (and Web) industry is unrelentingly ageist, at least as much as, say, theoretical physics (where, I've been told, you're over the hill by your mid-20s), what the "cool kids" were doing quickly became the norm for the industry.

It may well be that "all this has happened before, and all this has happened again." That does not make it any more comfortable, or comforting, to those who are aware of how imperfect the present really is.

Oh, and by the way, Happy New Year! One is sorely tempted to say "It can't possibly be as bad as 2009 was..." Don't give in to the temptation. $DEITY tends to take that as a personal challenge... one that you or I can't possibly win.

Blast from the Past

Another in a continuing series...

Microcomputer(as PCs were called before the IBM PC) veterans of a certain vintage well remember that most counterintuitively productive of productivity tools, WordStar 3.3 (and earlier). The hegemon of its day, WordStar used what at first (and usually fifth) inspection appeared to be whimsical, arbitrary key combinations for commands. Ctrl K-H for Help was invariably what new users first memorised. All through the 1980s and well beyond, any word-processing software that came onto the market had some degree of WordStar-compatible commands, either as their main command set or as a bolt-on to wean folks onto the "new" way of doing things. This was even true for the first several releases of WordPerfect and of Microsoft Word. (Word today has several available add-ons to add WordStar command compatibility.)

Why was this so popular? As noted in the Wikipedia article:

...the "diamond" of Ctrl-S/E/D/X moved the cursors one character or line to the left, up, right, or down. Ctrl-A/F (to the outside of the "diamond") moved the cursor a full word left/right, and Ctrl-R/C (just "past" the Ctrl keys for up and down) scrolled a full page up/down. Prefacing these keystrokes with Ctrl-Q generally expanded their action, moving the cursor to the end/beginning of the line, end/beginning of the document, etc. Ctrl-H would backspace and delete. Commands to enable bold or italics, printing, blocking text to copy or delete, saving or retrieving files from disk, etc. were typically a short sequence of keystrokes, such as Ctrl-P-B for bold, or Ctrl-K-S to save a file. Formatting codes would appear on screen, such ^B for bold, ^Y for italics, and ^S for underscoring.

Although many of these keystroke sequences were far from self-evident, they tended to lend themselves to mnemonic devices (e.g., Ctrl-Print-Bold, Ctrl-blocK-Save), and regular users quickly learned them through muscle memory, enabling them to rapidly navigate documents by touch, rather than memorizing "Ctrl-S = cursor left."

Why is this relevant (or even interesting) today? Besides the lessons to be learnt about interface design, it's interesting to note how many editors out there still pay homage to WordStar. I stumbled across joe again this morning; it's available on essentially all Linux and BSD distributions, with versions built for other systems as well (e.g., Mac OS X and Cygwin/Windows), and source freely available if your platform isn't yet supported or you just like to tinker around on one that is.

What makes this fairly scary for us old-timers is just how quickly the old finger habits come back. If you had more than a year's experience beating your head against the original WordStar, I dare you to work with joe or its ilk for more than a few minutes before "how do I do...?" completely falls away from your thoughts and you're just typing as fast as you can think.

For that's the real beauty of this type of "primitive", what-you-see-isn't-what-you-get interface: you're not distracted by the ephemera of making your work appear "just so", and can actually focus on the work of writing. And that, in our click-and-drool modern interfaces, is what we've lost -- and no amount of clever code wizardry on the part of the interface designers can bring us back to that. Why? Because of basic human nature - if we see a button, at some point we'll want to push that button - "just to make things look better." And, all of a sudden, we notice that the entire morning has flown past while we were focusing on the first three paragraphs of a major report that's due this afternoon. Oops.

There's a reason why almost every tool aimed at professional writers -- people who make their living at x cents per word -- have "stripped down", minimalist interfaces, at least as an option. It's the same reason that far too many truly "old-school" writers give for writing on paper and then typing (or having someone type) their words into a computer: the fewer distractions you have, while still being able to do what you're trying to do, the more productive you'll be at it.

That concept extends far, far beyond the writing of prose -- and has too often been lost or forgotten in those other areas as well. Pity.

When Standing on the Shoulders of Giants, Don't Trip

I've been doing a lot of software installation lately, on Mac OS X and various BSDs and Linuxes. Doing so reminds me of one of the major banes of my life when developing for Windows. If you're a Windows usee, you're acutely familiar with the concept.

DLL Hell.

On Windows, as you install a dozen apps, each of them will try to install their own versions of many system-wide (or vendor-wide) libraries. The poster child for these is msvcrt.dll, the Microsoft C Runtime Library, upon which virtually everything in or running under Windows depends in some fashion. Many unhappy man-millenia have been spent by admins and PC owners the world over trying to resolve compatibility issues, usually introduced when a newly-installed app overwrites a more recent version with the earlier version that app shipped with. Other things — including other libraries the app may depend on or even Windows itself — may break since features and bug fixes they rely on (present in later versions of the library) are suddenly gone.

Why think about this, when I'm as likely to write (or even use) any Windows software in the next month as I am to win the local lottery without buying a ticket?

Because, one of the things I added to my new MacBook Pro was the MacPorts software, which is a Mac analogue to the BSD ports collection. The Linux equivalent to this is (more or less) your package manager, whether it be aptitude or yum or portage or whatever. Windows has no equivalent concept; every app you install is its own universe, or thinks it is (with the "benefits" noted earlier), and no central one-stop "update everything" capability.

Modern operating systems may not have DLL hell, but they do tend to have numerous different versions of different libraries and support frameworks installed. Since those can be accessed by apps as either "give me the most recent version" or "give me version x.y.z", no equivalent to DLL hell takes place. But it does tend to eat up disk space. And it makes it harder to mentally keep track of what's installed; installing the MacPorts edition of Bluefish, an HTML editor, installs some eighty other "ports" — direct or indirect dependencies of Bluefish (which itself is primarily a Linux app). Some of these dependencies are last-month recent updates; a few are several years old. But MacPorts determines which packages are needed by Bluefish (and its dependencies, and...) and installs the latest version (by default) of those dependencies, or the specific version asked for. Thus, several files with the same basic name, but different version numbers encoded into the filename, can coexist peacefully in virtually any modern OS.

Recent Microsoft system software, particularly the "managed" software using the .NET Framework, avoids most of the cavalier instances of DLL hell, but introduces other quirks and brittleness in compensation.

But I'm still a bit unnerved by a software package — any package mdash; that has 80+ designated dependencies. I'm certainly thankful that I don't have to manage that dependency matrix myself.

If modern software development is "standing on the shoulders of giants", to appropriate a Newton paraphrase, then careless dependency introduction would be the functional equivalent of drunken tap-dancing: almost impossible not to trip.

Reuse, Renew, Recycle - Data Structures Edition

Everything gets recycled these days, including acronyms...

Whether you're developing Web pages in your language of choice, writing a new game to take on the world, or (insert your project here), at some point, you're going to deal with structured data.

You're quite likely, at some point, to deal with fairly complex, nested structured data - for configuration, to preserve the state of objects within your software, or whatever. In recent years, "complex, nested structured data" has often been coded up in XML. This can save your coding bacon; there are quite a few nice tools out there to work with XML, insulating you from having to deal with the "raw" <tag ...="" attribute="value">data<nested_tag>more data</nested_tag></tag> soup.

But at some point you may well end up having to hand-edit the thing — you may want to pull something out for reuse, or you suspect that one of the tools you're using is messing up in some way... it happens. And even during that bright, wonderful time when you're building a POC (proof of concept/ possibly overly c***tastic; see, I said we could reuse), you wind up humping a lot of "extraneous" data around that just really wraps a few small things. Ugh.

Even with compression, you're still taking a hit; you've just moved it from the pipe to the CPU. Twice.

I got handed a client project a little over a year ago that used this nifty notation called YAML. While YAML was apparently originally developed for Ruby apps, it's evolved; there are now support tools/libraries for everything from PHP to Visual Basic to Haskell. "YAML Ain't Markup Language," but it can express very complex data structures; nesting is defined by indentation, item types by decorations. (See the spec or this PHP implementation for more detail.)

For projects which have been using binary data storage for non-BLOB data, simply because XML was seen as overkill, this might well simplify things. Likewise, if you're pushing data around between different apps/platforms, thinking "I maybe oughta look at JSON", this is for you.

I've been using it primarily in PHP to encode various data and content models, and it has allowed me to dramatically simplify my code in several places - reducing complexity, inefficiency, and possible bugs. The Ruby and Python crowds have been using this for a while; PHP folk have tended to look at it suspiciously, since it came from Over in Those Languages.

---

Good tools work in any language.

---

Reuse? I was talking about recycled acronyms earlier?

At about the same time I found YAML the non-markup-language markup language, I also came across a nifty CSS framework with the highly original name of YAML. This code, which makes it pretty easy to style sites using Yet Another Multicolumn Layout, is a set of CSS (2.x) stylesheets, with minimal graphics and JavaScript thrown into the box. What it gives you is a nice, consistent layout on just about any browser (including hideously-broken variants of MS Internet Exploder) which you can slice and dice pretty much at will. It even degrades sensibly for non-CSS, non-JS browsers — like the Googlebot. If you know how to read CSS reasonably well and this takes you more than an hour to grok, you're working too hard. Browse the docs, the samples, the tutorials, or the online community rumblings, and you'll be productive in no time.

And yes, you can use both together on the same Web dev project. Just make sure you keep your tools straight. After all, YAML is not at all the same thing as YAML.

Two steps forward, three steps back

Alternate title: ARRRRRGGGGGHHHHHHH!!!!!

Both of you Gentle Readers may have noticed that I've been away from the blog for a while, and that a few posts that were previously published have gone missing. I've been busy fighting some other fires for a while, and my current network access has lacked the stability and efficiency that local propaganda would have you expect.

This evening (Wednesday 18th) I came across a piece of nifty-looking software, MacJournal by Mariner Software. It looks great — software that would let me compose/revise blog posts offline, in a native Mac app with nice organizational features and so on, at an attractive price, and with a 15-day evaluation period thrown in, just so you can try before you buy.

"Cool," I thought; "I'll be able to multitask on my shiny new MBP that's coming Any Day Now™."

Downloading and installing the eval copy went just as you'd expect; drag an icon into a folder, wait for the "Copying" progress dialog to go away, and it's done. Standard Mac user experience; nothing to see here, folks — unless you were expecting a Windows-style "Twenty Questions" installation.

I decided to do a really simple, trivial first exercise: select the five posts I'd written so far in a tutorial series; add the keyword ("label" in Blogger.com parlance) tutorial to them; save them back to Blogger. No animals were harmed in the performance of this experiment, and very explicitly, no content was directly, intentionally edited. (Note the qualifiers; they're important.)

(insert "train wreck" sound effects here.)

The first two parts were (relatively) unmolested; they didn't have any code blocks in them. The latter three did, however, and those were completely deformed. Numerous span elements were added, particularly around links (MacJournal seems to think links shouldn't be underlined, ever). Other formatting was changed; in particular, code tags were replaced by spans that set the font size to 13 points.

WTF?

It's going to take me a bit of noodling around in the software to figure out how to change the defaults to something that makes sense (at least for me), and until then, I'm back to editing in the browser. If the evaluation period expires before I'm happy with the configuration, then I'll comply with the license and blow it off my system. I'd really rather not do that; the feature list looks good, the interface is clean, and best of all, I don't have this Could not contact blogger.com line underneath my editing area as I type.

I understand that MacJournal, like most apps, has default ways of laying things out and working with things. I'm well aware of the difference between an "import" and a "copy" of something. But... I believe very strongly that the first rule of software, as medicine, should be "First, do no harm" — and that includes "don't mess with my formatting without even putting up a confirmation dialog asking my permission!" I really don't think that's too much to ask, or too hard to implement — and doing so would a) make a much more positive initial user experience by b) showing that you've thought things through well enough that c) your still-potential user isn't looking at an hour or two of careful, detailed work just to get back to where he was before he touched your product &emdash; or, rather, it touched his work.

Like most Mac users, I've gotten spoiled by how well most software on this platform is thought through to the tiniest details. Like most, I get annoyed when I have to deal with Windows or Linux apps that simply aren't thought through at all, apparently. (Spend a week with Microsoft Office or, even better, Apple iWork on the Mac; I dare you to go back to Office on Windows and be happy with it.) To run into a Mac app that fails such a simple use case so spectacularly (granted, in its default configuration) simply beggars explanation.

You want to start a tutorial; well, you know...

Not as catchy as the Beatles' Revolution, even if the meter works.... oh well....

Continuing from the first post in this tutorial. What do I think is important when starting to demonstrate some code? As with most writing, it depends on the audience. For the purpose of this series of posts, I'm assuming that you fit comfortably in or near the following:

• You're comfortable with HTML and XML doesn't make you run screaming from the room;
• You have a basic understanding of databases; you've run across SQL before and understand the basic concepts;
• You understand PHP; you've written some code before;
• You understand the concepts of "object-oriented development", "patterns", "best practices" and ideally "test-driven development" (usually abbreviated as "TDD"), even though you may not have loads of experience (yet) with them; and crucially
• You want to improve your ability to write code that you can refine and possibly reuse over time.

The assumption that you know or at least are interested in PHP is a given, since that's the language we'll be using here.

What will you need to have installed and available to follow along?

1. Access to a system with PHP 5.2 or higher, available both from the command line and the Web server (via a module or CGI);
2. The PHPUnit and MDB2_Driver_mysql modules installed and available;
3. A text editor of your choice;
4. The ability to create PHP scripts and HTML files and have those accessible from the Web server as well as the command line.

These should all be pretty obvious to more experienced PHP developers, but making sure that we're both operating from the same set of assumptions — and no others — greatly reduces the likelihood of confusion and breakage along the way. Many of you haven't yet dealt much with unit tests using PHPUnit or similar systems; that's going to be a starting point for us.

Time v. Money v. Risk v. Frustration

I just spent three hours beating my head against The Joys of Wi-Fi Networking. No doubt because it saved $50 or so, the HDB public-housing flat I live in here in Singapore (which are "affectionately" known as 'chicken coops' for their quality and structural integrity) didn't put telephone jacks in every room. Nor did they run conduit between rooms so that suchlike could be added later. The upshot of this is that my DSL modem, in the living/dining room, is 15 meters and two concrete-slab walls away from my study/office, in a bedroom (with no phone jack and two power outlets).

Up to now, this hasn't been an insurmountable problem, because the DSL modem/802.11g router the government-linked telco sells you when you open a DSL subscription could punch a (barely) usable signal through those two concrete walls; desktop and laptop computers strewn around the study were on the LAN, with quite serviceable WPA2 encryption to keep private bits private. The router (in the living room) claimed to have a 14 Mbps connection to The Net; the aggregate total bandwidth available in the study ranged from 3 to 5 Mbps; not fantastic but usable. The world was in a survivable state of chaotic flux.

Then, yesterday morning, FedEx delivered a bright, shiny (actually used, in a Target bag rather than original box, no longer in the Polycom US catalog but still serviceable) Polycom SoundPoint 501 SIP phone, courtesy of my newest client (thanks, Nathan, Matt and Margaret - I'm not kvetching, honest!). An hour of work with jackhammer and flamethrower cleared sufficient space on the desk for the new trophy. Now to plug it in and fire it up.

Plug it in? To what? My ear?

Just to make sure I had my head straight, I looked at what it would take to run line from point P to R. Two solid concrete/rebar walls, routing around various doors/windows, across the span of the living room.... no, that wasn't going to happen, certainly not by Thursday. Put the phone in the living room? Not a chance. Wait a minute - they have these things called wireless Ethernet bridges; I should be able to get one of those, stick it in the study, connect the phone to it and it to the existing wireless LAN, and I'm good to go.

I went out to the local "IT super-mall", Funan. It's the best place in Singapore to buy electronic gear of whatever variety from reasonably reputable dealers. After visiting some 25 different shops, large and small, I was firmly reacquainted with one of the basic facts of Singapore life.

This is a firmly Stalinist economy in the areas that matter. All imports (which is to say, anything of value) are brought in through a small number of generally well-connected exclusive distributors. When one shop says "finished, already!" (the de facto motto of the city), everybody is going to say the same thing. That is, if you're fortunate enough to even run into some salesperson who actually understands what you're looking for. I could find things easier (and often cheaper!) in Vietnam.

Then I happened to pop into one hole-in-the-wall one-man shop, Crystal Systems. The "one man" said "say, I understand what you're trying to do; I did exactly that for a firm here recently; all you need are two routers." The archetypal light-bulb moment. (Remember, I'm a software craftsman, not a hardware/network engineer.) This is where the "time v. money v. ..." of the title comes in. What I should have done was to buy two identical brand-new 802.11n routers. That would have cost me about S$190 (~US$136 or so), and would have given me a perfect excuse to replace my aging 2Wire 2700HGV-2, But this has already been a bit of an expensive month - new hard drive, various software - so after some consideration I decided to try to salvage the existing router (apparent POS though it may be) and just spend the minimum necessary. So I went home with one S$69 TP-LINK TL-WR340G router.

It features the now-customary browser-based setup - just remember to use a hard-wired link rather than wireless (d'oh!), and it's almost as easy as falling off your seat. Bridging mode is pretty obvious; the TP-LINK wants to know the MAC address of the router you're connecting to, along with its encryption setup.

In bridge mode, the TL-WR340G only supports the obsolescent WEP encryption, not the current and generally superior WPA2. The 2Wire - along with every device currently connected to it, obviously - uses WPA2, and does not support a mixed WPA2/WEP environment.

Back to the store I go tomorrow; the only question now is: one new Netgear N router (which can fall back to 802.11g) or two?

Responding while Forbidden; Gender and Other Issues in OSS and PHP

This post is what would have been a response to a post on Elizabeth Naramore's blog, which she titled Gender in IT, OSS and PHP, and How it Affects Us *All*. Quite a good post, actually, with a long and often thoughtful (but as often thoughtless) comment thread following. I was hoping to respond to a comment on the post, but that apparently is no longer allowed, even though the "Leave a Reply" form at the bottom of the page is functional. There's also a mysterious "Login/Password" pair on the page, but no indication of which ID one uses, or how to go about getting one.

Following is the content of the reply-that-wasn't: I (perhaps unreasonably) think this has some points worth pondering. Please do read the original post first and then come back here - where the "comment" feature definitely does work.

---

@A Girl: Great that you're doing AP CompSci next year. As someone who's been in the Craft for 30 years, I have a great sentimental attachment to your idea that "teachers and professors have the chance to shape the mindsets of their students towards women in the industry". If we were a true profession, where essentially all practitioners have a certain common level and content of educational background combined with qualifying experience (e.g., apprenticeship, internship), I'd agree wholeheartedly.

The fact is that many if not most of the people in the industry - both the "coders in the trenches" and the ex-coders who got promoted into management ("because they were such great coders" - thereby removing two qualified people from an organization)... far too many of these people have noformal education in CS (or, often, anything else). And by the time they realize how important that might be, they're old enough that they're facing ageism in the workplace already - they're not confident enough to put the "big hole in the middle of [their] career" and "go back" to school. It doesn't help that schools the world over do such a lousy job of outreach and marketing to those potential students - they're focused on the Executive MBAs and other graduate-level returning students, who can have their pricey programs paid for by their employer. Joe or Jane Schmuck trying to keep head above water in the face of cut-throat competition from planeloads of new arrivals with mimeographed certifications, who've been taught their entire lives to never think out of the box to begin with... things start getting really tough out there. I'm not surprised that enrollment in CS programs is down. I'm amazed beyond words that it's still as high as it is; a less starry-eyed observer might expect the number of CS majors to closely track, say, majors in Phoenician economics.

A lot of the new entrants into CS and IT over the last ten years or so have degrees - they're just not in the "obvious" field. Someone, and I wish I could find the original, wrote an article in one of the industry mags (like C/C++ Users Journal, not IEEE Computer) that, to be good at software in the modern era, one needed to have exposure to "behavioral science, psychology, linguistics, human factors, sociology, philosophy, rhetoric, ethnology, ethnography, information theory, economics, organizational politics, and a dozen other things - and please, please learn to write competently in English!" - I've had that taped above my display for years now. So it's not that we're not educated; the problem - and it is a problem - is that there is no universal common body of knowledge for software "engineering" - which is one of the necessary precursors of any true profession. We're not going to have a CBOK without either a broad consensus within the industry, or imposition of a system from outside (and very narrowly-focused) forces in government or the larger economy. Given the prevailing social and political attitudes of current practitioners ("herding libertarian-poseur cats" is a phrase not infrequently heard), that would seriously disrupt the Craft and, by extension, any industry or field dependent upon software (which by now is pretty much everything).

How to solve the problem - and, in so doing, help redress the pandemic sexism, racism and ageism (in huge parts of the world, recruiting with explicit age limits is perfectly legal, and here in South Asia, you're old for coding at 28)? I've got no idea. When I first started doing this, I thought that within the next thirty years or so (from 1979), we'd be able to turn this informal craftwork that had taken the industry away from the "educated CS types" and turn it into a real profession. Now? I'd say we're 30 to 50 years away from now - unless we have a software equivalent of the New London School explosion and a "solution" gets imposed from outside. We need to grow up, and quickly.

We Interrupt This Program...

We interrupt this tutorial to interject an observation about goals, methods and promises. Goals we have for ourselves as people and as professionals; the method we use to pursue those dreams; perhaps most importantly, the promises we make, both to ourselves and to our customers, about what we're doing and why.

I consider this after reading the Website for some consultants who've done some (relatively nice, relatively low-key) link-dropping on LinkedIn. I'm not naming them here, not because I don't want to draw attention to them (their own site is very clean and well-done), but because the point that I'm going to be making here isn't just limited to them - we, as a craft and an industry of Web development, have some Serious Problems™.

The "problem" is nicely summarized by this group's mission statement:

Our mission is to produce the perfect implementation of your dreams.

What could possibly be wrong with that?

As a goal, implied but left unspoken, absolutely nothing; both as practitioners and as clients, we tend to set ever-higher goals for ourselves. Indeed, that's the only way the "state of the art" - any "art" - can advance. But we who practice the art and craft of software (including Web) development (as opposed to the engineering discipline of hardware development) have a history of slashed-beyond-reality schedules and budgets coupled with a tendency for stakeholders not to hear "if all goes well" as a condition to our latest schedule estimate. We have a history, perceived and actual, of promising more than we can deliver. Far more attention is paid by non-technical people to the "failures" and "broken promises" of software than to things done right. For a craft whose work is accruing increasing public-policy and -safety implications, the effect of unrealistic expectations, brought about by poor communication and technical decisions being made by people who aren't just technically ignorant but proud of the fact, is disturbing. What started as a slow-motion train wreck has now achieved hypersonic speeds, and represents a clear and present danger to the organisational health and safety of all stakeholders.

I don't mean to say that projects always fail, but an alarming number of them do. If, say, dams or aircraft were built with the same overall lack of care and measurable engineering precision that is the norm in commercial software development, we'd have a lot more catastrophic floods, and a lot few survivors fleeing the deluge by air. When I entered this craft thirty years ago (last May), I was soon led to believe that we were thirty to fifty years away from seeing a true profession of "software engineering". As a time frame beginning now, in 2009, I now think that is almost laughably optimistic.

Why have things gotten worse when we as a tool-building and -using society need them to get better? Some people blame "The Microsoft Effect" - shipping software of usually-dubious quality to consumers (as opposed to 'customers') who have bought into the (false) idea that they have no realistic choice.

It's more pervasive than that; commercial software development merely reflects the fashion of the business "community" that supports it, which has bought into one of the mantras of Guy Kawasaki's "The Art of Innovation", namely "don't worry, be crappy." Not that Kawasaki is giving bad advice, but his precondition is being ignored just as those of other software people have been: the key sentence in his "don't worry, be crappy" paragraph is "An innovator doesn't worry about shipping an innovative product with elements of crappiness if it's truly innovative" (emphasis mine). In other words, if you really are going to change the world, nobody will notice if your Deus ex Machina 1.0 has clay feet as long as you follow up quickly with a 1.1 that doesn't...and follow that with a 2.0 that changes the game again. But that space between 1.0 and 1.1 has to be fast, Kawasaki argues (in the next paragraph, titled "Churn, Baby, Churn"), and the version after that has to come along before people (like possible competitors) start saying things like "well, he just brought out 1.1 to fix the clay feet in 1.0." If the customers see that you're bringing out new versions as fast as they can adapt to the previous ones, but that each new version is a vastly superior, revelatory experience compared to the earlier release that they were already delighted by, they'll keep giving you enough money for you to finish scaling the "revolutionary" cliff and take a (brief) rest with "evolutionary" versions. Business has not only forgotten how important that whole process is to their continued survival, but they've removed the capability for their bespoke software (and Web) infrastructure to use and reuse that model. All that remains is "it's ok if we ship crap; so does everybody else." That's the kind of thinking that made General Motors the world-bestriding Goliath it is today - as opposed to the wimpy also-ran it was (emphatically NOT) half a century ago. We really don't need any more businesses going over that sort of cliff.

What we do need, and urgently, are two complementary, mutually dependent things. We need a sea change in the attitude of (most) businesses, even technology businesses, towards software - to realise and acknowledge that the Pointy-Haired Boss is not merely a common occurrence in the way business manages software, but actively threatens the success of any project (and business) so infested. Just as businesses at some point realise that "paying any price to cut costs" is an active threat to their own survival, they need to apply that reality to their view of and dealings with the technical infrastructure that increasingly enables their business to function at all.

Both dependent on that and as an enabler of that change, the software and Web development industry really needs to get its house in order. We need to get away from the haphazard by-guess-and-by-golly estimation and monitoring procedures in use by the majority of projects (whose elaborate Microsoft Project plans and PowerPoint decks bear less and less resemblance to reality as the project progresses) and enforce use of the tools and techniques that have been proven to work, and have an organised, structured quest to research improvements and New Things.. Despite what millions of business cards and thousands of job advertisements the world over proclaim, there is no true discipline of "software engineering", any more than there was "oilfield engineering" in widespread use before the New London School explosion of 1937. Over 295 people died in that blast; we have software-controlled systems that, should they fail, could in fact hurt or kill many more - or cause significant, company- or industry-ruinous physical damages. We should not wait for such an event before "someone" (at that point, almost certainly an outside governmental or trans-governmental entity) says "These are the rules." While I understand and agree with the widespread assertion that certification tests in their present form merely demonstrate an individual's capability to do well on such tests, we do need a practical, experiential system - probably one modelled on the existing systems for engineering, law or medicine. Not that people should work 72-hour shifts; there's enough of that already. But rather that there should be a progression of steps from raw beginner to fully-trusted professional, with a mix of educational and experiential ingredients to ascend that progression, and continuing educational and certificating processes throughout one's entire career. The cost for this is going to have to be accepted as "part of the system" by business; if business wants properly competent engineers, and not just the latest boatload of unknowns with mimeographed vendor certs, then they're going to have to realize that that benefit does not come without cost to all sides. The free ride is over - for all the stakeholders at the table.

Tutorials, best practices and staying current

A gent by the name of Brian Carey has written a very nice little tutorial on "Creating an Atom feed in PHP", and gotten it published on the IBM DeveloperWorks site. In the space of about ten pages, Brian gives a stratospheric overview of what Atom is and why PHP is a good language for developing Atom-aware apps, and then gets into the tutorial - defining a MySQL database table to hold the data used to 'feed' the Atom feed, and writing code to get the data out and put it into the form that a reader such as NetNewsWire expects an Atom feed to be in.

Now, to be fair, Brian describes himself as "an information systems consultant who specializes in the architecture, design, and implementation of Java enterprise applications", and the paper is clearly meant as a whirlwind tutorial, not to be taken by the careful/experienced reader as necessarily production-quality code. And if this had been published in, say, 2002 or so, I'd have thought it a great how-to for banging out some PHP 4 code. But this is 2009 PHP 4 is a historical artifact, and blogs and industry journals of seemingly every stripe are decrying the poor quality (security, maintainability, etc.) of PHP code... much of which is still written as if it were the turn of the century, ignoring PHP 5's numerous new features and the best practices that both spawned them and grew from them.

So what's really wrong with doing things like they did in Ye Olden Tymes™?

• Procedural code makes it harder to make changes (fix bugs, add features, change to reflect new business rules) and be certain that those changes don't introduce new defects. This is largely because...
• While it is possible to test procedural code using automatable tools like PHPUnit, it's a lot harder and more complex than testing a clean, object-oriented design.
• Hard-coding everything, interspersing 'magic values' throughout code, is a major hindrance to future reuse - or present debugging;
• Using quick-and-dirty, old-style database APIs exposes you to the risk of input data that is more 'dirty' than it should be - opening the door to SQL injection and other nastiness;
• Not staying current exposes your code to the risk that it's either using features that have since been deprecated or removed entirely, or (arguably worse), the risk that new features (such as standard library additions) make some of your existing code redundant at best.

Each of these, to varying degrees, is true of much of the PHP code I've read in the last couple of years, including that in the DeveloperWorks paper. For example, the DW paper's code makes use of the old-style PHP4 mysql_* API rather than the more abstract/portable MDB2 database abstraction layer. And there's also the rather terse implementation of a date3339() function for converting a timestamp to RFC 3339 output format, that's now nicely handled through the standard PHP DateTime class.

How would I have done things instead? Read the next few posts in this blog to find out. And, of course, comments are always welcome.

Smokin' Linux? Roll Your Own!

As people who've encountered the "business end" of Linux have known for some time, the system (in whichever distribution you prefer, greatly rewards (some would say 'requires') tinkering and customisation. This can be done with any Linux system, really; some distros, like LinuxFromScratch and, to a lesser degree, Gentoo and its derivatives, explicitly assume that you will be customizing their base system according to your specific needs.

Now the major distros are getting into it. There have been various Ubuntu and Fedora customisation kits on the Net, but none as far as I can tell that are as directly supported (or easy to use) as from OpenSUSE, the "community-supported" offering from Novell (who also offer SUSE Linux Enterprise Desktop and Server.

Visit the OpenSUSE site, and prominently visible is a link to the OpenSUSE Build Service, which "allows developers to package software for all major Linux distributions", or at least those that use rpm packaging, the packaging system used by Red Hat, Mandriva, CentOS, and other similar systems. But that's not all...

SUSE now have a new service, SUSE Studio, which allows users to create highly customized systems based on either the community (OpenSUSE) or enterprise versions of SUSE Linux. These "appliances" can be put together on the basis of "patterns", such as lamp_server (LAMP, or Linux/Apache/MySQL/PHP Web server) or technical_writing (which includes numerous tools like Docbook). You can even supply your own (either self-built or acquired elsewhere) RPM packages to include in the appliance you're building, and SUSE Studio will deal with the dependency matching (warning you if packages are required that aren't either among its standard set or uploaded by you).

Startup scripts, networking, basically anything that is usually handled through the basic installation or post-installation configuration - all can be configured within the SUSE Studio Web interface.

And then, when you've got your system just the way you want it, you can build it as either an ISO (CD/DVD) image to be downloaded and burned onto disc, or as a VM image for two of the most popular VM systems (VMWare and Xen).

But wait, there's more...

Using a Flash-enabled browser, you can even "test drive" your appliance, testing it while running (transparently) in an appropriate VM hosted within the SUSE Studio infrastructure. Especially if you have a relatively slow connection, this will let you do preliminary "smoke testing" without having to download the actual image to your local system. Once you're ready to do so, of course, downloading is very nearly a single-click affair. Oh, and you're given (presently) 15 GB of storage for your various builds - so you can easily do comparative testing.

What don't I like about it? In the couple of hours I've been messing around with it today, there's really only one nagging quibble: When you do the "test drive" of your new creation, the page you're running it in is a standard, non-secure http Web page. The page warns you that any data and keystrokes sent will not be encrypted, and recommends the use of ssh if that is a concern (by which most people will think https). But there's no obvious way to switch, and shutting down the running appliance (which starts by the time you read the warning) involves keystrokes and so on...

In fairness, this is still very clearly a by-invitation beta offering (but you can ask for an invite), and some rough edges are to be expected. I'm sure I'll run into another one or two as things go on. I'm equally certain that all the major problems will be smoothed out before SUSE Studio goes into general public availability.

So, besides the obvious compulsive hackers and the people building single-purpose appliance-type systems, who would really make use of this?

One obvious use case, which the SUSE Studio site describes, is as a canned demo of a software system. If you're an ISV, you can add your software or Web app to a SUSE Studio appliance, lock down the OS image to suit (encrypting file systems and so on), and hand out your discs at your next trade show (or have them downloadable from your Website). No worries about installing or uninstalling from prospective customers' systems; boot from the CD (or load it into a VM) and they're good to go.

Another thought that hit me this morning was for use as an interview filter. This can be in either of two distinct modes. First, you might be looking for people who are really familiar with how Linux works. Write up the specs of a SUSE Studio appliance (obviously more demanding than just the click-and-drool interface) and use an app of your own devising to validate the submitted entries. This validation could be automated in any of several ways.

The second possible interview filter would be as a programming/Web dev system. As a variation on the "ISV" example above, you load up an appliance with a set of tools and/or source files, ready to be completed and/or fixed by your candidates. They start up the appliance (either a live CD or VM), go through your instructions for the test, and then submit results (probably an encrypted [for authentication] archive of all the files they've touched, as determined by the base system tools) via email or FTP. On your end, you have a script that unpacks the submission into a VM and uses the appropriate automated testing tools to validate it. I can even see this as a business model for someone who offers this capability as a service to companies wishing to have a better filter for prospective candidates than resume-keyword matching - which as we all know is practically useless due to the high number of both false negatives and false positives.

What do you all think?

The Debate between Adequacy and Excellence

I was clicking through my various feeds hooked into NetNewsWire, in this case The Apple Core column on ZDNet, when I came across this item, where the writer nicely summed up the perfectly understandable strategy Microsoft have always chosen and compared that with Apple and the Mac. Go read the original article (on Better Living without MS Office and then read the comment.

As I've commented on numerous times in this blog and elsewhere (notably here), I'm both very partial to open standards (meaning open data formats, but usually expressed in open source implementations) and to the Apple Mac. As I've said before, and as the experience of many, many users I've supported on all three platforms bears out, the Mac lets you get more done, with less effort and irritation along the way, than either Windows or Linux as both are presently constructed.

But the first two paragraphs of this guy's comment (and I'm sorry that the antispam measures on ZDNet apparently don't permit me to credit the author properly) made me sit up and take notice, because they are a great summation of how I currently feel about the competing systems:

The Macs vs. PC debate has been going on for about 25 years or so, but the underlying debate is much older. What we are really discussing is the difference between adequacy and excellence. While I doubt I would want to be friends with Frank Lloyd Wright or Steve Jobs, both represent the exciting belief in what is possible. While Bill Gates and Steve Ballmer rake in billions, their relative impact on the world of ideas is miniscule.

Bill Gates understands that business managers are on the whole are a practical, albeit uninspired and short-sighted bunch. By positioning Microsoft early on to ride into the enterprise with the implicit endorsement of one of the biggest, longest-lived, and influential suppliers of business equipment, Gates was able to secure Microsoft's future. Microsoft's goal has never seemed to me to be to change the world, only to provide a service that adequately meets business needs. Microsoft has also shown from early on a keen awareness that once you get people to use your product, your primary goal is not to innovate to keep your customers, but, rather to make leaving seem painful and even scary. Many companies do this, but Microsoft has refined this practice into an art.

He then expands on this theme for four more paragraphs, closing with

Practically speaking Microsoft is here to stay. But I am glad that Apple is still around to keep the computer from becoming dreary, to inspire people to take creative risks, to express themselves, and to embrace the idea that every day objects, even appliances like the computer, can be more than just the sum of their functions.

Aux barricades! it may or may not be, depending on your existing preferences and prejudices. But it does nicely sum up, more effectively and efficiently than I have been able to of late, the reasons why Apple is important as a force in the technology business. Not that Microsoft is under imminent threat of losing their lifeblood to Apple; their different ways of looking at the world and at the marketplace work against that more effectively than any regulator could. But the idea that excellence is and should be a goal in and of itself, that humanity has a moral obligation to "continually [reach] well past our grasp", should stir passion in anyone with a functioning imagination. Sure, Microsoft have a commanding lead in businesses, especially larger ones — though Apple's value proposition has become much better there in the last ten years or so; it's hard to fight the installed base, especially with an entrenched herd mentality among managers. But, we would argue, that does not argue that Apple have failed, any more than the small number of buildings designed by Frank Lloyd Wright and his direct professional disciples argue for his irrelevance in architecture. If nobody pushes the envelope, if nobody makes a habit of reaching beyond his grasp, how will the human condition ever improve? For as Shaw wrote,

The reasonable man adapts himself to the world. The unreasonable man persists in trying to adapt the world to himself. All progress, therefore, depends upon the unreasonable man.

And that has been one of my favourite quotes for many years now.

Web Standards DO Save, Then and Now

It's been known in the Web-development community for several years now that well-designed, semantic, standards-compliant Websites use dramatically less resources (such as bandwidth) than 1997-era tangles of nested tables and invalid HTML. But it's still refreshing to read confirmation that that truth is pretty universally applicable - especially when that reading doesn't depend on what we now know as the "latest and greatest" Web browsers.

Take, for example, this five-year-old article by Jim Ramsey, then-Webmaster of the San Francisco Examiner newspaper's Website. In the section titled Simplify, Man!, he writes:

This is what a basic link in our navigation looked like late last year, before standards:

<tr>
<td class="navmenu" height="18"
  onClick="javascript:rolloutNav(this);document.location='/home/index.cfm'"
  onMouseOver="javascript:rolloverNav(this);"
  onMouseOut="javascript:rolloutNav(this); " colspan="2">
<a href="/home/index.cfm" class="nav">HOME</a></td> </tr><tr>
<td bgcolor="#EEEEEE" class="navmenuspacer" colspan="2">
<img src="../site_images/spacer.gif" width="1" height="2"></td> </tr>

Now take a look at what an Examiner navigation link looks like now:

<li><a href="/home/">Home</a></li>

That’s a big difference. In fact, the first one is so bad, I’m almost embarrassed to include it here. And what did I get for all that extra stuff? Basically, nothing. The JavaScript triggers the rollover effect and the table cells control the spacing. All of that can be done using styles.

Let’s take another example. Here’s how a link to a story in the Arts section looked before standards:

<img src="../site_images/sfex/homekickerarrow.gif" width="6" height="8">
<span class="kicker">Movie Review: Dickie Roberts<br></span>
<a href="../templates/story.cfm?displaystory=1&storyname=090503a_dickie"
    class="headlinesm">Problem 'Child'</a>
<hr noshade size="1" color="#EEEEEE">

Here’s the same thing following standards:

<h5>Movie Review: Hero</h5>
<h4><a href="/article/index.cfm/i/082704a_hero">Holding out for a 'Hero'</a></h4>

Again, once it is styled, the second version can be made to look identical to the first. When you can simplify markup in this way, it starts to make a big difference in bandwidth.

Comparing last year’s table-based site to our new standards-based one, the amount of information on our homepage is strikingly similar. Both contain basically the same elements and yet the HTML is 13K smaller on the CSS-version at 19.6K.

As a result, even though our traffic was about 40% higher in July 2004 than in September 2003, our bandwidth was almost exactly the same for those two months.

"[E]ven though our traffic was about 40% higher...our bandwidth was almost exactly the same..." What other development practice allows you to simultaneously:

• boost your traffic by nearly half without similar increases in bandwidth costs;
• improve your search-engine results without expensive, error-prone twiddling;
• open your site to a potentially much wider audience, by not limiting what platform or browser your audience uses; and
• significantly reduce the cost and complexity of maintaining your site?

As one colleague put it in an IM discussion, "people who develop like it's 1997 shouldn't be surprised if their revenues and page views don't exceed 1997 levels. Especially if they weren't around in 1997."

I'm not (quite) to the point of some of the more, um, evangelical developers out there in equating broken, invalid sites with actionable incompetence — but if our craft has serious hopes of making it out of "hobbyist" status in the eyes of non-technical business people, to where they treat practitioners as members of a profession, then we need to get some meaningful, practical, well-defined standards. This (standard-compliant, semantic development) is at or very near the top of my list of such standards.

Final note: I find it deeply ironic that Google have apparently redesigned the posting interface for Blogger. Whereas previously I have been able to post from any Linux, Mac or iPhone browser, switch between HTML editing and WYSIWYG view, and use all the other goodness...the new CSS and JavaScript seem to fail in any of the half-dozen Mac browsers I've tested today. One step forward, two steps back is especially painful when one started by walking directly away from the very edge of a cliff.

Tools, Continued

UPDATED 2 November 2010; see last paragraph.

---

This blog, fairly obviously, is published on blogger.com, which is now owned by Google. Blogger.com is geared primarily towards people who want to write but don't want to have to worry about the nitty-gritty technical details involved (such as HTML, CSS and so on). Sign up, click the 'New Post' button, and off you go...almost easier than falling out of your chair.

For the Web wizards among you, you can get into the guts of how your blog and posts are laid out and formatted; there are several in-the-tin and numerous third-party templates that can be used to style your blog any way you like it, and those can then be hand-tweaked by you to get things just so.

As with any click-and-go interface, the Blogger new-post window (what I'm typing in right now) has a "Preview" button (or actually, link); click on it and you'll see a more-or-less-reasonable facsimile of what your breathless prose will look like to the next visitor who stumbles across your blog. I did say "more-or-less-reasonable"...

One thing the preview area doesn't do — sensibly in hindsight — is to apply your template's formatting to the post being previewed. So that, for instance, if your template specifies that you want 9-point Gentium Book Basic, Times New Roman or Times (in that order) for your body, you won't see it that way in the preview.

What you also won't see — and this is what tripped me up for the longest time — is other styling for the post body, particularly justification. You may notice that all my posts are displayed with justified left and right margins; the default, and most commonly used setting, is for a justified left and ragged right margin, which I find unattractive. I've tweaked the template I use several times over the years to try and get the effect I was looking for. Each time, the post-preview showed no changes to the text formatting, and so I undid the change without viewing the entire blog normally. I have been instead hand-wrapping the content of an entire post in a <div>...</div> element pair whose only reason for existence was to set text-align: justify;.

This isn't the first time I've blogged about my learning-experiences-that-shouldn't-have-been, and just to be perfectly clear about this, I'm not trying to dis Blogger.com about this. The preview-while-composing feature is merely to let you see the content of the post you're working on without all the editing framework around it. It won't, and arguably shouldn't try to render that content in its final form. You do, after all, have the ability to go back and edit posts you've already published.

This is, at its heart, a cautionary tale for those of us tasked with making the Web easier to use for a wide variety of users. Be careful especially with interface design. Recognise that, barring explicit cues to the contrary, people's assumptions about How Things Work on your site may have only a nodding acquaintance with your own — but if you have some appropriate description in the right place, users are often happy to adjust. But there has to be a balance, or your 'power users' will feel like they're being stifled. How you achieve that balance is, of course, up to you to find out. Good luck.

UPDATE: Blogger have, since I wrote the original post, changed their preview feature (at least for "Blogger in Draft," their next-gen interface, which I use and recommend.) When you click the Preview button now, Blogger opens a new tab (or window, if you're old-school enough not to use tabs) with the preview of your draft post. This preview is exactly as a visitor to the published post would see it, with the exception of a great big Preview stripe in the upper-left corner. A wonderful improvement; thanks, guys!

Misadventures with Customer "Service": With Service like This, Why Bother?

Doing a job search using various Websites became really popular about ten years ago as an easy way for candidates and companies to find each other without any (visibly obvious) middlemen directly involved. I understand that in some geographic and employment areas, it's still a useful tool. I've had myself registered on fewer than half a dozen for several years, even though my last 6 jobs were all found through other means.

Job sites aren't typical consumer Websites in that they're really trying to target two distinct groups of customers: companies with jobs on offer (who generally pay the bills) and jobseekers (who generally don't). Even though jobseekers don't have any up-front money dangling in front of the site operator, wise (or experienced) operators of such systems know that they should "take care" of these site users well, since:

• Companies will be less likely to pay to advertise on general-audience sites with few users, and
• Users who have negative experiences (on any site) are often motivated to "spread the word".

Any business which is heavily dependent on its Website for revenue or for customer service knows (or quickly learns) that locking actual or potential customers out of their site through defective, easily-correctable site implementation has a direct and negative impact. The pathologically clueless operators don't seem to know or care why this happens; they just notice their monthly traffic figures dwindling over time, with revenues to match.

This even applies, although to a lesser extent, when the market being served by the site encourages a lack of competition. This is the case quite often here in South Asia, and notably in Singapore. Only two media companies serve the English-language market here, both with apparent Government ties. Broadcast television and radio are similarly throttled. This becomes apparent to anyone who spends more than a few days here. Individuals are engaged in frenetic, all-consuming competition with each other, while favoured companies and industries don't (visibly) engage in such grubbiness.

All of this went through my mind again over the last few days, after I'd had a series of consistently unpleasant experiences with one particular Website operator. Necessary background: I have been essentially Windows-free (and thus virus-free and botnet-free) for some five years now, initially using Linux systems like Ubuntu and then the Apple Mac. What has made this change practical, for me and for tens of millions of other people, has been the emergence of open standards in desktop computing, particularly with regard to open data formats. A classic example with which anyone reading this is familiar is HTTP. Any device, from large mainframes to appliances to mobile phones, can be used to produce or access information via HTTP, most commonly in the form of Web pages. Other "standards", like MP3 for audio or Microsoft Office document formats, are proprietary to a company or organisation but widely implemented by competing or collaborating systems. This reliance on actual or de facto standards, as much as any other single factor, is what has enabled much of modern technology, particularly the Internet (which many think of as synonymous with the World Wide Web).

Not everybody has always "played fair" with these standards, however. Microsoft, in particular, have been infamous for their commonly-used practice of "embrace, extend and extinguish" has led to several instances where they support an existing standard (such as HTML for Web pages or Kerberos for network authentication) and either introduce incompatible features or defective versions of standard features into their implementation. This has the effect of "locking in" users who support Microsoft's implementation rather than the actual standard. These usees, as they are often called, face significant actual or perceived costs and difficulties were they to switch away from their now-Microsoft-specific infrastructure. Microsoft have by no means been the only ones to do this sort of thing; they are, however, often quite brazen about it — and their usees don't always have an accurate understanding of the true costs of their "investment."

In particular, during the "dark ages" of Web development in the mid- to late 1990s, it was common for sites to have text on each page proclaiming "Best viewed with Microsoft Internet Explorer"; the tools which they were using to create the Web pages (often Microsoft FrontPage), did not always (or easily) generate valid HTML, CSS or standard JavaScript — all of which are needed to make interactive, "dynamic" Web sites work. As the relevant standards have become better-known to amateur and semiprofessional Web developers, the tools used by professional developers mature, and those using Web browsers become more aware of the importance of these standards, we have seen an increasing Web "Enlightenment", where once again, anybody (or anything, like a Google search crawler) can access all content for a site.

There are exceptions to this, however, just as there are trailing-edge sluggards in any social or technical change. These can give excuses about lack of resources, perceived lack of need ("We're doing fine the way things are") and so on — which quite often attempt to mask either ignorance of the issues involved or, at a fundamental level, disrespect for their current and actual customers. When a well-equipped, motivated (outside) organisation is in a position to gain from that ignorance or contempt — as Microsoft for so long was with regard to Web development and standards — those "sluggards" will continue to pat themselves on the back for their "prudence" and "conservatism" — right up to and often past the point at which the business dies.

One such organization appears to be the company which owns and operates the JobsDB.com group of employment Web sites. With "specialised" sites for Singapore, Thailand, the Philippines, Australia and other countries, and an apparently large advertising and promotion budget, it's safe to speculate that millions of people have at least seen their advertisements, if not actually made use of their site. Those who actually do try to browse for a job there, however, can face significant problems — if they're using anything other than Internet Explorer on Windows as their browser. (And the message seems to be finally getting through to many users — Microsoft Internet Explorer and Windows itself cause numerous security and stability problems that are comparatively unheard of on any other system.)

I browsed jobsdb.com.sg, the Singapore-themed version of their online job service. I got as far as entering my details in an application form for a position, when I ran into insurmountable difficulties. Numerous pages display poorly in any of the browsers I tried. "Obviously using archaic design," I thought, "but as long as the function isn't too badly broken, I can still deal with this." I eventually got to the a page with a form to be filled in by those wishing to apply for a specific job (at SingTel) and ran into fatal problems. I clicked on the link to report a problem with the site, and included the following description:

SingTel application page misbehaves horribly with Safari 4.0.2/Mac. An ASP alert box, (URL omitted for this blog post), is displayed in a new tab, taking the entire browser window; the message "Please correct the field(s) with red exclamation mark(s) (!). You can click on the exclamation mark for instruction and input relevant information." indicates that there was an error on the initial page (which is still open in its original tab), but no red exclamation marks appear.

Very 1998ish.

Sixteen hours after filing the initial report, and getting an auto-generated email indicating that it had been received and entered into their ticketing system, I get the following email from "sg-cshelpdesk@jobsdb.com.sg"):

Dear Sir / Madam,

Please clear the cache of your browser by deleting the cookies and temporary internet files, enable the java and javascript, disable the popup blocker, open a new browser and try from there.

Otherwise, please try other browsers like firefox or Internet Explorer 6.0 & above to access the Career Portal and verify that your Internet Settings are configured correctly against http://sg.career.jobsdb.com/faq/documents/Internet_Explorer_Settings.pdf. Then open a new browser and submit from there.

Best Regards, Customer Service

Absolutely no problem-specific content whatsoever. This could easily have been (and in my opinion probably was) an automatically-generated message that was sent when no relevant response would be forthcoming within 24 hours after filing the problem (the '16:44' in the timestamp, or 4:45 PM local time, sounds suspiciously like "it's the end of the day, nobody got to this, blow him off and hope he goes away.") There continues to be an explicit assumption within the fluff mail that I'm using Internet Exploder; even though the second paragraph mentions "firefox" (sic), the PDF link supplied says that it relates to IE.

I'm getting used to companies, particularly the all-pervasive oligopolies, here in Singapore treating the customer with contempt. Knowing that no living human would probably ever read it, however, I did send one last complaint email.

Dear Customer “Service”,

Did you actually read the report I filed? Here are a few Statistically Improbable Phrases to clue you in:

• “standards-compliant (non-IE) browsers”. I tested your site with ten different browsers on four different operating systems. In no case did it operate correctly. By the way, Microsoft does not make a version of Internet Exploder for non-Windows systems.
• “Safari” — Safari is the standard, comes-with-the-system browser for Macs and iPhones, and is available for those wishing to upgrade their Internet experience on Windows as well.
• “an ASP alert box...is displayed in a new tab” — obviously not what was intended by the coder; however, competent JS wouldn’t do this.

I am a Web developer with nearly 15 years experience. I have been teaching Web development for ten years. If a student of mine from my very first class, back in 1999, had submitted work with such grossly defective JavaScript, he would have been required to redo it. You see, even since the early days of the World Wide Web, there have been standards that even Microsoft can comply with – though they also “support” their own proprietary, non-standard version of HTML and JavaScript and have encouraged careless or inexperienced “developers” to lock themselves in, breaking sites for anyone using a non-IE browser. That includes half the smartphones on the planet, every single non-Windows desktop or laptop system, and search engines like Google.

You should take http://www.amazon.com/Novelty-Sign-Brain-before-engaging/dp/B000K62V92 as a gentle but firm recommendation.

Sincerely,

No, I don't expect a relevant reply. Yes, I do feel better. No, I won't be using JobsDB or any related site, and will happily explain to any who ask my opinion precisely why.

A final note: There is a nice little site called yougetsignal.com which offers a nice set of network (Internet) diagnostic and information tools. One page on the site, http://www.yougetsignal.com/tools/web-sites-on-web-server/, does what's called a "reverse IP address lookup". I used this page to find out what other sites are hosted by the same server as jobsdb.com.sg. Try it yourself; the results included apparently all "country-specific" jobsdb.com sites — as well as the site for Target, an American retailer. Interesting.

Not that I expect it, but if any of the parties mentioned here were to reply to what I've written here, I'd be happy to publish their response.