Trust, Tools, and Tripping Over Shoelaces While Dancing

On the shoulders of giants, that is.

While "a poor craftsman blames his tools", sometimes those tools lead you down the garden path to such a degree that you don't realise that you're doing a Wile E Coyote impersonation until about the third bounce off the canyon floor. For your amusement and edification, if not abject stupefaction, I present a subset of a passing spec file. This is CoffeeScript, using teaspoon and mocha for specs. It's deceptively straightforward, IMAO:

#= require 'gateway'

describe 'Gateway class', ->

  beforeEach ->
    @subject = new window.meldd.Gateway()
    @name = 'foo'
    @value = $('body')

  # other specs elided...

  describe 'has a "useGroup" method that', ->

    it 'takes one parameter', ->
      expect(@subject.useGroup).to.have.length 1

    describe 'when called using', ->

      it 'a nonexistent group name, returns an empty object hash', ->
        group = @subject.useGroup 'not found'
        expect(group).to.be.an 'object'
        expect(group).to.be.empty()

      describe 'a group with one key, returns an object', ->

        beforeEach ->
          @subject.register 'foo', 'bar', 'group1'
          @group = @subject.useGroup 'group1'

        it 'hash', ->
          expect(@group).to.be.an 'object'

        it 'with the correct single item name as the only key', ->
          expect(@subject.groups['group1'][0]).to == 'bar'
          expect(Object.keys(@group)).to == ['foo']

        it 'with the correct item value as the only value', ->
          expect(@group['foo']).to == 'bar'

      describe 'two groups with one identical key, returns', ->

        beforeEach ->
          @subject.register 'this', @, 'group1', 'group2'
          @group1 = @subject.useGroup 'group1'
          @group2 = @subject.useGroup 'group2'

        it 'the identical object in both groups', ->
          # equality to avoid "Converting circular structure to JSON" error
          expect(@group1['this']).to == @
          expect(@group2['this']).to == @
          # identity; they're the exact same object instance
          expect(@group1['this']).to.be @group2['this']

Pretty straightforward, and both of you Scripters reading this can probably code up a proper Gateway.useGroup method without knowing much about the rest of the class. I'd bet the rent that your method would not closely resemble this:

  useGroup: (param) ->
    {}

And yet...

What. The. Fox?!?

The Formerly Magical Trackpad

My Apple Magic Trackpad died tonight, aged less than six months, after a week-long illness. Numerous battery transplants, resets and reboots were unable to revive the patient. In lieu of flowers, donations to its replacement are now being gratefully accepted.

Whiskey. Tango. Fox?

Hardware — any hardware, but especially Apple kit — isn't supposed to die after mere months of daily use. I expect better from systems, and especially from Apple. Were I anywhere else on the planet besides Second World Singapore, I would have a reasonable degree of confidence that those expectations would be fulfilled. PAPaganda notwithstanding, this is the Potemkin city where having the highest-paid politicians on the planet doesn't mean that products or services (beyond political control-related security "services") are up to the same levels as would be expected in, say, Tanzania.

So I'm back to using the mouse that I'd been using for nearly two years before buying the Trackpad. With the wheel and five buttons (that I know of), it's certainly adequate for most ordinary-user or Web-developer tasks in Snow Leopard.

But trackpads are The Future™, a reality emphasised by the new OS X 10.7 "Lion" release, and by what's rumoured for Windows 8. While mice are comfortable and familiar to essentially everyone who's used a Mac in the last 27 years, or Windows PC in the last 15 or so, there are actions and gestures that the trackpad is simply a more natural fit for, and to which a trackpad user's "muscle memory" rapidly adapts. Going back to the mouse feels both familiar and foreign, not unlike attempting to ride a bicycle for the first time in thirty years (another recent experience).

So, I'll probably buy a replacement trackpad fairly soon. That is, if I can stay online long enough to get useful work done; we've had intermittent failures in the DSL service here of late. (Or am I allowed to say "intermittent successful access"? This is Second World Singapore, a family-run company town that dislikes any attention paid to the clay feet of its mighty statuary (and I'm not dissing the Merlion here).

Sigh.

Windows? Check. Skype? Check. Quality? Oops…

Several cultures share variations of a proverb which reminds us that you cannot build a house on sand and have it endure the test of time — or even the next tide.

Skype learned that the hard way a couple of weeks ago. Like all peer-to-peer, or P2P, networks, Skype relies on (a varying subset of) its users' systems to handle routine network-housekeeping chores, including routing. Fine; that's the way many applications have done things for more than a decade. Skype have been living unusually close to the edge in two respects: they have an unusually large, widely-distributed number of simultaneous users relative to other P2P networks, and it implicitly trusts the reliability of its Windows-based application and the underlying Windows operating system when performing at any needed scale. This is what sent the waves crashing in upon the sand-borne house.

Microsoft Windows is, inarguably, a wildly successful software system; along with Microsoft's Office set of applications, it is the basis of Microsoft's billions of dollars a year in revenue and profit. But just because something is successful or because it's made by a major corporation doesn't necessarily mean it's well-designed or -built. (See, for example, the Ford Pinto.)

By relying on the (always-questionable) stability and reliability of each of the Windows PCs acting as a "supernode" in its network architecture, Skype was setting itself on a path to eventual inevitable, foreseeable large-scale failure.

Many of the well-known problems with keeping Windows stable and running properly have to do with the ridiculous ease of developing, distributing and using malware; in some countries (such as Singapore), estimated infection rates of Internet-connected Windows PCs routinely hit or exceed 90%. Besides the obvious threat to any "secure" information on those systems (credit card info, login IDs, and so on), those infected Windows systems have a relatively large share of their processing power diverted for the benefit of someone other than the legitimate user or of applications other than those that s/he has authorised.

Another cause of instability and other problems, as with the Skype outage, is that it is very difficult to develop, deploy and maintain even moderately complex applications under Windows with any real means of asserting that the software is reliable and robust. The only way to gain any such real assurance is by testing as many hardware and software combinations with your software as you can, and hoping that you've found all the "most likely" problems. (This is why Microsoft and others employ thousands of software testers, in addition to a massive automated-testing infrastructure.) And, though Microsoft have made some relatively amazing improvements in this area (compare, say, Windows 98 to Windows 7), even they can't catch everything — and their development process and system architecture essentially require them to. Every Windows usee in the last 25 years has seen the "Blue Screen of Death;" that is merely an acknowledgement by Windows that it's completely lost its mind, usually after innumerable random acts of lesser senility.

Skype were bitten by both ends of this issue: a new release of their client application for Windows had defects that were not caught during development or internal testing, and by deploying that defective application on an operating system that provides little-to-no protection against maliciously or inadvertently defective applications, a large "time bomb" started ticking away, unnoticed by all until its spectacular detonation on or around 22 December 2010.

For a Skype client to be a "supernode," it cannot be behind a "firewall" that uses network address translation to isolate its clients from the big, bad Net. (There are ways around that, too, of course; but they're generally very black-hat techniques that don't scale well on a commercial basis.) A client does not have to be running on Windows to be a supernode, but given the demographics of Skype's user base, and the larger PC user base in general, the majority are.

And so, when Skype's defective Windows client ran into mortal difficulties on Windows' defective platform, the ordinary user PCs that were acting as supernodes started dropping like flies. Other Skype clients, as with any proper P2P network, simply rerouted to other supernodes. However, since many (if not most) of the Windows clients were behind NAT firewalls and there were (and are) a relatively small percentage of non-Windows clients available to serve as supernodes, the number of "leaf node" clients chasing the decreasing number of supernodes put the remaining systems under an unsustainable load, and so those Skype clients (with their supernode capabilities) dropped off the network.

Had there been a more heterogenous mix of Skype clients so that those available to perform supernode duties were not almost exclusively Windows PCs, the failure might well have been mitigated if not eliminated altogether. Or, were it practical to develop a (reasonably) guaranteed-stable application on Windows, the defect that initiated the failure cascade might well never have occurred. But by "putting all their eggs in one basket" which was known to have large, gaping holes in it, Skype were guaranteeing that their metaphorical sidewalk would eventually be covered hip-deep in broken eggs.

Monocultures in technology, as in farming, are generally Bad Things; a threat which can attack any member of a monoculture can attack all such members. (Think of the Windows malware-infection rates as an example of this.) Single points of commonality (or of control) are always eventual single points of failure.

The Skype failure could well have been a reminder of these long-known basic truths. The commercial interests involved, however, ensure that such a lesson will go unheeded. It will therefore be repeated; very possibly not with Skype, but with some other aspect of our grossly defective information-technology ecosystem. Think of it as "Unsafe at Any Speed" meeting Groundhog Day until something is done to prevent yet another China Syndrome.

Proof Against Most Idiots

Fair warning: Geekish laudatory rant ahead. Still with me? Good!

I suppose it says something less than complimentary about our Craft that, when things actually work in a sensible fashion, recovering from Stupid User Actions™, it's surprising enough to be noteworthy. But I seriously doubt that I'm the only one who's noticed this.

Case in point: yesterday, I started downloading the Debian Linux testing-version DVDs. Yes, plural; there are 8 of them., as the standard software repository comes along with the test build. Anyway...

As most of us do, I was multitasking pretty heavily last night; when I knocked off, I just put my iMac to sleep as usual. It was long after I had gone to bed that I remembered, "hey, wasn't I downloading a passel of Linux DVDs when I slept the system?"

Expecting to have several "interesting" error messages displayed when I woke the system up (if in fact the file system hadn't been somehow damaged... yes, I was used by Windows for far too long), I powered up the system and was about to log in when BAM!... 15 minutes of phone calls. Thoroughly distracted afterwards, I log in, check my email, read a couple of new bug reports for a project I'm working on, and then remember "hey! everything's working!"

I switch to the Terminal window for the download (which was using GNU wget 1.12 if you care), to be greeted by the following display (excerpting):

--2010-08-15 19:45:15--  http://cdimage.debian.org/cdimage/weekly-builds/amd64/iso-dvd/debian-testing-amd64-DVD-6.iso
Resolving cdimage.debian.org (cdimage.debian.org)... 130.239.18.163, 130.239.18.173
Connecting to cdimage.debian.org (cdimage.debian.org)|130.239.18.163|:80... connected.
HTTP request sent, awaiting response... 302 Found
Location: http://caesar.acc.umu.se/cdimage/weekly-builds/amd64/iso-dvd/debian-testing-amd64-DVD-6.iso [following]
--2010-08-15 19:45:20--  http://caesar.acc.umu.se/cdimage/weekly-builds/amd64/iso-dvd/debian-testing-amd64-DVD-6.iso
Resolving caesar.acc.umu.se (caesar.acc.umu.se)... 2001:6b0:e:2018::142, 130.239.18.142
Connecting to caesar.acc.umu.se (caesar.acc.umu.se)|2001:6b0:e:2018::142|:80... failed: No route to host.
Connecting to caesar.acc.umu.se (caesar.acc.umu.se)|130.239.18.142|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 4696719360 (4.4G) [application/octet-stream]
Saving to: “debian-testing-amd64-DVD-6.iso”

89% [===============================================================================================>           ] 4,214,102,016 --.-K/s   in 13h 40m 

2010-08-16 09:26:01 (83.6 KB/s) - Read error at byte 4214102016/4696719360 (Operation timed out). Retrying.

--2010-08-16 09:26:04--  (try: 2)  http://caesar.acc.umu.se/cdimage/weekly-builds/amd64/iso-dvd/debian-testing-amd64-DVD-6.iso
Connecting to caesar.acc.umu.se (caesar.acc.umu.se)|130.239.18.142|:80... connected.
HTTP request sent, awaiting response... 206 Partial Content
Length: 4696719360 (4.4G), 482617344 (460M) remaining [application/octet-stream]
Saving to: “debian-testing-amd64-DVD-6.iso”

100%[++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++==========>] 4,696,719,360 1.26M/s   in 6m 26s  

2010-08-16 09:32:50 (1.19 MB/s) - “debian-testing-amd64-DVD-6.iso” saved [4696719360/4696719360]

Note the report of a read error at 09:26:04 on 2010-08-16. The software was complaining that it had to recover from a timeout. Yes, I'm sure ten hours or so significantly exceeds whatever timeout value had been coded into wget... but it never missed a beat (or a byte; the SHA1 checksum matched afterwards).

I'm including the obligatory "don't try this at home, kids!" warning... but, if you aren't sitting in front of Windows, isn't it nice to know you can?

The Decline of the Internet, Part MMMDIC

Rant mode: on. Some effort will be made to keep ear-steam to a minimum.

Internet Relay Chat (better known as IRC), specifically Undernet, was one of the very first things I discovered on the (then-)text-mode-only Internet, Over the next 20 years or so, I've gone through varying levels of activity (including writing an IRC client for OS/2, the operating system that brought technology to a marketing fight (as in, "bringing a pocket knife to a firefight")./p>

One of the changes that happened on Usenet (and other networks) when the Net started getting used by larger hordes of people, with varying intentions, was to institute some form of registration. You'd visit a bare-bones Web site (such as the one for Undernet, fill in a few items on a form, and wait for an email to pop into your inbox stating that your ID was active (known to the bot managing authentication on the servers). I'd used the same "nick" (nickname), JeffD for at least as long as I or my logs can remember (certainly before 1998), and formalized this when the channels I frequented (chiefly #usa) started requiring it.

Now, I (like to think I) do have a life outside IRC and the Net in general; I've been known to go as long as a year without signing on in Undernet. Not a problem; things Just Worked™, in almost Mac-like fashion.

Until today. I fired up my current-favourite client program (Colloquy, highly recommended); it connected to a random Undernet server (in Budapest in this case), and sent the customary PRIVMSG (private message) to the bot that handles authentication, with my username and password. Instead of the usual "authentication accepted" message coming back, I see "X NOTICE I don't know who JeffD is."

Well, blowups happen. I go to the Undernet CService page I mentioned earlier; it's a whiz-bang series of 7 pages now. I go through it, enter my desired username, password, and an email address that I can be reached at (the same one I've had since text-mode days), and am finally presented with this marvel of clarity:

Congratulations! Something went wrong. Sorry.

I'd bet heavily that the reason for the breakage would, on investigation, turn out to be a case of unchecked assumptions no longer being valid, combined with a lack of human management. Nobody has the time for community-without-a-pricetag anymore, and that in a nutshell is what IRC is

And, two hours later, I still haven't received the expected email. Feh.

Rant mode: standby.

The Sun Has Set on a Great Brand

Just a quick note pointing out, once again, that when many of us foresaw gloom and doom from the Oracle acquisition of Sun Microsystems....we were being hippie-pie-in-the-sky optimists.

Item: A couple of days ago, I tried to log into my Sun Developer Network account, so I could grab the latest "official" Java toolset for a Linux VM I was building. I couldn't log in; neither the password that was saved in my password manager nor any other password I could think of would work. I then went through the recover-lost-password form, expecting a reset email to pop into my inbox within a minute or two. An hour later, I had to go do other things. Yesterday, no email.

Finally, today (Tuesday morning), this gem appears in my inbox:

Date: Sun, 30 May 2010 10:07:20 -0700 (PDT)
From: login-autoreply@sun.com
To: jdickey@seven-sigma.com
Message-ID: <12602411.48691.1275239240752.JavaMail.daemon@mailhost>
Subject: Your Sun Online Account Information
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
X-Source-IP: rcsinet15.oracle.com [148.87.113.117]
X-CT-RefId: str=0001.0A090205.4C029B48.0062,ss=1,fgs=0
X-Auth-Type: Internal IP

Thank you for contacting Sun Online Account Support.

We're sorry but a user name for your account could not be determined.

We would be happy to look into this further for you.  Please contact us at @EMAIL_FEEDBACK_CONTACT@.  We apologize for any inconvenience.

Thank you,
Sun Online Account Support

Oracle isn't even bothering to fill in (no doubt database-fed) form-mail fields to keep in touch with current and possible future Sun-originated customers, and the login-autoreply address is no doubt a black hole slaved to /dev/null. If they had sent an HTML email with 48-point red F*** YOU!!! as the entire content, the message received could not have been more clear: you don't matter, go away, throw your money, time and attention at somebody else.

I plan to do precisely that, and make clear to all and sundry precisely why.

Let's Do The Time Warp Agai-i-i-i-n!! (Please, $DEITY, no...)

For those who may somehow not be aware of it, LinkedIn is a (generally quite good) professionally-oriented social-networking site. This is not Facebook, fortunately. It's not geared towards teenagers raving about the latest corporate boy band du jour. It often can be, however, a great place to network with people from a variety of vocational, industry and/or functional backgrounds to get in contact with people, share information, and so on.

One of the essential features of LinkedIn is its groups, which are primarily used for discussions and job postings. In the venerable Usenet tradition, these discussions can have varying levels of insightful back-and-forth, or they can degenerate into a high-fidelity emulation of the "Animal House" food fight. As with Usenet, they can often give the appearance of doing both at the same time. Unlike Usenet, one has to be a member of LinkedIn to participate.

One of the (several) groups I follow is LinkedPHPers, which bills itself as "The Largest PHP Group" on LinkedIn. Discussions generally fall into at least one of a very few categories:

• How do I write code to solve "this" problem? (the 'professional' version of "Help me do my homework");

• What do people know/think about "this" practice or concept?

• I'm looking for work, or people to do work; does anybody have any leads?

As veterans of this sort of discussion would expect, the second type of discussion can lead to long and passionate exchanges with varying levels of useful content (what became known on Usenet as a "flame war.") The likelihood of such devolution seems to be inversely proportional to its specificity and proportionally to the degree which the concept in question is disregarded/unfamiliar/unknown to those with an arguable grasp of their Craft.

It should thus be no surprise that a discussion on the LinkedPHPers group of "Procedural vs Object Oriented PHP Programming" would start a flame war for both of the above reasons. With 58 responses over the past month as I write this, there are informational gems of crystal clarity buried in the thick, gruesome muck of proud ignorance. As Abraham Lincoln is reported to have said, "Better to remain silent and be thought a fool than to speak out and remove all doubt."

What's my beef here? Simply that this discussion thread is re-fighting a war that was fought and settled over a quarter-century ago by programming in general. The reality is that any language that has a reasonable implementation of OOP (with encapsulation/access control, polymorphism and inheritance, in that order by my reckoning) should be used in that way.

Several of the posts trot out the old canard about a performance 'penalty' when using OOP. In practice, that's true of only the sharpest edge cases – simple, tiny, standalone classes that should never have been developed that way because they don't provide a useful abstraction of a concept within the solution space, generally by developers who are not professionally knowledgeable of the concepts involved and quite often by those copying and pasting code they don't understand into their own projects (which they also don't understand). That bunch sharply limited the potential evolution and adoption of C++ in the '80s and '90s, and many of their ideological brethren have made their home in Web development using PHP.

Yes, I know that "real" OOP in PHP is a set of tacked-on features, late to the party; first seriously attempted in PHP 4, with successively evolving implementations in 5.0, 5.2 and 5.3, with the semi-mythological future PHP 6 adding many new features. I know that some language features are horribly unwieldy (which is why I won't use PHP namespaces in my own code; proven idea, poor implementation). But taken as a whole, it's increasingly hard to take the Other Side ("we don' need no steeeenkin' objects") at all seriously.

The main argument for ignoring the "ignore OOP" crowd is simply this: competent, thoughtful design using OOP gives you the ability to know and prove that your code works as expected, and data is accessed or modified only in the places and ways that are intended. OOP makes "software-as-building-blocks" practical, a term that first gained currency with the Simula language in the mid-1960s. OOP enables modern software proto-engineering practices such as iterative development, continuous integration and other "best practices" that have been proven in the field to increase quality and decrease risk, cost and complexity.

The 'ignore OOP in PHP' crowd like to point to popular software that was done in a non-OOP style, such as Drupal, a popular open-source Web CMS. But Drupal is a very mature project, by PHP standards; the open-source project seems to have originated in mid-2000, and it was apparently derived from code written for a project earlier still. So the Drupal code significantly predates PHP 5, if not PHP 4 (remember, the first real whack at OOP in PHP). Perusing the Drupal sources reveals an architecture initially developed by some highly experienced structured-programming developers (a precursor discipline to OOP); their code essentially builds a series of objects by convention, not depending on support in the underlying language. It is a wonder as it stands – but I would bet heavily that the original development team, if tasked with re-implementing a Web CMS in PHP from a blank screen, would use modern OO principles and the underlying language features which support them.

And why would such "underlying language features" exist and evolve, especially in an open-source project like PHP, if there was not a real, demonstrable need for them? Saying you're not going to do OOP when using PHP is metaphorically akin to saying you intend to win a Formula One race without using any gear higher than second in the race.

Good luck with that. You might want to take a good, hard look at what your (more successful) colleagues are doing, adopt what works, and help innovate your Craft further. If you don't, you'll continue to be a drag on progress, a dilettante intent upon somehow using a buggy whip to accelerate your car.

It doesn't work that way anymore.

Time v. Money v. Risk v. Frustration

I just spent three hours beating my head against The Joys of Wi-Fi Networking. No doubt because it saved $50 or so, the HDB public-housing flat I live in here in Singapore (which are "affectionately" known as 'chicken coops' for their quality and structural integrity) didn't put telephone jacks in every room. Nor did they run conduit between rooms so that suchlike could be added later. The upshot of this is that my DSL modem, in the living/dining room, is 15 meters and two concrete-slab walls away from my study/office, in a bedroom (with no phone jack and two power outlets).

Up to now, this hasn't been an insurmountable problem, because the DSL modem/802.11g router the government-linked telco sells you when you open a DSL subscription could punch a (barely) usable signal through those two concrete walls; desktop and laptop computers strewn around the study were on the LAN, with quite serviceable WPA2 encryption to keep private bits private. The router (in the living room) claimed to have a 14 Mbps connection to The Net; the aggregate total bandwidth available in the study ranged from 3 to 5 Mbps; not fantastic but usable. The world was in a survivable state of chaotic flux.

Then, yesterday morning, FedEx delivered a bright, shiny (actually used, in a Target bag rather than original box, no longer in the Polycom US catalog but still serviceable) Polycom SoundPoint 501 SIP phone, courtesy of my newest client (thanks, Nathan, Matt and Margaret - I'm not kvetching, honest!). An hour of work with jackhammer and flamethrower cleared sufficient space on the desk for the new trophy. Now to plug it in and fire it up.

Plug it in? To what? My ear?

Just to make sure I had my head straight, I looked at what it would take to run line from point P to R. Two solid concrete/rebar walls, routing around various doors/windows, across the span of the living room.... no, that wasn't going to happen, certainly not by Thursday. Put the phone in the living room? Not a chance. Wait a minute - they have these things called wireless Ethernet bridges; I should be able to get one of those, stick it in the study, connect the phone to it and it to the existing wireless LAN, and I'm good to go.

I went out to the local "IT super-mall", Funan. It's the best place in Singapore to buy electronic gear of whatever variety from reasonably reputable dealers. After visiting some 25 different shops, large and small, I was firmly reacquainted with one of the basic facts of Singapore life.

This is a firmly Stalinist economy in the areas that matter. All imports (which is to say, anything of value) are brought in through a small number of generally well-connected exclusive distributors. When one shop says "finished, already!" (the de facto motto of the city), everybody is going to say the same thing. That is, if you're fortunate enough to even run into some salesperson who actually understands what you're looking for. I could find things easier (and often cheaper!) in Vietnam.

Then I happened to pop into one hole-in-the-wall one-man shop, Crystal Systems. The "one man" said "say, I understand what you're trying to do; I did exactly that for a firm here recently; all you need are two routers." The archetypal light-bulb moment. (Remember, I'm a software craftsman, not a hardware/network engineer.) This is where the "time v. money v. ..." of the title comes in. What I should have done was to buy two identical brand-new 802.11n routers. That would have cost me about S$190 (~US$136 or so), and would have given me a perfect excuse to replace my aging 2Wire 2700HGV-2, But this has already been a bit of an expensive month - new hard drive, various software - so after some consideration I decided to try to salvage the existing router (apparent POS though it may be) and just spend the minimum necessary. So I went home with one S$69 TP-LINK TL-WR340G router.

It features the now-customary browser-based setup - just remember to use a hard-wired link rather than wireless (d'oh!), and it's almost as easy as falling off your seat. Bridging mode is pretty obvious; the TP-LINK wants to know the MAC address of the router you're connecting to, along with its encryption setup.

In bridge mode, the TL-WR340G only supports the obsolescent WEP encryption, not the current and generally superior WPA2. The 2Wire - along with every device currently connected to it, obviously - uses WPA2, and does not support a mixed WPA2/WEP environment.

Back to the store I go tomorrow; the only question now is: one new Netgear N router (which can fall back to 802.11g) or two?

Professionalism, Web development, and giving oxy to morons

Whereas a poor craftsman will blame his tools, poor tools will handicap even the most skilled craftsman.

As I insinuated in my previous post, I'm getting up to speed on the Zend Framework, the "900-kg elephant" of PHP application frameworks.

One major bone I have to pick with the ZF team is with regard to documentation: each time I've checked the site in the last couple of months, there's been an apparently current HTML version (now clocking in at some 300 HTML pages). There is also a PDF version, the promise of which is used as an enticement to register for their content distribution network (and, presumably, marketing info). As of this moment, however, the framework is at version 1.8.0, but the PDF version of the programmer's reference manual only covers version 1.6.0 (from September, 2008); some 12 releases earlier. It no longer fully matches the actual code, to the point where it is not difficult for a new developer to get deeply confused.

After spending a half-hour browsing the HTML version of the document, I am unable to find any declaration as to which version of the Framework is documented. However, the README.TXT file included with the source distribution states that it covers the 1.8 release, revision 15226, released on April 30, 2009. Classes which are listed in the README as being new, such as Zend_Filter_Encrypt, are documented in the HTML programmer's guide. Establishing a match between the (HTML) doc and the current code is non-trivial, however. While it may be argued that people unfamiliar with browsing a Subversion repository are not likely to be common within Zend's target audience, I would indirectly refute that: a product release, particularly one with a strong industry following, should be

• properly documented;
• easy for a (prospective) user to verify that he has the complete package; and
• with a definite, intuitive learning curve.

In my view, the Zend Framework fails on at least two of these points. The assertion within large segments of the PHP community that it is the "gold standard" of PHP application frameworks should be a disturbing, cautionary omen: if Web development, particularly PHP development, wishes to be taken seriously by the software industry at large, then some major improvements and attitude shifts need to occur quickly, publicly and effectively. It is still far too easy for potential developers outside the "early-adopter" leading edge to scoff that PHP development (and, by extension, Web development as a whole) is still far too immature and amateurish to be taken seriously. As someone who has developed professionally in PHP for some ten years now, that is a disturbing state of affairs; one that I would love to see (and participate in) a free-ranging discussion of.

Maybe not eating 'crow', specifically, but..... DUCK!!!

As in, "bend over, here it comes again..."

One of the things I have greatly appreciated about the Mac, especially with OS X, is how simple and straightforward software management is, compared to Linux and especially Windows (where every system change is a death-defying adventure against great odds). Operating system or Apple-supplied apps need an update? Software Update is as painless as it gets: the defaults Just Work in proper Mac fashion, but you can set your own schedule, along with a few other options. There is a well-established convention for third-party apps to check for updates via a Web service "phoning home" at app startup; this has been very easy to deal with. Application and file layout is regular and sensible; libraries and resources are generally grouped in bundles at the system or user level. After a few years of DLL hell in Windows and library mix-and-match in Linux, this was shaping up to be a real pleasure.

Then, as some of you know, I updated Mac OS X on my iMac from 10.5.5 to 10.5.6. As expected, that apparently went as smooth as glass. I even blogged about it. XCode worked; MS Office 2008 for the Mac worked; Komodo Edit worked; all my IM clients worked; all seemed customarily wonderful in the omniverse. I even started up Mail; it opened normally and happily downloaded my regular mail and Google mail, just as it had done every day for months. (I didn't actually open any messages then; that will turn out to be important.) Satisfied that everything Just Worked as always, I went back to working on a project for a few hours before turning in for the night.

Next morning, I went through the usual routine. Awake the Mac from hibernation; log in; start Yahoo, MSN and Skype; start Mail; open Komodo; open Web browsers (Safari, Opera and Camino) and I'm ready to get started. First thing...here's an interesting-sounding email message; let's open that up and... *POOF* — Mail crashes.

WTF? It started up just fine; I even got the "Message for you, Sir" Monty Python WAV I'd set Mail to use as my new-mail-received notification. I start Mail again. Picking a different message, I double-click it in the inbox. A window frame opens with the message title, sits empty for a few hundred milliseconds, then Mail goes away again. Absolutely, totally repeatable. Reboot changes nothing. Safe Boot (the Mac equivalent of Windows' "safe mode") changes nothing. The cold fingers of panic stroke my ribs like Glenn Gould at the piano. On a bad-karma scale of 0 to 10, initial reaction is an "O my God"; we're not dead, but we're hurt bad; the karma has definitely run over the dogma. 

The next couple of days are spent using my ISP's Webmail service, and a set of Python scripts I'd previously written to search mailbox contents — Apple Mail, like any sensible email program, adheres to established standard formats. If I'd been using Microsoft Lookout! in a similar situation, I'd have been up the creek.

Finally, I come across some Web-forum items that indicate that GPGMail needs to be updated; if it's not, Mail will crash under OS X 10.5.6 — which is exactly what was happening. (If you're not using GPGMail, GNU Privacy Guard, or any of the various GPG interfaces for Windows such as Enigmail for Mozilla Thunderbird, you don't know how many people are recording and/or reading your email — but if it transits a server in the US or UK, it's guaranteed that it will be.

Installing the upgraded GPGMail bundle was the work of less than two minutes (hint: remove or rename the old bundle before copying the new one over. You probably don't need the insurance, but consider how we got here...). Then start up Mail as usual. It should, once again, Just Work — complete with being able to read and reply to messages, with or without GPG signatures.

OK, so what lessons can we take away from this experience, both as users and developers?

Time Machine may well be the single most rave-worthy piece of software I've touched in 30 years, but it can't (obviously, easily) do everything, and in a crisis, even experienced users may well not want to risk bringing too much (or too little) "back from history". There's definitely a market for addons to TM to do things like "look in my user and system library directories, the Application directory structure, MacPorts, etc., and bring application Foo back to the state it was in last Tuesday morning, but leave my data files as they are." I almost certainly could do that with the bare interface -- but, especially since it was "broken" as part of an OS upgrade, and (with the Windows/Linux experience fresh in mind) not comfortable exploring hidden dependencies... I was without my main email system for three days. Sure, I had workarounds -- that I wouldn't have had if I'd been in a stock Windows situation -- but that's not really the point, is it?

Also, app developers (Mac or other), add this to your "best practices" list: If your software uses any sort of plug-in/add-on architecture, where modules are developed independently of the main app, then you can have dependency issues. The API you make available to your plugin developers will change over time (or your application will stagnate); if you make it easy for them (and your users) to deal with your latest update, you'll be more successful. There's (at least) two ways to go about doing this:

The traditional "brute force" approach. Have a call you can use to tell plugins what version of the app is running, and allow them to declare whether or not they're compatible with that version. Notify the user about any that don't like your new version. For examples of this, see the way Firefox and friends deal with their plugins. Yes, it works, but it's not very flexible; a new version may come out that doesn't in fact modify any of the APIs you care about — which means that the plugin should work even though it was developed against version 2.4 of your app and you're now on 4.2.

Alternatively, a more fine-grained approach. Group your API into smaller, functional service areas (such as, say, address-book interface or encryption services for an email program). Have your plug-in API support a conversational approach.

1. The app calls into the plugin, asking it which services it needs and what versions of each it supports.
2. The app parses the list it gets back from the plugin. If the app version is later than the supported range for a specific feature identified by the plugin, add that to a "possibly unsupported" list. (If the app version is earlier than the range supported by the plugin, assume that it's not supported and go on to check the next one.)
3. If the "possibly unsupported" plugin list is empty, go ahead and continue bringing up the app, loading the plugins normally; you're done with this checklist.
4. For each item in the "possibly unsupported" list, determine whether the API for each feature required for the plugin has changed since the plugin was explicitly supported. (This is how a plugin for an earlier release, say 2.4, could work just fine with a later version, like 4.2.) If there's no change in the APIs of each feature required by the plugin, remove that plugin from the "possibly unsupported" list.
5. If any plugins remain in the list, check if there's an updated version of that plugin on the Net. This might be done using a simple web-service-to-database-query on your Web server. If your Web server knows of an update, ask the user for permission to install it. If the user declines, or no upgrade is available, unload the plugin. (You'll check again next time the app is started; maybe there's an update by then.)
6. Once the status of each plugin has been established, and compatible plugins loaded, finish starting up your app.

Of course, there are various obvious optimisations and convenience features that can be built into this. Any presentation to the user can and likely should be aggregated; "here's a list of the plugins that I wasn't able to load and couldn't find updates for." Firefox and friends are a good open-source example of this. The checks for plugin updates can also be scheduled, so as not to slow down every app startup. This might be daily, weekly, twice a month, whatever; the important thing is to let the user configure that schedule and view a list of plugins that are installed but not active.

As I started this post by saying, I've been very favorably impressed by Mac apps' ease of use (including installation and maintenance). Mail fell down and couldn't get up again without outside assistance; this is unusual. The fact that this was caused by a plugin and that Mail could not detect and work around the conflict just amazes me; I expect more from Apple. I'm not ready to decrease my use of the Mac because this happened — but I am going to pay more attention to how things work under the hood. The fact that I have to even be aware of this -- which is one of the features that hitherto distinguished the Mac from the grubbier Windows and Linux alternatives -- is worrisome.

Again, your comments are welcome.

Does anybody else have a problem with this?

If you've got an ssh connection to a Debian or Ubuntu Linux box handy, and you have sudo privileges on that box, try this little experiment:

1. ssh to your box as an ordinary user;
2. sudo su to get a root prompt (you should be asked for your password - this is important);
3. as soon as you get the root prompt, exit back to normal user, then exit your ssh session entirely.

Now, here's the scary part:

1. ssh to that same box again right away, as the same user;
2. sudo su to get a root prompt again.

Why is this scary? Because the second time you ask for a root prompt, you're not prompted for a password. This means that, not only does the actual Linux box require access and user security appropriate to its function, but so does every device that can ssh into it with a rootable user! I'm sure this isn't in any way new, but in 10+ years of using Linux, I just now encountered that scenario for the very first time. As Linux is becoming more popular, and more users are marching up the 'power user' scale, this is something that should be paid attention to - especially in a business environment. Yowza!

If you can't extend it, is it really an eXtensible HTML?

Arrrrrrrrrrrrrggggggghhhhhhhhhhhhhhh!!!!! So much for consistency....

As anybody who has worked with me in the last 3-4 years well knows, I have been an enthusiastic advocate of DocBook as a documentation markup vocabulary for various purposes, and by extension, XML-based tools for all manner of things (Apache Ant and so on).

One feature I use regularly in Docbook XML source documents is the internal subset, which lets you define entities and include files defining entities not part of the original DTD. So, for instance, my standard software.ent file has an entry of

<ENTITY mswindows '<ulink url="http://www.apple.com/switch">Microsoft Windows</ulink>'>;

This way, anywhere in a DocBook file that includes that entity definition, I can type &mswindows; and, when the document is transformed (into XHTML, PDF, RTF or whatever), the desired link and text will appear in place of the entity. This is an obvious lifesaver when you want to include, for instance, links to glossary definitions for unfamiliar terms scattered through a document.

Fine. But the current state of XHTML (the XML-based successor to HTML) simply doesn't support it. It does not appear to be possible to have an XHTML document with an internal subset parsed correctly by any current major browser on Windows, Linux or the Macintosh. Various Google searches such as this one produce links to pages that say, with varying levels of emphasis and literal wording, "you can't use internal subsets for XHTML that is to be rendered by a browser". It seems that in the force-fit of XML to HTML that produced XHTML, the concept of different "streams", or purposes, for documents was introduced. XHTML which is to be rendered in a Web browser has one set of limitations (including the internal subset); whereas XHTML conformant to the same definition documents which is to be processed as "pure XML" has another.

To say that this sucks is to use that colloquialism as an extreme understatement, akin to saying that tsunamis are wet. This limitation closes off an entire range of applications that would use dynamically-generated XHTML as browser-viewable data in the same spirit as XML generally (without writing an otherwise redundant app to parse and reformat the data). The benefits — and they are significant — of an XML-based browser markup language are (in my view) seriously degraded by foolishness like this.

Of course, several of you are already thinking, I could just use XSLT instead. I had previously wondered why PHP and other Web scripting languages included support for XSLT processing. Now I know, I guess.

If anybody has any corrections or other good ideas, please let me know.