Two steps forward, three steps back

Alternate title: ARRRRRGGGGGHHHHHHH!!!!!

Both of you Gentle Readers may have noticed that I've been away from the blog for a while, and that a few posts that were previously published have gone missing. I've been busy fighting some other fires for a while, and my current network access has lacked the stability and efficiency that local propaganda would have you expect.

This evening (Wednesday 18th) I came across a piece of nifty-looking software, MacJournal by Mariner Software. It looks great — software that would let me compose/revise blog posts offline, in a native Mac app with nice organizational features and so on, at an attractive price, and with a 15-day evaluation period thrown in, just so you can try before you buy.

"Cool," I thought; "I'll be able to multitask on my shiny new MBP that's coming Any Day Now™."

Downloading and installing the eval copy went just as you'd expect; drag an icon into a folder, wait for the "Copying" progress dialog to go away, and it's done. Standard Mac user experience; nothing to see here, folks — unless you were expecting a Windows-style "Twenty Questions" installation.

I decided to do a really simple, trivial first exercise: select the five posts I'd written so far in a tutorial series; add the keyword ("label" in Blogger.com parlance) tutorial to them; save them back to Blogger. No animals were harmed in the performance of this experiment, and very explicitly, no content was directly, intentionally edited. (Note the qualifiers; they're important.)

(insert "train wreck" sound effects here.)

The first two parts were (relatively) unmolested; they didn't have any code blocks in them. The latter three did, however, and those were completely deformed. Numerous span elements were added, particularly around links (MacJournal seems to think links shouldn't be underlined, ever). Other formatting was changed; in particular, code tags were replaced by spans that set the font size to 13 points.

WTF?

It's going to take me a bit of noodling around in the software to figure out how to change the defaults to something that makes sense (at least for me), and until then, I'm back to editing in the browser. If the evaluation period expires before I'm happy with the configuration, then I'll comply with the license and blow it off my system. I'd really rather not do that; the feature list looks good, the interface is clean, and best of all, I don't have this Could not contact blogger.com line underneath my editing area as I type.

I understand that MacJournal, like most apps, has default ways of laying things out and working with things. I'm well aware of the difference between an "import" and a "copy" of something. But... I believe very strongly that the first rule of software, as medicine, should be "First, do no harm" — and that includes "don't mess with my formatting without even putting up a confirmation dialog asking my permission!" I really don't think that's too much to ask, or too hard to implement — and doing so would a) make a much more positive initial user experience by b) showing that you've thought things through well enough that c) your still-potential user isn't looking at an hour or two of careful, detailed work just to get back to where he was before he touched your product &emdash; or, rather, it touched his work.

Like most Mac users, I've gotten spoiled by how well most software on this platform is thought through to the tiniest details. Like most, I get annoyed when I have to deal with Windows or Linux apps that simply aren't thought through at all, apparently. (Spend a week with Microsoft Office or, even better, Apple iWork on the Mac; I dare you to go back to Office on Windows and be happy with it.) To run into a Mac app that fails such a simple use case so spectacularly (granted, in its default configuration) simply beggars explanation.

You want to start a tutorial; well, you know...

Not as catchy as the Beatles' Revolution, even if the meter works.... oh well....

Continuing from the first post in this tutorial. What do I think is important when starting to demonstrate some code? As with most writing, it depends on the audience. For the purpose of this series of posts, I'm assuming that you fit comfortably in or near the following:

• You're comfortable with HTML and XML doesn't make you run screaming from the room;
• You have a basic understanding of databases; you've run across SQL before and understand the basic concepts;
• You understand PHP; you've written some code before;
• You understand the concepts of "object-oriented development", "patterns", "best practices" and ideally "test-driven development" (usually abbreviated as "TDD"), even though you may not have loads of experience (yet) with them; and crucially
• You want to improve your ability to write code that you can refine and possibly reuse over time.

The assumption that you know or at least are interested in PHP is a given, since that's the language we'll be using here.

What will you need to have installed and available to follow along?

1. Access to a system with PHP 5.2 or higher, available both from the command line and the Web server (via a module or CGI);
2. The PHPUnit and MDB2_Driver_mysql modules installed and available;
3. A text editor of your choice;
4. The ability to create PHP scripts and HTML files and have those accessible from the Web server as well as the command line.

These should all be pretty obvious to more experienced PHP developers, but making sure that we're both operating from the same set of assumptions — and no others — greatly reduces the likelihood of confusion and breakage along the way. Many of you haven't yet dealt much with unit tests using PHPUnit or similar systems; that's going to be a starting point for us.

Time v. Money v. Risk v. Frustration

I just spent three hours beating my head against The Joys of Wi-Fi Networking. No doubt because it saved $50 or so, the HDB public-housing flat I live in here in Singapore (which are "affectionately" known as 'chicken coops' for their quality and structural integrity) didn't put telephone jacks in every room. Nor did they run conduit between rooms so that suchlike could be added later. The upshot of this is that my DSL modem, in the living/dining room, is 15 meters and two concrete-slab walls away from my study/office, in a bedroom (with no phone jack and two power outlets).

Up to now, this hasn't been an insurmountable problem, because the DSL modem/802.11g router the government-linked telco sells you when you open a DSL subscription could punch a (barely) usable signal through those two concrete walls; desktop and laptop computers strewn around the study were on the LAN, with quite serviceable WPA2 encryption to keep private bits private. The router (in the living room) claimed to have a 14 Mbps connection to The Net; the aggregate total bandwidth available in the study ranged from 3 to 5 Mbps; not fantastic but usable. The world was in a survivable state of chaotic flux.

Then, yesterday morning, FedEx delivered a bright, shiny (actually used, in a Target bag rather than original box, no longer in the Polycom US catalog but still serviceable) Polycom SoundPoint 501 SIP phone, courtesy of my newest client (thanks, Nathan, Matt and Margaret - I'm not kvetching, honest!). An hour of work with jackhammer and flamethrower cleared sufficient space on the desk for the new trophy. Now to plug it in and fire it up.

Plug it in? To what? My ear?

Just to make sure I had my head straight, I looked at what it would take to run line from point P to R. Two solid concrete/rebar walls, routing around various doors/windows, across the span of the living room.... no, that wasn't going to happen, certainly not by Thursday. Put the phone in the living room? Not a chance. Wait a minute - they have these things called wireless Ethernet bridges; I should be able to get one of those, stick it in the study, connect the phone to it and it to the existing wireless LAN, and I'm good to go.

I went out to the local "IT super-mall", Funan. It's the best place in Singapore to buy electronic gear of whatever variety from reasonably reputable dealers. After visiting some 25 different shops, large and small, I was firmly reacquainted with one of the basic facts of Singapore life.

This is a firmly Stalinist economy in the areas that matter. All imports (which is to say, anything of value) are brought in through a small number of generally well-connected exclusive distributors. When one shop says "finished, already!" (the de facto motto of the city), everybody is going to say the same thing. That is, if you're fortunate enough to even run into some salesperson who actually understands what you're looking for. I could find things easier (and often cheaper!) in Vietnam.

Then I happened to pop into one hole-in-the-wall one-man shop, Crystal Systems. The "one man" said "say, I understand what you're trying to do; I did exactly that for a firm here recently; all you need are two routers." The archetypal light-bulb moment. (Remember, I'm a software craftsman, not a hardware/network engineer.) This is where the "time v. money v. ..." of the title comes in. What I should have done was to buy two identical brand-new 802.11n routers. That would have cost me about S$190 (~US$136 or so), and would have given me a perfect excuse to replace my aging 2Wire 2700HGV-2, But this has already been a bit of an expensive month - new hard drive, various software - so after some consideration I decided to try to salvage the existing router (apparent POS though it may be) and just spend the minimum necessary. So I went home with one S$69 TP-LINK TL-WR340G router.

It features the now-customary browser-based setup - just remember to use a hard-wired link rather than wireless (d'oh!), and it's almost as easy as falling off your seat. Bridging mode is pretty obvious; the TP-LINK wants to know the MAC address of the router you're connecting to, along with its encryption setup.

In bridge mode, the TL-WR340G only supports the obsolescent WEP encryption, not the current and generally superior WPA2. The 2Wire - along with every device currently connected to it, obviously - uses WPA2, and does not support a mixed WPA2/WEP environment.

Back to the store I go tomorrow; the only question now is: one new Netgear N router (which can fall back to 802.11g) or two?

Responding while Forbidden; Gender and Other Issues in OSS and PHP

This post is what would have been a response to a post on Elizabeth Naramore's blog, which she titled Gender in IT, OSS and PHP, and How it Affects Us *All*. Quite a good post, actually, with a long and often thoughtful (but as often thoughtless) comment thread following. I was hoping to respond to a comment on the post, but that apparently is no longer allowed, even though the "Leave a Reply" form at the bottom of the page is functional. There's also a mysterious "Login/Password" pair on the page, but no indication of which ID one uses, or how to go about getting one.

Following is the content of the reply-that-wasn't: I (perhaps unreasonably) think this has some points worth pondering. Please do read the original post first and then come back here - where the "comment" feature definitely does work.

---

@A Girl: Great that you're doing AP CompSci next year. As someone who's been in the Craft for 30 years, I have a great sentimental attachment to your idea that "teachers and professors have the chance to shape the mindsets of their students towards women in the industry". If we were a true profession, where essentially all practitioners have a certain common level and content of educational background combined with qualifying experience (e.g., apprenticeship, internship), I'd agree wholeheartedly.

The fact is that many if not most of the people in the industry - both the "coders in the trenches" and the ex-coders who got promoted into management ("because they were such great coders" - thereby removing two qualified people from an organization)... far too many of these people have noformal education in CS (or, often, anything else). And by the time they realize how important that might be, they're old enough that they're facing ageism in the workplace already - they're not confident enough to put the "big hole in the middle of [their] career" and "go back" to school. It doesn't help that schools the world over do such a lousy job of outreach and marketing to those potential students - they're focused on the Executive MBAs and other graduate-level returning students, who can have their pricey programs paid for by their employer. Joe or Jane Schmuck trying to keep head above water in the face of cut-throat competition from planeloads of new arrivals with mimeographed certifications, who've been taught their entire lives to never think out of the box to begin with... things start getting really tough out there. I'm not surprised that enrollment in CS programs is down. I'm amazed beyond words that it's still as high as it is; a less starry-eyed observer might expect the number of CS majors to closely track, say, majors in Phoenician economics.

A lot of the new entrants into CS and IT over the last ten years or so have degrees - they're just not in the "obvious" field. Someone, and I wish I could find the original, wrote an article in one of the industry mags (like C/C++ Users Journal, not IEEE Computer) that, to be good at software in the modern era, one needed to have exposure to "behavioral science, psychology, linguistics, human factors, sociology, philosophy, rhetoric, ethnology, ethnography, information theory, economics, organizational politics, and a dozen other things - and please, please learn to write competently in English!" - I've had that taped above my display for years now. So it's not that we're not educated; the problem - and it is a problem - is that there is no universal common body of knowledge for software "engineering" - which is one of the necessary precursors of any true profession. We're not going to have a CBOK without either a broad consensus within the industry, or imposition of a system from outside (and very narrowly-focused) forces in government or the larger economy. Given the prevailing social and political attitudes of current practitioners ("herding libertarian-poseur cats" is a phrase not infrequently heard), that would seriously disrupt the Craft and, by extension, any industry or field dependent upon software (which by now is pretty much everything).

How to solve the problem - and, in so doing, help redress the pandemic sexism, racism and ageism (in huge parts of the world, recruiting with explicit age limits is perfectly legal, and here in South Asia, you're old for coding at 28)? I've got no idea. When I first started doing this, I thought that within the next thirty years or so (from 1979), we'd be able to turn this informal craftwork that had taken the industry away from the "educated CS types" and turn it into a real profession. Now? I'd say we're 30 to 50 years away from now - unless we have a software equivalent of the New London School explosion and a "solution" gets imposed from outside. We need to grow up, and quickly.

We Interrupt This Program...

We interrupt this tutorial to interject an observation about goals, methods and promises. Goals we have for ourselves as people and as professionals; the method we use to pursue those dreams; perhaps most importantly, the promises we make, both to ourselves and to our customers, about what we're doing and why.

I consider this after reading the Website for some consultants who've done some (relatively nice, relatively low-key) link-dropping on LinkedIn. I'm not naming them here, not because I don't want to draw attention to them (their own site is very clean and well-done), but because the point that I'm going to be making here isn't just limited to them - we, as a craft and an industry of Web development, have some Serious Problems™.

The "problem" is nicely summarized by this group's mission statement:

Our mission is to produce the perfect implementation of your dreams.

What could possibly be wrong with that?

As a goal, implied but left unspoken, absolutely nothing; both as practitioners and as clients, we tend to set ever-higher goals for ourselves. Indeed, that's the only way the "state of the art" - any "art" - can advance. But we who practice the art and craft of software (including Web) development (as opposed to the engineering discipline of hardware development) have a history of slashed-beyond-reality schedules and budgets coupled with a tendency for stakeholders not to hear "if all goes well" as a condition to our latest schedule estimate. We have a history, perceived and actual, of promising more than we can deliver. Far more attention is paid by non-technical people to the "failures" and "broken promises" of software than to things done right. For a craft whose work is accruing increasing public-policy and -safety implications, the effect of unrealistic expectations, brought about by poor communication and technical decisions being made by people who aren't just technically ignorant but proud of the fact, is disturbing. What started as a slow-motion train wreck has now achieved hypersonic speeds, and represents a clear and present danger to the organisational health and safety of all stakeholders.

I don't mean to say that projects always fail, but an alarming number of them do. If, say, dams or aircraft were built with the same overall lack of care and measurable engineering precision that is the norm in commercial software development, we'd have a lot more catastrophic floods, and a lot few survivors fleeing the deluge by air. When I entered this craft thirty years ago (last May), I was soon led to believe that we were thirty to fifty years away from seeing a true profession of "software engineering". As a time frame beginning now, in 2009, I now think that is almost laughably optimistic.

Why have things gotten worse when we as a tool-building and -using society need them to get better? Some people blame "The Microsoft Effect" - shipping software of usually-dubious quality to consumers (as opposed to 'customers') who have bought into the (false) idea that they have no realistic choice.

It's more pervasive than that; commercial software development merely reflects the fashion of the business "community" that supports it, which has bought into one of the mantras of Guy Kawasaki's "The Art of Innovation", namely "don't worry, be crappy." Not that Kawasaki is giving bad advice, but his precondition is being ignored just as those of other software people have been: the key sentence in his "don't worry, be crappy" paragraph is "An innovator doesn't worry about shipping an innovative product with elements of crappiness if it's truly innovative" (emphasis mine). In other words, if you really are going to change the world, nobody will notice if your Deus ex Machina 1.0 has clay feet as long as you follow up quickly with a 1.1 that doesn't...and follow that with a 2.0 that changes the game again. But that space between 1.0 and 1.1 has to be fast, Kawasaki argues (in the next paragraph, titled "Churn, Baby, Churn"), and the version after that has to come along before people (like possible competitors) start saying things like "well, he just brought out 1.1 to fix the clay feet in 1.0." If the customers see that you're bringing out new versions as fast as they can adapt to the previous ones, but that each new version is a vastly superior, revelatory experience compared to the earlier release that they were already delighted by, they'll keep giving you enough money for you to finish scaling the "revolutionary" cliff and take a (brief) rest with "evolutionary" versions. Business has not only forgotten how important that whole process is to their continued survival, but they've removed the capability for their bespoke software (and Web) infrastructure to use and reuse that model. All that remains is "it's ok if we ship crap; so does everybody else." That's the kind of thinking that made General Motors the world-bestriding Goliath it is today - as opposed to the wimpy also-ran it was (emphatically NOT) half a century ago. We really don't need any more businesses going over that sort of cliff.

What we do need, and urgently, are two complementary, mutually dependent things. We need a sea change in the attitude of (most) businesses, even technology businesses, towards software - to realise and acknowledge that the Pointy-Haired Boss is not merely a common occurrence in the way business manages software, but actively threatens the success of any project (and business) so infested. Just as businesses at some point realise that "paying any price to cut costs" is an active threat to their own survival, they need to apply that reality to their view of and dealings with the technical infrastructure that increasingly enables their business to function at all.

Both dependent on that and as an enabler of that change, the software and Web development industry really needs to get its house in order. We need to get away from the haphazard by-guess-and-by-golly estimation and monitoring procedures in use by the majority of projects (whose elaborate Microsoft Project plans and PowerPoint decks bear less and less resemblance to reality as the project progresses) and enforce use of the tools and techniques that have been proven to work, and have an organised, structured quest to research improvements and New Things.. Despite what millions of business cards and thousands of job advertisements the world over proclaim, there is no true discipline of "software engineering", any more than there was "oilfield engineering" in widespread use before the New London School explosion of 1937. Over 295 people died in that blast; we have software-controlled systems that, should they fail, could in fact hurt or kill many more - or cause significant, company- or industry-ruinous physical damages. We should not wait for such an event before "someone" (at that point, almost certainly an outside governmental or trans-governmental entity) says "These are the rules." While I understand and agree with the widespread assertion that certification tests in their present form merely demonstrate an individual's capability to do well on such tests, we do need a practical, experiential system - probably one modelled on the existing systems for engineering, law or medicine. Not that people should work 72-hour shifts; there's enough of that already. But rather that there should be a progression of steps from raw beginner to fully-trusted professional, with a mix of educational and experiential ingredients to ascend that progression, and continuing educational and certificating processes throughout one's entire career. The cost for this is going to have to be accepted as "part of the system" by business; if business wants properly competent engineers, and not just the latest boatload of unknowns with mimeographed vendor certs, then they're going to have to realize that that benefit does not come without cost to all sides. The free ride is over - for all the stakeholders at the table.

Tutorials, best practices and staying current

A gent by the name of Brian Carey has written a very nice little tutorial on "Creating an Atom feed in PHP", and gotten it published on the IBM DeveloperWorks site. In the space of about ten pages, Brian gives a stratospheric overview of what Atom is and why PHP is a good language for developing Atom-aware apps, and then gets into the tutorial - defining a MySQL database table to hold the data used to 'feed' the Atom feed, and writing code to get the data out and put it into the form that a reader such as NetNewsWire expects an Atom feed to be in.

Now, to be fair, Brian describes himself as "an information systems consultant who specializes in the architecture, design, and implementation of Java enterprise applications", and the paper is clearly meant as a whirlwind tutorial, not to be taken by the careful/experienced reader as necessarily production-quality code. And if this had been published in, say, 2002 or so, I'd have thought it a great how-to for banging out some PHP 4 code. But this is 2009 PHP 4 is a historical artifact, and blogs and industry journals of seemingly every stripe are decrying the poor quality (security, maintainability, etc.) of PHP code... much of which is still written as if it were the turn of the century, ignoring PHP 5's numerous new features and the best practices that both spawned them and grew from them.

So what's really wrong with doing things like they did in Ye Olden Tymes™?

• Procedural code makes it harder to make changes (fix bugs, add features, change to reflect new business rules) and be certain that those changes don't introduce new defects. This is largely because...
• While it is possible to test procedural code using automatable tools like PHPUnit, it's a lot harder and more complex than testing a clean, object-oriented design.
• Hard-coding everything, interspersing 'magic values' throughout code, is a major hindrance to future reuse - or present debugging;
• Using quick-and-dirty, old-style database APIs exposes you to the risk of input data that is more 'dirty' than it should be - opening the door to SQL injection and other nastiness;
• Not staying current exposes your code to the risk that it's either using features that have since been deprecated or removed entirely, or (arguably worse), the risk that new features (such as standard library additions) make some of your existing code redundant at best.

Each of these, to varying degrees, is true of much of the PHP code I've read in the last couple of years, including that in the DeveloperWorks paper. For example, the DW paper's code makes use of the old-style PHP4 mysql_* API rather than the more abstract/portable MDB2 database abstraction layer. And there's also the rather terse implementation of a date3339() function for converting a timestamp to RFC 3339 output format, that's now nicely handled through the standard PHP DateTime class.

How would I have done things instead? Read the next few posts in this blog to find out. And, of course, comments are always welcome.

Smokin' Linux? Roll Your Own!

As people who've encountered the "business end" of Linux have known for some time, the system (in whichever distribution you prefer, greatly rewards (some would say 'requires') tinkering and customisation. This can be done with any Linux system, really; some distros, like LinuxFromScratch and, to a lesser degree, Gentoo and its derivatives, explicitly assume that you will be customizing their base system according to your specific needs.

Now the major distros are getting into it. There have been various Ubuntu and Fedora customisation kits on the Net, but none as far as I can tell that are as directly supported (or easy to use) as from OpenSUSE, the "community-supported" offering from Novell (who also offer SUSE Linux Enterprise Desktop and Server.

Visit the OpenSUSE site, and prominently visible is a link to the OpenSUSE Build Service, which "allows developers to package software for all major Linux distributions", or at least those that use rpm packaging, the packaging system used by Red Hat, Mandriva, CentOS, and other similar systems. But that's not all...

SUSE now have a new service, SUSE Studio, which allows users to create highly customized systems based on either the community (OpenSUSE) or enterprise versions of SUSE Linux. These "appliances" can be put together on the basis of "patterns", such as lamp_server (LAMP, or Linux/Apache/MySQL/PHP Web server) or technical_writing (which includes numerous tools like Docbook). You can even supply your own (either self-built or acquired elsewhere) RPM packages to include in the appliance you're building, and SUSE Studio will deal with the dependency matching (warning you if packages are required that aren't either among its standard set or uploaded by you).

Startup scripts, networking, basically anything that is usually handled through the basic installation or post-installation configuration - all can be configured within the SUSE Studio Web interface.

And then, when you've got your system just the way you want it, you can build it as either an ISO (CD/DVD) image to be downloaded and burned onto disc, or as a VM image for two of the most popular VM systems (VMWare and Xen).

But wait, there's more...

Using a Flash-enabled browser, you can even "test drive" your appliance, testing it while running (transparently) in an appropriate VM hosted within the SUSE Studio infrastructure. Especially if you have a relatively slow connection, this will let you do preliminary "smoke testing" without having to download the actual image to your local system. Once you're ready to do so, of course, downloading is very nearly a single-click affair. Oh, and you're given (presently) 15 GB of storage for your various builds - so you can easily do comparative testing.

What don't I like about it? In the couple of hours I've been messing around with it today, there's really only one nagging quibble: When you do the "test drive" of your new creation, the page you're running it in is a standard, non-secure http Web page. The page warns you that any data and keystrokes sent will not be encrypted, and recommends the use of ssh if that is a concern (by which most people will think https). But there's no obvious way to switch, and shutting down the running appliance (which starts by the time you read the warning) involves keystrokes and so on...

In fairness, this is still very clearly a by-invitation beta offering (but you can ask for an invite), and some rough edges are to be expected. I'm sure I'll run into another one or two as things go on. I'm equally certain that all the major problems will be smoothed out before SUSE Studio goes into general public availability.

So, besides the obvious compulsive hackers and the people building single-purpose appliance-type systems, who would really make use of this?

One obvious use case, which the SUSE Studio site describes, is as a canned demo of a software system. If you're an ISV, you can add your software or Web app to a SUSE Studio appliance, lock down the OS image to suit (encrypting file systems and so on), and hand out your discs at your next trade show (or have them downloadable from your Website). No worries about installing or uninstalling from prospective customers' systems; boot from the CD (or load it into a VM) and they're good to go.

Another thought that hit me this morning was for use as an interview filter. This can be in either of two distinct modes. First, you might be looking for people who are really familiar with how Linux works. Write up the specs of a SUSE Studio appliance (obviously more demanding than just the click-and-drool interface) and use an app of your own devising to validate the submitted entries. This validation could be automated in any of several ways.

The second possible interview filter would be as a programming/Web dev system. As a variation on the "ISV" example above, you load up an appliance with a set of tools and/or source files, ready to be completed and/or fixed by your candidates. They start up the appliance (either a live CD or VM), go through your instructions for the test, and then submit results (probably an encrypted [for authentication] archive of all the files they've touched, as determined by the base system tools) via email or FTP. On your end, you have a script that unpacks the submission into a VM and uses the appropriate automated testing tools to validate it. I can even see this as a business model for someone who offers this capability as a service to companies wishing to have a better filter for prospective candidates than resume-keyword matching - which as we all know is practically useless due to the high number of both false negatives and false positives.

What do you all think?

The Debate between Adequacy and Excellence

I was clicking through my various feeds hooked into NetNewsWire, in this case The Apple Core column on ZDNet, when I came across this item, where the writer nicely summed up the perfectly understandable strategy Microsoft have always chosen and compared that with Apple and the Mac. Go read the original article (on Better Living without MS Office and then read the comment.

As I've commented on numerous times in this blog and elsewhere (notably here), I'm both very partial to open standards (meaning open data formats, but usually expressed in open source implementations) and to the Apple Mac. As I've said before, and as the experience of many, many users I've supported on all three platforms bears out, the Mac lets you get more done, with less effort and irritation along the way, than either Windows or Linux as both are presently constructed.

But the first two paragraphs of this guy's comment (and I'm sorry that the antispam measures on ZDNet apparently don't permit me to credit the author properly) made me sit up and take notice, because they are a great summation of how I currently feel about the competing systems:

The Macs vs. PC debate has been going on for about 25 years or so, but the underlying debate is much older. What we are really discussing is the difference between adequacy and excellence. While I doubt I would want to be friends with Frank Lloyd Wright or Steve Jobs, both represent the exciting belief in what is possible. While Bill Gates and Steve Ballmer rake in billions, their relative impact on the world of ideas is miniscule.

Bill Gates understands that business managers are on the whole are a practical, albeit uninspired and short-sighted bunch. By positioning Microsoft early on to ride into the enterprise with the implicit endorsement of one of the biggest, longest-lived, and influential suppliers of business equipment, Gates was able to secure Microsoft's future. Microsoft's goal has never seemed to me to be to change the world, only to provide a service that adequately meets business needs. Microsoft has also shown from early on a keen awareness that once you get people to use your product, your primary goal is not to innovate to keep your customers, but, rather to make leaving seem painful and even scary. Many companies do this, but Microsoft has refined this practice into an art.

He then expands on this theme for four more paragraphs, closing with

Practically speaking Microsoft is here to stay. But I am glad that Apple is still around to keep the computer from becoming dreary, to inspire people to take creative risks, to express themselves, and to embrace the idea that every day objects, even appliances like the computer, can be more than just the sum of their functions.

Aux barricades! it may or may not be, depending on your existing preferences and prejudices. But it does nicely sum up, more effectively and efficiently than I have been able to of late, the reasons why Apple is important as a force in the technology business. Not that Microsoft is under imminent threat of losing their lifeblood to Apple; their different ways of looking at the world and at the marketplace work against that more effectively than any regulator could. But the idea that excellence is and should be a goal in and of itself, that humanity has a moral obligation to "continually [reach] well past our grasp", should stir passion in anyone with a functioning imagination. Sure, Microsoft have a commanding lead in businesses, especially larger ones — though Apple's value proposition has become much better there in the last ten years or so; it's hard to fight the installed base, especially with an entrenched herd mentality among managers. But, we would argue, that does not argue that Apple have failed, any more than the small number of buildings designed by Frank Lloyd Wright and his direct professional disciples argue for his irrelevance in architecture. If nobody pushes the envelope, if nobody makes a habit of reaching beyond his grasp, how will the human condition ever improve? For as Shaw wrote,

The reasonable man adapts himself to the world. The unreasonable man persists in trying to adapt the world to himself. All progress, therefore, depends upon the unreasonable man.

And that has been one of my favourite quotes for many years now.

Web Standards DO Save, Then and Now

It's been known in the Web-development community for several years now that well-designed, semantic, standards-compliant Websites use dramatically less resources (such as bandwidth) than 1997-era tangles of nested tables and invalid HTML. But it's still refreshing to read confirmation that that truth is pretty universally applicable - especially when that reading doesn't depend on what we now know as the "latest and greatest" Web browsers.

Take, for example, this five-year-old article by Jim Ramsey, then-Webmaster of the San Francisco Examiner newspaper's Website. In the section titled Simplify, Man!, he writes:

This is what a basic link in our navigation looked like late last year, before standards:

<tr>
<td class="navmenu" height="18"
  onClick="javascript:rolloutNav(this);document.location='/home/index.cfm'"
  onMouseOver="javascript:rolloverNav(this);"
  onMouseOut="javascript:rolloutNav(this); " colspan="2">
<a href="/home/index.cfm" class="nav">HOME</a></td> </tr><tr>
<td bgcolor="#EEEEEE" class="navmenuspacer" colspan="2">
<img src="../site_images/spacer.gif" width="1" height="2"></td> </tr>

Now take a look at what an Examiner navigation link looks like now:

<li><a href="/home/">Home</a></li>

That’s a big difference. In fact, the first one is so bad, I’m almost embarrassed to include it here. And what did I get for all that extra stuff? Basically, nothing. The JavaScript triggers the rollover effect and the table cells control the spacing. All of that can be done using styles.

Let’s take another example. Here’s how a link to a story in the Arts section looked before standards:

<img src="../site_images/sfex/homekickerarrow.gif" width="6" height="8">
<span class="kicker">Movie Review: Dickie Roberts<br></span>
<a href="../templates/story.cfm?displaystory=1&storyname=090503a_dickie"
    class="headlinesm">Problem 'Child'</a>
<hr noshade size="1" color="#EEEEEE">

Here’s the same thing following standards:

<h5>Movie Review: Hero</h5>
<h4><a href="/article/index.cfm/i/082704a_hero">Holding out for a 'Hero'</a></h4>

Again, once it is styled, the second version can be made to look identical to the first. When you can simplify markup in this way, it starts to make a big difference in bandwidth.

Comparing last year’s table-based site to our new standards-based one, the amount of information on our homepage is strikingly similar. Both contain basically the same elements and yet the HTML is 13K smaller on the CSS-version at 19.6K.

As a result, even though our traffic was about 40% higher in July 2004 than in September 2003, our bandwidth was almost exactly the same for those two months.

"[E]ven though our traffic was about 40% higher...our bandwidth was almost exactly the same..." What other development practice allows you to simultaneously:

• boost your traffic by nearly half without similar increases in bandwidth costs;
• improve your search-engine results without expensive, error-prone twiddling;
• open your site to a potentially much wider audience, by not limiting what platform or browser your audience uses; and
• significantly reduce the cost and complexity of maintaining your site?

As one colleague put it in an IM discussion, "people who develop like it's 1997 shouldn't be surprised if their revenues and page views don't exceed 1997 levels. Especially if they weren't around in 1997."

I'm not (quite) to the point of some of the more, um, evangelical developers out there in equating broken, invalid sites with actionable incompetence — but if our craft has serious hopes of making it out of "hobbyist" status in the eyes of non-technical business people, to where they treat practitioners as members of a profession, then we need to get some meaningful, practical, well-defined standards. This (standard-compliant, semantic development) is at or very near the top of my list of such standards.

Final note: I find it deeply ironic that Google have apparently redesigned the posting interface for Blogger. Whereas previously I have been able to post from any Linux, Mac or iPhone browser, switch between HTML editing and WYSIWYG view, and use all the other goodness...the new CSS and JavaScript seem to fail in any of the half-dozen Mac browsers I've tested today. One step forward, two steps back is especially painful when one started by walking directly away from the very edge of a cliff.

Tools, Continued

UPDATED 2 November 2010; see last paragraph.

---

This blog, fairly obviously, is published on blogger.com, which is now owned by Google. Blogger.com is geared primarily towards people who want to write but don't want to have to worry about the nitty-gritty technical details involved (such as HTML, CSS and so on). Sign up, click the 'New Post' button, and off you go...almost easier than falling out of your chair.

For the Web wizards among you, you can get into the guts of how your blog and posts are laid out and formatted; there are several in-the-tin and numerous third-party templates that can be used to style your blog any way you like it, and those can then be hand-tweaked by you to get things just so.

As with any click-and-go interface, the Blogger new-post window (what I'm typing in right now) has a "Preview" button (or actually, link); click on it and you'll see a more-or-less-reasonable facsimile of what your breathless prose will look like to the next visitor who stumbles across your blog. I did say "more-or-less-reasonable"...

One thing the preview area doesn't do — sensibly in hindsight — is to apply your template's formatting to the post being previewed. So that, for instance, if your template specifies that you want 9-point Gentium Book Basic, Times New Roman or Times (in that order) for your body, you won't see it that way in the preview.

What you also won't see — and this is what tripped me up for the longest time — is other styling for the post body, particularly justification. You may notice that all my posts are displayed with justified left and right margins; the default, and most commonly used setting, is for a justified left and ragged right margin, which I find unattractive. I've tweaked the template I use several times over the years to try and get the effect I was looking for. Each time, the post-preview showed no changes to the text formatting, and so I undid the change without viewing the entire blog normally. I have been instead hand-wrapping the content of an entire post in a <div>...</div> element pair whose only reason for existence was to set text-align: justify;.

This isn't the first time I've blogged about my learning-experiences-that-shouldn't-have-been, and just to be perfectly clear about this, I'm not trying to dis Blogger.com about this. The preview-while-composing feature is merely to let you see the content of the post you're working on without all the editing framework around it. It won't, and arguably shouldn't try to render that content in its final form. You do, after all, have the ability to go back and edit posts you've already published.

This is, at its heart, a cautionary tale for those of us tasked with making the Web easier to use for a wide variety of users. Be careful especially with interface design. Recognise that, barring explicit cues to the contrary, people's assumptions about How Things Work on your site may have only a nodding acquaintance with your own — but if you have some appropriate description in the right place, users are often happy to adjust. But there has to be a balance, or your 'power users' will feel like they're being stifled. How you achieve that balance is, of course, up to you to find out. Good luck.

UPDATE: Blogger have, since I wrote the original post, changed their preview feature (at least for "Blogger in Draft," their next-gen interface, which I use and recommend.) When you click the Preview button now, Blogger opens a new tab (or window, if you're old-school enough not to use tabs) with the preview of your draft post. This preview is exactly as a visitor to the published post would see it, with the exception of a great big Preview stripe in the upper-left corner. A wonderful improvement; thanks, guys!

Misadventures with Customer "Service": With Service like This, Why Bother?

Doing a job search using various Websites became really popular about ten years ago as an easy way for candidates and companies to find each other without any (visibly obvious) middlemen directly involved. I understand that in some geographic and employment areas, it's still a useful tool. I've had myself registered on fewer than half a dozen for several years, even though my last 6 jobs were all found through other means.

Job sites aren't typical consumer Websites in that they're really trying to target two distinct groups of customers: companies with jobs on offer (who generally pay the bills) and jobseekers (who generally don't). Even though jobseekers don't have any up-front money dangling in front of the site operator, wise (or experienced) operators of such systems know that they should "take care" of these site users well, since:

• Companies will be less likely to pay to advertise on general-audience sites with few users, and
• Users who have negative experiences (on any site) are often motivated to "spread the word".

Any business which is heavily dependent on its Website for revenue or for customer service knows (or quickly learns) that locking actual or potential customers out of their site through defective, easily-correctable site implementation has a direct and negative impact. The pathologically clueless operators don't seem to know or care why this happens; they just notice their monthly traffic figures dwindling over time, with revenues to match.

This even applies, although to a lesser extent, when the market being served by the site encourages a lack of competition. This is the case quite often here in South Asia, and notably in Singapore. Only two media companies serve the English-language market here, both with apparent Government ties. Broadcast television and radio are similarly throttled. This becomes apparent to anyone who spends more than a few days here. Individuals are engaged in frenetic, all-consuming competition with each other, while favoured companies and industries don't (visibly) engage in such grubbiness.

All of this went through my mind again over the last few days, after I'd had a series of consistently unpleasant experiences with one particular Website operator. Necessary background: I have been essentially Windows-free (and thus virus-free and botnet-free) for some five years now, initially using Linux systems like Ubuntu and then the Apple Mac. What has made this change practical, for me and for tens of millions of other people, has been the emergence of open standards in desktop computing, particularly with regard to open data formats. A classic example with which anyone reading this is familiar is HTTP. Any device, from large mainframes to appliances to mobile phones, can be used to produce or access information via HTTP, most commonly in the form of Web pages. Other "standards", like MP3 for audio or Microsoft Office document formats, are proprietary to a company or organisation but widely implemented by competing or collaborating systems. This reliance on actual or de facto standards, as much as any other single factor, is what has enabled much of modern technology, particularly the Internet (which many think of as synonymous with the World Wide Web).

Not everybody has always "played fair" with these standards, however. Microsoft, in particular, have been infamous for their commonly-used practice of "embrace, extend and extinguish" has led to several instances where they support an existing standard (such as HTML for Web pages or Kerberos for network authentication) and either introduce incompatible features or defective versions of standard features into their implementation. This has the effect of "locking in" users who support Microsoft's implementation rather than the actual standard. These usees, as they are often called, face significant actual or perceived costs and difficulties were they to switch away from their now-Microsoft-specific infrastructure. Microsoft have by no means been the only ones to do this sort of thing; they are, however, often quite brazen about it — and their usees don't always have an accurate understanding of the true costs of their "investment."

In particular, during the "dark ages" of Web development in the mid- to late 1990s, it was common for sites to have text on each page proclaiming "Best viewed with Microsoft Internet Explorer"; the tools which they were using to create the Web pages (often Microsoft FrontPage), did not always (or easily) generate valid HTML, CSS or standard JavaScript — all of which are needed to make interactive, "dynamic" Web sites work. As the relevant standards have become better-known to amateur and semiprofessional Web developers, the tools used by professional developers mature, and those using Web browsers become more aware of the importance of these standards, we have seen an increasing Web "Enlightenment", where once again, anybody (or anything, like a Google search crawler) can access all content for a site.

There are exceptions to this, however, just as there are trailing-edge sluggards in any social or technical change. These can give excuses about lack of resources, perceived lack of need ("We're doing fine the way things are") and so on — which quite often attempt to mask either ignorance of the issues involved or, at a fundamental level, disrespect for their current and actual customers. When a well-equipped, motivated (outside) organisation is in a position to gain from that ignorance or contempt — as Microsoft for so long was with regard to Web development and standards — those "sluggards" will continue to pat themselves on the back for their "prudence" and "conservatism" — right up to and often past the point at which the business dies.

One such organization appears to be the company which owns and operates the JobsDB.com group of employment Web sites. With "specialised" sites for Singapore, Thailand, the Philippines, Australia and other countries, and an apparently large advertising and promotion budget, it's safe to speculate that millions of people have at least seen their advertisements, if not actually made use of their site. Those who actually do try to browse for a job there, however, can face significant problems — if they're using anything other than Internet Explorer on Windows as their browser. (And the message seems to be finally getting through to many users — Microsoft Internet Explorer and Windows itself cause numerous security and stability problems that are comparatively unheard of on any other system.)

I browsed jobsdb.com.sg, the Singapore-themed version of their online job service. I got as far as entering my details in an application form for a position, when I ran into insurmountable difficulties. Numerous pages display poorly in any of the browsers I tried. "Obviously using archaic design," I thought, "but as long as the function isn't too badly broken, I can still deal with this." I eventually got to the a page with a form to be filled in by those wishing to apply for a specific job (at SingTel) and ran into fatal problems. I clicked on the link to report a problem with the site, and included the following description:

SingTel application page misbehaves horribly with Safari 4.0.2/Mac. An ASP alert box, (URL omitted for this blog post), is displayed in a new tab, taking the entire browser window; the message "Please correct the field(s) with red exclamation mark(s) (!). You can click on the exclamation mark for instruction and input relevant information." indicates that there was an error on the initial page (which is still open in its original tab), but no red exclamation marks appear.

Very 1998ish.

Sixteen hours after filing the initial report, and getting an auto-generated email indicating that it had been received and entered into their ticketing system, I get the following email from "sg-cshelpdesk@jobsdb.com.sg"):

Dear Sir / Madam,

Please clear the cache of your browser by deleting the cookies and temporary internet files, enable the java and javascript, disable the popup blocker, open a new browser and try from there.

Otherwise, please try other browsers like firefox or Internet Explorer 6.0 & above to access the Career Portal and verify that your Internet Settings are configured correctly against http://sg.career.jobsdb.com/faq/documents/Internet_Explorer_Settings.pdf. Then open a new browser and submit from there.

Best Regards, Customer Service

Absolutely no problem-specific content whatsoever. This could easily have been (and in my opinion probably was) an automatically-generated message that was sent when no relevant response would be forthcoming within 24 hours after filing the problem (the '16:44' in the timestamp, or 4:45 PM local time, sounds suspiciously like "it's the end of the day, nobody got to this, blow him off and hope he goes away.") There continues to be an explicit assumption within the fluff mail that I'm using Internet Exploder; even though the second paragraph mentions "firefox" (sic), the PDF link supplied says that it relates to IE.

I'm getting used to companies, particularly the all-pervasive oligopolies, here in Singapore treating the customer with contempt. Knowing that no living human would probably ever read it, however, I did send one last complaint email.

Dear Customer “Service”,

Did you actually read the report I filed? Here are a few Statistically Improbable Phrases to clue you in:

• “standards-compliant (non-IE) browsers”. I tested your site with ten different browsers on four different operating systems. In no case did it operate correctly. By the way, Microsoft does not make a version of Internet Exploder for non-Windows systems.
• “Safari” — Safari is the standard, comes-with-the-system browser for Macs and iPhones, and is available for those wishing to upgrade their Internet experience on Windows as well.
• “an ASP alert box...is displayed in a new tab” — obviously not what was intended by the coder; however, competent JS wouldn’t do this.

I am a Web developer with nearly 15 years experience. I have been teaching Web development for ten years. If a student of mine from my very first class, back in 1999, had submitted work with such grossly defective JavaScript, he would have been required to redo it. You see, even since the early days of the World Wide Web, there have been standards that even Microsoft can comply with – though they also “support” their own proprietary, non-standard version of HTML and JavaScript and have encouraged careless or inexperienced “developers” to lock themselves in, breaking sites for anyone using a non-IE browser. That includes half the smartphones on the planet, every single non-Windows desktop or laptop system, and search engines like Google.

You should take http://www.amazon.com/Novelty-Sign-Brain-before-engaging/dp/B000K62V92 as a gentle but firm recommendation.

Sincerely,

No, I don't expect a relevant reply. Yes, I do feel better. No, I won't be using JobsDB or any related site, and will happily explain to any who ask my opinion precisely why.

A final note: There is a nice little site called yougetsignal.com which offers a nice set of network (Internet) diagnostic and information tools. One page on the site, http://www.yougetsignal.com/tools/web-sites-on-web-server/, does what's called a "reverse IP address lookup". I used this page to find out what other sites are hosted by the same server as jobsdb.com.sg. Try it yourself; the results included apparently all "country-specific" jobsdb.com sites — as well as the site for Target, an American retailer. Interesting.

Not that I expect it, but if any of the parties mentioned here were to reply to what I've written here, I'd be happy to publish their response.

Expanding the Omniverse

Anybody who's worked with me in about the last 25 years knows that I've been preaching the idea that software craftfolk should never stop learning. Further, I've always believed that learning new languages or tools is one of the easiest ways to accomplish this, keeping the mind supple and open to new ways of doing things. And by and large, I've kept this up, learning enough of a new or long-neglected language to at least be able to read and patch code every two or three months. (You do the math.)

One of the languages that I learned rather early on in its lifecycle is Ruby. (Wikipedia has a rather good article summarizing the history, if you're unfamiliar with it.) Early 1.0-1.4 or thereabouts releases were interesting — they showed the power and promise of the concepts that Ruby is built on, without the bloated inscrutability of, say, perl. As so often has happened, I'd learned just enough to be dangerous, and then got sucked back into the charnel house that is Microsoft Windows development. By the time I had the time and inclination to start messing with Ruby again, an unfortunate thing happened.

Ruby on Rails.

Not that Rails isn't a great tool for building 37signals-type Websites; it clearly is. But it became a victim of its own hype and started being used for everything imaginable — famously including Twitter (a hype explosion in its own right.) Rails was enough to push me — and apparently a good number of other folks — away from Ruby and onto other languages, notably Python. And so I spent the bulk of the next couple of years in PHP, Python, C++, Objective-C, D (another bit of unsung genius), and managed to keep busy.

We developers have a well-known cliché for what drives us to do new things or participate in development projects; "scratching an itch". To scratch an itch in this context is to solve a problem that we ourselves are facing, or to do something that otherwise interests us. What got me motivated to get back up to speed in Ruby wasn't the Rails hype, or even seeing all the nice APIs that Twitter and Repertoire had made available. My problem was a bit simpler and more immediate.

Apple's Mail app started crashing under the load I was giving it. For about a year, I'd had a mail store that averaged about 2 GB and I was getting on the order of 600 to 800 emails a day. I don't mean to be critical of the app; it just wasn't designed to do what I was asking it to, certainly not when sharing 2 GB of RAM with the rest of the system. By the time I migrated away from Mail, I had over 400 filtering rules defined, to slice and dice incoming emails into appropriate folders where I could deal with them as I chose.

In late May, 2009, I up and migrated my email from Apple Mail to Microsoft Entourage. For those of you whose only exposure to Microsoft email software has been Exchange or Lookout! ("Outlook") Express, you're in for a pleasant shock. Entourage runs quite happily in the system as it is (even if I can't run certain other apps at the same time without upgrading RAM), and doesn't give me the maddening ten-minute freezes that were common with Apple Mail as it tried to figure out what to do next. Importing my existing mail store was a breeze. The filtering rules even came along for the ride, and Microsoft's rules editor is a real treat. There was, however, one small problem.

Somehow, the ordering of rules had become scrambled during the import, and after a few weeks of hand-editing to fix the biggest problems, I started looking for a program that would let me import, export, reorder and bulk-edit Entourage's rules. So far, I haven't been able to find one. (If anybody knows of anything useful, please add a comment or email me.) OK, I thought, no problem; nearly everything on the Mac is scriptable. I should just need to learn how to access Entourage's rule set from AppleScript or something similar, then I can write the tool I want. Not trivial, but certainly something that seemed conceptually quite practical.

So I started learning AppleScript, and casting about for tools and sample code that talked to Entourage from AppleScript. While searching, I ran across Matt Neuberg's site. Dr. Neuberg wrote the definitive guide on AppleScript — and then found something that worked better for him: rb-appscript. He's written an online book about it (eventually to be published on dead trees). At this point, I said "ok, let's get started with Ruby again and see what we can do."

If anybody has any pointers or suggestions, please comment. Thanks for reading.

"You can have my...when you pry it out of my cold, dead hands"

One of the, shall we say, unusual things about being in this line of work is that you develop stronger-than-is-healthy bonds to particular bits and pieces of technology, both hardware and software. For example, I think that anybody who's worked on a Mac as their main system for a year or so would take a catastrophic productivity hit if they were required to work in a Windows-only environment. Further, I assert, based on experience and observation, that this hit would actually increase in severity commensurate with previous Windows experience, as the nature of the problems and hassles encountered on a continuous basis would be that much more familiar.

But, believe it or not, this isn't a hit piece on Windows. If anything, Mac OS X takes the brunt here, in more or less a continuation of a previous post.

Don't get me wrong. The Mac itself is still one of the very few things I'd plug into that quote in the title. But the differences between it and the BSD Unix heritage it's based on, at an architecture-implementation level, sometimes drive me up the wall; it's as though I'm caught in the old Saturday Night Live "It's a floor wax AND a dessert topping!" skit.

Case in point: the Web developer's Swiss Army Ginsu Knife, PHP. On Unix and Linux systems, it's very straightforward to build a scripting engine that contains just the features and extensions needed, with security and debugging features added in to taste. The last few releases have even made that process humanly feasible for Windows usees. On Mac OS X, however, it's a different story entirely. (Item: As of Friday 11 July, the search "mac php configuration" returned some 5.22 million hits. Applying Sturgeon's Second Law to this leaves us with at least half a million pleas for and/or offers of help.)

The practical result is that, on Mac OS X, one uses one of the various prebuilt binaries for PHP (from Apple, MacPorts, MAMP or similar — or goes without. If you're interested enough in PHP to want it to work on a Mac, you probably have some urgent deadline breathing down your neck; you don't have time to figure out how/why there are unique runes and incantations involved in making it work. This is the dark flip side of "almost everything Just Works"; the things that don't Just Work tend to average the entire experience out.

So, what's the most efficient solution for the reasonably serious Mac-loving Web developer? Simple: max out the memory in your Mac (and I do mean "as much as Steve&Co let you put in and not one byte less, blastit!"). Then, add one of the software tools that's going to be on that Mac somebody pries out of my "cold, dead hands"; VMWare Fusion. Once you have Fusion installed, grab an ISO image of your BSD version or Linux distro of choice, install it, and use that for your PHP and other web-dev activities.

Better still, you can still use your fave Mac editor, browser and so on during development in the VM; you'll want to set up ssh on your VM, and then use sshfs to mount your Linux/BSD filesystem as a disk volume in your Mac. From that point — a file is a file is a file; you're just taking advantage of the *ix VM's tools. This is how I do my PHP 5.3 and PHP 6 testing now.

And, after going through all this extra effort, you may well pray that the new version of Mac OS X enables a better native solution. I know I do.

Nothing is perfect in this world, and all technologies have speed bumps in some fashion. The good bad thing about OS X is that there aren't nearly as many as in some other systems I could name — but when you hit the ones that are there, you hit them hard.

---

EDITED 2010/11/03: At the time this was originally written, there was a prerelease bit of software that lots of people, including the vendor and the opportunistic authors and publisher of at least one programming book, thought was going to eventually become PHP version 6. Not quite; one of the long-promised features for PHP 6 went down a rabbit hole and ultimately killed the whole thing. Most of the usable "PHP 6" features were incorporated into version 5.3.2.

The Best Tool for the Job

One of the nice things about growing up around (almost exclusively) men who were master mechanics, carpenters or other such highly skilled tradesmen was that I developed an appreciation both for "the best tool for the job at hand" and "making do with what's available" — and whichever of these applied, accomplishing the task at hand to the best of anyone's ability.

As I've progressed through my software and Web career, I've become highly opinionated about the tools I use, just like any other experienced software craftsperson I've ever known. You and I might use different tools to accomplish what functionally is the same task, but so long as we each have practical, experiential bases for those preferences, we should just go ahead and get what needs doing done. (There's an argument in there for open standards as a requisite for that to happen, but that's another post.)

Too many people who should know better have religious-level devotion to or hostility towards certain companies and/or products. Yes, that includes me; I know I've said some pretty inflammatory things, usually when I felt someone was expressing a religious belief masked as a technical opinion. No doubt they've felt the same about me and any others who were incautious enough to oppose their evangelism (or reactionism, depending on the circumstances). In general, it should be pretty evident to everyone with a personal or professional involvement in IT or personal electronics that trends are driven as much by "what I say three times is true!" as what actually can be shown to be true. That's how mediocre-at-best products become "industry Leaders"; inertia and close-mindedness set in, reinforced by a well-funded, continuous and strident marketing/branding campaign.

I was having a discussion about this online recently, with a former associate who's long had me pegged as an ABMer ("Anything but Microsoft"). I can understand how he formed that opinion; I've long complained about the (innumerable) defects in the "market-leading" operating system, and about how slowly progress has been made in cleaning up the most egregious faults (such as security). But I've also worked at Microsoft in Redmond — three different times — and I've always been impressed by the number of truly gifted people working there. They've had their triumphs and tragedies (anyone used Microsoft Bob lately?). They've had to deal with widely differing process and management effectiveness as they transfer between or liaise with different groups. They've ignored a lot of what has been done outside the company, but they've also created some amazing things inside; too many of which unfortunately never make it into public products.

And the quality of their work product varies as much as any of the factors that go into it. Cases in point: compare, say, Windows Vista with Windows Mobile or the XBox; compare Microsoft Outlook (forever known as "Lookout!" to security/admin people) with Entourage; compare Word for Windows to Word for the Mac — what I understand is a completely different code base (and visibly so) that "just happens" to be able to flawlessly read and write documents shared with Word for Windows.

I also reread a blog post I wrote last December where I detailed the issues I was starting to have with Apple's own Mail app for the Mac. I have a mail store that's hovered somewhere above 2 GB for the last year. I receive 100-200 legitimate emails per day (and up to 700 spams). I presently have over 230 filtering rules defined for how to handle all that mail. Those rules have been built up over the last five years or so — first using Mozilla Thunderbird, then Apple's Mail.app, and now a new system; a progression that also speaks eloquently about the value of open standards. I have never, to my knowledge, lost a saved message whilst transferring from one package to its successor. The few hiccups each transition has had with filtering rules have all been relatively easy to find and fix, with the newest app making that process breathtakingly simple.

The new mail app? As you've no doubt guessed, Microsoft Entourage. It, like every other Mac app I've ever used, Just Works as expected (at least until you get out to the far, bleeding edges). If Microsoft made Windows and Office for Windows as well as they make Entourage (and the rest of their Office:Mac products), they really wouldn't have to worry about competition — and they'd richly deserve that. The market-friendly price for their Mac product (where their major, worthy competitor sells for US$79) is just icing on the cake.

I don't hate Microsoft. I just wish they would stick to what they do as well or better than anyone else, and leave the crappy products that can never be anything but hypersonic train wrecks — like Windows and Internet Exploder. I wish that ever more fervently every time I'm asked to help some hapless Windows usee fix "why my computer doesn't work". That would also make Microsoft's long-suffering stockholders — including current employees, former employees and myself, among others — feel a lot better.

Another item from the "That's obvious - in hindsight" dept.

Since upgrading to Safari 4 (why haven't you yet?), I ran into a problem with the single add-in that I've bothered keeping in Safari — Pith Helmet. If you're familiar with AdBlock Plus on Firefix, you've got the basic idea; an add-in to your browser(s) of choice that lets you block advertising, annoying Flash, or pretty much anything else you can identify by file name (e.g., "*.swf*) or by URL (e.g., "http://www.doubleclick.net"), and "magically" removes it from the final content displayed by your browser. This feature has gotten so popular that several browsers are building in more-or-less-competent versions of it by default.

Getting back to the problem... PithHelmet, the Safari ad blocker, was incompatible with Safari 4 because the framework it depended on, called SIMBL, has not had an update since October 2006 — which, as far as Safari or WebKit, equates to "forever". So I do a Google search for "greasekit safari 4", and then start whittling down the results (English language only, please, and only within the last year). Eventually, I ran across a forum post (which I have since lost) saying basically "Yes, PH crashes Safari 4 but has anybody else tried out Fanboy's AdBlock CSS sheet?"

Which, if you know anything about Web development, should have sent the palm of your hand rocketing toward your forehead in a major "D'oh! moment. Of course! Why didn't I (or we all) think of that about 6 or 8 years back?

For those of you who aren't as knowledgeable about the detailed workings of your browser, let me explain. Every Web browser, probably since at least Netscape 1.0, has included support for "user style sheets"; where individual users (or organisations of such users) can choose to instruct their browsers to display certain specific content differently than it was originally specified. For this to work, the user in question (or someone s/he depends on) have to be very literate in HTML and particularly CSS, the languages of Web pages. To do simple blocking, like 'block all SWF files", isn't hard with modern browsers, but the Web developers themselves can make life significantly easier by following modern "best practices". The "practices" particularly relevant here are "add 'id' attributes to all structural and semantic elements." (hmm; what's the difference between that and "all elements"? Another blog post...) If the developer does that, including for the body tag, then it's very easy for even a neophyte user to start filtering just what's wanted.... and learn something about how Web pages work into the bargain.

Thanks for reading — and commenting.

Remember to test your testing tools!

I've been doing some PHP development lately that involves a lot of SPL, or Standard PHP Library exceptions. I do test-driven development for all the usual reasons, and so make heavy use of the PHPUnit framework. One great idea that the developer of PHPUnit had was to add a test-case method called setExpectedException(), which should eliminate the need for you (the person writing the test code) to do an explicit try/catch block yourself. Tell PHPUnit what you expect to see thrown in the very near future, and it will handle the details.

But, as the saying says, every blessing comes with a curse (and vice versa). The architecture of PHPUnit pretty well seems to dictate that there can only be one such caught exception in a test method. In other words, you can't set up a loop that will repeatedly call a method and pass it parameters that you expect it to throw on; the first time PHPUnit's behind-the-scenes exception-catcher catches the exception you told it was coming, it terminates the test case.

Oops. But if you think about it, pretty expectable (pardon the pun). For PHPUnit to catch the exception, the exception has to get thrown and unwind the call stack past your test-case method. That makes it very difficult (read: probably impossible to do reliably inside PHPUnit's current architecture) to resume your test-case code after the call that caused the exception to be thrown — which is what you'd want if you were looping through these things.

This leaves you, of course, with the option of writing try/catch blocks yourself — which you were hoping to avoid but which still works precisely as expected.

Moral of the story: Beware magic bullets. They tend to blow up in your face when you least expect it.

News Flash: Microsoft Reinvents Eiffel, 18 Years On

One of the major influences on the middle third of my career thus far was Bertrand Meyer's Eiffel programming language and its concept of design by contract. With such tools, for the first time (at least as far as I was aware), entire classes of software defects could be reliably detected at run time (dynamic checking) and/or at compile time (static checking). I worked on a couple of significant project teams in the mid- to late '90s that used Eiffel quite successfully. Further, it impacted my working style in other languages; for several years, I had a reputation on C and C++ projects for putting far more assert statements than was considered usual by my colleagues. More importantly, it made me start thinking in a different way about how to create working code. Later, as I became aware of automated testing, continuous integration and what is now called agile development, they were all logical extensions of the principles I had already adopted.

This all happened over a period of 15 or so years, in a field where anyone with more than 2 or 3 years' experience is considered "senior". But for me, and most other serious practitioners who I knew and worked with, two to three years was really just about as long as it took to answer more questions than we raised. That, in most crafts, is considered one of the signs of becoming a journeyman rather than a wet-behind-the-ears apprentice.

Then, a few hours ago, I was reading a blog entry by one David R. Heffelfinger which mentioned a project at Microsoft DevLabs called "SmallBasic". Another project that the same organization developed is called "Code Contracts"; there's a nice little set of tools (which will be built into the upcoming Visual Studio 2010 product), and a nice introductory video. Watch the video (you'll need Silverlight to view it), and then do some research on Eiffel and design-by-contract and so on, and it's very difficult not to see the similarities.

So, on the one hand, I'm glad that .NET developers will finally be getting support for 20-year-old concepts (by the time significant numbers of developers use VS 2010 and .NET 4.0). Anything that helps improve the developer and user experiences on Windows (or, in fact, any platform) is by definition a Good Thing™.

On the other hand, I see more evidence of Microsoft's historical Not Invented Here mentality; beating the drum for "new and wonderful ideas for Windows development" that developers on other platforms have been using effectively for some time. While the Code Contracts project indirectly credits Eiffel — the FAQ page links to Spec# at Microsoft Research, which lists Eiffel as one of its influences — it would have been nice to see acknowledgement and explanation of precursor techniques be made more explicitly. Failure to do so merely reinforces the wisdom of Santayana as applied to software: "Those who cannot remember the past are condemned to repeat it", as well as "Fanaticism consists in redoubling your efforts when you have forgotten your aim." This last is something that we who wish to improve our craft would do well to remember.

What do you all think?

Jaw-Droppers - Blast from the Past

Just when you thought it was safe to forget that the 1970s ever existed... this gem shows up on the XML Daily Newslink, a mailing list I follow intermittently. (Actually, this was included in the XMLDN from Wed 11 Feb — an indication of how "closely" I've been following lately.)

Developing a CICS-Based Web Service
G. Subrahmanyam, G. Mokhasi, S. Kusumanchi; SOA World Magazine

Web services have opened opportunities to integrate the applications at an enterprise level irrespective of the technology they have been implemented in. IBM's CICS transaction server for z/OS v3.1 can support web services. It can help expose existing applications as web services or develop new functionality to invoke web services. One of the commonly used protocols for CICS web services is SOAP for CICS. It enables the communication of applications through XML. It supports as a service provider and service consumer independent of platform and language. SOAP for CICS enables CICS applications to be integrated with the enterprise via web services as part of lowering the cost of integration and retaining the value of the legacy application. SOAP for CICS also comes along with the implementation encoder and decoder.

"SOAP for CICS"? Give it a REST, guys. On the other hand.... preserving this by-now more-solid-than-most-rocks 1969-era technology IS a signal achievement; an example of engineering stability on par with Soyuz.

On the other hand... support for legacy technologies like this looks set to become increasingly expensive and risky over time, since apparently:

1. Numerous surveys published in the last few years indicate that the sizable majority of "old mainframe" tech folk will have retired by 2010 (do the math on the years), and

2. partly as a result of (1), users risk become increasingly dependent on outsourced Indian providers — hardly conducive to effective project control.

In my own professional view, technological preservation activities like this are mainly useful for one thing: they can serve as the pattern against which a re-implementation of the business process using more currently-supportable tech can be verified. And once an organization has done this for its most valuable legacy systems (which wouldn't have been preserved this long if they weren't so valuable), the actual and perceived risk of migrating to new technologies as conditions warrant drops dramatically. After all, if you (or your internal colleagues) have successfully brought your line-of-business systems from CICS via REST to, say, PHP or Java, you're a lot more comfortable with the idea of migrating to whatever the mid-to-trailing technology is in ten years' time — while the support costs of the (then) existing system are still manageable.

Are any of you actually involved in any technological archaeology like this? When I was younger, I used to brag that half the systems I'd worked with were older than I was, but that stopped being true about 1995. (AFAIK, I was one of the last guys to professionally touch a working IBM 360 mainframe, in about 1989.)

(Original article at this entry's title link, which is also here.)

Professionalism, Web development, and giving oxy to morons

Whereas a poor craftsman will blame his tools, poor tools will handicap even the most skilled craftsman.

As I insinuated in my previous post, I'm getting up to speed on the Zend Framework, the "900-kg elephant" of PHP application frameworks.

One major bone I have to pick with the ZF team is with regard to documentation: each time I've checked the site in the last couple of months, there's been an apparently current HTML version (now clocking in at some 300 HTML pages). There is also a PDF version, the promise of which is used as an enticement to register for their content distribution network (and, presumably, marketing info). As of this moment, however, the framework is at version 1.8.0, but the PDF version of the programmer's reference manual only covers version 1.6.0 (from September, 2008); some 12 releases earlier. It no longer fully matches the actual code, to the point where it is not difficult for a new developer to get deeply confused.

After spending a half-hour browsing the HTML version of the document, I am unable to find any declaration as to which version of the Framework is documented. However, the README.TXT file included with the source distribution states that it covers the 1.8 release, revision 15226, released on April 30, 2009. Classes which are listed in the README as being new, such as Zend_Filter_Encrypt, are documented in the HTML programmer's guide. Establishing a match between the (HTML) doc and the current code is non-trivial, however. While it may be argued that people unfamiliar with browsing a Subversion repository are not likely to be common within Zend's target audience, I would indirectly refute that: a product release, particularly one with a strong industry following, should be

• properly documented;
• easy for a (prospective) user to verify that he has the complete package; and
• with a definite, intuitive learning curve.

In my view, the Zend Framework fails on at least two of these points. The assertion within large segments of the PHP community that it is the "gold standard" of PHP application frameworks should be a disturbing, cautionary omen: if Web development, particularly PHP development, wishes to be taken seriously by the software industry at large, then some major improvements and attitude shifts need to occur quickly, publicly and effectively. It is still far too easy for potential developers outside the "early-adopter" leading edge to scoff that PHP development (and, by extension, Web development as a whole) is still far too immature and amateurish to be taken seriously. As someone who has developed professionally in PHP for some ten years now, that is a disturbing state of affairs; one that I would love to see (and participate in) a free-ranging discussion of.

OMFG, or Holy Deforestation, Batman!

As some of you know, I'm working on a book on Web development, using off-the-shelf tools (frameworks, template engines, JavaScript libraries, etc.) to leverage semantic, standards-compliant, accessible, search-friendly Websites. (That's more a matter of adjusting your development philosophy and workflow than anything else, but I digress). As part of that, I'e been doing a (reasonably) comprehensive review of PHP 5 application frameworks. You might have heard of ezComponents or CakePHP, but the 900-kg elephant in the room is definitely the Zend Framework. It ships with the Dojo JavaScript toolkit, but doesn't make it excessively difficult to mix and match others (Scriptaculous, Prototype, jQuery, etc.) if desired.

And here's another reason to call ZF the '900-kg elephant' — the programmer's reference guide (for version 1.7) weighs in at a <sarcasm>svelte</sarcasm> 1170 pages. Don't print this at home, folks. Better yet, just don't print it... either browse it online or download the PDF. Save a forest or six. For you old-timers, this will remind you quite a bit of US DOD Standard 2167A, "fondly" remembered as "documentation by the boxcar load".