Why is "doing things right" a "thing"? Whither professionalism?

Over on the Reddit r/programming subreddit, /u/jfalvarez posted a link to Tom Stuart's great presentation at the Scottish Ruby Conference 2014, entitled "The DHH Problem".

If you're involved with Ruby or Web development in any way, it's well worth three and a half minutes of your time.

On the Reddit post where Mr Alvarez posted the link to the video, another Redditor, dventimi, asked

Why is this a thing?

I disagree with the later poster who snarked "People like gossip." It's much, much more important than that.

What follows is the content of my reply to "why is this a thing?" I'd love to have a discussion with you about it, either here or on the original post.

---

Because people are finally waking up to the fact that the coding style and philosophy espoused by DHH are disastrous in non-toy applications maintained over any significant period of time.

That this should have been obvious to all, but wasn't discussed as such, once Twitter, after having conflated Rails with Ruby as a whole, left Ruby because they couldn't make Rails do what they needed to no matter how many times DHH came onsite and insulted their intelligence, speaks eloquently to how broken things are.

Most master software developers I've met and worked with have been mediocre-at-best promoters, including self-promoters. DHH has the opposite problem: his initial extraction of Rails from Basecamp's codebase "worked", for apps very much like Basecamp at the time. His fixed, what some have called "trolling", attitudes and the personality they describe, prevent him from seeing things from any other perspective or from growing his skills beyond a certain point. The echo chamber of acolytes and zealots his sublime self-promotional talents have allowed him to accrete around himself "ensure" that he never has to.

That's a problem for the rest of us, who have deadlines, work to finish within those deadlines, and paycheques we'd like to be able to cash that we won't get unless we meet those deadlines with that work. Especially for those of us who wish to improve our craft and, in whatever way we can, nudge ourselves and others towards helping the craft of software development more closely approximate a true engineering discipline.

Paying excessive heed to the opinions-masquerading-as-principles brayed by DHH, and twisting ourselves to fit inside The Rails Way, actively impede all of the above tasks. Tom Stuart was absolutely right to thank DHH for producing Rails and promoting it sufficiently that the rest of us get to write Ruby for a living…just before tearing DHH a new one, in the polite, discreet way only a well-educated Brit can, for getting so much of Rails and how software should be written so disastrously wrong.

Thank you, Tom Stuart. An entire industry may well be in your debt.

Take Only Pictures; Leave Only Footprints

If you've ever visited a National Park or many state parks in the United States, or been involved with a nature-oriented community group, you've likely heard that saying many times. For those who haven't, the meaning should be obvious: leave the shared place that you're moving through in at least as good condition as you found it, so that the next people to travel that way can enjoy it as much as you did.

That applies to environments other than the great outdoors, of course. The phrase popped into my mind earlier today as I was poking at a jQuery plugin to add a context menu to a Web app I'm developing.

It's great that there's so many free software tools out there; I've been using them for more than 25 years, and I've developed more than a few. Sometimes, however, one is reminded of the wisdom of Oliver Wendell Holmes when he said, "Learn from the mistakes of others… you can't live long enough to make them all yourself." But in the grand software tradition, we often give it our best effort.

This is a reasonably well-known plugin, one of several that offer to solve a common problem in more-or-less similar fashion. And if you use it precisely in the way that the author expected, it does the job. But even then, if you're using it in a Web app or site that customers are paying money for, you might hope that they never hit the "View Source" menu item in their browser. And, to be fair, this is not intended as a slam against Matt Kruse or his code; most of the other plugins I've looked at in the last couple of days have at least as many "quirks" and assumptions.

"It's just a context menu," I hear you say. "What could possibly go wrong?…go wrong?…go wrong?…(sound of tape snapping)" (with apologies to Westworld, a movie with many lessons on the art and craft of software development.)

If you're adding the menu to your page when it's first loaded, and you want a static menu, with the options the same for all uses of the context menu, fine. If you want a more dynamic menu, and don't mind attaching functions to the menu that know enough of the detailed state of your page/app that they can generate the properly-adjusted menu items each time the menu is displayed, that works, too. But the main assumptions are that the plugin's main method will be called once and only once on a page, and that you really don't care about the markup being added to the end of your page. The first assumption, particularly in a rich front end, is likely to be naïve; the second is, bluntly, an assault on your professionalism.

What's so bad? Here's the plugin's output for a very simple menu, reformatted for clarity:

<table cellspacing="0" cellpadding="0" style="display: none;">
  <tbody>
    <tr>
      <td>
        <div class="context-menu context-menu-theme-vista">
          <div class="context-menu-item " title="">
            <div class="context-menu-item-inner" style="">One</div>
          </div>
          <div class="context-menu-item " title="">
            <div class="context-menu-item-inner" style="">Two</div>
          </div>
          <div class="context-menu-item " title="">
            <div class="context-menu-item-inner" style="">Three</div>
          </div>
        </div>
      </td>
    </tr>
  </tbody>
</table>

<div class="context-menu-shadow" style="display: none; position: absolute; z-index: 9998; opacity: 0.5; background-color: black;"></div>

If you read HTML and CSS, you're probably nodding your head, thinking "that looks simple enough; hey, hang on a minute…" That "hang on" is where you start seeing any of several issues:

1. We have a table, containing a (rarely-used) tbody, containing a single tr, containing a single td, containing a series of nested divs which contain the menu items. If he'd implemented the entire menu as a table, he might have been able to claim that he was trying to support 1997-era browsers. (Would jQuery 1.4 work in Netscape Navigator 4?) Failing that, it's a mess. Nesting divs (or more semantically appropriate elements) has been a Solved Problem for a decade or so.

2. Even though this is implicitly intended to be fire-and-forget markup added to the DOM at page-creation time, it's disturbing that there are no identifiers for any of the table elements. If you wanted to, say, remove the table later, you'd have to use code like

   $('.context-menu').parent().parent().parent().parent().remove()

   Ick. And then you'd have to go clean up the shadow div separately.

3. Speaking of the shadow div being separate from the menu-enclosing table, why not wrap both in an (identified) div, so that you (and your clients' code) can treat the menu as a single entity?

4. Doing that would also let you dynamically delete and replace the menu, based on changes in the state of your app, without needing to define menu-item handler functions that violate the Principle of Least Knowledge.

5. Another benefit of that would be if your context menu was generated more than once (possibly because the content needed changing or items needed disabling), you'd never have more than one copy of the menu in the DOM. As it is now, you can have an arbitrarily large number, with only the most recently-added being the "active" one. Ugh. This would be more likely to happen on a dev box running a test suite, but still. Ugh.

6. Particularly in a behaviour-driven or test-driven development (BDD or TDD) environment, being able to test/validate the markup and the item-handler logic separately as well as together is important. Doing so with this plugin (and again, to be fair, most of the others) eliminates the normal-use workflow from consideration.

7. One feature of this particular plugin is that it supports having you define your own HTML for the menu and pass it in. But the example given is too simple to be useful as a menu. Browsing the plugin source code seems to indicate that there is no event-handler support for menus defined in this manner; you'd have to iterate through your context menu items, assigning event handlers for at least the click event. If you're going to do all that work, why bother with this plugin?

8. A table and divs. Enclosed and in parallel. (Careful wiping that green sludge off your monitor; that's your brain that just exploded.)

Menus are "important" enough that HTML 5 has its own set of elements dedicated just to marking up menus. However, sensible, reasonably semantic standard patterns for use with HTML 4 and XHTML 1.x have evolved that address the issues I've mentioned (among many others). The markup for the example menu above could have been written as:

  <div class="context-menu-container" id="someId">
    <ul class="context-menu context-menu-theme-vista">
      <li class="context-menu-item">
        <span class="context-menu-item-inner">One</span>
      </li>
      <li class="context-menu-item">
        <span class="context-menu-item-inner">Two</span>
      </li>
      <li class="context-menu-item">
        <span class="context-menu-item-inner">Three</span>
      </li>
    </ul>
    <div class="context-menu-shadow"></div>
  </div>

One outermost div, with an id so your plugin doesn't get confused if you have multiple menus on a page, but one div so you can work with it as a single unit. An unordered list containing list items corresponding to the menu items, since that's the closest HTML4/XHTML 1 comes to HTML5's menu semantics. The Script that builds the thing can adapt the styles and attributes at the class+id CSS level, eliminating the need for hard-coded monsterpieces such as that style attribute for the table.

If you're going to write a jQuery plugin, live in jQuery. You can dynamically modify styles, attributes, content, even whole sections of how your page is rendered in the browser, without touching the basic markup. You can put the style bits that you intend to remain constant into a CSS file and link those styles to individual DOM fragments via classes and ids. That also keeps your HTML and CSS clean and cacheable.

If we don't write clean code, then having a lightning-fast browser on a petabit-Ethernet connection won't matter; users and reviewers will still complain that "your app is slow". Think before, as, and after you code, and remember:

First, do no harm.

Patterns and Anti-Patterns: Like Matter and Anti-Matter

Well, that's a few hours I'd like to have over again.

As both my regular readers know, I've long been a proponent of agile software development, particularly with respect to my current focus on Web development using PHP.

One tool that I, and frankly any PHP developer worth their salt, use is PHPUnit for unit testing, a central practice in what's called test-driven development or TDD. Essentially, TDD is just a bit of discipline that requires you to determine how you can prove that each new bit of code you write works properly — before writing that new code. By running the test before you write the new code, you can prove that the test fails... so that, when you write only the new code you intend, a passing test indicates that you've (quite likely) got it right.

At one point, I had a class I was writing (called Table) and its associated test class (TableTest). Once I got started, I could see that I would be writing a rather large series of tests in TableTest. If they remained joined in a single class, they would quickly grow quite long and repetitive, as several tests would verify small but crucial variations on common themes. So, I decided to do the "proper" thing and decompose the test class into smaller, more focused pieces, and have a common "parent" class manage all the things that were shared or common between them. Again, as anyone who's developed software knows, this has been a standard practice for several decades; it's ordinarily the matter of a few minutes' thought about how to go about it, and then a relatively uneventful series of test/code/revise iterations to make it happen.

What happened this afternoon was not "ordinary." I made an initial rewrite of the existing test class, making a base (or "parent") class which did most of the housekeeping detail and left the new subclass (or "child" class) with just the tests that had already been written and proven to work. (That's the key point here; I knew the tests passed when I'd been using a single test class, and no changes whatever were made to the code being tested. It couldn't have found new ways to fail.)

Every single test produced an error. "OK," I thought, "let's make the simplest possible two-class test code and poke around with that." Ten minutes later, a simplified parent class and a child class with a single test were producing the same error.

The simplified parent class can be seen on this page, and the simplified child class here. Anybody who knows PHP will likely look at the code and ask, "what's so hard about that?" The answer is, nothing — as far as the code itself goes.

What's happening, as the updated comments on pastebin make clear, is that there is a name collision between the ''data'' item declared as part of my TableTest class and an item of the same name declared as part of the parent of that class, PHPUnit's PHPUnit_Framework_TestCase.

In many programming languages, conflicts like this are detected and at least warned about by the interpreter or compiler (the program responsible for turning your source code into something the computer can understand). PHP doesn't do this, at least not as of the current version. There are occasions when being able to "clobber" existing data is a desirable thing; the PHPUnit manual even documents instances where that behaviour is necessary to test certain types of code. (I'd seen that in the manual before; but the significance didn't immediately strike me today.)

This has inspired me to write up a standard issue-resolution procedure to add to my own personal Wiki documenting such things. It will probably make it into the book I'm writing, too. Basically, whenever I run into a problem like this with PHPUnit or any other similar interpreted-PHP tool, I'll write tests which do nothing more than define, write to and read from any data items that I define in the code that has problems. Had I done that in the beginning today, I would have saved myself quite a lot of time.

Namely, the three hours it did take me to solve the problem, and the hour I've spent here venting about it.

Thanks for your patience. I'll have another, more intelligent, post along shortly. (That "more intelligent" part shouldn't be too difficult now, should it?)

In Praise of Robust Tools and of The Future™

One of the best points about doing Web development in PHP is that it's so widely used; several respectable estimates by organizations that get paid to find these things out say that some varying number north of 50% of all sites on the World Wide Web use PHP as their implementation language. This includes numerous content management systems, or CMS, such as WordPress, Joomla! and Drupal.

One of the far-less-good points about doing Web development in PHP is that it's so widely used; the (evolving) "standard" set of tools, in true open-source fashion, are assembled and maintained by an ad-hoc band of individual and small-corporate luminaries, on infrastructure that worked just fine back when any given server was hit a couple of thousand times a year; by continuing to rely on (what to the outside Net appear to be) individual servers without terribly huge pipes connecting them to the larger Internet, the infrastructure completely fails when that's scaled up by a factor of a thousand or ten.

Web developers using PHP are highly dependent on an extension architecture/platform called PEAR, the "PHP Extension and Application Repository." Singular. Well, for any given extension or application, it's singular. So, as the user base scales upwards, popular tools that go through a vigorous release cycle, the servers they're hosted on (and network choke points between large subsets of users and those servers) start having reliability problems; transfers slow to a crawl, or fizzle out entirely. (One such transfer this evening proceeded at a sedate 600 bytes per second. Not kilobytes: bytes. That's 1980s dial-up speed.)

Developers in other languages have similar tools; Rubyists have gems available from sites like rubygems.org; Pythonistas have eggs. But somehow I hear a lot less grumbling, and do less of my own, when the subjects of gems or eggs come up; they seem to Just Work™ – which indicates that the network infrastructure is a lot more robust; either larger "pipes" and/or more, distributed servers such as with a CDN.

Trying to access servers like this from Second World cities like Singapore that apparently devote more resources to content filtering/monitoring than easing congestion doesn't help either. Hey, SingNet and M"D"A, how come it takes ten hops to get off an island that's barely that many miles across? If we want to plug Singapore into the "new global economy," having grotesquely under-resourced connections is not helpful.

But back to the main subject: If PHP is going to continue its phenomenally successful growth, then the infrastructure is going to have to decentralise, aggressively. Having essential tools like PEAR and the PHPUnit repository available only from a single point ensures that single points of failure will continue to seriously compromise the PHP ecosystem. No access to servers, no access to tools. No access to tools, then development of PHP artifacts with reasonable efficiency and economy is severely degraded.

In other words, the status quo is a limiting factor on future growth and success. However, as many will be quick to point out, building infrastructure costs money – for hardware, for connectivity, for the paid labour of skilled craftspeople and professionals to create, install, operate and maintain this enhanced infrastructure. No explicit means of funding such an endeavour presently exists, at least not to my knowledge.

So where do we go from here, PHP community?>

I Thought Standard Libraries Were Supposed to be Better...

...than hand coding. Either the PHP folks never got that memo, or I'm seriously misconceptualising here.

Case in point: I was reading through Somebody Else's Code™, and I saw a sequence of "hand-coded" assignments of an empty string to several array entries, similar to:

    $items[ 'key2' ] = '';
    $items[ 'key1' ] = '';
    $items[ 'key6' ] = '';
    $items[ 'key3' ] = '';
    $items[ 'key8' ] = '';
    $items[ 'key5' ] = '';
    $items[ 'key4' ] = '';
    $items[ 'key7' ] = '';

I thought, "hey, hang on; there's a function to do easy array merges in the standard library (array_merge); surely it'd be faster/easier/more reliable to just define a (quasi-)constant array and merge that in every time through the loop?"

Fortunately, I didn't take my assumption on blind faith; I wrote a quick little bit to test the hypothesis:

$count = 1e5;
$data = array(
        'key2' => '',
        'key1' => '',
        'key6' => '',
        'key3' => '',
        'key8' => '',
        'key5' => '',
        'key4' => '',
        'key7' => '',
        );
$realdata = array();

$start = microtime( true );
for ( $loop = 0; $loop < $count; $loop++ )
{
    $realdata = array_merge( $realdata, $data );
};
$elapsed = microtime( true ) - $start;
printf( "%ld iterations with array_merge took %7.5f seconds.\n", $count, $elapsed );

$start = microtime( true );
for ( $loop = 0; $loop < $count; $loop++ )
{
    $data[ 'key2' ] = '';
    $data[ 'key1' ] = '';
    $data[ 'key6' ] = '';
    $data[ 'key3' ] = '';
    $data[ 'key8' ] = '';
    $data[ 'key5' ] = '';
    $data[ 'key4' ] = '';
    $data[ 'key7' ] = '';
};
$elapsed = microtime( true ) - $start;
printf( "%ld iterations with direct assignment took %7.5f seconds.\n", $count, $elapsed );

I ran the tests on a nearly two-year-old iMac with a 3.06 GHz Intel Core 2 Duo processor, 4 GB of RAM, OS X 10.6.4 and PHP 5.3.1 (with Zend Engine 2.3.0). Your results may vary on different kit, but I would be surprised if the basic results were significantly re-proportioned. The median times from running this test program 20 times came out as:

Assignment process	Time (seconds) for 100,000 iterations
array_merge	0.41995
Hand assignment	0.15569

So, the "obvious," "more readable" code runs nearly three times slower than the existing, potentially error-prone during maintenance, "hand assignment." Hang on, if we used numeric indexes on our array, we could use the array_fill function instead; how would that do?

Adding the code:

    $data2 = array();
    $data2[ 0 ] = '';
    $data2[ 1 ] = '';
    $data2[ 2 ] = '';
    $data2[ 3 ] = '';
    $data2[ 4 ] = '';
    $data2[ 5 ] = '';
    $data2[ 6 ] = '';
    $data2[ 7 ] = '';
$start = microtime( true );
for ( $loop = 0; $loop < $count; $loop++ )
{
    $data2 = array_fill( 0, 8, '' );
};
$elapsed = microtime( true ) - $start;
printf( "%ld iterations with array_fill took %7.5f seconds.\n", $count, $elapsed );

produced a median time of 0.21475 seconds, or some 37.9% slower than the original hand-coding.

For folks coming from other, compiled languages, such as C, C++, Ada or what-have-you, this makes no sense whatsoever; those languages have standard libraries that are not only intended to produce efficiently-maintainable code, but (given reasonably mature libraries) efficiently-executing code as well. PHP, at least in this instance, is completely counterintuitive (read: counterproductive): if you're in a loop that will be executed an arbitrary (and arbitrarily large) number of times, as the original code was intended to be, you're encouraged to write code that invites typos, omissions and other errors creeping in during maintenance. That's a pretty damning indictment for a language that's supposedly at its fifth major revision.

If anybody knows a better way of attacking this, I'd love to read about it in the comments, by email or IM.

Let's Do The Time Warp Agai-i-i-i-n!! (Please, $DEITY, no...)

For those who may somehow not be aware of it, LinkedIn is a (generally quite good) professionally-oriented social-networking site. This is not Facebook, fortunately. It's not geared towards teenagers raving about the latest corporate boy band du jour. It often can be, however, a great place to network with people from a variety of vocational, industry and/or functional backgrounds to get in contact with people, share information, and so on.

One of the essential features of LinkedIn is its groups, which are primarily used for discussions and job postings. In the venerable Usenet tradition, these discussions can have varying levels of insightful back-and-forth, or they can degenerate into a high-fidelity emulation of the "Animal House" food fight. As with Usenet, they can often give the appearance of doing both at the same time. Unlike Usenet, one has to be a member of LinkedIn to participate.

One of the (several) groups I follow is LinkedPHPers, which bills itself as "The Largest PHP Group" on LinkedIn. Discussions generally fall into at least one of a very few categories:

• How do I write code to solve "this" problem? (the 'professional' version of "Help me do my homework");

• What do people know/think about "this" practice or concept?

• I'm looking for work, or people to do work; does anybody have any leads?

As veterans of this sort of discussion would expect, the second type of discussion can lead to long and passionate exchanges with varying levels of useful content (what became known on Usenet as a "flame war.") The likelihood of such devolution seems to be inversely proportional to its specificity and proportionally to the degree which the concept in question is disregarded/unfamiliar/unknown to those with an arguable grasp of their Craft.

It should thus be no surprise that a discussion on the LinkedPHPers group of "Procedural vs Object Oriented PHP Programming" would start a flame war for both of the above reasons. With 58 responses over the past month as I write this, there are informational gems of crystal clarity buried in the thick, gruesome muck of proud ignorance. As Abraham Lincoln is reported to have said, "Better to remain silent and be thought a fool than to speak out and remove all doubt."

What's my beef here? Simply that this discussion thread is re-fighting a war that was fought and settled over a quarter-century ago by programming in general. The reality is that any language that has a reasonable implementation of OOP (with encapsulation/access control, polymorphism and inheritance, in that order by my reckoning) should be used in that way.

Several of the posts trot out the old canard about a performance 'penalty' when using OOP. In practice, that's true of only the sharpest edge cases – simple, tiny, standalone classes that should never have been developed that way because they don't provide a useful abstraction of a concept within the solution space, generally by developers who are not professionally knowledgeable of the concepts involved and quite often by those copying and pasting code they don't understand into their own projects (which they also don't understand). That bunch sharply limited the potential evolution and adoption of C++ in the '80s and '90s, and many of their ideological brethren have made their home in Web development using PHP.

Yes, I know that "real" OOP in PHP is a set of tacked-on features, late to the party; first seriously attempted in PHP 4, with successively evolving implementations in 5.0, 5.2 and 5.3, with the semi-mythological future PHP 6 adding many new features. I know that some language features are horribly unwieldy (which is why I won't use PHP namespaces in my own code; proven idea, poor implementation). But taken as a whole, it's increasingly hard to take the Other Side ("we don' need no steeeenkin' objects") at all seriously.

The main argument for ignoring the "ignore OOP" crowd is simply this: competent, thoughtful design using OOP gives you the ability to know and prove that your code works as expected, and data is accessed or modified only in the places and ways that are intended. OOP makes "software-as-building-blocks" practical, a term that first gained currency with the Simula language in the mid-1960s. OOP enables modern software proto-engineering practices such as iterative development, continuous integration and other "best practices" that have been proven in the field to increase quality and decrease risk, cost and complexity.

The 'ignore OOP in PHP' crowd like to point to popular software that was done in a non-OOP style, such as Drupal, a popular open-source Web CMS. But Drupal is a very mature project, by PHP standards; the open-source project seems to have originated in mid-2000, and it was apparently derived from code written for a project earlier still. So the Drupal code significantly predates PHP 5, if not PHP 4 (remember, the first real whack at OOP in PHP). Perusing the Drupal sources reveals an architecture initially developed by some highly experienced structured-programming developers (a precursor discipline to OOP); their code essentially builds a series of objects by convention, not depending on support in the underlying language. It is a wonder as it stands – but I would bet heavily that the original development team, if tasked with re-implementing a Web CMS in PHP from a blank screen, would use modern OO principles and the underlying language features which support them.

And why would such "underlying language features" exist and evolve, especially in an open-source project like PHP, if there was not a real, demonstrable need for them? Saying you're not going to do OOP when using PHP is metaphorically akin to saying you intend to win a Formula One race without using any gear higher than second in the race.

Good luck with that. You might want to take a good, hard look at what your (more successful) colleagues are doing, adopt what works, and help innovate your Craft further. If you don't, you'll continue to be a drag on progress, a dilettante intent upon somehow using a buggy whip to accelerate your car.

It doesn't work that way anymore.

I thought OSS was supposed to break down walls, not build them.

Silly me. I've only been using, evangelizing and otherwise involved in open source software for 15 or 20 years, so what do I know?

In reaction to the latest feature article in DistroWatch Weekly, I'm angry. I'm sad. Most of all, I feel sorry for those losers who would rather keep their pristine little world free of outside involvement, especially when that involvement is with the express intent of not making it easy for non-übergeeks to use open source software — in this case, OpenBSD. OpenBSD, like its cousins FreeBSD and NetBSD, is one of the (current) "major" base versions of BSD Unix. While the latter two have had numerous other projects that have taken their software as a base, fewer have branched from "Open" BSD, as can be seen in this comparison on Wikipedia. Recently, two OpenBSD enthusiasts have attempted to address this, only to be flamed mercilessly for their trouble.

The DistroWatch feature previously mentioned concerns GNOBSD, a project created by Stefan Rinkes, whose goal plainly was to make this highly stable and secure operating system, with a lineage that long predates Linux, accessible to a wider audience (who don't use Mac OS X - based (indirectly) on both FreeBSD and NetBSD).

For his troubles, Mr. Rinkes was the subject of repeated, extreme, egregiously elitist flaming, this thread being but one example. He was eventually forced to withdraw public access to the GNOBSD disc image, and adding a post stating that he did not "want to be an enemy of the OpenBSD Project."

Reading along the thread on the openbsd-misc mailing list brings one to a post by one "FRLinux", linking to another screaming flamewar summarized here, with the most directly relevant thread starting with a message titled "ComixWall terminated. Another hard worker with the intent of creating an OpenBSD-based operating system that was easy for "ordinary people" to use, promptly incurred the wrath of the leader of the OpenBSD effort, Theo de Raadt, with numerous other "worthies" piling on.

WTF?!?

OK, I get it. The OpenBSD folks don't want anyone playing in their sandbox providing access to the OpenBSD software (itself supposedly completely open source, free software) that might in any way compete with "official" OpenBSD downloads or DVD/CD sales. They especially don't want any possibility of the Great Unwashed Masses™ — i.e., anyone other than self-professed, officially-blessed übergeeks — from playing with "their" toys.

OK, fine. It has been a large part of my business and passion to evangelize open source and otherwise free software as an enabler for the use of open data standards by individuals and businesses. I have supported and/or led the migration, in whole or in part, of several dozen small to midsize businesses from a completely closed, proprietary software stack (usually Microsoft Windows and Microsoft Office for Windows) to a mix of other (open and proprietary) operating systems and applications. OpenBSD has historically been high on my list of candidates for clients' server OS needs; if you can manage a Windows Server cluster with acceptable performance and stability, your life will be easier managing almost anything else — provided that you're not locked in to proprietary Windows-only applications and services. In these days of "cloud" computing and "software as a service", the arguments against Ye Olde Standarde are becoming much more compelling.

I just won't be making those arguments using OpenBSD anymore. And to my two clients who've expressed interest in BSD on the desktop for their "power" developers over the next couple of years, I'll be pitching other solutions to them...and explaining precisely why.

Because, out here in the real world, people don't like dealing with self-indulgent assholes. That's a lesson that took this recovering geek (in the vein of "recovering alcoholic") far too many years to learn. And people especially don't like trusting their business or their personal data to such people if they have any choice at all.

And, last I checked, that was the whole point of open standards, open source, and free software: giving people choices that allow them to exercise whatever degree of control they wish over their computing activities. I'd really hate to think that I've spent roughly half my adult life chasing a myth. Too many people have put too much hard work into far too many projects to let things like this get in our way.

Fixing A Tool That's Supposed To Help Me Fix My Code; or, Shaving a Small Yak

Well, that's a couple of hours I'd really like to have back.

One of the oldest, simplest, most generally effective debugging tools is some form of logging. Writing output to a file, to a database, or to the console gives the developer a window into the workings of code, as it's being executed.

The long-time "gold standard" for such tools has been the Apache Foundation's log4j package, which supports Java. As with many tools (e.g., PHPUnit), a good Java tool was ported to, or reimplemented in, PHP. log4php is uses the same configuration files and (as much as possible) the same APIs as log4j. However, as PHP and Java are (significantly) different languages, liberties had to be taken in various places. Add to this the fact that PHP has been undergoing significant change in the last couple of years (moving from version 5.2 to the significantly different 5.3 as we wait for the overdue, even more different, 6.0), and a famous warning comes to mind when attempting to use the tool.

Here be dragons.

The devil, the saying goes, is in the details, and several details of log4php are broken in various ways. Of course, not all the breakage gives immediately useful information on how to repair it.

Take, as an example, the helper class method LoggerPatternConverter::spacePad(), reproduced here:

    /**
     * Fast space padding method.
     *
     * @param string    $sbuf      string buffer
     * @param integer   $length    pad length
     *
     * @todo reimplement using PHP string functions
     */
    public function spacePad($sbuf, $length) {
        while($length >= 32) {
          $sbuf .= $GLOBALS['log4php.LoggerPatternConverter.spaces'][5];
          $length -= 32;
        }
        
        for($i = 4; $i >= 0; $i--) {    
            if(($length & (1<<$i)) != 0) {
                $sbuf .= $GLOBALS['log4php.LoggerPatternConverter.spaces'][$i];
            }
        }

        // $sbuf = str_pad($sbuf, $length);
    }

Several serious issues are obvious here, the most egregious of which is acknowledged in the @todo note: "reimplement using PHP string functions." The $GLOBALS item being referenced is initialized at the top of the source file:

$GLOBALS['log4php.LoggerPatternConverter.spaces'] = array(
    " ", // 1 space
    "  ", // 2 spaces
    "    ", // 4 spaces
    "        ", // 8 spaces
    "                ", // 16 spaces
    "                                " ); // 32 spaces

If you feel yourself wanting to vomit upon and/or punch out some spectacularly clueless coder, you have my sympathies.

The crux of the problem is that the function contains seriously invalid code, at least as of PHP 5.3. Worse, the error messages that are given when the bug is exercised are extremely opaque, and a Google search produces absolutely no useful information.

The key, as usual, is in the stack trace emitted after PHP hits a fatal error.

To make a long story short(er), the global can be completely eliminated (it's no longer legal anyway), and the code can be refactored so:

    public function spacePad(&$sbuf, $length) {
        $sbuf .= str_repeat( " ", $length );
    }

Of course, this makes the entire method redundant; the built-in str_repeat() function should be called wherever the method is called.

I'll make that change and submit a patch upstream tomorrow... errr, later today; it's coming up on 1 AM here now.

But at least now I can use log4php to help me trace through the code I was trying to fix when this whole thing blew up in my face.

At least it's open source under a free license.

Blast from the Past

Another in a continuing series...

Microcomputer(as PCs were called before the IBM PC) veterans of a certain vintage well remember that most counterintuitively productive of productivity tools, WordStar 3.3 (and earlier). The hegemon of its day, WordStar used what at first (and usually fifth) inspection appeared to be whimsical, arbitrary key combinations for commands. Ctrl K-H for Help was invariably what new users first memorised. All through the 1980s and well beyond, any word-processing software that came onto the market had some degree of WordStar-compatible commands, either as their main command set or as a bolt-on to wean folks onto the "new" way of doing things. This was even true for the first several releases of WordPerfect and of Microsoft Word. (Word today has several available add-ons to add WordStar command compatibility.)

Why was this so popular? As noted in the Wikipedia article:

...the "diamond" of Ctrl-S/E/D/X moved the cursors one character or line to the left, up, right, or down. Ctrl-A/F (to the outside of the "diamond") moved the cursor a full word left/right, and Ctrl-R/C (just "past" the Ctrl keys for up and down) scrolled a full page up/down. Prefacing these keystrokes with Ctrl-Q generally expanded their action, moving the cursor to the end/beginning of the line, end/beginning of the document, etc. Ctrl-H would backspace and delete. Commands to enable bold or italics, printing, blocking text to copy or delete, saving or retrieving files from disk, etc. were typically a short sequence of keystrokes, such as Ctrl-P-B for bold, or Ctrl-K-S to save a file. Formatting codes would appear on screen, such ^B for bold, ^Y for italics, and ^S for underscoring.

Although many of these keystroke sequences were far from self-evident, they tended to lend themselves to mnemonic devices (e.g., Ctrl-Print-Bold, Ctrl-blocK-Save), and regular users quickly learned them through muscle memory, enabling them to rapidly navigate documents by touch, rather than memorizing "Ctrl-S = cursor left."

Why is this relevant (or even interesting) today? Besides the lessons to be learnt about interface design, it's interesting to note how many editors out there still pay homage to WordStar. I stumbled across joe again this morning; it's available on essentially all Linux and BSD distributions, with versions built for other systems as well (e.g., Mac OS X and Cygwin/Windows), and source freely available if your platform isn't yet supported or you just like to tinker around on one that is.

What makes this fairly scary for us old-timers is just how quickly the old finger habits come back. If you had more than a year's experience beating your head against the original WordStar, I dare you to work with joe or its ilk for more than a few minutes before "how do I do...?" completely falls away from your thoughts and you're just typing as fast as you can think.

For that's the real beauty of this type of "primitive", what-you-see-isn't-what-you-get interface: you're not distracted by the ephemera of making your work appear "just so", and can actually focus on the work of writing. And that, in our click-and-drool modern interfaces, is what we've lost -- and no amount of clever code wizardry on the part of the interface designers can bring us back to that. Why? Because of basic human nature - if we see a button, at some point we'll want to push that button - "just to make things look better." And, all of a sudden, we notice that the entire morning has flown past while we were focusing on the first three paragraphs of a major report that's due this afternoon. Oops.

There's a reason why almost every tool aimed at professional writers -- people who make their living at x cents per word -- have "stripped down", minimalist interfaces, at least as an option. It's the same reason that far too many truly "old-school" writers give for writing on paper and then typing (or having someone type) their words into a computer: the fewer distractions you have, while still being able to do what you're trying to do, the more productive you'll be at it.

That concept extends far, far beyond the writing of prose -- and has too often been lost or forgotten in those other areas as well. Pity.

When Standing on the Shoulders of Giants, Don't Trip

I've been doing a lot of software installation lately, on Mac OS X and various BSDs and Linuxes. Doing so reminds me of one of the major banes of my life when developing for Windows. If you're a Windows usee, you're acutely familiar with the concept.

DLL Hell.

On Windows, as you install a dozen apps, each of them will try to install their own versions of many system-wide (or vendor-wide) libraries. The poster child for these is msvcrt.dll, the Microsoft C Runtime Library, upon which virtually everything in or running under Windows depends in some fashion. Many unhappy man-millenia have been spent by admins and PC owners the world over trying to resolve compatibility issues, usually introduced when a newly-installed app overwrites a more recent version with the earlier version that app shipped with. Other things — including other libraries the app may depend on or even Windows itself — may break since features and bug fixes they rely on (present in later versions of the library) are suddenly gone.

Why think about this, when I'm as likely to write (or even use) any Windows software in the next month as I am to win the local lottery without buying a ticket?

Because, one of the things I added to my new MacBook Pro was the MacPorts software, which is a Mac analogue to the BSD ports collection. The Linux equivalent to this is (more or less) your package manager, whether it be aptitude or yum or portage or whatever. Windows has no equivalent concept; every app you install is its own universe, or thinks it is (with the "benefits" noted earlier), and no central one-stop "update everything" capability.

Modern operating systems may not have DLL hell, but they do tend to have numerous different versions of different libraries and support frameworks installed. Since those can be accessed by apps as either "give me the most recent version" or "give me version x.y.z", no equivalent to DLL hell takes place. But it does tend to eat up disk space. And it makes it harder to mentally keep track of what's installed; installing the MacPorts edition of Bluefish, an HTML editor, installs some eighty other "ports" — direct or indirect dependencies of Bluefish (which itself is primarily a Linux app). Some of these dependencies are last-month recent updates; a few are several years old. But MacPorts determines which packages are needed by Bluefish (and its dependencies, and...) and installs the latest version (by default) of those dependencies, or the specific version asked for. Thus, several files with the same basic name, but different version numbers encoded into the filename, can coexist peacefully in virtually any modern OS.

Recent Microsoft system software, particularly the "managed" software using the .NET Framework, avoids most of the cavalier instances of DLL hell, but introduces other quirks and brittleness in compensation.

But I'm still a bit unnerved by a software package — any package mdash; that has 80+ designated dependencies. I'm certainly thankful that I don't have to manage that dependency matrix myself.

If modern software development is "standing on the shoulders of giants", to appropriate a Newton paraphrase, then careless dependency introduction would be the functional equivalent of drunken tap-dancing: almost impossible not to trip.