Bootstrap and alternates are GREAT! Except when they lead to...

...divitis like this (in a Slim template):

Yes, that's eleven levels of indentation for the link within the .list-group-item list item. Can *you* glance at that and grok it in reasonable fullness without spending ten minutes rebuilding the DOM in your head? I can't, and I wrote the sucker yesterday.

What's the problem? Nesting styles is the problem. Using line 23 as an example, we have, in reverse order:

• an anchor link;
  • within a list-item tag;
    • within a loop (that arguably doesn't count, but does kick the indentation in a level);
      • within an unordered list;
        • within a .row div;
          • within a .col-md-10 div;
            • within another .row div;
              • within a .col-md-12 div
                • within an outermost, containing .row div.

Got all that? At least we're using something like Slim and not, $DEITY forfend I'd need to use something that looked more like HTML soup, like ERB.

Why bother? Because, as far as I can tell, that's what it takes to work with nested columns in Bootstrap. Everything's within an outermost row, in which you define columns; within any given column, you can use another row to break the containing column into sub-columns. Remember when, in ancientest days of the Web (circa 2001 at latest), we started telling people not to use tables for layout because it led to inscrutably crufty hairballs? They're baaaaaack!

There's got to be a better way. Isn't there? Isn't there?!?!?

In other news...

By this time next week, after over ten years of blogging on Blogger.com, I expect to have the process of moving my blogs (and my Tumblr, and many of my G+ posts well underway, if not yet completed. Why? I'd like to

1. Reduce my Google footprint or, more precisely, the footprint GOOG leaves on me;
2. Be able to point people to one place where they can see all my non-code online writing;
3. Use a markup language that's less actively hostile to writing than HTML is; with Markdown and Jekyll on Github Pages, I get that;
4. Have more control over that "one place" than any of the Google-based "places" presently give me; and
5. Be able to embed Gists or other bits of code in my writing and have them look better than the embedded Gist at the top of this post.

It's not for everybody. But if you've been using Blogger in HTML-editing (rather than WYSIWYG) mode, as I have since the beginning, you can certainly learn (and benefit from and enjoy) Markdown and Jekyll.

Why is "doing things right" a "thing"? Whither professionalism?

Over on the Reddit r/programming subreddit, /u/jfalvarez posted a link to Tom Stuart's great presentation at the Scottish Ruby Conference 2014, entitled "The DHH Problem".

If you're involved with Ruby or Web development in any way, it's well worth three and a half minutes of your time.

On the Reddit post where Mr Alvarez posted the link to the video, another Redditor, dventimi, asked

Why is this a thing?

I disagree with the later poster who snarked "People like gossip." It's much, much more important than that.

What follows is the content of my reply to "why is this a thing?" I'd love to have a discussion with you about it, either here or on the original post.

---

Because people are finally waking up to the fact that the coding style and philosophy espoused by DHH are disastrous in non-toy applications maintained over any significant period of time.

That this should have been obvious to all, but wasn't discussed as such, once Twitter, after having conflated Rails with Ruby as a whole, left Ruby because they couldn't make Rails do what they needed to no matter how many times DHH came onsite and insulted their intelligence, speaks eloquently to how broken things are.

Most master software developers I've met and worked with have been mediocre-at-best promoters, including self-promoters. DHH has the opposite problem: his initial extraction of Rails from Basecamp's codebase "worked", for apps very much like Basecamp at the time. His fixed, what some have called "trolling", attitudes and the personality they describe, prevent him from seeing things from any other perspective or from growing his skills beyond a certain point. The echo chamber of acolytes and zealots his sublime self-promotional talents have allowed him to accrete around himself "ensure" that he never has to.

That's a problem for the rest of us, who have deadlines, work to finish within those deadlines, and paycheques we'd like to be able to cash that we won't get unless we meet those deadlines with that work. Especially for those of us who wish to improve our craft and, in whatever way we can, nudge ourselves and others towards helping the craft of software development more closely approximate a true engineering discipline.

Paying excessive heed to the opinions-masquerading-as-principles brayed by DHH, and twisting ourselves to fit inside The Rails Way, actively impede all of the above tasks. Tom Stuart was absolutely right to thank DHH for producing Rails and promoting it sufficiently that the rest of us get to write Ruby for a living…just before tearing DHH a new one, in the polite, discreet way only a well-educated Brit can, for getting so much of Rails and how software should be written so disastrously wrong.

Thank you, Tom Stuart. An entire industry may well be in your debt.

Going Forward; Avoiding the Cliff While Looking for Cliff's Notes

Over on the Ruby sub-Reddit, I got into a discussion (mostly with Jelly_Jim, the OP, and with realnti) about the conflation in far too many minds of Ruby, the programming language, with Ruby on Rails, the famously-"opinionated" Web application framework. While pontificating on Jelly_Jim's original question, I described my frustration-bordering-on-antipathy with the traditional (known as "The Rails Way") application structure; listed a few presentations, books and blog posts that have influenced me greatly, and described my own take on the Kübler-Ross model for Ruby-based (and at least initially, largely Rails-based) Web development.

That discussion, over the course of roughly a day, helped me figure out how our app is going to survive the expected Slashdot effect, transitioning from a pretty traditional, megalithic Rails app to an app that uses Rails components in a sharply different, more recognisably "object-oriented", service-based architecture.

Leaving aside my more prosaic experiential difficulties with Rails (I really loathe ActiveRecord as commonly used), and taking into account the "Uncle" Bob Martin keynote, the various blog posts I referenced in the original Reddit post, and a couple of books I mentioned in a later reply, I think I've possibly hit upon a way to get where I want us to go.

That's the easy part, of course; it only took me never-mind-how-many months. The "interesting" part is going to be getting there — understaffed, on a shoestring budget for the moment even by startup standards (another problem I'm working), and working a schedule for two and a half years that would land a healthy twenty-year-old in hospital by now. Comments/suggestions are, as always, greatly appreciated.

---

Basic Architectural Principles

• Loosely-coupled, granular architecture FTW. That means, inter alia,
  • Nearly the entire app to be packaged as Gems used by an otherwise minimal application;
  • Rails mountable engines to package assets (particularly CoffeeScript/JavaScript) and major components;
  • Plan and design for updating and provisioning components which communicate across natural architectural seams.
• Hexagonal, "ports-and-adapters", or "clean" conceptual architecture; all dependencies point inward. Your domain entities, at the centre of the system, will change less than details like the delivery mechanism (Web UI, phone app, etc) or the database (which the domain shouldn't even be able to prove exists as such).
• By adopting a heavily job-story-oriented view of the workflows; with the use of online tools like Trello, WebSequenceDiagrams.com; and with supporting code tools like ActiveInteraction and so on, we should be a lot more productive and a lot more Agile than we have been during our forty (calendar time: two) years lost in the wilderness.
• And oh, yeah, just to keep things interesting: we've got to keep regularly demonstrating forward user-visible progress, for all the obvious reasons and a couple I really can't go into here.

---

How We Get There, v. 28.0

First, since the major focus of late has been on some unique (and not-so-unique) CoffeeScript code, start there. Separate the existing Script code out into four engine-based Gems, following/adapting Derek Prior's example. These levels are, from conceptual bottom to top:

1. A "gateway" CoffeeScript class we've written as a poor man's RequireJS, that gives us just enough pseudo-dynamic referencing of data/code without changing anything the Rails asset pipeline assumes;
2. Our directly DOM-aware Script code, as one or possibly two Gems (one is a major crown jewel; we might decide to split off utility/support code into a separate Gem, so as not to touch one when maintaining the other);
3. The code proxying our internally-used message queue (decoupling "FTW", remember?) and the Ajax code to communicate with the Rails back end; and
4. The code implementing various domain/service objects sitting atop the Script stack; one Gem per service.

Next, adapt the relatively-straightforward (as yet) Ruby/Rails code to honour the architectural principles listed earlier. This is where tools like ActiveInteraction pull their weight (and solve several existing pain points in the process).

Just to make sure we really do understand where we're going with this, take our first user story-specific code (including interactor) and package it up as a Gem that should then Just Work as in the previous step. Start thinking about whether we really want traditionally-packaged Gems or unbuilt dependencies that remain part of the single app repository.

Proceed to reimplement and repackage the (again, relatively few) job stories already existing as was done in the preceding step.

Start knocking down the remaining Trello cards as we complete our app.

We've been saying since the Very Beginning that anything that we were building that wasn't inextricably tied to proprietary, commercially-valuable code (or that gives too much of a hint into any unique details it may have) "should be" openly released, but hadn't yet figured out a feasible, cleanly reusable way to do that. If we've done our jobs properly up to this point, we have that way now.

Push the "deploy and launch" Big Green Button. We've bloody well earned it by now.

---

Any Suggestions or Comments?

Clean Code. "I do not think that word means what you think it means."

From my copy of Clean Code: A Handbook of Agile Software Craftsmanship (the paper copy of which (barely) predates my work in Rails. The highlighted paragraph is what I'd highlighted in the paper copy:

Active Record

Active Records are special forms of DTOs. They are data structures with public (or bean- accessed) variables; but they typically have navigational methods like save and find. Typically these Active Records are direct translations from database tables, or other data sources.

Unfortunately we often find that developers try to treat these data structures as though they were objects by putting business rule methods in them. This is awkward because it creates a hybrid between a data structure and an object.

The solution, of course, is to treat the Active Record as a data structure and to create separate objects that contain the business rules and that hide their internal data (which are probably just instances of the Active Record).

I've been increasingly uncomfortable with the way Rails encourages people to conflate the domain model objects with persistence in Rails' core component, ActiveRecord. Running across this bit again makes me think of Inigo Montoya from The Princess Bride: You keep using that word. I do not think it means what you think it means. Judging from my own projects and those I've participated in and observed on Github, the fact that ActiveRecord does not properly implement or encourage the Active Record pattern is a primary root cause of many bugs and much confusion and heartburn.

Happy New Year.

Tap-Dancing Among the Land Mines

…is best done very carefully, and with the knowledge that not everything works as you'd like it do. That's a good metaphor for my week this week.

Working on our new platform and its (eventual) variety of products and supporting tools, mostly in Ruby using Rails, some of the absolutely key elements are primarily front-end code, written as CoffeeScript running in the browser. Typically enough, we make heavy use of jQuery, various plugins, and other libraries. When understood and used correctly and they work as expected, they're great for reducing the effort needed to accomplish new and useful things.

The moderately-sized application I'm presently working on uses over 125 third-party components, many of which are dependencies of components we directly/intentionally use. Of those, some three dozen components have yet to reach a "1.0" release level. That's a lot of trust we put into people we've never met and teams completely outside our organisation. The fact that this is typical to the point of being mundane among Ruby shops is one of the phenomena most commonly cited by new Ruby devs and managers as "mind-blowing".

When everything works properly, it's awesome in a very different way than anything I've done in my nearly 35-year career. When anything fails to work properly, it's equally spectacular; just in a very different way.

Consider today. I've been working on a Script component (which I expect to publish in due course as a jQuery plugin itself). Being a proper behaviour-driven development project, I've been writing specs using Mocha and running them with Teaspoon. Teaspoon lets you run your spec suite either in a browser or from the command line. The browser-based interface is easy and convenient (almost fun) to work with during development; with a mouse click, you can focus on just the spec you're working with at the moment, or step back and run a group of related specs or your entire suite. After finally finding and killing the last of never-mind-how-many typos and silly errors (CoffeeScript, alas, has not yet implemented proper DWIM support), I had the component working. It was clean. It was more than fast enough for my purposes. And, most importantly, it worked properly, against every condition I threw at it.

That is, when I was running Teaspoon from the browser. The command-line invocation, needed for continuous integration among other things, failed in a very odd fashion. See, for example, this bit of screen output:

[jeffdickey@Jeffs-Prolog-iMac select_demo (wip)]$ teaspoon -q | tapout
Started w/ Seed: 0
..................F

1. FAIL markup from selectedMarkup()

    Unknown
    
    TypeError: 'undefined' is not a function (evaluating 'currentNode.contains(last)') in http://127.0.0.1:58247/assets/selection_wrapper.js?body=1 (line 111)
    
    window.meldd.SelectionWrapper for rich single-paragraph content, returns the correct markup from selectedMarkup().#:0


Finished in 0.092s (206.522 test/s, 0.004842s avg.)

19 tests: 18 pass, 1 fail, 10 exit, 0 todo, 0 omit

Now, bear in mind that when Teaspoon was running inside the browser, no such error message was displayed; all specs passed. The code being complained about looked something like:

  getMarkupUntil: (currentNode, contentString, last, offset) ->
    while currentNode?
      if currentNode.contains(last) # <-- THIS IS THE LINE BEING COMPLAINED ABOUT
        if currentNode == last
          # do something...
        else  # currentNode contains last but is not last
          # do something else...
      else # currentNode does not contain last but we're still looking
          # do something different...
    # return our results

Now, currentNode.contains is making use of a bog-standard HTML DOM function that's been in every major browser for years. Even Internet Exploder 5, from the Dark Ages of 1999, managed to support it! What could possibly go wrong?!?

Tap dancing on the shoulders of not-always-giants, it turns out.

Teaspoon, by default, uses PhantomJS as a "headless Webkit browser workalike with JavaScript support". If you're going to automate testing of your Web-based application (or any of several other related activities), this is the tool you want to use. Except, o course, when it doesn't work correctly. I changed the getMarkupUntil method I showed earlier to test my hunch:

  getMarkupUntil: (currentNode, contentString, last, offset) ->
    while currentNode?
      return null unless currentNode.contains?
      if currentNode.contains(last) # <-- THIS IS THE LINE BEING COMPLAINED ABOUT
        if currentNode == last
          # do something...
        else  # currentNode contains last but is not last
          # do something else...
      else # currentNode does not contain last but we're still looking
          # do something different...
    # return our results

The new line returns a null value from the function if and only if currentNode has no property (data or method) named contains. I reasoned that this would (fairly obviously) cause the calling code to fail, and the difference in the failure message (from what was shown previously) would confirm to me that this was indeed the problem. Again, running the specs from inside the browser presented no problems. Running from the command line, however, yielded

[jeffdickey@Jeffs-Prolog-iMac select_demo (wip)]$ teaspoon -q | tapout
Started w/ Seed: 0
..................F

1. FAIL markup from selectedMarkup()

    Unknown
    
    TypeError: Type error in http://127.0.0.1:58271/assets/selection_wrapper.js?body=1 (line 174)
    
    window.meldd.SelectionWrapper for rich single-paragraph content, returns the correct markup from selectedMarkup().#:0


Finished in 0.090s (211.111 test/s, 0.004737s avg.)

19 tests: 18 pass, 1 fail, 10 exit, 0 todo, 0 omit

As expected, the null value caused other code to die.

I then browsed the Teaspoon documentation, which pointed out that although PhantomJS was used by default, Selenium could be used instead, by installing the selenium-webdriver Gem and updating the project configuration accordingly. After re-bundling the app, I could run my spec suite from the command line. Hurrah!

Well, golf-clap applause at least. We'd had previous problems with Selenium testing the Script code as run from within a Web app. We'll burn that bridge again when it falls on us. Again.

"Easy Credit" IS "Easy Debt". It sucks for finance. It sucks just as much for your code.

One of the great things about the Rails ecosystem is that there are lots of different tools that can save you time and pain during development. Some of them work by embracing and extending the Rails standard bits, like replacing (nearly) bog-standard ApplicationController subclasses with subclasses of InheritedResources::Base. Some of them instead swap out pieces of the original Rails stack, like replacing ActionView with Erector or something similar. I wouldn't be at all surprised if my current app that mixes the two practices (in different areas) isn't at all unusual.

But whichever route or routes you choose, you need to constantly remain aware of that choice, especially when considering new tools you'd like to incorporate into your app. And you may also find that the tools you're using, while superb for their intended purpose, complicate other choices you make in the course of developing your project. A tool which was the greatest painkiller since morphine when you first picked it up, now seems to be "getting in the way" as you get farther along in development.

As practitioners of agile development, this shouldn't scare or even surprise us (though our non-technical stakeholders may initially be less sanguine). Good agile architecture says that everything in your app other than the raw business rules is an implementation detail that should be testable and fully replaceable in isolation.

Novice to journeyman Rails developers may be forgiven at this point for saying "but that's the way Rails is; it all works together, even though you can swap pieces out as you like". A majority of Rails developers have at least experimented with swapping out Test::Unit for RSpec, or erb for Haml or Slim. More adventurous still are the folks who swap out the ORM, replacing ActiveRecord with, say, DataMapper or the aforementioned Erector for ActionView. Why are they trying such things? Because they're asking (quoting Wynne again), "what does a modular Rails architecture look like?" Or, "we know we've hit the wall with our current (connected) architecture; everything we touch introduces new tech debt if not outright breakage, but we don't know how to do anything else and stay true to The Rails Way."

That "wall" you've hit has nothing to do with the number of lines of code you've written nor the number of unique visitors your production app serves per day nor the revenue your app is bringing in the door. You've realised that your ability to keep bringing in that revenue, let alone increasing it, is dependent on you somehow reconciling your understanding of "agile development" and "software engineering-style practices" with "the Rails Way". One of these things is not like the others. If you haven't yet, watch both of those videos. Particularly in Matt Wynne's talk from about 5:30 where he talks about the crossover point between a "connected" architecture being cheaper/faster/better giving way to a "modular" one… that rings a lot of bells for me. Likewise when Uncle Bob rails against Rails apps screaming "I'm a Rails app, never mind what I do". I've done several projects, never mind how many, where I've gotten to that point (which Wynne describes as "hang on; I don't want to work on that application anymore. Something's seriously wrong with it") and my reaction to the realisation that "something is seriously wrong" has been to stop, look at everything I'm doing and evaluating how well I'm following convention, "best practice" and so on, come to the conclusion that I'm doing everything exactly the way I've been taught it should be done, and deciding that that proves the fault is in my understanding of the problem, so I start over, from scratch, trying to figure out where I fell off the cliff. With one application, that involved at least five restarts over the course of a year and a half… and I must have always "fallen off the cliff" at the same spot, because I never saw it coming before I realised I was at the bottom of that cliff once again. "Uncle Bob" Martin's keynote (from Ruby Midwest 2011) does a great job of explaining why an experienced OO software developer like me has always felt uncomfortable with Rails' interpretation of the "Model-View-Controller" architecture. By writing your app centred on what the app is for rather than how the app presents itself on the Web, you make your life a lot easier and more enjoyable. Looking back at my commit history over the last year and a half, I see that well over 2/3 of the times I was "stuck" for more than a couple of hours had more to do with "how do I adapt what I'm trying to do to how Rails wants me to do things" than "how do I implement (this feature)". That should, and has been, setting off all kinds of alarm bells; I'd just drank enough of the Flavor Aid to think that pain level was "normal" for Rails. It shouldn't be.

Going forward, my strategy for Rails development is going to work something like this:

1. Model the domain objects (Martin's "interactors", "entities", "boundaries" and "entity gateways", after Jacobsen "control objects") as POROs (Plain Old Ruby Objects);
2. Wrap those up in Gems whose use can be demonstrated by specs, supplemented by a simple (Web or JSON or command-line; doesn't matter) example or set of examples.
3. Plug those into a Web app using the current framework of choice (Rails, Padrino, JoeBobsAwesomeFramework, or whatever), whose sole purpose is to implement the "client" side of the boundary layer and present what comes back over the fence.

Now we're back to something more closely approximating classical agile development that focuses on delivering business value, not playing mind games with a tool that insists on making most of the important decisions for you, and makes them in ways that make it harder for your app to be scalable, performant, or fun to work on. Or even for your app to be finished.

Thoughts?

---

P.S. Oh, and as far as InheritedResources::Base, that truly useful tool for Rails app development I mentioned before? We'll continue to use it as a default, but if we choose to use tools that "get in the way" (like wisper) by wanting all your methods explicitly defined, we'll switch back to ApplicationController, on the theory that making the puzzle one piece smaller without affecting how the other pieces fit together is a Good Thing™. But that's another decision we want to defer as long as possible; "good architecture" is all about deferring as many decisions as possible as long as possible, while still getting business value done.

Tools Save You Time, Except When They Don't

Two of the greatest boons yet to hit Railskind, IMAO, have been Platformatec's Devise and inherited_resources. Surprisingly, given how closely they're related, they don't always play nice together. If you ever get a failing feature spec that's just modifying non-password data on your user-profile page, it can really confuse you until you do some Googling around. (That's how you got here, yes?)

You'll need to update your User controller so that it doesn't send empty password fields to Devise for the update action. This won't affect creating Users; you want Devise to reject empty passwords there. Here's what I have for my users_controller.rb (also available as a Gist):

---

# Actions for Users. Not necessarily a finer distinction one way or another yet.
class UsersController < InheritedResources::Base
  # FIXME respond_to :json # should spec JSON actions, too
  respond_to :html
  load_and_authorize_resource

  def update
    # Devise has the convention that, for updates where the password is NOT to
    # be modified, the password and password-confirmation fields MUST NOT be
    # submitted. inherited_resources, of course, just sees two text fields and
    # wraps them up along with the rest of the form. Oops.
    remove_password_parameters if password_fields_are_empty?
    super
  end

  private

  def password_fields_are_empty?
    params['user']['password'].empty? and
      params['user']['password_confirmation'].empty?
  end

  def remove_password_parameters
    params['user'].delete 'password'
    params['user'].delete 'password_confirmation'
  end

end

This is still a heck of a lot cleaner/shorter than what I'd have written for a controller subclassed directly from ApplicationController. But is it universally better, or better for your specific needs? Thoughts on that in the next post.

Tap-Dancing on the Shoulders of Giants, Cont'd

(after Bernard of Chartres)

Some days you wake up feeling old. Some things that happen during the day make you feel much, much older.

In times technologically ancient but historically recent, people wrote computer programs by writing code for virtually every instruction to be executed by that program. Later, as capabilities grew (and we learned better), we used underlying systems and libraries to reduce the amount of new code we had to write for increasingly complex programs. Later still, networks and the Internet augmented the resources we could use in our creations. As anyone who's written desktop applications for a recent major operating system can attest, the complexity of these applications can meet and often exceed what one person can keep in his or her head. Consequently, a high priority became "doing more with less", while still delivering an efficient, usable, correct-enough system.

Recent development, especially Web development, has taken that priority and raised it to an imperative. To be a good developer, you don't just have to be able to code well, you must be able to efficiently find, evaluate, learn, use, and collaborate on further development of these components. Write code once, that's fine. Write code twice, write it again in a more easily reusable form, and rewrite the first two to use the new one. What used to take a team months and thousands of lines of code can now be accomplished by a single developer in days or weeks, with dozens to scores of lines of more readily understandable code — that uses ever-more-complete and -complex underlying libraries.

I was reminded of this yet again by a recent blog post by David Guttman (with accompanying Heroku demo and source code). There's a moderate amount of framework-type boilerplate around it, but the source file I linked does all the heavy lifting.

It's less than fifty source lines of CoffeeScript. Fifty. Lines.

Not since the heyday of APL has a fifty-line program done so much. However, fifty lines of CoffeeScript is far more readily understandable by mere mortals than fifty characters of APL. (Browse this to get an idea of what I'm talking about.)

When I was a wee lad in uni, I wrote code, in ALGOL 68 on a Burroughs B6700, to drive a Tektronix graphics display to produce graphics similar to Guttman's. My program, when complete, nearly filled two 1600-character-per-inch 9-track tapes. (Yes, those erratically whirring reels you see in old newsreels and oddball posts on YouTube.) Each tape stored about 40 megabytes. (Less space than the music file I'm listening to at the moment; about 1/8 of a CD altogether.)

That project took the better part of two school quarters, about four months.

You can still spend months researching if you have to learn some Brave (totally) New Domain and a Strange New Language at the same time, but to actually write a fifty-line program? Days? Hours?

And that brings me to the point of the title; too many of us, not just younger developers, now take these new tools for granted. The problem in doing so, in not understanding how things were done before, blinds us to important details about the now-modern tools we use today. By not having a clear understanding of the problems solved by our tools, by not understanding the tradeoffs and alternatives involved in developing the tools we use every day, it's practically impossible for us to make the most effective use of those tools. And that can have some serious, project-endangering consequences, obvious in hindsight to those who have hit them at least once or twice..

We all stand on the shoulders of giants. Don't get too cocky with your tap-dancing, lest you follow in the shuddering footfalls of one Wile E. Coyote.

Take Only Pictures; Leave Only Footprints

If you've ever visited a National Park or many state parks in the United States, or been involved with a nature-oriented community group, you've likely heard that saying many times. For those who haven't, the meaning should be obvious: leave the shared place that you're moving through in at least as good condition as you found it, so that the next people to travel that way can enjoy it as much as you did.

That applies to environments other than the great outdoors, of course. The phrase popped into my mind earlier today as I was poking at a jQuery plugin to add a context menu to a Web app I'm developing.

It's great that there's so many free software tools out there; I've been using them for more than 25 years, and I've developed more than a few. Sometimes, however, one is reminded of the wisdom of Oliver Wendell Holmes when he said, "Learn from the mistakes of others… you can't live long enough to make them all yourself." But in the grand software tradition, we often give it our best effort.

This is a reasonably well-known plugin, one of several that offer to solve a common problem in more-or-less similar fashion. And if you use it precisely in the way that the author expected, it does the job. But even then, if you're using it in a Web app or site that customers are paying money for, you might hope that they never hit the "View Source" menu item in their browser. And, to be fair, this is not intended as a slam against Matt Kruse or his code; most of the other plugins I've looked at in the last couple of days have at least as many "quirks" and assumptions.

"It's just a context menu," I hear you say. "What could possibly go wrong?…go wrong?…go wrong?…(sound of tape snapping)" (with apologies to Westworld, a movie with many lessons on the art and craft of software development.)

If you're adding the menu to your page when it's first loaded, and you want a static menu, with the options the same for all uses of the context menu, fine. If you want a more dynamic menu, and don't mind attaching functions to the menu that know enough of the detailed state of your page/app that they can generate the properly-adjusted menu items each time the menu is displayed, that works, too. But the main assumptions are that the plugin's main method will be called once and only once on a page, and that you really don't care about the markup being added to the end of your page. The first assumption, particularly in a rich front end, is likely to be naïve; the second is, bluntly, an assault on your professionalism.

What's so bad? Here's the plugin's output for a very simple menu, reformatted for clarity:

<table cellspacing="0" cellpadding="0" style="display: none;">
  <tbody>
    <tr>
      <td>
        <div class="context-menu context-menu-theme-vista">
          <div class="context-menu-item " title="">
            <div class="context-menu-item-inner" style="">One</div>
          </div>
          <div class="context-menu-item " title="">
            <div class="context-menu-item-inner" style="">Two</div>
          </div>
          <div class="context-menu-item " title="">
            <div class="context-menu-item-inner" style="">Three</div>
          </div>
        </div>
      </td>
    </tr>
  </tbody>
</table>

<div class="context-menu-shadow" style="display: none; position: absolute; z-index: 9998; opacity: 0.5; background-color: black;"></div>

If you read HTML and CSS, you're probably nodding your head, thinking "that looks simple enough; hey, hang on a minute…" That "hang on" is where you start seeing any of several issues:

1. We have a table, containing a (rarely-used) tbody, containing a single tr, containing a single td, containing a series of nested divs which contain the menu items. If he'd implemented the entire menu as a table, he might have been able to claim that he was trying to support 1997-era browsers. (Would jQuery 1.4 work in Netscape Navigator 4?) Failing that, it's a mess. Nesting divs (or more semantically appropriate elements) has been a Solved Problem for a decade or so.

2. Even though this is implicitly intended to be fire-and-forget markup added to the DOM at page-creation time, it's disturbing that there are no identifiers for any of the table elements. If you wanted to, say, remove the table later, you'd have to use code like

   $('.context-menu').parent().parent().parent().parent().remove()

   Ick. And then you'd have to go clean up the shadow div separately.

3. Speaking of the shadow div being separate from the menu-enclosing table, why not wrap both in an (identified) div, so that you (and your clients' code) can treat the menu as a single entity?

4. Doing that would also let you dynamically delete and replace the menu, based on changes in the state of your app, without needing to define menu-item handler functions that violate the Principle of Least Knowledge.

5. Another benefit of that would be if your context menu was generated more than once (possibly because the content needed changing or items needed disabling), you'd never have more than one copy of the menu in the DOM. As it is now, you can have an arbitrarily large number, with only the most recently-added being the "active" one. Ugh. This would be more likely to happen on a dev box running a test suite, but still. Ugh.

6. Particularly in a behaviour-driven or test-driven development (BDD or TDD) environment, being able to test/validate the markup and the item-handler logic separately as well as together is important. Doing so with this plugin (and again, to be fair, most of the others) eliminates the normal-use workflow from consideration.

7. One feature of this particular plugin is that it supports having you define your own HTML for the menu and pass it in. But the example given is too simple to be useful as a menu. Browsing the plugin source code seems to indicate that there is no event-handler support for menus defined in this manner; you'd have to iterate through your context menu items, assigning event handlers for at least the click event. If you're going to do all that work, why bother with this plugin?

8. A table and divs. Enclosed and in parallel. (Careful wiping that green sludge off your monitor; that's your brain that just exploded.)

Menus are "important" enough that HTML 5 has its own set of elements dedicated just to marking up menus. However, sensible, reasonably semantic standard patterns for use with HTML 4 and XHTML 1.x have evolved that address the issues I've mentioned (among many others). The markup for the example menu above could have been written as:

  <div class="context-menu-container" id="someId">
    <ul class="context-menu context-menu-theme-vista">
      <li class="context-menu-item">
        <span class="context-menu-item-inner">One</span>
      </li>
      <li class="context-menu-item">
        <span class="context-menu-item-inner">Two</span>
      </li>
      <li class="context-menu-item">
        <span class="context-menu-item-inner">Three</span>
      </li>
    </ul>
    <div class="context-menu-shadow"></div>
  </div>

One outermost div, with an id so your plugin doesn't get confused if you have multiple menus on a page, but one div so you can work with it as a single unit. An unordered list containing list items corresponding to the menu items, since that's the closest HTML4/XHTML 1 comes to HTML5's menu semantics. The Script that builds the thing can adapt the styles and attributes at the class+id CSS level, eliminating the need for hard-coded monsterpieces such as that style attribute for the table.

If you're going to write a jQuery plugin, live in jQuery. You can dynamically modify styles, attributes, content, even whole sections of how your page is rendered in the browser, without touching the basic markup. You can put the style bits that you intend to remain constant into a CSS file and link those styles to individual DOM fragments via classes and ids. That also keeps your HTML and CSS clean and cacheable.

If we don't write clean code, then having a lightning-fast browser on a petabit-Ethernet connection won't matter; users and reviewers will still complain that "your app is slow". Think before, as, and after you code, and remember:

First, do no harm.

It's Not Just "JavaScript" Anymore

Late to the party again, as usual, but…

I've been reading up on ECMAScript 5, the standardised language historically and popularly known as "JavaScript". Even though I'm doing most of my work in CoffeeScript now, ECMAScript is relevant because that's what CoffeeScript compiles down to. (At least, to a 'safe', 'good bits' subset thereof). So here I've gone and mentioned three different names for "very similar" languages in the same paragraph. When I mean one in particular from here out, I'll name it; otherwise Script refers to all three to the best of my knowledge. Anyway...

One of the things I find myself doing, and reading in just about everybody else's code in most languages, is a prudent sanity-check validation before doing something "dangerous" that profoundly changes the state of the system based on the state of various objects within it. A decidedly non-trivial part of that is usually simple validation of individual property values; is the someFutureDate value actually greater than the value for Now? Should we check that every time we're about to make an important decision based on the value of someFutureDate? What a pain…especially knowing that the one time you forget to do that check often turns out to be the root cause of a critical bug?

ECMAScript 5 has completely rethought how properties on objects are implemented. They can be made read-only, non-enumerable, non-modifiable once set. Perhaps more interestingly, they can now be defined either using a simple value as before, or by defining getters and setters, as is done in several other languages. What this buys you, at the cost of a bit of relocated complexity, is the ability to validate assigned values without the assigning code doing anything other than assigning a value to the property. The domain logic relevant to a specific property of an object can now be coupled more tightly to the property and the details less visible or relevant to outside code, in effect creating a conceptual "mini-class" around a property and its getter/setter logic that is transparent to outside code.

This and several other features of ECMAScript 5 now make it easier to write nice, fine-grained, SOLID code in ECMAScript 5 than was previously possible in any Script dialect. Huge win all around.

Which brings up questions. How could support for property descriptors and the like be added to CoffeeScript? Should they be added? Would support for these underlying ECMAScript features require changes to the CoffeeScript compiler itself, or could it be achieved less intrusively?

'Least Knowledge' May Be More than You Think

I was chatting on IRC the other day and got into a discussion that stuck in the back of my mind until now. This guy was a fresh convert to the ideas of SOLID and to the Law of Demeter (or Principle of Least Knowledge). Now, these are principles I hold at least as strongly as the average experienced developer, and we eventually agreed to disagree over the propriety of code such as a line I just wrote (which in fact prompted this post).

Consider the Ruby code:

    data[:original_content] = article.content.lines.first.rstrip

This says:

1. "take whatever your article is;
2. get whatever its content attribute is or method returns;
3. (assuming that is a String or something that can be treated as one,) get its lines enumeration;
4. get the first entry in that enumeration;
5. (again, assuming it's a String or workalike,) strip any trailing whitespace from that string; and
6. stuff it into the collection data, indexed by the symbol :original_content."

It's not hard to argue that this is a bad code smell, being a long list of assumed return types and so on, except for one thing: in my view, the LoD does not cover sequencing a series of standard library calls.

Look at the code again. The bit that references my code is the fragment

  data[:original_content] = article.content

That fragment is clean; article is an object and content is an attribute of or method on that object. This code assumes that the value of article.content is a standard String. As a standard class, it's virtually guaranteed not to change in backwards-incompatible ways over the useful life of the code. What then? Standard library calls!

1. String#lines returns a standard Enumerator instance;
2. Enumerable#first (via the earlier Enumerator) returns a standard String instance again; and finally
3. String#rstrip cleans off any trailing whitespace.

Nowhere are used any internal APIs that could change. Breaking that one line apart into 5 would produce repetitive, non-semantic, anti-Ruby-best-practices code of the style pandemic on large Java projects. We're in Ruby (and Rails) for a reason: part of that reason is that the language idioms encourage an almost literary expressiveness that makes more explicit the reality that a program is a conversation between the humans developing and maintaining it, that just happens to be executable by a computer. Literate, semantic programming is like behaviour-driven development; once you've wrapped your head around it and see how much better a programmer it makes you, you really don't want to go back to BDUF jousting in Java or PHP.

On further review...

Now, if you argue that the LoD is an aid in reducing coupling between classes, which reduces per-statement complexity, I can see that. But, in the example above, we're back to the fact that there is only one dot before you start jamming together standard library calls. Had I instead written:

content = article.content lines = content.lines first_line = lines.first data[:original_content] = first_line.rstrip

then I'd argue that I've introduced three unnecessary local variables when only one (the second) could foreseeably change. If I later modified the return value from article.content to something other than a Stringlike object, then probably the assignment to lines would have to change. But only that one line — the same level of change that would be required in my current code.

Comments?

Once more into the breach, dear colleagues, once more…

…or How I Learned to Stop Worrying and Love Serving Aboard Kobayashi Maru; a history lesson.

Once again, I've had an interesting couple of months. Between modern Singapore's regular effect on my health, some insane work and the opportunity to get even more insane work if my two best references weren't indefinitely unavailable (but hey, Thailand is usually lovely this time of year…or anytime, actually).

Ahem.

I am rediscovering a love for developing in Ruby after losing touch with it some ten years ago. In the Ruby 1.5 days, things like class variables and the hook system were either new and shiny, or had finally been thrashed into something both usable and beautiful. If programming was what you did to earn a living, then programming in Ruby was something that you thanked $DEITY for every day because, after all, how many people in this world get to use beautiful tools that make you measurably better at your craft while blowing your mind on a regular basis, and get paid for the privilege?

The Fall

Then Ruby on Rails came along, with the clueless-business-media hype and cult of personality built around G-d Himself, or at least the three-initial version of same as anointed by said media, and Things Changed:

New acquaintance: So what do you do for a living?

Me: I write computer software, mostly using Ruby, or when I have to, Delphi or C++.

N.A.: Oh, you're a Ruby on Rails programmer.

So I left Ruby behind. Delphi lasted for a while, until its competitor's lock on the default system combined with some spectacularly bad corporate timing to relegate it to an "oh yeah, I heard of that once" niche. And then came this gawky new kid called PHP.

PHP wasn't exactly "new" when I first got into it; version 3.x had been out for some time, and the in-beta version 4 had class-based object-oriented programming. It was more primitive than an 1898 Duryea automobile to a Porsche driver, but you could see that the basic principles were at least in sight. And so, as the dot-com bubble was starting to really inflate, I jumped into PHP with a vengeance. It wasn't a product of one of the existing tech-industry corporate titans and it wasn't tied to a single operating system or Web server (though it did seem to work best with the then-early Linux OS and Apache server).

The great thing about PHP is that just about anybody can poke at it for a while, and at the end have something that (seems to) work. The pathologically execrable thing about PHP is that the barrier to entry is microscopically low, lower even than for Visual Basic 6 back in the day. And so, the inevitable result was (and is) that you have literally hundreds of thousands, if not millions, who get jobs by saying "Yeah, I know PHP; I've done (this little site) and (that steaming mess with a lot of Flashy bling on it)." In reality, there are thousands, at most, of good PHP developers out there, with a few more thousands actively working at improving their craft.

Purgatory

So PHP 4 had a crappy object model that anybody could poke at or ignore at will. PHP 5, from mid-2004, started to get its head on straight with respect to both OOP and what it really took to do PHP well, but lots of damage had already been done. Innumerable client projects had either failed or become incredible maintenance/performance nightmares due to the shoddy code that PHP, and the community that grew up around it, encouraged mediocre/inexperienced/inattentive developers to write.

And then, in mid-2009, PHP 5.3 came into the world, and it was a glorious golden statue with legs of brown, wet mud. Many of us who wanted to see PHP evolve into a "properly" object-oriented language, along the path that it had been following, found much to rejoice in. Support for closures. Better internationalisation support. Far better garbage collection. A rearrangement and winnowing of the extension and application repository system that was the closest thing PHP has to Python's eggs or Ruby's gems.

PHP 5.3 also introduced what had to have been the most-requested new feature for years: namespaces. Namespaces are one solution to the problem of allowing the development team to organise collections and hierarchies of classes to both make them easier to work with conceptually, and of mitigating possible naming conflicts (class Foo in namespace Bar is "obviously" distinct from class Foo in namespace Barney.)

However, this is also where the legs of the "golden statue" were transformed into wet mud: the way in which the PHP namespace features work is so semantically and visually jarring, with so many inconsistencies visible to both the experienced pre-5.3 PHP developer and the experienced developer of object-oriented software in other languages, that it quickly became a laughingstock and a millstone. A necessary millstone, but one of which many writers of both code and prose waxed eloquent in their righteous, intricately-justified derision.

A New Hope

And this eventually served as a wake-up call to a number of those who view PHP as just another tool in their toolkit, as opposed to either a cash cow to be milked or a semi-religious icon to be polished and cared for in the precise fashion that the High Priests of Zend decree, or, those who simply never learned enough to care. Seven to ten years is a long time to spend in any one language for an experienced developer, and quite a number of highly-visible PHP community stalwarts have been publicly participating in and contributing to other communities: various JVM languages like Groovy, Scala and Clojure; Objective-C; C#; and Ruby.

A few short months ago, I had urgent need to re-immerse myself in Ruby, learn modern Rails, and make myself ready in all respects to participate in Ruby on Rails projects at a senior or leading level. This gave rise to a series of fortunate events, to paraphrase Lemony Snicket. I discovered that Ruby 1.9 is now a very advanced, mature language with solid experience-based best practices that are sensible and self-consistent. I discovered that Rails 3.1 is an incredibly productive way to get a Web site or application up and running, and that the ways in which it encourages you to code and think do not have the same propensity to inflict mortal wounds as, say, the overly-trusting PHP journeyman developer. Rails itself has grown to be a much larger community that no longer revolves around a single individual, or even a small group of individuals, as that it did seven years ago or as too many other languages do now. And, importantly, many of the things that make Rails great would not be practical, or even possible, were it built on and in any other language than Ruby

Above all, Ruby gives hope to those few, mindful of Knuth's saying that "programming is a literary act", understand that computer programs are written to be read and modified by humans, with computer execution almost a secondary concern over the life of a project. If you care about thinking creatively, if you enjoy having your mind regularly blown in ways that challenge you to actively and continuously improve your mastery of the craft of software development, you are going to love Ruby, and Rails.

An Evangelist Repurposed

And so, to any who are pondering a new Web project, who see how overwhelming the mindshare of PHP is and how obviously negligent (even to non-technical eyes) is far too much PHP code and, by extension, those who made such code available, I would ask you to strongly consider Ruby, and Rails, even if your team has little or no experience in them.

After all, quality and intelligence should, in any world worth living in, be major competitive advantages — or, rather, by rights, their lack must needs be mortal.

A Blessing is ALWAYS A Curse (and Vice Versa)

I've recently started as Senior Architect at Savant Degrees, a Singapore-based Web consultancy. Quite often, I feel like my job is as much "Senior Curmudgeon" as anything. According to Wiktionary.org, a curmudgeon is "[a]n ill-tempered (and frequently old) person full of stubborn ideas or opinions." I'm undoubtedly the oldest person among any of my colleagues that I've met, and I was told fairly explicitly that I was being hired on the basis of the variety and depth of my experience — i.e., for my ideas and opinions. And, like most people, I think I'm only stubborn when I'm right.

Naturally enough, I'm doing a lot of learning, too. This is my first experience with Groovy, a nicely dynamic language which runs atop the Java VM. Like any language worth its weight, a rich ecosystem is growing around it. Like the ecosystems around most languages, there is at least one ultra-massive tool that takes new folk some time to wrap their minds around. With PHP, for instance, it's the Zend Framework, which arguably makes the C++ Standard Library look minimalistic. With Java, it's pretty much anything having to do with J2EE1 (to which Groovy itself is a reaction).

The biggest tool I've yet stubbed my toe on in the Groovy world is, unquestionably, Gradle2. In the grand Zend tradition, the PDF version of the user guide is nearly 300 pages long and, also like Zend, is considerably out-of-date. It's considerably easier to argue that Gradle justifies the bulk, however, since Gradle is a "let's build anything" build manager/dependency tracker/dessert topping/floor wax. The basics can be understood in half a day, even if you've only nodding familiarity with Groovy3. To truly master it, however, requires significant extra time — as much as you feel you derive benefit from.

And that (finally) brings me to the point of this post. The group I'm working with now has some very talented and moderately experienced developers in it; they're open to new ideas and new ways of doing things, more than is the local cultural norm. In our craft, that's not just a Good Thing™; it's necessary for continuing success4. But that enthusiasm is tempered by a keen consciousness of impending deadlines, particularly the "curse of success;" a month, later this year, when we will be spread extremely thin amongst several important projects. The ways we've done things have been successful, but we see the down-sides of success as well.

So, new ways of doing things, and the hope of new, increasing success. New things like continuous integration using automated builds and unit tests, among other such recently-popular tools and techniques. (They're popular because they generally work better than Ye Olde Ways.) I'm arguing for the team to use tools that will, at the click of a button or the tick of the clock, automatically check out, build, test and package the code and associated resources for our project. The team is new to this, having used "traditional" "waterfall" processes in the past. Knowing one is to be hanged, as Mark Twain said, tends to focus the mind wonderfully; a rational person (or team) will avoid being put in that position. My job, so to speak, is to make the tools that help us cut the hangman's rope, and teach their use. Presenting already-busy people with several hundred pages of documentation is probably not the most effective way to do that.

So, at the moment, the most pressing blessing is that we have such wonderful tools available, with more on the way. Most categories of tools have several at-least-usable alternatives available; it's very easy to go into information overload. When that happens, often you just choose to use what some exemplar, say a tool vendor, use themselves. It becomes tragically ironic when a development group pursuing a more agile process has their agility and effectiveness limited by their "agile" tools. (I don't mean to be commenting on any specific company or team here; I've seen this effect on numerous projects over the years.)

Choosing deliberately increases the odds that you will choose wisely, and those odds will increase along with your experience. Experience is perhaps the ultimate professional blessing-and-curse-combined; "you never learn anything from doing it right the first time." I prefer the Nietzsche:

That which does not kill you, makes you stronger.

We're not dead yet — so we must be strong, yes? Another "stubborn idea and opinion."

Comments?

Footnotes:

1. Now known as JEE, because "Java 2" (1.2) is so last decade; we're on 1.6 now. (Return)

2. No, I haven't looked at Grails yet. Being a close of Ruby on Rails, I expect that to be the proverbial "kitchen sink with subdivisions attached." Fortunately (?), we're a Tomcat shop. (Return)

3. But, of course, experience counts; Gradle rewards your groovy Groovy skills with more efficiently expressive power in your project automation. (Return)

4. There's another curmudgeonly opinion for you. (Return)

It's Not Lucid For Me Anymore

For those of you a bit limited in your English knowledge, lucid means "easily understood; completely intelligible or comprehensible."

My experience bringing up some Web servers on the latest release of Ubuntu Linux, release 10.10 aka "Maverick Meerkat" (replacing 10.04, "Lucid Lynx") left me feeling scalped by Apache (the Web server software).

It's (usually) quite straightforward to bring up a Web site in a server's main document root directory, the system-wide, (hopefully) secured directory that Apache loads files from when you browse a simple Web URL such as http://www.example.com/.

Many modern Web sites, and the tools used to build them, depend heavily on Apache's mod_rewrite module, which takes the URL requested by your browser and "rewrites" it into something quite different for security purposes and/or to enable various Web application frameworks to function. Your browser's request for http://www.example.com/ may be presented to the server as if you'd typed http://server1.example.com/webapps/whatever/index.php&node=1. All this is transparent FM to you and, more importantly, to the search engines (Google and friends).

OK, fine. Anybody who's been doing Web work at all knows this. If you've used the Web server on your own system (Apache comes standard with every Apple Mac or Linux system; you Windows usees will have to do it for/to yourselves). The ''mod_rewrite'' syntax makes "arcane" a laughable understatement, but again, no biggie. If you're using just about any framework, it comes with (or documents how to create) the configuration file needed, which generally requires minimal-to-no twiddling from you. Again, as long as you're working at the system document root.

What will send you on a magical voyage of discovery, or an extreme bout of Google-fu, is if you're not working at the system root. Maybe you've got more than one Web site or app you're hosting on a single server (generally using "virtual hosts". More likely, you want to be able to serve a Web page or site from your own user-level directory. Apache has a module for that, too; it's called mod_userdir. And as long as you're dealing with static Web pages, it's drop-dead simple. That makes sense; Apache has been around quite nearly as long as the Web has; all the easy problems and most of the reasonably hard ones have long been solved by now.

As you can probably guess from the build-up here, mixing all the above ingredients (Apache, a framework, mod_rewrite and mod_userdir is a fairly reliable way to ensure that "hilarity ensues". This is so not so much because the combination inherently poses challenges (reasonable experienced developers may differ on that), but because the interaction is quite often (read: until and unless decisively proven otherwise) platform-specific. By "platform-specific," I mean "virtually certain to differ between various distributions of Linux, not to mention Mac OS X, BSD Unix (FreeBSD, OpenBSD, etc.) and Windows."

You may have made it all work on a dozen systems before, only to find that the next system you try displays the performance characteristics of an in-process actuation of an IED. You'll flail around, you'll feel like a "stupid newbie," you'll get on IRC and pound on Google. And sooner or later, because you are a stubborn SOB (aren't you?), you'll find (situation-specific) Enlightenment.

And so, I wish to announce my heart-felt thanks to user FRuso on the Ubuntu user forums who posted this response to The Question. It's a different answer than for openSUSE or Fedora, and please don't do this to an OS X system. But if you're one of the Teeming Millions™ (well, Hundreds, anyway) trudging down this well-worn trail and you're trying to make Ubuntu work, this will help out.

Because, after all, if every similar platform worked the same way, people might actually get work done. If it was that easy, ordinary Joe-Sixpack users might start setting up their own servers, instead of relying on corporate-sanctioned and -supported hosting providers to centralise everything. And then, people might actually exchange ideas and grow their minds a bit! Who knows?

Ahhh, it's "the Internet" (actually, the Web). Most people don't care, as long as they can view the latest bread and circuses on YouTube or Hulu.

Redundancy is Repetitive, Inefficient and Counterproductive

But you already know that.

I've long been a supporter of free and open source software, even contributing to a few projects. However, my enthusiasm has cooled a bit in the last several years, as the time I've been able to devote to such projects has both dwindled and been divided among more projects.

I'm continually flabbergasted by the number of open-source projects that cover the same ground, ad infinitum, often with little to no apparent differentiation between them. By continually "reinventing the same wheel," these small projects usually manage to put out a small number of alpha- or beta-quality releases, and then cease to make visible progress. This could be because the original developer(s) became frustrated or distracted; because the project succumbed to internal politics (a leading cause of FLOSS project death); or merely because the developer(s) realised that they were deep in the rut of a path that has been travelled many, many times before.

This flash of obviousness came about as I've been researching and evaluating PHP frameworks for Web development. I've been developing and maintaining sites using a couple of these, notably Kohana 2.3, but for various reasons have been reluctant to make the switch to the now-current 3.0 release. I decided to do a relatively formal, structured evaluation of several of the frameworks out there, rather than just pounce on the first I came across that looked "reasonable." (Silly me, but oh well.) Of course, as my list of frameworks to look at grew from six, to twenty, to considerably more, I became uncomfortable; torn between the reality that I'm doing this all myself (and devoting significantly less than 100% time to the endeavour), but at the same time wanting to remain open to pleasant surprises. After all, "pleasant surprises" were how I stumbled upon Kohana in the first place, and though I am strongly considering using another tool in its place (again, for non-technical reasons), contributing to a megalithic monoculture is not my idea of "adding value."

Today I did something that I should have done a week or two ago: I went out to SourceForge and got a list of all the PHP frameworks registered on the site.

Oh, my effing $DEITY! "Searching gives 1316 results."

Obviously, not all of those are appropriate (there are numerous reinventions of the CMS wheel, for instance); many can be filtered simply because they haven't published updates recently enough to meet my criteria, and so on, but that's all rather beside my current point.

Open-source projects are (in)famously both insular and prolific, leading to the kind of nonsense I ran into on SourceForge. Commercial products have a built-in inhibitor of this particular pathology: if a market gets divided among so many competitors that nobody can cover their costs, then in very short order the existing products will either consolidate into fewer, more sustainable alternatives or die altogether. Open-source projects, particularly those started by people inexperienced in such projects, fail to realise that they too are competing for resources just as much as their commercial brethren. In the FLOSS case, the finite, competed-for resource is time, both of (potential) contributors and of passive users (what the commercial world calls "customers").

Some open-source projects recognise this. In the Linux and Unix/BSD desktop worlds, the "market" for desktop managers (that provide the windows, icons, menus, and so on that are the UI to the user) has been dominated by two projects, KDE and GNOME. Though there are other choices with solid offerings and active communities, a developer can choose to develop for either KDE or GNOME with the confidence that virtually anyone in his target audience will be able to run his software on their system of choice. Consequently, most projects that have to make a choice, choose KDE or GNOME.

The Web is a different beast entirely, and while standards are for the most part very well-defined, the "barrier to entry" for those seeking to do things (allegedly) differently is very low. Some in the PHP Web development community advise new developers to tackle creating their own framework, as a learning tool. I suspect that that's how many of the SourceForge listings originated. I also happen to think that the advice itself is naïve, akin to a new auto mechanic being advised to build his own engine to gain a better understanding of those from Toyota or Ford. What I would recommend instead is that a new developer learn one or two of the more widely-used frameworks, browse around for another two or three that seem interesting, and start contributing to the project(s) of her choice. Virtually all such projects recognise their need for more help, as well as the fact that "we were all newbies once."

Consolidation in the PHP-framework field, while maintaining the low barrier to entry, should be a Good Thing™: to a greater degree than commercial products, FLOSS projects should succeed on their merits, technical and otherwise. As more developers join a well-run project community, the project's software should continue to both improve and find wider user/customer use. The ability to "do your own thing" should be kept, if only as a check against the use of marketing hype over technical merit.

But I believe that relatively small, informal "A- and B-List" groups of selections would be good for projects and developers (and developers' clients); a lot of what I've come across in the last few days would be on my "P-List," if not "ZZZZ."

Oh yeah, by the way: this sort of maturation and shared effort would do wonders for our Craft's progress towards true professional status. Any of us who've had our schedules and budgets tinkered with by people who admit they know nothing of what we do should understand the desirability of that goal. Given the degree to which software permeates public life and policy, that "desirability" has become a dire necessity; it's just more expedient for people to pretend that isn't really the case. Stuxnet, anyone?

IE9 and Standards? Nothing to see here, folks; move along...

This is a re-post of a comment I left to the post "HTML5 Support in Internet Explorer 9 on Louis Lazaris' Impressive Webs blog (which I highly recommend for people working in or interested in Web development). I've slightly edited a few places for clarity. The comments that had been left by people were almost universally complaining about Microsoft in general and IE's continuing history of imperfect-at-best compliance with either-you-do-or-you-might-as-well-not-try standards. Having worked in and with Microsoft on several occasions, and being active in a number of open-source projects besides, I have a slightly different view, as I state below. Thanks for reading.

---

Actually, I'm amazed that it's progressed as much as it has.

Let me be clear; I'm not in any way praising IE; the industry as a whole needs to bury it — decisively, irretrievably and imminently. We as development professionals need to have the professional self-respect to tell current and potential clients, "We develop to current Web standards that are supported across various browsers and platforms. To the degree that any particular release of Microsoft Internet Exploder supports the markup, styling and behaviour of any particular site, that's well and good. However, without monumental additional time and budget resources, no attempt will be made to provide IE support when those resources could instead be used to improve the experience for users of modern browsers."

I firmly believe that IE hasn't progressed as far as, say, Chromium, can be laid firmly at the feet of Microsoft's existing code base. Microsoft's developers and managers are fully experienced with the reality that complete green-field rewrites of existing projects almost never succeed. They've got a code base where the list of major Windows modules and subsystems that do not have dependency relationships with IE could be read aloud comfortably in a single breath. That was done initially by choice; now, it doesn't matter how well-meaning the intentions or competent the team are, they have to live with the codebase they have. It's all legacy code. The developers at Microsoft are (with rare exception) not morons, but living in the sewer while they're trying to make something that can stand next to the shiny new browsers next door has to be a psychologically toxic exercise. Their baby is blowing up the Web left and right; they know it, and they know they can't do a damned thing about it without staging a coup d'état, replacing dozens of levels of management and senior executives and fundamentally changing the culture of the organisation. Don't hold your breath waiting for that to happen.

That isn't sour grapes or a diss against the IE developers; it's simple reality. Microsoft do some amazing things — just not so much for (or on) Windows. Unfortunately for all of us, Windows and Office for Windows are the herds of cash cows for Microsoft, and anything that could be seen as even potentially disrupting that would get shot down faster than you can say S-E-C; the investors would never stand for it. And, with the system and the rules the way they are, they'd be perfectly right. Innovation isn't as important to the bean-counters (or the regulators) as "maintaining and enhancing shareholder value," and MSFT have had enough problems with that lately. (Just compare their share values to, say, AAPL over the last ten years.) Doing anything "risky" is simply politically (and legally, really) impossible.

So, no matter how many times the IE team jump up and down and say "look at all our neat new features," without mentioning the standard features left unimplemented because they pulled off numerous miracles just making what they have work, the response has to be "nothing to see here, folks; move along."

And move along we shall.

Don't Waste My Time.

What follows is a critique and a complaint, not a rant. What's the difference, you ask? A rant, in my view, is a primal scream of frustration and near-impotent rage that often doesn't let a few (allegedly) non-essential facts get in the way of a good thrashing. It also takes far more energy than I'm able to summon at present, for reasons that may become clear below.

---

As I've mentioned previously, I've been kicking the tires on the (relatively new) Kohana 3.0 application framework for PHP Web development. I'd previously used (and enthused about) the 2.3.x ("KO2") release; I was itching to get my hands dirty with 3.0.1 After a couple of dozen hours spent plinking away at it, however, I'm reevaluating my choices.

Two of the things that endeared the earlier system to me were the good-to-excellent, copious documentation and the active, always-willing-to-help user community (which, quite frankly, are the backbones of any open-source project).

KO3 is a different proposition entirely than KO2. As I write this, the eighth bug-fix update to 3.0 is available on the Web site. Since this is a new "the same, but different" project, all the users are starting fresh, so there can't be the same level of everyone-helps-everyone-else camaraderie as with KO2. This puts a foreseeable, far greater burden on the documentation to help people get productive as quickly and effectively as possible. However, the documentation, quite charitably, remains in a beta-quality state. This is true in comparison to both the earlier release and to other PHP Web frameworks such as Symfony2, CakePHP, FLOW3 and (notoriously) Zend. With most of these other frameworks, as with KO2, it was a quick, straightforward process figuring out how to get from the "hello-world" style demos, to being able to create a useful-but-simple site, to branching out from there. It's taken four times longer to get half as far with KO3 as with KO2.

Judging by comments in the Kohana support forums, I'm not alone in this; the documentation has been roundly panned by virtually all users who've bothered to comment. There's been far too much defensive "no, it isn't bad, you just need to read a bit more, and use the source, Luke" attitude from the project folks. During the KO2 lifecycle, the attitude I understood from their responses was more along the lines of "we know there are a few problems; we're working on them," quickly followed by "take a look at this." I don't know if 3.0 is so much more complex than 2.x that they simply don't have the bandwidth to document things to their previous standards. Frankly, I don't care any more.

I've decided that my existing projects that have been started in Kohana 2.3 will remain there; possibly moving to 2.4 when it becomes the officially-supported release. But I do not plan to invest any more time and effort into Kohana 3.0, at least until the new project has had some time to mature. I fully recognise the potentially self-defeating attitude inherent in that viewpoint. Virtually any open-source project depends on its community to "plug the holes" that the "official" project maintainers don't have time for or deliberately leave as open questions for the community. Well-run community projects are extremely collaborative, communication-rich environments.

Other projects take a "vendor/product" approach, essentially saying "Here's everything you'll need, soup-to-nuts; we'd really appreciate it if you built around the edges and took things deeper in your direction of interest, but the core that we're responsible for is solid." Those "vendors", the Zends and Sensio Labs of the world, have rich open-source offerings that they use as a platform to build (or offer) what actually pays the bills.

While I have a strong philosophical and experiential preference for community-driven (or at least -oriented) projects, there have to be times when I just want to Get Things Done rather than embark on an endless voyage of discovery.3 It is at those times that I'll reach for something that "just works" well enough for me to accomplish whatever it is I'm trying to do at the moment, whether it's to write a book or to bring up a client's Web site. I know and accept that any new tool, or new version of a tool I've used previously, will require a certain amount of time to "get up to speed" on. I don't know everything (or necessarily anything) before I learn it; the most I can hope for (and what I really do insist on) is that things make sense within the logical and semantic frameworks4 that they're expressed in, and that that expression is accessible, complete and correct enough that I can feel like I'm making progress. This invariably involves some sort of documentation; whether a "conventional" or online manual, a Wiki, or a user forum; the format is less important to me than the attributes I mentioned earlier.

Kohana 3.0, as it currently stands, does not meet that standard for me. And so I'm back in a feverish learn-and-evaluate mode with at least two other frameworks. I have projects to finish, and I have several chapters of a book I'm working on that had initially been written around Kohana 2, and will now need to be substantially redone.5

I intend to give each of those new candidate frameworks the same amount of time that it took me to get productive in Kohana 2.x (which was significantly less than the "industry leader," as I've previously mentioned). This is going to be an interesting week and a half or so, in the Chinese-curse sense of the word.

Footnotes:

1. Technical reasons for moving to KO3 included the switch from a model-view-controller (MVC) architecture to hierarchical MVC (or HMVC); if you know what these mean, you should know this is a Very Big Deal™. Additionally, I've found it a Very Bad Thing to tie myself to a legacy (obsolescent) project, and the end of KO2 is being made very plain on the Kohana Web site.(Return)

2. If the new, pre-release Symfony 2 code is as good as the documentation, we're in for a real treat.

3. I am typing this on a Mac rather than a Linux box, after all (though I have three Linux VMs running at the moment).(Return)

4. This implies, of course, that there are "logical and semantic frameworks" sufficiently visible, understandable and relevant to the task at hand. (Return)

5. I certainly don't want to fall into the same trap as a previous series of PHP books, which relied on the obsolete and inadequate Ulysses framework. Ulysses has been justly and productively panned, which has to reflect poorly on the aforementioned book (which I happen to own) and its authors. (Return)

In Praise of Robust Tools and of The Future™

One of the best points about doing Web development in PHP is that it's so widely used; several respectable estimates by organizations that get paid to find these things out say that some varying number north of 50% of all sites on the World Wide Web use PHP as their implementation language. This includes numerous content management systems, or CMS, such as WordPress, Joomla! and Drupal.

One of the far-less-good points about doing Web development in PHP is that it's so widely used; the (evolving) "standard" set of tools, in true open-source fashion, are assembled and maintained by an ad-hoc band of individual and small-corporate luminaries, on infrastructure that worked just fine back when any given server was hit a couple of thousand times a year; by continuing to rely on (what to the outside Net appear to be) individual servers without terribly huge pipes connecting them to the larger Internet, the infrastructure completely fails when that's scaled up by a factor of a thousand or ten.

Web developers using PHP are highly dependent on an extension architecture/platform called PEAR, the "PHP Extension and Application Repository." Singular. Well, for any given extension or application, it's singular. So, as the user base scales upwards, popular tools that go through a vigorous release cycle, the servers they're hosted on (and network choke points between large subsets of users and those servers) start having reliability problems; transfers slow to a crawl, or fizzle out entirely. (One such transfer this evening proceeded at a sedate 600 bytes per second. Not kilobytes: bytes. That's 1980s dial-up speed.)

Developers in other languages have similar tools; Rubyists have gems available from sites like rubygems.org; Pythonistas have eggs. But somehow I hear a lot less grumbling, and do less of my own, when the subjects of gems or eggs come up; they seem to Just Work™ – which indicates that the network infrastructure is a lot more robust; either larger "pipes" and/or more, distributed servers such as with a CDN.

Trying to access servers like this from Second World cities like Singapore that apparently devote more resources to content filtering/monitoring than easing congestion doesn't help either. Hey, SingNet and M"D"A, how come it takes ten hops to get off an island that's barely that many miles across? If we want to plug Singapore into the "new global economy," having grotesquely under-resourced connections is not helpful.

But back to the main subject: If PHP is going to continue its phenomenally successful growth, then the infrastructure is going to have to decentralise, aggressively. Having essential tools like PEAR and the PHPUnit repository available only from a single point ensures that single points of failure will continue to seriously compromise the PHP ecosystem. No access to servers, no access to tools. No access to tools, then development of PHP artifacts with reasonable efficiency and economy is severely degraded.

In other words, the status quo is a limiting factor on future growth and success. However, as many will be quick to point out, building infrastructure costs money – for hardware, for connectivity, for the paid labour of skilled craftspeople and professionals to create, install, operate and maintain this enhanced infrastructure. No explicit means of funding such an endeavour presently exists, at least not to my knowledge.

So where do we go from here, PHP community?>

Phing! It's a Dessert Topping! It's a Floor Wax! No, it's efPhing something!

I want my hour back. No, seriously; that's what happens when you don't touch a tool for a while, but you spend a lot of time with its "kissin' cousin."

Phing, if you're new-ish to serious PHP development, is your ultimate Swiss Army Ginsu Chainsaw™. It's a "build tool" that lets you automate pretty near anything, especially having to do with PHP. There are several dozen "tasks", like the PhpCodeSnifferTask, the IfTask, and of course the PhingTask. There is also good documentation on extending Phing; adding all sorts of new tasks and other bits, and people have run with the ball.

Put another way, if you're coming from the Java omniverse, Phing is analogous to Apache Ant, with at least as vibrant a community effort throwing stuff over the wall.

Sometimes the things that get thrown over the wall blow up, though. Sometimes that isn't Phing's phault, though that's the fish-eye lens you're looking through as you grapple with the problem. FOr instance, the PEAR coding standards as interpreted by PHP CodeSniffer require tags in the internal documentation you must produce to comply with the standard that are not supported by the documentation generator they're ostensibly intended for. You'll see your Phing script break on that rock — unless you "change the conditions of the test." Figuring out the (multiple) flags, option settings and occult incantations necessary to make things run smoothly will take you some time to figure out, unless you just did it last week and wrote copious notes in your wiki.

But the absolute break-down-and-laugh-until-you-cry moment came from the way Phing handles "custom properties," the symbols you can define to make your Phing life easier in various ways, like having common, shared policies and processes across projects, with specific values set on a per-project basis. You whip up a new "property file" as part of a new project, and reuse your existing XML "build file." Ah, but there is one sizable bump to trip the unwary or rushed: Properties are specified in a sensible XML format when included in the build file, but separate, included "property files" look like old-style Windows .ini files. And $DEITY help you if you forget, because Phing sure won't.

Let's say you have a "drop-dead simple" build.xml file, like this:

<?xml version="1.0" encoding="UTF-8"?>

<project name="demo1" default="demo">

<if>
    <equals arg1="${usepropfile}" arg2="false" />
    <then>
        <property name="foo" value="baz" />
    </then>
    <else>
        <property file="build.properties" />
    </else>
</if>

    <target name="demo">
        <echo message="Demo target; foo= ${foo}." />
    </target>
</project>

If you screw up the build.properties file — say, by specifying your values in the same sort of XML format:

    <property name="foo" value="quuz" />

you'll see a most sublimely confarkled message:

    ....
     [echo] Demo target; foo= ${foo}.
    ....

At this point, you'll either have a D'oh! moment, or you'll start chasing your tail. Choose Door #2, and you could be at it a while.

The answer, as often, is RTFM. Appendix F (!) of the Phing manual defines the "Property File Format," which is our venerable foil the .ini file with a few twists (like being able to define properties that incorporate the values of other properties – just like in build.xml).

Would it really have been too complicated to use the same format (i.e., the XML tags) in the property file as in the build file? You could even deal with making it valid XML with minimal effort. But no...

I actually did learn this properly a couple of years ago, when Phing was new and kind of shiny. It's only become more powerful since then. Just don't hold three lighted M-80s in your hand.

Why rant about this? I've long firmly believed that one of the main duties of a master craftsman, in any craft – including software development – is to occasionally fall into the weeds, pick himself up, and remind the young journeymen and apprentices "don't do what I just did; you'll make yourself look silly, or worse." Better for one person to do it who can be reasonably expected to pick himself up, than for a dozen others to fall in with no idea how deep they're getting.