Bootstrap and alternates are GREAT! Except when they lead to...

...divitis like this (in a Slim template):

Yes, that's eleven levels of indentation for the link within the .list-group-item list item. Can *you* glance at that and grok it in reasonable fullness without spending ten minutes rebuilding the DOM in your head? I can't, and I wrote the sucker yesterday.

What's the problem? Nesting styles is the problem. Using line 23 as an example, we have, in reverse order:

• an anchor link;
  • within a list-item tag;
    • within a loop (that arguably doesn't count, but does kick the indentation in a level);
      • within an unordered list;
        • within a .row div;
          • within a .col-md-10 div;
            • within another .row div;
              • within a .col-md-12 div
                • within an outermost, containing .row div.

Got all that? At least we're using something like Slim and not, $DEITY forfend I'd need to use something that looked more like HTML soup, like ERB.

Why bother? Because, as far as I can tell, that's what it takes to work with nested columns in Bootstrap. Everything's within an outermost row, in which you define columns; within any given column, you can use another row to break the containing column into sub-columns. Remember when, in ancientest days of the Web (circa 2001 at latest), we started telling people not to use tables for layout because it led to inscrutably crufty hairballs? They're baaaaaack!

There's got to be a better way. Isn't there? Isn't there?!?!?

In other news...

By this time next week, after over ten years of blogging on Blogger.com, I expect to have the process of moving my blogs (and my Tumblr, and many of my G+ posts well underway, if not yet completed. Why? I'd like to

1. Reduce my Google footprint or, more precisely, the footprint GOOG leaves on me;
2. Be able to point people to one place where they can see all my non-code online writing;
3. Use a markup language that's less actively hostile to writing than HTML is; with Markdown and Jekyll on Github Pages, I get that;
4. Have more control over that "one place" than any of the Google-based "places" presently give me; and
5. Be able to embed Gists or other bits of code in my writing and have them look better than the embedded Gist at the top of this post.

It's not for everybody. But if you've been using Blogger in HTML-editing (rather than WYSIWYG) mode, as I have since the beginning, you can certainly learn (and benefit from and enjoy) Markdown and Jekyll.

Rails, Circular Logic and Incremental Big Bang Architecture

(Initially published on my Tumblr, which I'm finding increasingly usable. Thoughts on Tumblr vs Blogger.com?)

---

Rails is a fantastic solution to a well-understood class of problem: the CRUD-oriented (or REST-oriented) query/report/update Web application, circa 2004–2006. Its philosophy of curation over configuration, despite having been likened to the report-producing languages of old (such as IBM RPG II, means that you really only have two things to worry about: your domain business logic and page transitions (which is another way of saying “your domain business logic, as it applies to presentation”).

And yet, the farther your app gets from that sweet spot,, either by decoupling the UI and using a fancy JavaScript framework; by needing to support user inputs other than typing into form fields and clicking buttons; or even the distance imposed by being nearly ten years on from the “sweet spot” and so many tools and other bits of code have changed around it — the harder it gets to do what you want to do in The Rails Way. Lots of people much smarter than me, from Robert C “Uncle Bob” Martin to Jim Weirich to Sandi Metz to Matt Wynne and a host of others, have spent many person-months thinking about, grappling with and writing about how to mitigate Rails, the knots it ties you into and the practices that much of the rest of our craft has built consensus on as being essential for good practice, that you can’t use in Rails without superhuman effort. Because the apps we’re building now, even the relatively simple ones, aren’t largely analogous to Basecamp (the application from which Rails was originally extracted).

So here’s the problem I’m facing, and the problem you’ve more than likely faced if you’ve done a non-trivial app in Rails.

If you’ve done any medium-to-large-scale development in other systems, especially in a decent OO language, one of your high priorities is going to be separating your domain logic from your “infrastructure” logic, including layering your presentational logic related to your data models separately from the data models themselves. Small is beautiful, and Ruby has some great tools to let you do that, including in particular the whole universe of Gems. Rails, however, makes it bloody hard to build what the rest of the craft would view as modular apps that follow SOLID principles. The retort to that is generally along the lines of “Rails is omakase” — which is fine as long as your app is living within that circa 2004–2006 sweet spot I mentioned earlier. The clock in the upper-right corner of my display here informs me that today is Saturday 7 June 2014 as I write this. Oops.

So we have great ideas like Hexagonal Rails , Architecture: The Lost Years and so on, that showed ways, with varying degrees of universality, to “get a handle on” the application/system you were developing.

But, can you answer what your application does? Not “what is its data model”, “what are the different types of users and permissions”, but a pithy sentence or two that gets across the idea of what makes your application different than every other CRUDtastic app out there, Rails or otherwise. Because if you can’t, none of this Rails-survival lore will help you.

And, equally importantly, as Jim Weirich points out in his Cincy.rb talk [Decoupling from Rails](http://youtu.be/tg5RFeSfBM4), is the question of “has this ever been successfully done before? If so, how? If not, what’s been the stumbling block that we have to stand back and work around using something else?” Because, in all fairness to Uncle Bob, if there aren’t examples of his built-on-Rails architecture in the wild for people to poke at, projector-ware can be inspirational but not terribly practical. For instance, as Jim points out at about 10 minutes into his presentation, Uncle Bob’s architecture shows where Rails controllers and views go, but ActiveRecord “models” don’t seem to be there at all*. Separating domain logic from persistence just happens to be one of the hardest nuts to crack in a Rails app to begin with. Hmmm.

Is the solution more decoupling? Jim spends the rest of that talk isolating and extracting what several teams publish as “interactors”, or support for what I call in my own apps “domain service objects” or DSOs. These help, a lot; having “skinny” controllers that don’t really know anything at all about your app beyond how to hand data off to classes that do lets you refactor your business rules efficiently by putting their control logic all in one place separate from Rails. Similarly, implementing façades or “decorators” with presentational logic around your models, as with Draper and others, helps you write clean, business-logic-free view templates.

But you’ve still got ActiveRecord to deal with in your models, and it leads you down the garden path and over the cliff with things like, oh, relations. When you have, say, a Blog model that has_many :posts and a Post model that belongs_to :blog, what you’ve got is a circular dependency; you can’t make changes to your Posts with certainty that those changes are isolated from your Blog, or vice versa, because each depends on the other. There have been numerous serious attempts to effect such a decoupling; perhaps the most widely known is Ruby Object Manager, whose Version 2 has been in the works for over two years now. It’s led by one of the greats in the Ruby community, Piotr Solnica, but it’s a Hard Problem. Especially when coexisting with Rails. People even resort to uncomfortable hacks like this to Get Things Done. There has to be a better way, and monorail (traditional Rails all-in-one non-modular app) is not that way.

And there are other, seemingly orthogonal issues to deal with. Authentication? Authorisation? Several solutions that are widely used exist for these in the “traditional” Rails “monorail” application style; how will adapting your architecture affect these, and how will you implement them?

Anybody?

Rails is Rails is NOT Rails. Oh, my aching head.

For a couple of decades, I was into Linux. One of the ways you could tell good distributions and (especially) good documentation for those distributions is that they were (usually) careful to make notes like

This is a new feature as of kernel 2.6, first introduced in QuuxLinux in Version 6.0.

or

That API changed in kernel 2.4 and again in kernel 3.0, first introduced in QuuxLinux in Versions 2.0 and 11.0 respectively.

Now I'm a Ruby and (pro tem) Rails developer, and all that is old is new again, and you don't always know it.

I just spent a day and a half of highly-energetic tail-chasing, working at getting my code and specs for (what is so far) a "toy" demo project to understand Rails routing when namespaces are introduced. Namespaces, which are (as far as I can tell) functionally equivalent to Ruby modules, let you provide an additional level of separation between different parts of your app. They're also practically essential for writing Ruby Gems of any complexity, which is a basic requirement for "Vendor Everything" — an important tool in structuring your apps for maintainability and reusability. So "supporting namespaces" was an essential, if not universally obvious, part of the exploration that I'd embarked upon.

As I write in the project README, I want to "namespace everything". I also write "no code in the absence of a failing spec", so obviously my specs need to work properly with namespaces. Believing in six impossible things before breakfast is trivial in comparison, if you've never done it before.

Well, how better to learn than with a demo project? And how better for my current and future employers to judge my progress and reasoning than by making such (non-proprietary) explorations in the full glare of public scrutiny and ridicule, à la GitHub?

After accomplishing this first tiny step of The Mission, I'm even more convinced than ever that part of what makes Rails development "interesting" is that there's such a mix of information about Rails 2.3, Rails 3.x and Rails 4 out in the wild, treated as though they are slight variations on a common theme, rather than three different products unified mainly by a shared product name.

For those who care, the initial plea for help was on Reddit's /r/ruby sub-Reddit. Initially I'd written up this Gist to document the files I was using and the output I was getting. After reading /u/Nitrodist's response, I made some changes, as reflected in this Gist. Finally, after further prodding from Nitrodist and others, and intensive BFI, I determined the two changes needed to my original code. Documented in this final Gist, and pulling together information from several sources referencing different versions of Rails, this *early step* in the exploration has been completed — with several times the expected effort.

Things have to get better, don't they? Only if you're not aware that Captain Edward A Murphy was a raving optimist.

Going Forward; Avoiding the Cliff While Looking for Cliff's Notes

Over on the Ruby sub-Reddit, I got into a discussion (mostly with Jelly_Jim, the OP, and with realnti) about the conflation in far too many minds of Ruby, the programming language, with Ruby on Rails, the famously-"opinionated" Web application framework. While pontificating on Jelly_Jim's original question, I described my frustration-bordering-on-antipathy with the traditional (known as "The Rails Way") application structure; listed a few presentations, books and blog posts that have influenced me greatly, and described my own take on the Kübler-Ross model for Ruby-based (and at least initially, largely Rails-based) Web development.

That discussion, over the course of roughly a day, helped me figure out how our app is going to survive the expected Slashdot effect, transitioning from a pretty traditional, megalithic Rails app to an app that uses Rails components in a sharply different, more recognisably "object-oriented", service-based architecture.

Leaving aside my more prosaic experiential difficulties with Rails (I really loathe ActiveRecord as commonly used), and taking into account the "Uncle" Bob Martin keynote, the various blog posts I referenced in the original Reddit post, and a couple of books I mentioned in a later reply, I think I've possibly hit upon a way to get where I want us to go.

That's the easy part, of course; it only took me never-mind-how-many months. The "interesting" part is going to be getting there — understaffed, on a shoestring budget for the moment even by startup standards (another problem I'm working), and working a schedule for two and a half years that would land a healthy twenty-year-old in hospital by now. Comments/suggestions are, as always, greatly appreciated.

---

Basic Architectural Principles

• Loosely-coupled, granular architecture FTW. That means, inter alia,
  • Nearly the entire app to be packaged as Gems used by an otherwise minimal application;
  • Rails mountable engines to package assets (particularly CoffeeScript/JavaScript) and major components;
  • Plan and design for updating and provisioning components which communicate across natural architectural seams.
• Hexagonal, "ports-and-adapters", or "clean" conceptual architecture; all dependencies point inward. Your domain entities, at the centre of the system, will change less than details like the delivery mechanism (Web UI, phone app, etc) or the database (which the domain shouldn't even be able to prove exists as such).
• By adopting a heavily job-story-oriented view of the workflows; with the use of online tools like Trello, WebSequenceDiagrams.com; and with supporting code tools like ActiveInteraction and so on, we should be a lot more productive and a lot more Agile than we have been during our forty (calendar time: two) years lost in the wilderness.
• And oh, yeah, just to keep things interesting: we've got to keep regularly demonstrating forward user-visible progress, for all the obvious reasons and a couple I really can't go into here.

---

How We Get There, v. 28.0

First, since the major focus of late has been on some unique (and not-so-unique) CoffeeScript code, start there. Separate the existing Script code out into four engine-based Gems, following/adapting Derek Prior's example. These levels are, from conceptual bottom to top:

1. A "gateway" CoffeeScript class we've written as a poor man's RequireJS, that gives us just enough pseudo-dynamic referencing of data/code without changing anything the Rails asset pipeline assumes;
2. Our directly DOM-aware Script code, as one or possibly two Gems (one is a major crown jewel; we might decide to split off utility/support code into a separate Gem, so as not to touch one when maintaining the other);
3. The code proxying our internally-used message queue (decoupling "FTW", remember?) and the Ajax code to communicate with the Rails back end; and
4. The code implementing various domain/service objects sitting atop the Script stack; one Gem per service.

Next, adapt the relatively-straightforward (as yet) Ruby/Rails code to honour the architectural principles listed earlier. This is where tools like ActiveInteraction pull their weight (and solve several existing pain points in the process).

Just to make sure we really do understand where we're going with this, take our first user story-specific code (including interactor) and package it up as a Gem that should then Just Work as in the previous step. Start thinking about whether we really want traditionally-packaged Gems or unbuilt dependencies that remain part of the single app repository.

Proceed to reimplement and repackage the (again, relatively few) job stories already existing as was done in the preceding step.

Start knocking down the remaining Trello cards as we complete our app.

We've been saying since the Very Beginning that anything that we were building that wasn't inextricably tied to proprietary, commercially-valuable code (or that gives too much of a hint into any unique details it may have) "should be" openly released, but hadn't yet figured out a feasible, cleanly reusable way to do that. If we've done our jobs properly up to this point, we have that way now.

Push the "deploy and launch" Big Green Button. We've bloody well earned it by now.

---

Any Suggestions or Comments?

Clean Code. "I do not think that word means what you think it means."

From my copy of Clean Code: A Handbook of Agile Software Craftsmanship (the paper copy of which (barely) predates my work in Rails. The highlighted paragraph is what I'd highlighted in the paper copy:

Active Record

Active Records are special forms of DTOs. They are data structures with public (or bean- accessed) variables; but they typically have navigational methods like save and find. Typically these Active Records are direct translations from database tables, or other data sources.

Unfortunately we often find that developers try to treat these data structures as though they were objects by putting business rule methods in them. This is awkward because it creates a hybrid between a data structure and an object.

The solution, of course, is to treat the Active Record as a data structure and to create separate objects that contain the business rules and that hide their internal data (which are probably just instances of the Active Record).

I've been increasingly uncomfortable with the way Rails encourages people to conflate the domain model objects with persistence in Rails' core component, ActiveRecord. Running across this bit again makes me think of Inigo Montoya from The Princess Bride: You keep using that word. I do not think it means what you think it means. Judging from my own projects and those I've participated in and observed on Github, the fact that ActiveRecord does not properly implement or encourage the Active Record pattern is a primary root cause of many bugs and much confusion and heartburn.

Happy New Year.

No, You Can't Always Get What You Want...

…but as Mick Jagger and Keith Richards remind us, if you try "sometimes you get what you need". Any software developer lives with that on a daily basis; the implications of that are one of the major things that separate our craft from the profession of engineering. I had several of those "implications" blow up in my face this week as I was working on (what else?) a Rails app.

Or, more properly, an app that uses Rails. Because what the majority of my time has been spent with for the last few weeks has not been Rails, or even Ruby, but trying to get the front-end features that have to be implemented in JavaScript (or CoffeeScript or another language that compiles to JavaScript) (generically here, Script) because they're dependent on user-agent and DOM events. The last time I went down this road, I wound up with an app that had five lines of CoffeeScript for every two of Ruby; this one is well along that path. That's relevant; I suspect that there are far more non-trivial apps that use Rails in production than there are Rails apps; for anything beyond a simple pseudo-blog or relatively straightforward database front end, you sometimes wonder what the early Rails core team was thinking when they established some of these "conventions" that trump "configuration". Granted, there are a lot of people using that "sweet spot" for Rails app development, and more power to them, but I'd bet heavily that there are at least as many who started out trying to do an app that uses Rails and fairly quickly find themselves working around or against things as much as they work with them.

A prime example of this is front-end work. Rails has support for *Script and the asset pipeline as ballyhooed features. Once you start writing more than a few dozen lines of Script code in three or four classes or modules, you quickly come to the conclusion that the people who designed and held court over how Rails works really didn't have much more Script experience than using snippets of jQuery or Prototype.js to "jazz up" a form. Heck, the "unobtrusive JavaScript" feature is essentially useless outside forms. What I'm working with at the moment is an app that intelligently reacts to a user selecting content with the mouse, based on the position of the content, the permissions and authorisation of the user, and other factors. Having a few Script classes (and associated specs) to deal with that introduces issues that may not be readily apparent in the beginning, but will bite your team over time.

The JavaScript community have developed several module loaders to work around the lack of such a feature in the core language (for now, at least); there seems to be something of a transition underway from CommonJS to RequireJS, with several alternatives providing variations on the theme. This is important for several obvious reasons, including the ability that programmers have enjoyed in various languages since at least the 1960s to develop, test and maintain cooperating code modules (objects, classes, etc) relatively independently. By having each such module load, and know about, only the other modules it uses, those modules (should) remain unaffected by other modules in the system, even indirect dependencies that continue to work as their dependents expect. Programming 101 stuff; nothing new for anyone reading this.

But, and it's a very large but, the Web prizes a site's ability to perform as few network accesses as practical to get its resources loaded and start doing useful work. To this end, Rails since version 3.1 has included the asset pipeline, which has the effect (broadly) of concatenating and compressing Script files, CSS stylesheets, and other resources so that they can all be loaded in a single request. This is done using an add-in Gem (component) called Sprockets. This works quite well for its intended purpose and truly is one of the nicest things about working in Rails — until you start writing modular Script code.

If you know that the asset pipeline, used by default, means that everything is loaded by the time any of your Script front-end code starts running, but you want to break your code into modules, the "obvious" choice is to create a lot of "global" objects for your classes and shared data and such. If you're an experienced developer, this feels wrong, almost dirty. You might then try only having a single global that functions as a top-level "namespace" for your application, with artefacts spreading out in a sensibly-organised tree below that. This path leads to names like @com.hudsucker.proxy.activation.foo.bar.baz.joebob.and.his.third.cousin. Maintaining such, or even keeping it straight in your head (and your teammates') quickly becomes an "interesting challenge". You quickly realise that there must be a better way, simply because there have been other developers before you who have remained (apparently) sane.

But to use loaders such as RequireJS from Rails, you need components like requirejs-rails, which is wildly, justifiably "popular". Sane Script dependency management should be in Rails, you start thinking. And I'd agree with you. However…

The Might-As-Well-Be-Official Rails "Solution" to the problem is: don't do it that way. Write your front-end code in a traditional Script framework like Dojo or Node or Ember or JoeBobBriggsAmazingFramework or any of no doubt numerous other excellent choices. Use Rails "merely" as the back-end API server for persistence and auth and so on. And yes, with planning and preparation and an adequately-staffed and -trained team starting well before your deadline, that's a splendid choice. You may even realise that Rails is overkill for your needs and move to something like Sinatra or Padrino or, again, numerous others.

But…what if you don't have that time or that team, but you do have a mass of rigidly-organised, fragile Script code that you just promised would be part of a working app in two weeks' time? Assuming that you agree that resignation and suicide are suboptimal strategies, you could try what I'm in the middle of retrofitting our code to:

1. Create a single, gateway object that you use to get all your other objects. Let's call it a Gateway object. give it a global location at @ourapp_gateway. Give it two methods:
   1. a register method, that associates a string name with an arbitrary object/class/other collection of bits;
   2. a use method, that takes a string parameter and returns the item associated with that name.
2. Remove the Sprockets #= require modulename lines from each of your existing Script files, and replace them with immediate calls to @ourapp_gateway.use corresponding to each of the objects you need from Your Outside App, as well as calls to @ourapp_gateway.register to register the object(s) declared in that source file with the Gateway;
3. Add Sprockets #= require directives to your application.js, so that Sprockets actually loads the suckers;
4. Assign the return values from @ourapp_gateway.use to variables as would be done in Node or whatever.

Why bother, I hear you thinking. Because, even without a proper module loader, this still delivers at least one major benefit: it abstracts away the relationship between your artefacts, the DOM, and the asset pipeline. It gives you reasonable flexibility in moving things around within the DOM and with respect to source files by having the Gateway instance as an intermediary. And, if you're thoughtful, diligent and very, very lucky, it gives you a leg up when you do get around to using a proper module loader, by letting you slip that in behind your Gateway abstraction.

And if you think I'm getting somewhat jaded/disillusioned with my continuing Rails experience and integrating a Script-using front end that's not a complete app in itself…congratulations; you nailed it.

Anybody have any better ideas that they've actually made work?

Tap-Dancing Among the Land Mines

…is best done very carefully, and with the knowledge that not everything works as you'd like it do. That's a good metaphor for my week this week.

Working on our new platform and its (eventual) variety of products and supporting tools, mostly in Ruby using Rails, some of the absolutely key elements are primarily front-end code, written as CoffeeScript running in the browser. Typically enough, we make heavy use of jQuery, various plugins, and other libraries. When understood and used correctly and they work as expected, they're great for reducing the effort needed to accomplish new and useful things.

The moderately-sized application I'm presently working on uses over 125 third-party components, many of which are dependencies of components we directly/intentionally use. Of those, some three dozen components have yet to reach a "1.0" release level. That's a lot of trust we put into people we've never met and teams completely outside our organisation. The fact that this is typical to the point of being mundane among Ruby shops is one of the phenomena most commonly cited by new Ruby devs and managers as "mind-blowing".

When everything works properly, it's awesome in a very different way than anything I've done in my nearly 35-year career. When anything fails to work properly, it's equally spectacular; just in a very different way.

Consider today. I've been working on a Script component (which I expect to publish in due course as a jQuery plugin itself). Being a proper behaviour-driven development project, I've been writing specs using Mocha and running them with Teaspoon. Teaspoon lets you run your spec suite either in a browser or from the command line. The browser-based interface is easy and convenient (almost fun) to work with during development; with a mouse click, you can focus on just the spec you're working with at the moment, or step back and run a group of related specs or your entire suite. After finally finding and killing the last of never-mind-how-many typos and silly errors (CoffeeScript, alas, has not yet implemented proper DWIM support), I had the component working. It was clean. It was more than fast enough for my purposes. And, most importantly, it worked properly, against every condition I threw at it.

That is, when I was running Teaspoon from the browser. The command-line invocation, needed for continuous integration among other things, failed in a very odd fashion. See, for example, this bit of screen output:

[jeffdickey@Jeffs-Prolog-iMac select_demo (wip)]$ teaspoon -q | tapout
Started w/ Seed: 0
..................F

1. FAIL markup from selectedMarkup()

    Unknown
    
    TypeError: 'undefined' is not a function (evaluating 'currentNode.contains(last)') in http://127.0.0.1:58247/assets/selection_wrapper.js?body=1 (line 111)
    
    window.meldd.SelectionWrapper for rich single-paragraph content, returns the correct markup from selectedMarkup().#:0


Finished in 0.092s (206.522 test/s, 0.004842s avg.)

19 tests: 18 pass, 1 fail, 10 exit, 0 todo, 0 omit

Now, bear in mind that when Teaspoon was running inside the browser, no such error message was displayed; all specs passed. The code being complained about looked something like:

  getMarkupUntil: (currentNode, contentString, last, offset) ->
    while currentNode?
      if currentNode.contains(last) # <-- THIS IS THE LINE BEING COMPLAINED ABOUT
        if currentNode == last
          # do something...
        else  # currentNode contains last but is not last
          # do something else...
      else # currentNode does not contain last but we're still looking
          # do something different...
    # return our results

Now, currentNode.contains is making use of a bog-standard HTML DOM function that's been in every major browser for years. Even Internet Exploder 5, from the Dark Ages of 1999, managed to support it! What could possibly go wrong?!?

Tap dancing on the shoulders of not-always-giants, it turns out.

Teaspoon, by default, uses PhantomJS as a "headless Webkit browser workalike with JavaScript support". If you're going to automate testing of your Web-based application (or any of several other related activities), this is the tool you want to use. Except, o course, when it doesn't work correctly. I changed the getMarkupUntil method I showed earlier to test my hunch:

  getMarkupUntil: (currentNode, contentString, last, offset) ->
    while currentNode?
      return null unless currentNode.contains?
      if currentNode.contains(last) # <-- THIS IS THE LINE BEING COMPLAINED ABOUT
        if currentNode == last
          # do something...
        else  # currentNode contains last but is not last
          # do something else...
      else # currentNode does not contain last but we're still looking
          # do something different...
    # return our results

The new line returns a null value from the function if and only if currentNode has no property (data or method) named contains. I reasoned that this would (fairly obviously) cause the calling code to fail, and the difference in the failure message (from what was shown previously) would confirm to me that this was indeed the problem. Again, running the specs from inside the browser presented no problems. Running from the command line, however, yielded

[jeffdickey@Jeffs-Prolog-iMac select_demo (wip)]$ teaspoon -q | tapout
Started w/ Seed: 0
..................F

1. FAIL markup from selectedMarkup()

    Unknown
    
    TypeError: Type error in http://127.0.0.1:58271/assets/selection_wrapper.js?body=1 (line 174)
    
    window.meldd.SelectionWrapper for rich single-paragraph content, returns the correct markup from selectedMarkup().#:0


Finished in 0.090s (211.111 test/s, 0.004737s avg.)

19 tests: 18 pass, 1 fail, 10 exit, 0 todo, 0 omit

As expected, the null value caused other code to die.

I then browsed the Teaspoon documentation, which pointed out that although PhantomJS was used by default, Selenium could be used instead, by installing the selenium-webdriver Gem and updating the project configuration accordingly. After re-bundling the app, I could run my spec suite from the command line. Hurrah!

Well, golf-clap applause at least. We'd had previous problems with Selenium testing the Script code as run from within a Web app. We'll burn that bridge again when it falls on us. Again.

When Development DOES Go Horribly Wrong™

I just finished watching Ian Cooper's talk at NDC Oslo, TDD, where did it all go wrong. If you're doing any flavour of TDD (BDD, etc.), this will likely be one of the more important hours you spend this year. Ian helped me wrap my head around the details of what some of those voices screaming in the back of my mind as I (and likely you) write code, were trying to make me understand.

Bear in mind that I've been doing test-driven development for a decade, and some form of behaviour-driven, outside-in development for perhaps half that. I was once known as a "crazy blue-sky hacker", who composed symphonic-length pieces of software at a single sitting. (I'm explicitly disclaiming anything about the quality of that software; while it may have seemed sufficiently fit for purpose at the time, there almost certainly were bugs aplenty waiting the hapless maintainer.)

One of Ian's main points in the talk, if not the main point, is that test-driven development (and its successors such as BDD) should test behaviours of the system, not how every little internal method behaves. Lots of great insights here, from how to be more effective with fewer tests, to "write unit tests that focus on behaviours and thus can be used for acceptance". More bluntly, "the reason to test is a new behaviour, not a method or a class." I'm as guilty of that as anybody.

That last has been something I've been trying to work toward for well over a year. Unfortunately, one of the poisonously bad habits I've picked up working in Rails is the idea that there's a one-for-one mapping of specs/tests to implementation classes and methods (model, controller, helper, etc.). These are the trees that keep me from seeing the forest of what I'm ultimately trying to accomplish, and more often than not, have me polishing the nodules on each root of each tree.

One of the things that comes out of this understanding for Rails or, to be fair, for any framework in any language that focuses more on implementation classes than expressed behaviours, is that I for some time now have avoided use of the generators, particularly scaffolds, for core application components (models, controllers, views, etc). My preferred workflow has evolved to something like what I list below. Note that this is my existing workflow, and does not yet incorporate the new understanding of proper TDD.

1. I'll usually do a "first whack at" a model spec and bare-bones model, just to have something "specific" to work with in the later steps. I am beginning to see the way I have been doing this as a bad habit, particularly in "traditional" Rails. A Rails model should be concerned with persistence and validation, and an absolutely minimal amount beyond that. What I really want to start out with is my initial idea of a "business" domain object, which is Not The Same Thing;
2. I'll write what in the Rails world is called a "feature spec" and most of the rest of the omniverse knows as some form of "integration spec", modelling how a specific feature works. ("As a Member, I want to create a new BlogPost, to begin a discussion about a topic.") The initial "high-level" spec may or may not evolve into a series of specs which each demonstrate an individual sub-feature, but will always be expressed from the outside in, mocking what does not yet exist;
3. Next is a controller spec and the controller that it specifies, as a "traditional" developer-oriented spec (or set of specs). When done, I should be able to integrate this controller into the feature spec written previously (still using mock models) and demonstrate that the controller works;
4. Now, I re-look at the model(s) involved, turning those into "live" specs and "real" code that works correctly with what I've written thus far; my feature specs and controller specs still pass;
5. After the models and controllers are done and working with the feature specs, I'll do the views. I should have a very firm idea what's needed for these by now, as they've been used throughout (in mock or throwaway form) by the feature specs; this is where I write the code that emits the HTML, CSS and Script code for the UI.

That was my old workflow. I'm going to explore how to adapt and adjust this to more effectively and efficiently get the job done in light of Ian Cooper's talk and a (very near-future) reread of Kent Beck's Test-Driven Development: By Example. I say "effectively and efficiently" as my current project's spec-to-live-code ratio (in LOC) is approximately 13:4. I think most of us would agree that that's a very strong code smell.

"Easy Credit" IS "Easy Debt". It sucks for finance. It sucks just as much for your code.

One of the great things about the Rails ecosystem is that there are lots of different tools that can save you time and pain during development. Some of them work by embracing and extending the Rails standard bits, like replacing (nearly) bog-standard ApplicationController subclasses with subclasses of InheritedResources::Base. Some of them instead swap out pieces of the original Rails stack, like replacing ActionView with Erector or something similar. I wouldn't be at all surprised if my current app that mixes the two practices (in different areas) isn't at all unusual.

But whichever route or routes you choose, you need to constantly remain aware of that choice, especially when considering new tools you'd like to incorporate into your app. And you may also find that the tools you're using, while superb for their intended purpose, complicate other choices you make in the course of developing your project. A tool which was the greatest painkiller since morphine when you first picked it up, now seems to be "getting in the way" as you get farther along in development.

As practitioners of agile development, this shouldn't scare or even surprise us (though our non-technical stakeholders may initially be less sanguine). Good agile architecture says that everything in your app other than the raw business rules is an implementation detail that should be testable and fully replaceable in isolation.

Novice to journeyman Rails developers may be forgiven at this point for saying "but that's the way Rails is; it all works together, even though you can swap pieces out as you like". A majority of Rails developers have at least experimented with swapping out Test::Unit for RSpec, or erb for Haml or Slim. More adventurous still are the folks who swap out the ORM, replacing ActiveRecord with, say, DataMapper or the aforementioned Erector for ActionView. Why are they trying such things? Because they're asking (quoting Wynne again), "what does a modular Rails architecture look like?" Or, "we know we've hit the wall with our current (connected) architecture; everything we touch introduces new tech debt if not outright breakage, but we don't know how to do anything else and stay true to The Rails Way."

That "wall" you've hit has nothing to do with the number of lines of code you've written nor the number of unique visitors your production app serves per day nor the revenue your app is bringing in the door. You've realised that your ability to keep bringing in that revenue, let alone increasing it, is dependent on you somehow reconciling your understanding of "agile development" and "software engineering-style practices" with "the Rails Way". One of these things is not like the others. If you haven't yet, watch both of those videos. Particularly in Matt Wynne's talk from about 5:30 where he talks about the crossover point between a "connected" architecture being cheaper/faster/better giving way to a "modular" one… that rings a lot of bells for me. Likewise when Uncle Bob rails against Rails apps screaming "I'm a Rails app, never mind what I do". I've done several projects, never mind how many, where I've gotten to that point (which Wynne describes as "hang on; I don't want to work on that application anymore. Something's seriously wrong with it") and my reaction to the realisation that "something is seriously wrong" has been to stop, look at everything I'm doing and evaluating how well I'm following convention, "best practice" and so on, come to the conclusion that I'm doing everything exactly the way I've been taught it should be done, and deciding that that proves the fault is in my understanding of the problem, so I start over, from scratch, trying to figure out where I fell off the cliff. With one application, that involved at least five restarts over the course of a year and a half… and I must have always "fallen off the cliff" at the same spot, because I never saw it coming before I realised I was at the bottom of that cliff once again. "Uncle Bob" Martin's keynote (from Ruby Midwest 2011) does a great job of explaining why an experienced OO software developer like me has always felt uncomfortable with Rails' interpretation of the "Model-View-Controller" architecture. By writing your app centred on what the app is for rather than how the app presents itself on the Web, you make your life a lot easier and more enjoyable. Looking back at my commit history over the last year and a half, I see that well over 2/3 of the times I was "stuck" for more than a couple of hours had more to do with "how do I adapt what I'm trying to do to how Rails wants me to do things" than "how do I implement (this feature)". That should, and has been, setting off all kinds of alarm bells; I'd just drank enough of the Flavor Aid to think that pain level was "normal" for Rails. It shouldn't be.

Going forward, my strategy for Rails development is going to work something like this:

1. Model the domain objects (Martin's "interactors", "entities", "boundaries" and "entity gateways", after Jacobsen "control objects") as POROs (Plain Old Ruby Objects);
2. Wrap those up in Gems whose use can be demonstrated by specs, supplemented by a simple (Web or JSON or command-line; doesn't matter) example or set of examples.
3. Plug those into a Web app using the current framework of choice (Rails, Padrino, JoeBobsAwesomeFramework, or whatever), whose sole purpose is to implement the "client" side of the boundary layer and present what comes back over the fence.

Now we're back to something more closely approximating classical agile development that focuses on delivering business value, not playing mind games with a tool that insists on making most of the important decisions for you, and makes them in ways that make it harder for your app to be scalable, performant, or fun to work on. Or even for your app to be finished.

Thoughts?

---

P.S. Oh, and as far as InheritedResources::Base, that truly useful tool for Rails app development I mentioned before? We'll continue to use it as a default, but if we choose to use tools that "get in the way" (like wisper) by wanting all your methods explicitly defined, we'll switch back to ApplicationController, on the theory that making the puzzle one piece smaller without affecting how the other pieces fit together is a Good Thing™. But that's another decision we want to defer as long as possible; "good architecture" is all about deferring as many decisions as possible as long as possible, while still getting business value done.

Tools Save You Time, Except When They Don't

Two of the greatest boons yet to hit Railskind, IMAO, have been Platformatec's Devise and inherited_resources. Surprisingly, given how closely they're related, they don't always play nice together. If you ever get a failing feature spec that's just modifying non-password data on your user-profile page, it can really confuse you until you do some Googling around. (That's how you got here, yes?)

You'll need to update your User controller so that it doesn't send empty password fields to Devise for the update action. This won't affect creating Users; you want Devise to reject empty passwords there. Here's what I have for my users_controller.rb (also available as a Gist):

---

# Actions for Users. Not necessarily a finer distinction one way or another yet.
class UsersController < InheritedResources::Base
  # FIXME respond_to :json # should spec JSON actions, too
  respond_to :html
  load_and_authorize_resource

  def update
    # Devise has the convention that, for updates where the password is NOT to
    # be modified, the password and password-confirmation fields MUST NOT be
    # submitted. inherited_resources, of course, just sees two text fields and
    # wraps them up along with the rest of the form. Oops.
    remove_password_parameters if password_fields_are_empty?
    super
  end

  private

  def password_fields_are_empty?
    params['user']['password'].empty? and
      params['user']['password_confirmation'].empty?
  end

  def remove_password_parameters
    params['user'].delete 'password'
    params['user'].delete 'password_confirmation'
  end

end

This is still a heck of a lot cleaner/shorter than what I'd have written for a controller subclassed directly from ApplicationController. But is it universally better, or better for your specific needs? Thoughts on that in the next post.

FOCUS! Oooh, shiny, shiny...

That didn't last long; having a nice set of components to build on and being satisfied that it would carry us forward indefinitely…

As Ruby developers on Rails or Padrino or bare metal, we're absolutely spoilt for choice. As I write this, Rubygems claims 1,577,138,099 downloads of 56,183 gems since July 2009. (Compare to ~495 PHP packages in the official PEAR repo on Github.) If you can figure out what you're looking for well enough for a Google search, chances are excellent that you'll find something you can at least use; likely several somethings. As with all such, this is both a blessing and a curse.

The blessing is that the stuff is out there; it is mostly at least usable, especially for the gems that are at or past Version 1.0 (yes, there's a formal numbering system for gems); and if it doesn't do precisely what you want, chances are likewise excellent that you can wrap your own code around it to do your bidding, and/or pull a copy of the official source, make changes you'd like to see, and submit a pull request to tell the original developer(s) what you'd like to see pulled into the "official" version. ANd so on.

Back a couple of years ago when I was doing PHP for a living, it wasn't that hard to keep a rough idea of what was both available and usable on PEAR in my head, so that as I developed a project, I knew what I had available and could try to structure my code to work effectively with it. It's a lot easier when you can reverse that: think of what you want and how you'd like to work with it, and go run down the list of candidates to plug that hole.

But that touches on the other edge of the sword, and it's sharp. Doing all that browsing and cataloguing and selecting and integrating takes time and effort; time and effort that your customers (external or otherwise) generally assume you're putting into making a kick-ass product. Well, you are, sort of; you're just trying to figure out which giants' shoulders to stand on. Throw in the usual social media like Google+ and Twitter and so on, alerting you to the newest and shiniest of the new-and-shiny, and it's a wonder you get any "real work" done.

Memo to bosses: This is real work, and it's almost certainly less expensive real work than having your folks reinvent several dozen (or several hundred) wheels as they develop your Awesome Product, Version next.0. Deal.

But the "is it done yet?" bosses do have a very good point, devs. Promises have been made; contracts have been let; people's jobs (including yours) may well be on the line. Focus is necessary. You just have to learn to do it in a reasonably structured way, like your RSI breaks.

When you come across the latest shiny-shiny that you just know is going to make your job so much easier, better, more fun and profitable, add it to a list. When you're about to start on some new part of your project, glance at the list and see if it makes sense to use your new feature-thingy as a test bed for the new gem. Be careful that it can peacefully coexist with the rest of your codebase that hasn't been updated yet, and do remember to use a new feature branch. Set yourself a time limit for how long you'll allow yourself to get the new gem working. When that's expired, give yourself at most one extension (hey, you're exploring the not-yet-known-to-you, after all) and if it's still not walking and chewing gum at the same time, go back to where you branched and try again, the "old" way. This is so simple, yet not doing it can have a devastating effect on your schedule (and project, and…)

But given a choice between running the risk of distraction by a myriad assortment of Things That Generally Work, and trying to make one of a smaller number of Things That Should Work But Don't, Quite… well, that's what brought so many VB6 and Java folk to PHP in the first place, and what brought so many PHP people (including yours truly) into Ruby. Given sufficient resources and patience, you can make almost anything work in almost any language; I once saw a quite-playable version of the classic Star Trek game written in RPG II. I thought, and still think, that the guy who wrote it was absolutely nuts (especially since he knew at least three other languages that would have been easier/better aligned to the job), but *it worked*. So, as the adage goes, never argue with success. However, given the chance early enough in the project for it to make a difference, do respectfully suggest that jobs are best done with the available tools most closely aligned to the job. RPG is still in production for what it does best: complicated report generation from highly-structured data. PHP remains a valid choice for small projects, or for those companies unwilling to rely on non-local talent and willing to trade initial cost for quality. Ruby has numerous tools available, for just about anything, that challenge its practitioners to be better developers than they started out to be. Part of that challenge is the sheer wealth of tools available, and their haphazard discoverability. Part of it is looking back at code you wrote a few short months, or even weeks, ago; knowing it could be so measurably better; and having the self-discipline to leave it be until you have a business reason to change it.

Problems like that make my days more enjoyable than solutions in other languages generally have. Now if I could just turn this megalithic Rails app into something more componentised, service-supporting. Oooh, shiny, shiny…

It's Not Amtrak: Enjoy Riding the Rails

Historical note: Most of this post was written in the first week of October, 2011.

---

I've spent much of the last three weeks wrapping my head around Rails and Ruby after several years of mostly PHP with a couple of Groovy/Java side trips. It's been interesting, and a lot of fun so far.

I'm new to Ruby on Rails, but not to Ruby itself — though I took a long breather when Rails first became trendy. It was not at all clear to me at the time whether the (then-)popular trend among both the media and technology-involved people who Should Know Better to treat Ruby as essentially a synonym for "Ruby on Rails" was going to kill the language for other uses. It was quite clear to me that a huge chunk of early Rails culture was built around a cult-of-personality dedicated to Rails' primary creator, David Heinemeier Hansson. I try very hard to avoid cults; they almost invariably either spectacularly self-destruct (see Jonestown, Guyana) or get co-opted to serve unrelated or even opposing purposes (see State religions).

In about the last year or two, Rails has visibly grown up, both in capability and in outlook. There are other people who've stepped up and delivered great, widely-used code (does anybody know what happened to _why?), and Rails is now used in lots of systems both interesting and serious. I'd been thinking that maybe I ought to start getting myself up to scratch, particularly as I've been increasingly disenchanted with both the technical and sociopolitical aspects of PHP. The feelings of excitement, progress and everybody-pitch-in community that sustained PHP through its phenomenal growth in the last decade are, at best, flickering.

The final kick in the shorts was an interview cycle for new work with a great group that does, among other things, quite a bit of Rails work. Thus motivated, I grabbed copies of Agile Web Development with Rails, Fourth Edition and The RSpec Book and started going through them.

And promptly went out in search of better books. As of April, 2012, the ones I refer to most often are:

• Russ Olsen, Eloquent Ruby. Addison-Wesley Professional, 2011. Nearly 400 excellent pages that will help your Ruby code be more effective, expressive, pleasant to work with and easier to maintain; pick any five.
• Obie Fernandez, The Rails 3 Way; 2nd Edition. Addison-Wesley Professional, 2011. An encyclopedic yet guided introduction and continually-usable reference to Rails 3. Does not cover everything in 3.1/3.2, but will leave you comfortably able to adapt to the new changes.

One of the things I appreciate about Rails is the size and openness of the communities that have grown up around it. Everywhere I look, I find new code and tools to do useful things. I'm still on the upward slope of the "each Neat New Thing points to n other Neat New Things which point to…" curve, and I expect to be for quite some time. There are useful, useable if not always perfectly ideal tools to do things, and those tend to get adopted by (large chunks of) the community as de facto working standards. The whole "gem" ecosystem is chalk-and-Friday different (and dramatically better) than its PHP analogue; it's almost as though the "gems" folks looked at nearly everything PEAR gets wrong and made the conscious, deliberate decision to do the opposite. From the perspective of a newbie who's had painful experience with other systems, it Just (Seems Like It) Works.

Now, I'm sure that the rose-coloured spectacles will eventually pop out of their frames and get ground into dust. I'll find things I'm doing or using that just seem so bone-headed that I'll think, "I'm obviously well outside the ideal target audience for this". I seriously doubt that I'd think "This part of the system is so badly broken that it can't possibly be fixed"; again, this is a relatively new stack of tools with large, creative communities built around them. There will be alternatives to the particular whatever-it-is that's causing me pain, and there is a virtual certainty that I will be able to pull the offending piece out of my stack, put the new thing in, and continue working forward more effectively. I'm also convinced that, in the unlikely event that I'm such a black-swan corner case that I get motivated to write my own replacement, that it'll get visibility and feedback from others (if only to say "Hey, you just duplicated quteobscurename; go have a look at www.github.com/JoePalooka/quteobscurename — but hey, nicely done"). That would be more useful — and far more encouraging — than the usual PEAR mishmash of components that don't even all really work with the current language, written by people who believe that terse, sloppy, untested code is self-documenting.

---

Addition: April, 2012

And so it has come to pass; over the course of developing what will very soon be a commercial Web-based service, at least three initially-chosen components have been replaced by other, better and/or more suitable components. In all cases, the time involved in flailing around and deciding "this really has to go now" was several times greater than the hour or two it took to actually replace the old code with the new. That's been one of the big wins promised by object-oriented software development for roughly half a century. (Depending on who you ask, the first object-oriented language was either ALGOL X, in 1965, or Simula 67, in 1967. C++? Well over a decade later, and that's still older than most of its coders today.)

If At First You Don't Succeed, Try, Try Again

In a very real sense, that's a big part of what we do in test-driven or behaviour-driven development, isn't it?

It doesn't only apply to classes and projects. Teams work that way, too. You can have someone who, on paper, seems just great to help you out with the cat-herding problem that's a large part of any significant project. The interview is fantastic; he's 110% buzzword-compliant with what (you think) you need The New Hand to be able to pick up and run with. There might be a few minor quirks, niggles; a voice in the back of your mind standing up and clearing its throat obnoxiously, and when that doesn't get your attention, starts screaming loudly enough that surrounding folk reach for ear protection. When that doesn't get your attention — because, after all, you're talking to such a fine candidate — the voice might well be forgiven for giving up in disgust and seeking the farthest corner of your brain to sulk in. And give a good kick to, every now and again.

And then Something happens, usually a string of increasingly serious, increasingly rapidly-occurring Somethings, and you realise that this really isn't working out so well after all. The candidate-turned-colleague may well be a very nice person, but you just can't communicate effectively. Or the "incremental" test that she writes requires changes to a dozen different classes that, in turn, break 15 existing tests. If that doesn't make you go 'hmmm...', then maybe your other team members are concerned about his professionalism: consistently showing up late to meetings; giving a series of increasingly outlandish reasons for not coming into the office and working from home despite having previously promised not to, and so on. Seemingly before you know it, the project is at a virtual standstill, team dysfunction has reached soap-operatic heights, and at least one vital member of the team starts running around and shouting "off with her head!" All of which does absolutely wonderful things to your velocity and project schedule. Or maybe not so wonderful.

And the funny thing is that, at about that time, you remember that voice in the back of your mind that was trying to point out that something was rotten in this particular virtual state of Denmark from the beginning; you were just too optimistic, and likely too desperately stubborn, to recognise it at the time. If you're going to go about hiring someone, have a good rapport with your instincts. Know when to walk away; know when to run.

Long story short, we are looking for an awesome new team member. As you might have gathered from the last few blog posts, we're a Ruby on Rails shop, so you'd imagine that heavy Rails experience would be tops on our list, right? Not so fast, friend. I/we subscribe to the philosophy that source code (and the accompanying artefacts) are part of the rich communication that must continuously thrive between team members, past, present and future. These artifacts are unusual in that they are in fact executable (directly or indirectly) by a computer, but they're primarily a means of communication. As such, ability to communicate effectively in English trumps wizard-level Ruby skills. If we can't communicate effectively, it doesn't matter how good you are, does it?

What we're looking for at the moment can be summarised thusly:

1) Demonstrated professionalism. This does not mean "a long string of successful professional engagements", though obviously that doesn't hurt, especially if they individually lasted more than the sensible minimum. Do you generally keep your word? (Shtuff happens to everybody; nobody's perfect, but overall trends do matter.) Are colleagues secure in the knowledge that you routinely exceed expectations, or are they continuously making contingency plans?

2) Fluent, literate standard English at a level capable of understanding and conveying nuance and detail effectively and efficiently. This would be first on the list, were it not for the series of unfortunate events that led to this post being written at all. It is an absolutely necessary skill for each member of a truly successful team, and a skill that is becoming depressingly rare.

3) Customer-facing development experience, ideally in a high-availability, high-reliability environment. We're building a service that, if we're successful, people are going to "just assume" Works whenever they want it to, like flicking on the lights when you walk into a room at night. Those of us who've been around for a while remember the growing pains that many popular services have gone through; we have no plans whatever to implement a "fail whale".

4) Ruby exposure, with Rails 3 a strong preference. You can fill in along the edges as we go along, but if we can't have an informed discussion of best practices and how to apply them to our project, you're (and we're) hurting. If you're unclear on the concepts of RSpec or REST or Haml or _why, you're going to have to work really, really hard to convince us that we're not all wasting time we can't afford to. And, finally,

5) Deep OO experience in at least two different languages. If you can explain and contrast how you've used different systems' approaches to design by contract, encapsulation, metaprogramming, and so on, and have good ideas on how we can apply that to making our code even more spectacular, rock on. "Why isn't this #1 or #2 on your list," you might ask. If you can convince me why it should be, and why your skills are so obviously better than anybody else I'm likely to have walk in the virtual door... then you've already taken care of the others, anyway.

We're a startup, so we're not made of money. But since we're a startup, money isn't all we have to offer the right person. Know who that might be? Drop me a line at jeff.dickey at theprolog dot NOSPAM com. Those who leave LinkedIn-style "please hire me" comments to this post will be flamed, ridiculed, and disqualified.

Cover Yourself: Toolchains Are Agile, Too

As people who know me professionally and/or read my blog know well, I have been a (raucously) loud evangelist for test-first development (TDD, BDD, Scrum, whatever your flavour) for years now. If I write even an exploratory bit of code and don't have tests in place first, I get very uncomfortable. As complexity increases, without tests (preferably automated, repeatable tests), I argue that I simply can't know what's really going on, because I can't prove it.

A major corollary to this is test coverage reporting. If I can't see what's been tested and what hasn't, then in a very real sense nothing has been, since I can't document/prove what has been and what hasn't. And the better (more productive) teams I've worked in have established, and regularly hit, coverage testing better than 95%, with 100% being a common (and commonly attained) goal. (Edit: Note that this is for C0 and C1 test coverage; tools that cover C3 and C4 are rare to nonexistent in most languages, such as Ruby.)

As you may also know, I've been getting (back) into Ruby development, using Rails 3 on Ruby 1.9. Ruby's long-time de facto standard coverage tool for many years was rcov, which generally worked well. However, Rob Sanheim has stated that "RCov does not, and will not support C based Ruby 1.9.x implementations due to significant changes between 1.8 and 1.9.". He recommends either SimpleCov or CoverMe. Ripping out RCov and replacing it with CoverMeSimpleCov on a test project took all of five minutes and left me with attractive, functional, (so far apparently) quite accurate reports.

One of the basic principles of agile development is that the team must actively embrace constructive change as their project evolves. It's often easy for harried, hurried people to forget that that applies to their tools as much as it does to what they produce using those tools.

Just a thought as my evening winds down.

Well, that was fast.

Like many of you, I expect, I take a good look around at tools like editors when my needs change dramatically. A new system, or a language or app type I haven't worked with in a while, and I'll go out and see what the community is using (or at least buzzing about), narrow that down to a list of 2 or 3 to try, and start trying them out. Usually, I'll try one for a few days and then switch to the next one for a few days, until I've got lists of things I like and don't like, and make a decision. The last time I took a serious look at this was a couple of years ago, when I shelled out for Komodo IDE (which I still enthusiastically recommend to PHP developers in particular, by the way).

Recently, I've started working again intensely with Ruby, on a Mac instead of Linux as years earlier, and some quick research looking around mailing lists, group archives, and such, strongly suggested that TextMate was The Gold Standard.

But I'd never really done a head-to-head comparison, and I knew that BBEdit and RubyMine had passionate evangelists in the community.

So I downloaded the 30-day eval of RubyMine and fired it up.

I tried to open a Git project that I've worked with extensively in TextMate and from the Git command line. One immediate issue: copy-and-paste between the Mac pasteboard and the edit fields in the RubyMine dialog did not work. I then opened the project directory using RubyMine's "open a directory" feature; it found Git all right, but gave half a dozen red flags and refused to work further.

"What could cause such a reputed package to crap out so completely," I asked myself as I started browsing around inside the app directory. The answer became quickly obvious.

"Oh, it's a Java app." No native UI at all — though to be honest, you have to look closely at the UI widgets to tell.

It took less time to install the app, have it flame out, and uninstall it than it did to write this post.

The sooner we either a) rid the desktop world of Java or b) get the Java community to adopt usable desktop interoperability, the better. I'm years past the point where I care which option is selected, but one had better be.

Know when to walk away; know when to run

This started out as a reply to a comment on the LinkedPHPers group on LinkedIn; once I started writing, of course, it quickly grew beyond what was appropriate as a conversationally-inline comment. So I brought it over here. It's something I've been thinking about for a couple of weeks now, so let me get this anvil off my chest.

---

To R Matthew Songer1: I'd advise adding Ruby on Rails to that list of alternatives. I've been writing PHP for fun and profit since PHP 4 was still a future hope wrapped in hyperbole. Now that we've finally got a (mostly-)decent language in 5.3 for some serious software development, I'm burning out. Part of that burnout is due to what I see in the PHP community, part to geography, and part to other factors that approach "fit for use".

I've recently taken a post as Chief Engineer at a little startup nobody's heard of yet. Our prototype was done in PHP; it got the idea across well enough for our initial investors and customers to bang on our door. So the CEO and I thought, great, we'll find another senior2 PHP guy, write a real app (the prototype doesn't even have testable seams), and we're off.

If I were looking to hire a battalion of deputy junior assistant coders whose main qualification as PHP devs was being able to spell it whilst taking endless trips down the waterfall, I could do that. I was aiming higher: I wanted someone who knew his way around current best practices; who realises that being experienced is no excuse to stop learning aggressively; who understands how to build web apps that can evolve and scale; who looks for the best feasible way to do something instead of just the first one that pops into mind. I especially needed to find someone who was as BDD/TDD-infected as I am, and recognised the value of building tools and automation early (but incrementally) so that we'd be a lean, agile development machine when the rubber really needed to hit the road, a (very) few short weeks from now.

In the States, or in much of Europe, or basically anyplace outside Singapore, I'd probably have a decent chance of finding such a person. Here, not so much. I was especially concerned by the way PHP usage seems to be devolving in these parts. There are lots of folks who think that it's a good idea to reinvent Every. Single. Wheel. themselves, without really knowing all that much about what has gone before. Frameworks are a good example; if you aren't intensely aware of how good software, sites and Web apps get built; if you don't even bother with encapsulation or MVC or SOLID or on and on, how can you expect anybody who does know his Craft from a hole in the ground to take you or your "tool" seriously? It might wow the management rubes who proudly admit they don't know squat — but in the world as it is, or at least as it's going to exist in the very near future, those will (continue to) be a vanishing breed even here. Even in a Second World Potemkin village of a city-state where what you are and who you know is far more important in most situations than what you've done and what you know, you're still running a risk that somebody is going to come along who actually gets up in the morning and applies herself or himself to writing better stuff than they wrote yesterday. And, eventually, they're going to wipe the floor with you — either you as an individual or you as a society that remains stubbornly top-down-at-any-cost.3

In contrast, my experience talking with Ruby and Rails people here, even people with absolute minimal experience in Rails, is chalk-and-Friday different. For one illustrative example: one book that more than half the yes-I've-done-a-bit-of-Rails folk I've talked to here is Russ Olsen's Eloquent Ruby (ISBN 978-0-321-58410-6). Your average PHP dev — or C++ dev or Java dev — is fighting the dragons (schedule, complexity, communication, etc.) far too much to get out of fire-drill-coding mode and into writing. If you believe, as I do, that all truly competent software is written as a basis for and means of conversation between humans, and incidentally to be executed by a computer, then you know what I'm driving at here. Knuth's dictum that programming is a creative, literary act is too easily lost when you're just throwing yourself against the wall every day trying to see which of you will break first. (It's very rarely the wall.)

If you write software that can be read and understood, and intelligently expanded on or borrowed from, and your whole view of the omniverse changes. Your stress goes down, your written works' quality goes up, and you enjoy what you do a lot more. Automating away the repeated, detailed work and having a sensible process so you don't give yourself enough rope to shoot yourself in the foot regularly (with apologies to Alan Holub) is the most reliable way yet found to get your project schedule under your control instead of vice versa.

All this does however have one pre-supposition, which I have received numerous complaints on here locally: you and your team must be fully literate and fluent in a shared human language4. The vast majority of software to date seems to be associated with four such: Business Standard English, Russian, Japanese and Chinese. If your team shares a different language at a (near-)native level, with one or more of you having similar skills in the earlier four, you should be able to make do for yourself rather nicely. Having others build on your work, if desired, is going to be a bit more problematic. Poor communication has been at least a strong contributing cause, if not the root cause, of every failed project I have seen in my career. If you can't speak the customer's language effectively enough, with nuance, subtlety and clarity as needed, then your chances of project success are somewhat worse than my chances of winning the next six consecutive Toto jackpots — and I don't plan on buying any tickets.

Footnotes:

1. "I am sitting here tossing the options around for a business application, browser based, and trying to decide...PHP, Java, Python, .NET?" (Return)

2. Why insist on a senior guy when they're so rare here (at least the PHP variety)? Because I figure that with a senior dev coming on a month from now, we'd spend roughly half the time between now and our first drop-deadline writing magnificent code, and the other half building infrastructure and process to make building, testing and deploying the magnificent code we write straightforward enough not to distract the team from the creative acts of development. There's not enough time to wipe anybody's backside. (Return)

3. The phrase "at any cost" always reminds me of a company I contracted to back in the late '80s. My boss there had a (large) sign above his desk, "We will pay any price to cut costs." The company eventually paid the ultimate price — bankruptcy. (Return)

4. I'm describing what used to be called "college- or university-level language skills", but as any university teach will readily tell you, the skills exhibited by students have dropped precipitously and measurably in the last three decades or so. (Return)

DRY May Not Be Wet, But It Sure Is Cool

As any developer who values his time, sanity, or amicable relations with teammates who have to maintain his code knows, one of the cardinal rules of modern programming is "Don't Repeat Yourself", or DRY. This is obviously something that should make reading code easier. Obviously, therefore, it should be applied except where the lengths you go to to not repeat yourself make your code harder for someone else to read. The idioms of some languages help this more than those of others.

I was recently browsing through a (rather poor) Ruby programming book when this hit me between the eyes. Consider this ERB example from the book.

Pretty ugly, yes? Part of that ugliness is intrinsic to ERB; which is why people are leaving it in droves.

Compare this exact translation into Haml:

= form_for @product do |f|
  - if @product.errors.any?
    .error_explanation
      %h2= "#{pluralize(@product.errors.count, "error")} prohibited this product from being saved:"

      %ul
        - @product.errors.full_messages.each do |msg|
          %li= msg

  .field
    = f.label :title
    %br/
    = f.text_field :title

  .field
    = f.label :description
    %br/
    = f.text_area :description

  .field
    = f.label :image_url
    %br/
    = f.text_field :image_url

  .field
    = f.label :price
    %br/
    = f.text_field :price

  .actions
    = f.submit

A bit easier to understand, now that we've gotten most of the clutter out of the way, yes? Now, the repeating field definitions stick out like a sore thumb; they're almost the same, but not exactly. We've got three text_fields and one text_area, so a simple loop that just plugs in values won't quite cut it, will it? How about this:

= form_for @product do |f|
  - if @product.errors.any?
    .error_explanation
      %h2= "#{pluralize(@product.errors.count, "error")} prohibited this product from being saved:"

      %ul
        - @product.errors.full_messages.each do |msg|
          %li= msg

  - [:title, :description, :image_url, :price] do |name|
    .field
      = eval("f.label :#{name}")
      %br/
      = eval("f.text_field :#{name}") unless name == :description
      = eval("f.text_area  :#{name}") if     name == :description

  .actions
    = f.submit

If you're coming to Ruby from a PHP background, you've been conditioned not to use eval; there's all sorts of nastiness that could lurk there, especially when code is sloppy. (And PHP code is notorious for its sloppiness.) But in Ruby and Haml, this lets us solve a problem more eloquently than the two or three other possibilities that come to mind. Further, the idiom of the repeated-but-not-repeated text_field/text_area fragment

      = eval("f.text_field :#{name}") unless name == :description
      = eval("f.text_area  :#{name}") if     name == :description

should make it clear to even the most hurried reader that we're dealing with a special case. This is one of those situations where a helper method would be nuclear overkill. (If we were scattering dozens of these all-but-one-fields-is-the-same forms through an app, I'd revisit that stance.)

---

OK, but why blog about this in the first place? Well, the company I'm working with now is looking for a senior Ruby/Rails dev. Most of the CVs we're getting are from mid-level guys (if you're a Rails doyenne (i.e., female), please email!) who have done some Ruby and one or two other languages, generally PHP and Java. What I've learned, on both sides of the table, is that if you've got some experience in several languages that aren't all closely-related to each other, it's a lot easier for you to ramp up on any language you need. Any language will eventually go out of favour. Being able to code idiomatically in anything you need to, won't.

Once more into the breach, dear colleagues, once more…

…or How I Learned to Stop Worrying and Love Serving Aboard Kobayashi Maru; a history lesson.

Once again, I've had an interesting couple of months. Between modern Singapore's regular effect on my health, some insane work and the opportunity to get even more insane work if my two best references weren't indefinitely unavailable (but hey, Thailand is usually lovely this time of year…or anytime, actually).

Ahem.

I am rediscovering a love for developing in Ruby after losing touch with it some ten years ago. In the Ruby 1.5 days, things like class variables and the hook system were either new and shiny, or had finally been thrashed into something both usable and beautiful. If programming was what you did to earn a living, then programming in Ruby was something that you thanked $DEITY for every day because, after all, how many people in this world get to use beautiful tools that make you measurably better at your craft while blowing your mind on a regular basis, and get paid for the privilege?

The Fall

Then Ruby on Rails came along, with the clueless-business-media hype and cult of personality built around G-d Himself, or at least the three-initial version of same as anointed by said media, and Things Changed:

New acquaintance: So what do you do for a living?

Me: I write computer software, mostly using Ruby, or when I have to, Delphi or C++.

N.A.: Oh, you're a Ruby on Rails programmer.

So I left Ruby behind. Delphi lasted for a while, until its competitor's lock on the default system combined with some spectacularly bad corporate timing to relegate it to an "oh yeah, I heard of that once" niche. And then came this gawky new kid called PHP.

PHP wasn't exactly "new" when I first got into it; version 3.x had been out for some time, and the in-beta version 4 had class-based object-oriented programming. It was more primitive than an 1898 Duryea automobile to a Porsche driver, but you could see that the basic principles were at least in sight. And so, as the dot-com bubble was starting to really inflate, I jumped into PHP with a vengeance. It wasn't a product of one of the existing tech-industry corporate titans and it wasn't tied to a single operating system or Web server (though it did seem to work best with the then-early Linux OS and Apache server).

The great thing about PHP is that just about anybody can poke at it for a while, and at the end have something that (seems to) work. The pathologically execrable thing about PHP is that the barrier to entry is microscopically low, lower even than for Visual Basic 6 back in the day. And so, the inevitable result was (and is) that you have literally hundreds of thousands, if not millions, who get jobs by saying "Yeah, I know PHP; I've done (this little site) and (that steaming mess with a lot of Flashy bling on it)." In reality, there are thousands, at most, of good PHP developers out there, with a few more thousands actively working at improving their craft.

Purgatory

So PHP 4 had a crappy object model that anybody could poke at or ignore at will. PHP 5, from mid-2004, started to get its head on straight with respect to both OOP and what it really took to do PHP well, but lots of damage had already been done. Innumerable client projects had either failed or become incredible maintenance/performance nightmares due to the shoddy code that PHP, and the community that grew up around it, encouraged mediocre/inexperienced/inattentive developers to write.

And then, in mid-2009, PHP 5.3 came into the world, and it was a glorious golden statue with legs of brown, wet mud. Many of us who wanted to see PHP evolve into a "properly" object-oriented language, along the path that it had been following, found much to rejoice in. Support for closures. Better internationalisation support. Far better garbage collection. A rearrangement and winnowing of the extension and application repository system that was the closest thing PHP has to Python's eggs or Ruby's gems.

PHP 5.3 also introduced what had to have been the most-requested new feature for years: namespaces. Namespaces are one solution to the problem of allowing the development team to organise collections and hierarchies of classes to both make them easier to work with conceptually, and of mitigating possible naming conflicts (class Foo in namespace Bar is "obviously" distinct from class Foo in namespace Barney.)

However, this is also where the legs of the "golden statue" were transformed into wet mud: the way in which the PHP namespace features work is so semantically and visually jarring, with so many inconsistencies visible to both the experienced pre-5.3 PHP developer and the experienced developer of object-oriented software in other languages, that it quickly became a laughingstock and a millstone. A necessary millstone, but one of which many writers of both code and prose waxed eloquent in their righteous, intricately-justified derision.

A New Hope

And this eventually served as a wake-up call to a number of those who view PHP as just another tool in their toolkit, as opposed to either a cash cow to be milked or a semi-religious icon to be polished and cared for in the precise fashion that the High Priests of Zend decree, or, those who simply never learned enough to care. Seven to ten years is a long time to spend in any one language for an experienced developer, and quite a number of highly-visible PHP community stalwarts have been publicly participating in and contributing to other communities: various JVM languages like Groovy, Scala and Clojure; Objective-C; C#; and Ruby.

A few short months ago, I had urgent need to re-immerse myself in Ruby, learn modern Rails, and make myself ready in all respects to participate in Ruby on Rails projects at a senior or leading level. This gave rise to a series of fortunate events, to paraphrase Lemony Snicket. I discovered that Ruby 1.9 is now a very advanced, mature language with solid experience-based best practices that are sensible and self-consistent. I discovered that Rails 3.1 is an incredibly productive way to get a Web site or application up and running, and that the ways in which it encourages you to code and think do not have the same propensity to inflict mortal wounds as, say, the overly-trusting PHP journeyman developer. Rails itself has grown to be a much larger community that no longer revolves around a single individual, or even a small group of individuals, as that it did seven years ago or as too many other languages do now. And, importantly, many of the things that make Rails great would not be practical, or even possible, were it built on and in any other language than Ruby

Above all, Ruby gives hope to those few, mindful of Knuth's saying that "programming is a literary act", understand that computer programs are written to be read and modified by humans, with computer execution almost a secondary concern over the life of a project. If you care about thinking creatively, if you enjoy having your mind regularly blown in ways that challenge you to actively and continuously improve your mastery of the craft of software development, you are going to love Ruby, and Rails.

An Evangelist Repurposed

And so, to any who are pondering a new Web project, who see how overwhelming the mindshare of PHP is and how obviously negligent (even to non-technical eyes) is far too much PHP code and, by extension, those who made such code available, I would ask you to strongly consider Ruby, and Rails, even if your team has little or no experience in them.

After all, quality and intelligence should, in any world worth living in, be major competitive advantages — or, rather, by rights, their lack must needs be mortal.