Going Forward; Avoiding the Cliff While Looking for Cliff's Notes

Over on the Ruby sub-Reddit, I got into a discussion (mostly with Jelly_Jim, the OP, and with realnti) about the conflation in far too many minds of Ruby, the programming language, with Ruby on Rails, the famously-"opinionated" Web application framework. While pontificating on Jelly_Jim's original question, I described my frustration-bordering-on-antipathy with the traditional (known as "The Rails Way") application structure; listed a few presentations, books and blog posts that have influenced me greatly, and described my own take on the Kübler-Ross model for Ruby-based (and at least initially, largely Rails-based) Web development.

That discussion, over the course of roughly a day, helped me figure out how our app is going to survive the expected Slashdot effect, transitioning from a pretty traditional, megalithic Rails app to an app that uses Rails components in a sharply different, more recognisably "object-oriented", service-based architecture.

Leaving aside my more prosaic experiential difficulties with Rails (I really loathe ActiveRecord as commonly used), and taking into account the "Uncle" Bob Martin keynote, the various blog posts I referenced in the original Reddit post, and a couple of books I mentioned in a later reply, I think I've possibly hit upon a way to get where I want us to go.

That's the easy part, of course; it only took me never-mind-how-many months. The "interesting" part is going to be getting there — understaffed, on a shoestring budget for the moment even by startup standards (another problem I'm working), and working a schedule for two and a half years that would land a healthy twenty-year-old in hospital by now. Comments/suggestions are, as always, greatly appreciated.

---

Basic Architectural Principles

• Loosely-coupled, granular architecture FTW. That means, inter alia,
  • Nearly the entire app to be packaged as Gems used by an otherwise minimal application;
  • Rails mountable engines to package assets (particularly CoffeeScript/JavaScript) and major components;
  • Plan and design for updating and provisioning components which communicate across natural architectural seams.
• Hexagonal, "ports-and-adapters", or "clean" conceptual architecture; all dependencies point inward. Your domain entities, at the centre of the system, will change less than details like the delivery mechanism (Web UI, phone app, etc) or the database (which the domain shouldn't even be able to prove exists as such).
• By adopting a heavily job-story-oriented view of the workflows; with the use of online tools like Trello, WebSequenceDiagrams.com; and with supporting code tools like ActiveInteraction and so on, we should be a lot more productive and a lot more Agile than we have been during our forty (calendar time: two) years lost in the wilderness.
• And oh, yeah, just to keep things interesting: we've got to keep regularly demonstrating forward user-visible progress, for all the obvious reasons and a couple I really can't go into here.

---

How We Get There, v. 28.0

First, since the major focus of late has been on some unique (and not-so-unique) CoffeeScript code, start there. Separate the existing Script code out into four engine-based Gems, following/adapting Derek Prior's example. These levels are, from conceptual bottom to top:

1. A "gateway" CoffeeScript class we've written as a poor man's RequireJS, that gives us just enough pseudo-dynamic referencing of data/code without changing anything the Rails asset pipeline assumes;
2. Our directly DOM-aware Script code, as one or possibly two Gems (one is a major crown jewel; we might decide to split off utility/support code into a separate Gem, so as not to touch one when maintaining the other);
3. The code proxying our internally-used message queue (decoupling "FTW", remember?) and the Ajax code to communicate with the Rails back end; and
4. The code implementing various domain/service objects sitting atop the Script stack; one Gem per service.

Next, adapt the relatively-straightforward (as yet) Ruby/Rails code to honour the architectural principles listed earlier. This is where tools like ActiveInteraction pull their weight (and solve several existing pain points in the process).

Just to make sure we really do understand where we're going with this, take our first user story-specific code (including interactor) and package it up as a Gem that should then Just Work as in the previous step. Start thinking about whether we really want traditionally-packaged Gems or unbuilt dependencies that remain part of the single app repository.

Proceed to reimplement and repackage the (again, relatively few) job stories already existing as was done in the preceding step.

Start knocking down the remaining Trello cards as we complete our app.

We've been saying since the Very Beginning that anything that we were building that wasn't inextricably tied to proprietary, commercially-valuable code (or that gives too much of a hint into any unique details it may have) "should be" openly released, but hadn't yet figured out a feasible, cleanly reusable way to do that. If we've done our jobs properly up to this point, we have that way now.

Push the "deploy and launch" Big Green Button. We've bloody well earned it by now.

---

Any Suggestions or Comments?

If At First You Don't Succeed, Try, Try Again

In a very real sense, that's a big part of what we do in test-driven or behaviour-driven development, isn't it?

It doesn't only apply to classes and projects. Teams work that way, too. You can have someone who, on paper, seems just great to help you out with the cat-herding problem that's a large part of any significant project. The interview is fantastic; he's 110% buzzword-compliant with what (you think) you need The New Hand to be able to pick up and run with. There might be a few minor quirks, niggles; a voice in the back of your mind standing up and clearing its throat obnoxiously, and when that doesn't get your attention, starts screaming loudly enough that surrounding folk reach for ear protection. When that doesn't get your attention — because, after all, you're talking to such a fine candidate — the voice might well be forgiven for giving up in disgust and seeking the farthest corner of your brain to sulk in. And give a good kick to, every now and again.

And then Something happens, usually a string of increasingly serious, increasingly rapidly-occurring Somethings, and you realise that this really isn't working out so well after all. The candidate-turned-colleague may well be a very nice person, but you just can't communicate effectively. Or the "incremental" test that she writes requires changes to a dozen different classes that, in turn, break 15 existing tests. If that doesn't make you go 'hmmm...', then maybe your other team members are concerned about his professionalism: consistently showing up late to meetings; giving a series of increasingly outlandish reasons for not coming into the office and working from home despite having previously promised not to, and so on. Seemingly before you know it, the project is at a virtual standstill, team dysfunction has reached soap-operatic heights, and at least one vital member of the team starts running around and shouting "off with her head!" All of which does absolutely wonderful things to your velocity and project schedule. Or maybe not so wonderful.

And the funny thing is that, at about that time, you remember that voice in the back of your mind that was trying to point out that something was rotten in this particular virtual state of Denmark from the beginning; you were just too optimistic, and likely too desperately stubborn, to recognise it at the time. If you're going to go about hiring someone, have a good rapport with your instincts. Know when to walk away; know when to run.

Long story short, we are looking for an awesome new team member. As you might have gathered from the last few blog posts, we're a Ruby on Rails shop, so you'd imagine that heavy Rails experience would be tops on our list, right? Not so fast, friend. I/we subscribe to the philosophy that source code (and the accompanying artefacts) are part of the rich communication that must continuously thrive between team members, past, present and future. These artifacts are unusual in that they are in fact executable (directly or indirectly) by a computer, but they're primarily a means of communication. As such, ability to communicate effectively in English trumps wizard-level Ruby skills. If we can't communicate effectively, it doesn't matter how good you are, does it?

What we're looking for at the moment can be summarised thusly:

1) Demonstrated professionalism. This does not mean "a long string of successful professional engagements", though obviously that doesn't hurt, especially if they individually lasted more than the sensible minimum. Do you generally keep your word? (Shtuff happens to everybody; nobody's perfect, but overall trends do matter.) Are colleagues secure in the knowledge that you routinely exceed expectations, or are they continuously making contingency plans?

2) Fluent, literate standard English at a level capable of understanding and conveying nuance and detail effectively and efficiently. This would be first on the list, were it not for the series of unfortunate events that led to this post being written at all. It is an absolutely necessary skill for each member of a truly successful team, and a skill that is becoming depressingly rare.

3) Customer-facing development experience, ideally in a high-availability, high-reliability environment. We're building a service that, if we're successful, people are going to "just assume" Works whenever they want it to, like flicking on the lights when you walk into a room at night. Those of us who've been around for a while remember the growing pains that many popular services have gone through; we have no plans whatever to implement a "fail whale".

4) Ruby exposure, with Rails 3 a strong preference. You can fill in along the edges as we go along, but if we can't have an informed discussion of best practices and how to apply them to our project, you're (and we're) hurting. If you're unclear on the concepts of RSpec or REST or Haml or _why, you're going to have to work really, really hard to convince us that we're not all wasting time we can't afford to. And, finally,

5) Deep OO experience in at least two different languages. If you can explain and contrast how you've used different systems' approaches to design by contract, encapsulation, metaprogramming, and so on, and have good ideas on how we can apply that to making our code even more spectacular, rock on. "Why isn't this #1 or #2 on your list," you might ask. If you can convince me why it should be, and why your skills are so obviously better than anybody else I'm likely to have walk in the virtual door... then you've already taken care of the others, anyway.

We're a startup, so we're not made of money. But since we're a startup, money isn't all we have to offer the right person. Know who that might be? Drop me a line at jeff.dickey at theprolog dot NOSPAM com. Those who leave LinkedIn-style "please hire me" comments to this post will be flamed, ridiculed, and disqualified.

About me and my work at Cilix

I'm working on a lot of things for my work at Cilix, an engineering firm in Kuala Lumpur, Malaysia. First off, let me be clear on one thing: this blog is not officially sanctioned in any way by Cilix; this is 'just me'.

We call ourselves "A Knowledge Company". What that means, at least in my understanding, is that we apply professional knowledge and experience, augmented heavily by technology, to solve customers' knowledge-management and IT challenges. As such, we do a lot of writing — documents, Web pages, software, ad (nearly) infinitum.

We're a small shop as these things go, and our competition comes from much larger organisations with instant multinational name-brand recognition. Like any small firm, we have to win our first projects with a given client by promising — and delivering — a better value proposition than our competition. Where we get repeat business — again, like any similar firm — is by being agile, efficient, and above all, competent to the point of being unquestionably the least risky vendor for a particular solution.

Those attributes, in turn, lead us to consider issues like process, quality, and superlative knowledge of everything we are about. These issues, and how we as an organisation work through them, were what originally attracted me to the Company when I was approached and offered a position here. These issues are also the foci of what I expect to accomplish with this blog and the related collaboration tools (such as the Wiki).

---

I am also trying to evangelise and lead the implementation of open documentation and data-format standards at Cilix. This involves, among other things, migrating away from proprietary, binary formats like Microsoft Office documents to open, preferably text-based formats. As it happens, many of these open, text-based formats are based on XML vocabularies, such as Docbook and SVG.

Wby are text-based formats preferable? Lots of reasons:

• They are usually much more compact (and compressible) than comparable binary formats. Converting mostly-text Microsoft Word documents to Docbook equivalents often yields size reductions of 80% or more (think how much more convenient email attachments would be);
• They are usable with a wider variety of tools. I can throw a text file on my PalmPilot and fiddle with it far easier than a Microsoft Word document, for instance;
• The are more amenable to most version-control systems, particularly cvs and subversion. Instead of making copies of each version of a binary file, all that is required is to take the difference between two different versions of a text file — a much easier and more reliable operation. I have seen version control systems of all flavours — SourceSafe, cvs, Atria/Rational ClearCase — irretrievably corrupt binary files when insufficient care was taken by the configuration manager in dealing with binary files;
• They are more amenable to being stored in databases. Many databases (such as MySQL can return result sets packaged as XML fragments; this, combined with an XSLT parser and stylesheet, opens the door to some truly compelling presentation capabilities.

By taking advantage of these capabilities, we should be able to create better products with more predictable (and shorter) schedules without either greatly expanding the development team or pushing the present staff to the point of burnout. There is a saying in Silicon Valley in California, only partly tongue-in-cheek:

It isn't a startup until somebody dies

Here's hoping that's one "tradition" that's not exported anywhere outside the Valley.