Going Forward; Avoiding the Cliff While Looking for Cliff's Notes

Over on the Ruby sub-Reddit, I got into a discussion (mostly with Jelly_Jim, the OP, and with realnti) about the conflation in far too many minds of Ruby, the programming language, with Ruby on Rails, the famously-"opinionated" Web application framework. While pontificating on Jelly_Jim's original question, I described my frustration-bordering-on-antipathy with the traditional (known as "The Rails Way") application structure; listed a few presentations, books and blog posts that have influenced me greatly, and described my own take on the Kübler-Ross model for Ruby-based (and at least initially, largely Rails-based) Web development.

That discussion, over the course of roughly a day, helped me figure out how our app is going to survive the expected Slashdot effect, transitioning from a pretty traditional, megalithic Rails app to an app that uses Rails components in a sharply different, more recognisably "object-oriented", service-based architecture.

Leaving aside my more prosaic experiential difficulties with Rails (I really loathe ActiveRecord as commonly used), and taking into account the "Uncle" Bob Martin keynote, the various blog posts I referenced in the original Reddit post, and a couple of books I mentioned in a later reply, I think I've possibly hit upon a way to get where I want us to go.

That's the easy part, of course; it only took me never-mind-how-many months. The "interesting" part is going to be getting there — understaffed, on a shoestring budget for the moment even by startup standards (another problem I'm working), and working a schedule for two and a half years that would land a healthy twenty-year-old in hospital by now. Comments/suggestions are, as always, greatly appreciated.

---

Basic Architectural Principles

• Loosely-coupled, granular architecture FTW. That means, inter alia,
  • Nearly the entire app to be packaged as Gems used by an otherwise minimal application;
  • Rails mountable engines to package assets (particularly CoffeeScript/JavaScript) and major components;
  • Plan and design for updating and provisioning components which communicate across natural architectural seams.
• Hexagonal, "ports-and-adapters", or "clean" conceptual architecture; all dependencies point inward. Your domain entities, at the centre of the system, will change less than details like the delivery mechanism (Web UI, phone app, etc) or the database (which the domain shouldn't even be able to prove exists as such).
• By adopting a heavily job-story-oriented view of the workflows; with the use of online tools like Trello, WebSequenceDiagrams.com; and with supporting code tools like ActiveInteraction and so on, we should be a lot more productive and a lot more Agile than we have been during our forty (calendar time: two) years lost in the wilderness.
• And oh, yeah, just to keep things interesting: we've got to keep regularly demonstrating forward user-visible progress, for all the obvious reasons and a couple I really can't go into here.

---

How We Get There, v. 28.0

First, since the major focus of late has been on some unique (and not-so-unique) CoffeeScript code, start there. Separate the existing Script code out into four engine-based Gems, following/adapting Derek Prior's example. These levels are, from conceptual bottom to top:

1. A "gateway" CoffeeScript class we've written as a poor man's RequireJS, that gives us just enough pseudo-dynamic referencing of data/code without changing anything the Rails asset pipeline assumes;
2. Our directly DOM-aware Script code, as one or possibly two Gems (one is a major crown jewel; we might decide to split off utility/support code into a separate Gem, so as not to touch one when maintaining the other);
3. The code proxying our internally-used message queue (decoupling "FTW", remember?) and the Ajax code to communicate with the Rails back end; and
4. The code implementing various domain/service objects sitting atop the Script stack; one Gem per service.

Next, adapt the relatively-straightforward (as yet) Ruby/Rails code to honour the architectural principles listed earlier. This is where tools like ActiveInteraction pull their weight (and solve several existing pain points in the process).

Just to make sure we really do understand where we're going with this, take our first user story-specific code (including interactor) and package it up as a Gem that should then Just Work as in the previous step. Start thinking about whether we really want traditionally-packaged Gems or unbuilt dependencies that remain part of the single app repository.

Proceed to reimplement and repackage the (again, relatively few) job stories already existing as was done in the preceding step.

Start knocking down the remaining Trello cards as we complete our app.

We've been saying since the Very Beginning that anything that we were building that wasn't inextricably tied to proprietary, commercially-valuable code (or that gives too much of a hint into any unique details it may have) "should be" openly released, but hadn't yet figured out a feasible, cleanly reusable way to do that. If we've done our jobs properly up to this point, we have that way now.

Push the "deploy and launch" Big Green Button. We've bloody well earned it by now.

---

Any Suggestions or Comments?

Trust, Tools, and Tripping Over Shoelaces While Dancing

On the shoulders of giants, that is.

While "a poor craftsman blames his tools", sometimes those tools lead you down the garden path to such a degree that you don't realise that you're doing a Wile E Coyote impersonation until about the third bounce off the canyon floor. For your amusement and edification, if not abject stupefaction, I present a subset of a passing spec file. This is CoffeeScript, using teaspoon and mocha for specs. It's deceptively straightforward, IMAO:

#= require 'gateway'

describe 'Gateway class', ->

  beforeEach ->
    @subject = new window.meldd.Gateway()
    @name = 'foo'
    @value = $('body')

  # other specs elided...

  describe 'has a "useGroup" method that', ->

    it 'takes one parameter', ->
      expect(@subject.useGroup).to.have.length 1

    describe 'when called using', ->

      it 'a nonexistent group name, returns an empty object hash', ->
        group = @subject.useGroup 'not found'
        expect(group).to.be.an 'object'
        expect(group).to.be.empty()

      describe 'a group with one key, returns an object', ->

        beforeEach ->
          @subject.register 'foo', 'bar', 'group1'
          @group = @subject.useGroup 'group1'

        it 'hash', ->
          expect(@group).to.be.an 'object'

        it 'with the correct single item name as the only key', ->
          expect(@subject.groups['group1'][0]).to == 'bar'
          expect(Object.keys(@group)).to == ['foo']

        it 'with the correct item value as the only value', ->
          expect(@group['foo']).to == 'bar'

      describe 'two groups with one identical key, returns', ->

        beforeEach ->
          @subject.register 'this', @, 'group1', 'group2'
          @group1 = @subject.useGroup 'group1'
          @group2 = @subject.useGroup 'group2'

        it 'the identical object in both groups', ->
          # equality to avoid "Converting circular structure to JSON" error
          expect(@group1['this']).to == @
          expect(@group2['this']).to == @
          # identity; they're the exact same object instance
          expect(@group1['this']).to.be @group2['this']

Pretty straightforward, and both of you Scripters reading this can probably code up a proper Gateway.useGroup method without knowing much about the rest of the class. I'd bet the rent that your method would not closely resemble this:

  useGroup: (param) ->
    {}

And yet...

What. The. Fox?!?

Tap-Dancing Among the Land Mines

…is best done very carefully, and with the knowledge that not everything works as you'd like it do. That's a good metaphor for my week this week.

Working on our new platform and its (eventual) variety of products and supporting tools, mostly in Ruby using Rails, some of the absolutely key elements are primarily front-end code, written as CoffeeScript running in the browser. Typically enough, we make heavy use of jQuery, various plugins, and other libraries. When understood and used correctly and they work as expected, they're great for reducing the effort needed to accomplish new and useful things.

The moderately-sized application I'm presently working on uses over 125 third-party components, many of which are dependencies of components we directly/intentionally use. Of those, some three dozen components have yet to reach a "1.0" release level. That's a lot of trust we put into people we've never met and teams completely outside our organisation. The fact that this is typical to the point of being mundane among Ruby shops is one of the phenomena most commonly cited by new Ruby devs and managers as "mind-blowing".

When everything works properly, it's awesome in a very different way than anything I've done in my nearly 35-year career. When anything fails to work properly, it's equally spectacular; just in a very different way.

Consider today. I've been working on a Script component (which I expect to publish in due course as a jQuery plugin itself). Being a proper behaviour-driven development project, I've been writing specs using Mocha and running them with Teaspoon. Teaspoon lets you run your spec suite either in a browser or from the command line. The browser-based interface is easy and convenient (almost fun) to work with during development; with a mouse click, you can focus on just the spec you're working with at the moment, or step back and run a group of related specs or your entire suite. After finally finding and killing the last of never-mind-how-many typos and silly errors (CoffeeScript, alas, has not yet implemented proper DWIM support), I had the component working. It was clean. It was more than fast enough for my purposes. And, most importantly, it worked properly, against every condition I threw at it.

That is, when I was running Teaspoon from the browser. The command-line invocation, needed for continuous integration among other things, failed in a very odd fashion. See, for example, this bit of screen output:

[jeffdickey@Jeffs-Prolog-iMac select_demo (wip)]$ teaspoon -q | tapout
Started w/ Seed: 0
..................F

1. FAIL markup from selectedMarkup()

    Unknown
    
    TypeError: 'undefined' is not a function (evaluating 'currentNode.contains(last)') in http://127.0.0.1:58247/assets/selection_wrapper.js?body=1 (line 111)
    
    window.meldd.SelectionWrapper for rich single-paragraph content, returns the correct markup from selectedMarkup().#:0


Finished in 0.092s (206.522 test/s, 0.004842s avg.)

19 tests: 18 pass, 1 fail, 10 exit, 0 todo, 0 omit

Now, bear in mind that when Teaspoon was running inside the browser, no such error message was displayed; all specs passed. The code being complained about looked something like:

  getMarkupUntil: (currentNode, contentString, last, offset) ->
    while currentNode?
      if currentNode.contains(last) # <-- THIS IS THE LINE BEING COMPLAINED ABOUT
        if currentNode == last
          # do something...
        else  # currentNode contains last but is not last
          # do something else...
      else # currentNode does not contain last but we're still looking
          # do something different...
    # return our results

Now, currentNode.contains is making use of a bog-standard HTML DOM function that's been in every major browser for years. Even Internet Exploder 5, from the Dark Ages of 1999, managed to support it! What could possibly go wrong?!?

Tap dancing on the shoulders of not-always-giants, it turns out.

Teaspoon, by default, uses PhantomJS as a "headless Webkit browser workalike with JavaScript support". If you're going to automate testing of your Web-based application (or any of several other related activities), this is the tool you want to use. Except, o course, when it doesn't work correctly. I changed the getMarkupUntil method I showed earlier to test my hunch:

  getMarkupUntil: (currentNode, contentString, last, offset) ->
    while currentNode?
      return null unless currentNode.contains?
      if currentNode.contains(last) # <-- THIS IS THE LINE BEING COMPLAINED ABOUT
        if currentNode == last
          # do something...
        else  # currentNode contains last but is not last
          # do something else...
      else # currentNode does not contain last but we're still looking
          # do something different...
    # return our results

The new line returns a null value from the function if and only if currentNode has no property (data or method) named contains. I reasoned that this would (fairly obviously) cause the calling code to fail, and the difference in the failure message (from what was shown previously) would confirm to me that this was indeed the problem. Again, running the specs from inside the browser presented no problems. Running from the command line, however, yielded

[jeffdickey@Jeffs-Prolog-iMac select_demo (wip)]$ teaspoon -q | tapout
Started w/ Seed: 0
..................F

1. FAIL markup from selectedMarkup()

    Unknown
    
    TypeError: Type error in http://127.0.0.1:58271/assets/selection_wrapper.js?body=1 (line 174)
    
    window.meldd.SelectionWrapper for rich single-paragraph content, returns the correct markup from selectedMarkup().#:0


Finished in 0.090s (211.111 test/s, 0.004737s avg.)

19 tests: 18 pass, 1 fail, 10 exit, 0 todo, 0 omit

As expected, the null value caused other code to die.

I then browsed the Teaspoon documentation, which pointed out that although PhantomJS was used by default, Selenium could be used instead, by installing the selenium-webdriver Gem and updating the project configuration accordingly. After re-bundling the app, I could run my spec suite from the command line. Hurrah!

Well, golf-clap applause at least. We'd had previous problems with Selenium testing the Script code as run from within a Web app. We'll burn that bridge again when it falls on us. Again.

"You use the tools you have, not the tools you wish you had."

(with apologies to Donald Rumsfeld, and noting that this is frustrating precisely because we're so asymptotically close!

I forget who first said this, but "Experience is as much a catalogue of what doesn't work as well as you'd hoped as what does" might have had my career in mind. Or at least the 2013 segment of it.

A few short months ago, I came across Trello, by Joel Spolsky's Fog Creek Software. As with all great Web products, it started as a brilliantly simple idea (a Web version of the classic stick-user-story-index-cards-to-it whiteboard that every Agile project centres around) that's slowly sinking under a tsunami of well-intentioned customer feature requests.

So excuse me if I seem to be a)piling on in b)an arguably hypocritical manner when I say that, like most things, it's both a blessing and a curse. You use it for a while, you start thinking in terms of cards and such… and then you realise that it's really not going to help you as much as you expected. Partly because of features, alas, but more because of philosophy, or communication of same.

Some features that I would gladly pay to be able to use:

1. There doesn't appear to be a "show me a list of all cards on this board, sorted by card number (creation date), last modification, and so on" feature. Apparently, the workflow Trello expects you to adopt is to have your to-be-done and in-work cards in various lists, and your "done" cards (and lists) should be archived. If, instead, you prefer to have a "Done" list, you'll quickly find that it grows into a mess that you're going to be searching through using the (very handy in itself) filtering feature, but that requires you to remember keywords exactly and, more importantly, prevents the "aha!" moments that often come when you can look at a list of not-very-obviously-related information and draw new insights from what you're seeing.

2. Trello also have (generally deservedly) made a Big Deal out of their support for "Markdown Everywhere", but their parser doesn't support the feature that future-proofs Markdown itself; the ability to embed HTML inline seamlessly. This is especially noticeable for things like tables, which have no Markdown (or Trello) direct implementation.

   This lack means, for instance, that "rolling your own" card that contains a list of all cards in the board looks pretty cluttered and hard-to-read:

   

3. Checklists are almost optimally useful and easy to use. However, they suffer from one major misfeature: converting a checklist item to a separate card "removes the item and creates a card with the title of the item beneath the current card". It would be very useful to have Trello, instead of deleting the checklist item after the card has been created, replace the checklist item's text with a link to the new card. Many (most?) of our cards start out as blue-sky, high-level description of a functional feature. For example, "Any User, or any Guest, may view a list of all Articles with public status." That card had a six-item checklist on it, several of which turned out to be "worth" breaking out to separate cards. For each of those checklist-items-turned-cards, I went back and re-added an item to the checklist with a one-sentence summary of the card and a link (e.g., "Update the `Ability` model; see Card #47."), followed by editing the new "item" card so that its description started out like "This is a checklist item from Card #44."

   Why? In our process, each card (most granular tracked activity) should take a reasonably consistent amount of time, generally under a man-day of effort. When a single card instead takes 4-5 days (as has happened), it's easy for customer reps and investors to get the idea that we've slacked off because they understand our estimating process/goals. Being able to drill down (and up) from any given card, without the level of effort presently required on my part, helps avoid misunderstandings and time lost responding to panicked emails.

4. Trello allows you to "subscribe" to cards, lists and boards. This adds items to a list viewable when you're logged into trello. It would be much more useful if these "feeds" were available via RSS/Atom as well.

tl;dr: Trello is a great tool; so great, that many users (including me) are either pulling it either in directions it wasn't designed to go, or are demanding features that don't exist in the way we'd like them. I'll keep using it and recommending it for now, even though the time involved in running a project with it is scaling up prodigiously. That time, however, is less than the time it would take to work without Trello or something quite similar to it.

Lies, Damned Lies, and Truths Backed By Statistics

(with apologies to Mark Twain and Henry Du Pré Labouchère)

UPDATED 6 October 2014; see below. Original content follows:

This afternoon, I came across a fascinating post by Prem Sichanugrist on the Thoughtbot blog, courtesy of the ginormous informational firehose known as Twitter. In it, he discusses a new feature of Bundler 1.4.0, currently in prerelease, that lets you use multiple cores to install the Gems for your project.

Finally, you exult. One of the slowest (read: most network-bound) tools in the Ruby toolbox is getting at least some help. In his post, Prem recommends "setting the size" (number of concurrent bundle subtasks to run concurrently "to the number of CPU cores on your machine".

Don't do that! At least, not with doing some more exhaustive testing first. Not because the feature doesn't work (it does, splendidly) but because matching the --jobs option to the number of cores may well give you suboptimal performance.

Benchmarks

Following is a summary of the results on my current project on my main development system, a Mid-2011 iMac with a 3.1 GHz Core i5 CPU (4 cores) and 16 GB of RAM under OS X 10.8.4. The command lines used for benchmarking were

for BIJOBS in 1 2 3 4
do
  export T1=`date`
  rm -rf Gemfile.lock vendor/cache/* vendor/ruby/*
  bundle install --jobs $BIJOBS --path vendor
  bundle package --all
  bundle install --binstubs
  echo $T1
  date
end

Time to Install Gems with Different --jobs Settings

Value of	Elapsed	Time Savings from	Time Savings in % from
$BIJOBS	Time	$BIJOBS	$BIJOBS
1	10m 41s	0m 00s	0%
2	6m 21s	4m 20s	40.56%
3	4m 44s	5m 57s	55.69%
4	6m 31s	4m 10s	39.62%

As you can see, the trend was progressing nicely for 2 and 3 cores, at ~40% and ~56% time savings relative to --jobs 1 respectively. When --jobs had the value of 4, however, the savings was less than when using only two cores. Why?

My conjecture is that Bundler suffers a (relative) drop in efficiency if it has to swap out use of one of the CPU cores between bundling Gems and everything else. I suspect that many people will find the advice

Set the value of --jobs to one less than the number of cores in your CPU(s)

to be optimal.

Conclusion

It's good to have our tools use the available hardware resources (CPU cores, RAM, etc) more efficiently. But we should always remember that the obvious "best" configuration is not always the actual best configuration.

Thanks again to Prem Sichanugrist of Thoughtbot, as well as to Kohei Suzuki who Prem credits with the original patch enabling multiple cores in Bundler. Thanks, guys!

---

UPDATE — 6 October 2014: Matthew Rothenberg has published new benchmarks and a discussion for how things have (apparently) changed significantly with the current-as-I-write-this Bundler 1.7.3 and apparently since 1.5.3. I don't have the resources to replicate his test environment (local propaganda is titled "From Third World to First"; we made it halfway) but I don't doubt his results. It will be interesting to see what insights may be drawn from repeating his benchmarks on slower hardware and a far slower Internet connection. I expect things to become I/O-bound far more quickly than in his results, but I'll be benchmarking and reporting again by mid-October.

Thanks again, Matthew!

Tools Save You Time, Except When They Don't

Two of the greatest boons yet to hit Railskind, IMAO, have been Platformatec's Devise and inherited_resources. Surprisingly, given how closely they're related, they don't always play nice together. If you ever get a failing feature spec that's just modifying non-password data on your user-profile page, it can really confuse you until you do some Googling around. (That's how you got here, yes?)

You'll need to update your User controller so that it doesn't send empty password fields to Devise for the update action. This won't affect creating Users; you want Devise to reject empty passwords there. Here's what I have for my users_controller.rb (also available as a Gist):

---

# Actions for Users. Not necessarily a finer distinction one way or another yet.
class UsersController < InheritedResources::Base
  # FIXME respond_to :json # should spec JSON actions, too
  respond_to :html
  load_and_authorize_resource

  def update
    # Devise has the convention that, for updates where the password is NOT to
    # be modified, the password and password-confirmation fields MUST NOT be
    # submitted. inherited_resources, of course, just sees two text fields and
    # wraps them up along with the rest of the form. Oops.
    remove_password_parameters if password_fields_are_empty?
    super
  end

  private

  def password_fields_are_empty?
    params['user']['password'].empty? and
      params['user']['password_confirmation'].empty?
  end

  def remove_password_parameters
    params['user'].delete 'password'
    params['user'].delete 'password_confirmation'
  end

end

This is still a heck of a lot cleaner/shorter than what I'd have written for a controller subclassed directly from ApplicationController. But is it universally better, or better for your specific needs? Thoughts on that in the next post.

Well, that was fast.

Like many of you, I expect, I take a good look around at tools like editors when my needs change dramatically. A new system, or a language or app type I haven't worked with in a while, and I'll go out and see what the community is using (or at least buzzing about), narrow that down to a list of 2 or 3 to try, and start trying them out. Usually, I'll try one for a few days and then switch to the next one for a few days, until I've got lists of things I like and don't like, and make a decision. The last time I took a serious look at this was a couple of years ago, when I shelled out for Komodo IDE (which I still enthusiastically recommend to PHP developers in particular, by the way).

Recently, I've started working again intensely with Ruby, on a Mac instead of Linux as years earlier, and some quick research looking around mailing lists, group archives, and such, strongly suggested that TextMate was The Gold Standard.

But I'd never really done a head-to-head comparison, and I knew that BBEdit and RubyMine had passionate evangelists in the community.

So I downloaded the 30-day eval of RubyMine and fired it up.

I tried to open a Git project that I've worked with extensively in TextMate and from the Git command line. One immediate issue: copy-and-paste between the Mac pasteboard and the edit fields in the RubyMine dialog did not work. I then opened the project directory using RubyMine's "open a directory" feature; it found Git all right, but gave half a dozen red flags and refused to work further.

"What could cause such a reputed package to crap out so completely," I asked myself as I started browsing around inside the app directory. The answer became quickly obvious.

"Oh, it's a Java app." No native UI at all — though to be honest, you have to look closely at the UI widgets to tell.

It took less time to install the app, have it flame out, and uninstall it than it did to write this post.

The sooner we either a) rid the desktop world of Java or b) get the Java community to adopt usable desktop interoperability, the better. I'm years past the point where I care which option is selected, but one had better be.

Don't Waste My Time.

What follows is a critique and a complaint, not a rant. What's the difference, you ask? A rant, in my view, is a primal scream of frustration and near-impotent rage that often doesn't let a few (allegedly) non-essential facts get in the way of a good thrashing. It also takes far more energy than I'm able to summon at present, for reasons that may become clear below.

---

As I've mentioned previously, I've been kicking the tires on the (relatively new) Kohana 3.0 application framework for PHP Web development. I'd previously used (and enthused about) the 2.3.x ("KO2") release; I was itching to get my hands dirty with 3.0.1 After a couple of dozen hours spent plinking away at it, however, I'm reevaluating my choices.

Two of the things that endeared the earlier system to me were the good-to-excellent, copious documentation and the active, always-willing-to-help user community (which, quite frankly, are the backbones of any open-source project).

KO3 is a different proposition entirely than KO2. As I write this, the eighth bug-fix update to 3.0 is available on the Web site. Since this is a new "the same, but different" project, all the users are starting fresh, so there can't be the same level of everyone-helps-everyone-else camaraderie as with KO2. This puts a foreseeable, far greater burden on the documentation to help people get productive as quickly and effectively as possible. However, the documentation, quite charitably, remains in a beta-quality state. This is true in comparison to both the earlier release and to other PHP Web frameworks such as Symfony2, CakePHP, FLOW3 and (notoriously) Zend. With most of these other frameworks, as with KO2, it was a quick, straightforward process figuring out how to get from the "hello-world" style demos, to being able to create a useful-but-simple site, to branching out from there. It's taken four times longer to get half as far with KO3 as with KO2.

Judging by comments in the Kohana support forums, I'm not alone in this; the documentation has been roundly panned by virtually all users who've bothered to comment. There's been far too much defensive "no, it isn't bad, you just need to read a bit more, and use the source, Luke" attitude from the project folks. During the KO2 lifecycle, the attitude I understood from their responses was more along the lines of "we know there are a few problems; we're working on them," quickly followed by "take a look at this." I don't know if 3.0 is so much more complex than 2.x that they simply don't have the bandwidth to document things to their previous standards. Frankly, I don't care any more.

I've decided that my existing projects that have been started in Kohana 2.3 will remain there; possibly moving to 2.4 when it becomes the officially-supported release. But I do not plan to invest any more time and effort into Kohana 3.0, at least until the new project has had some time to mature. I fully recognise the potentially self-defeating attitude inherent in that viewpoint. Virtually any open-source project depends on its community to "plug the holes" that the "official" project maintainers don't have time for or deliberately leave as open questions for the community. Well-run community projects are extremely collaborative, communication-rich environments.

Other projects take a "vendor/product" approach, essentially saying "Here's everything you'll need, soup-to-nuts; we'd really appreciate it if you built around the edges and took things deeper in your direction of interest, but the core that we're responsible for is solid." Those "vendors", the Zends and Sensio Labs of the world, have rich open-source offerings that they use as a platform to build (or offer) what actually pays the bills.

While I have a strong philosophical and experiential preference for community-driven (or at least -oriented) projects, there have to be times when I just want to Get Things Done rather than embark on an endless voyage of discovery.3 It is at those times that I'll reach for something that "just works" well enough for me to accomplish whatever it is I'm trying to do at the moment, whether it's to write a book or to bring up a client's Web site. I know and accept that any new tool, or new version of a tool I've used previously, will require a certain amount of time to "get up to speed" on. I don't know everything (or necessarily anything) before I learn it; the most I can hope for (and what I really do insist on) is that things make sense within the logical and semantic frameworks4 that they're expressed in, and that that expression is accessible, complete and correct enough that I can feel like I'm making progress. This invariably involves some sort of documentation; whether a "conventional" or online manual, a Wiki, or a user forum; the format is less important to me than the attributes I mentioned earlier.

Kohana 3.0, as it currently stands, does not meet that standard for me. And so I'm back in a feverish learn-and-evaluate mode with at least two other frameworks. I have projects to finish, and I have several chapters of a book I'm working on that had initially been written around Kohana 2, and will now need to be substantially redone.5

I intend to give each of those new candidate frameworks the same amount of time that it took me to get productive in Kohana 2.x (which was significantly less than the "industry leader," as I've previously mentioned). This is going to be an interesting week and a half or so, in the Chinese-curse sense of the word.

Footnotes:

1. Technical reasons for moving to KO3 included the switch from a model-view-controller (MVC) architecture to hierarchical MVC (or HMVC); if you know what these mean, you should know this is a Very Big Deal™. Additionally, I've found it a Very Bad Thing to tie myself to a legacy (obsolescent) project, and the end of KO2 is being made very plain on the Kohana Web site.(Return)

2. If the new, pre-release Symfony 2 code is as good as the documentation, we're in for a real treat.

3. I am typing this on a Mac rather than a Linux box, after all (though I have three Linux VMs running at the moment).(Return)

4. This implies, of course, that there are "logical and semantic frameworks" sufficiently visible, understandable and relevant to the task at hand. (Return)

5. I certainly don't want to fall into the same trap as a previous series of PHP books, which relied on the obsolete and inadequate Ulysses framework. Ulysses has been justly and productively panned, which has to reflect poorly on the aforementioned book (which I happen to own) and its authors. (Return)

Phing! It's a Dessert Topping! It's a Floor Wax! No, it's efPhing something!

I want my hour back. No, seriously; that's what happens when you don't touch a tool for a while, but you spend a lot of time with its "kissin' cousin."

Phing, if you're new-ish to serious PHP development, is your ultimate Swiss Army Ginsu Chainsaw™. It's a "build tool" that lets you automate pretty near anything, especially having to do with PHP. There are several dozen "tasks", like the PhpCodeSnifferTask, the IfTask, and of course the PhingTask. There is also good documentation on extending Phing; adding all sorts of new tasks and other bits, and people have run with the ball.

Put another way, if you're coming from the Java omniverse, Phing is analogous to Apache Ant, with at least as vibrant a community effort throwing stuff over the wall.

Sometimes the things that get thrown over the wall blow up, though. Sometimes that isn't Phing's phault, though that's the fish-eye lens you're looking through as you grapple with the problem. FOr instance, the PEAR coding standards as interpreted by PHP CodeSniffer require tags in the internal documentation you must produce to comply with the standard that are not supported by the documentation generator they're ostensibly intended for. You'll see your Phing script break on that rock — unless you "change the conditions of the test." Figuring out the (multiple) flags, option settings and occult incantations necessary to make things run smoothly will take you some time to figure out, unless you just did it last week and wrote copious notes in your wiki.

But the absolute break-down-and-laugh-until-you-cry moment came from the way Phing handles "custom properties," the symbols you can define to make your Phing life easier in various ways, like having common, shared policies and processes across projects, with specific values set on a per-project basis. You whip up a new "property file" as part of a new project, and reuse your existing XML "build file." Ah, but there is one sizable bump to trip the unwary or rushed: Properties are specified in a sensible XML format when included in the build file, but separate, included "property files" look like old-style Windows .ini files. And $DEITY help you if you forget, because Phing sure won't.

Let's say you have a "drop-dead simple" build.xml file, like this:

<?xml version="1.0" encoding="UTF-8"?>

<project name="demo1" default="demo">

<if>
    <equals arg1="${usepropfile}" arg2="false" />
    <then>
        <property name="foo" value="baz" />
    </then>
    <else>
        <property file="build.properties" />
    </else>
</if>

    <target name="demo">
        <echo message="Demo target; foo= ${foo}." />
    </target>
</project>

If you screw up the build.properties file — say, by specifying your values in the same sort of XML format:

    <property name="foo" value="quuz" />

you'll see a most sublimely confarkled message:

    ....
     [echo] Demo target; foo= ${foo}.
    ....

At this point, you'll either have a D'oh! moment, or you'll start chasing your tail. Choose Door #2, and you could be at it a while.

The answer, as often, is RTFM. Appendix F (!) of the Phing manual defines the "Property File Format," which is our venerable foil the .ini file with a few twists (like being able to define properties that incorporate the values of other properties – just like in build.xml).

Would it really have been too complicated to use the same format (i.e., the XML tags) in the property file as in the build file? You could even deal with making it valid XML with minimal effort. But no...

I actually did learn this properly a couple of years ago, when Phing was new and kind of shiny. It's only become more powerful since then. Just don't hold three lighted M-80s in your hand.

Why rant about this? I've long firmly believed that one of the main duties of a master craftsman, in any craft – including software development – is to occasionally fall into the weeds, pick himself up, and remind the young journeymen and apprentices "don't do what I just did; you'll make yourself look silly, or worse." Better for one person to do it who can be reasonably expected to pick himself up, than for a dozen others to fall in with no idea how deep they're getting.