"You use the tools you have, not the tools you wish you had."

(with apologies to Donald Rumsfeld, and noting that this is frustrating precisely because we're so asymptotically close!

I forget who first said this, but "Experience is as much a catalogue of what doesn't work as well as you'd hoped as what does" might have had my career in mind. Or at least the 2013 segment of it.

A few short months ago, I came across Trello, by Joel Spolsky's Fog Creek Software. As with all great Web products, it started as a brilliantly simple idea (a Web version of the classic stick-user-story-index-cards-to-it whiteboard that every Agile project centres around) that's slowly sinking under a tsunami of well-intentioned customer feature requests.

So excuse me if I seem to be a)piling on in b)an arguably hypocritical manner when I say that, like most things, it's both a blessing and a curse. You use it for a while, you start thinking in terms of cards and such… and then you realise that it's really not going to help you as much as you expected. Partly because of features, alas, but more because of philosophy, or communication of same.

Some features that I would gladly pay to be able to use:

1. There doesn't appear to be a "show me a list of all cards on this board, sorted by card number (creation date), last modification, and so on" feature. Apparently, the workflow Trello expects you to adopt is to have your to-be-done and in-work cards in various lists, and your "done" cards (and lists) should be archived. If, instead, you prefer to have a "Done" list, you'll quickly find that it grows into a mess that you're going to be searching through using the (very handy in itself) filtering feature, but that requires you to remember keywords exactly and, more importantly, prevents the "aha!" moments that often come when you can look at a list of not-very-obviously-related information and draw new insights from what you're seeing.

2. Trello also have (generally deservedly) made a Big Deal out of their support for "Markdown Everywhere", but their parser doesn't support the feature that future-proofs Markdown itself; the ability to embed HTML inline seamlessly. This is especially noticeable for things like tables, which have no Markdown (or Trello) direct implementation.

   This lack means, for instance, that "rolling your own" card that contains a list of all cards in the board looks pretty cluttered and hard-to-read:

   

3. Checklists are almost optimally useful and easy to use. However, they suffer from one major misfeature: converting a checklist item to a separate card "removes the item and creates a card with the title of the item beneath the current card". It would be very useful to have Trello, instead of deleting the checklist item after the card has been created, replace the checklist item's text with a link to the new card. Many (most?) of our cards start out as blue-sky, high-level description of a functional feature. For example, "Any User, or any Guest, may view a list of all Articles with public status." That card had a six-item checklist on it, several of which turned out to be "worth" breaking out to separate cards. For each of those checklist-items-turned-cards, I went back and re-added an item to the checklist with a one-sentence summary of the card and a link (e.g., "Update the `Ability` model; see Card #47."), followed by editing the new "item" card so that its description started out like "This is a checklist item from Card #44."

   Why? In our process, each card (most granular tracked activity) should take a reasonably consistent amount of time, generally under a man-day of effort. When a single card instead takes 4-5 days (as has happened), it's easy for customer reps and investors to get the idea that we've slacked off because they understand our estimating process/goals. Being able to drill down (and up) from any given card, without the level of effort presently required on my part, helps avoid misunderstandings and time lost responding to panicked emails.

4. Trello allows you to "subscribe" to cards, lists and boards. This adds items to a list viewable when you're logged into trello. It would be much more useful if these "feeds" were available via RSS/Atom as well.

tl;dr: Trello is a great tool; so great, that many users (including me) are either pulling it either in directions it wasn't designed to go, or are demanding features that don't exist in the way we'd like them. I'll keep using it and recommending it for now, even though the time involved in running a project with it is scaling up prodigiously. That time, however, is less than the time it would take to work without Trello or something quite similar to it.

Lies, Damned Lies, and Truths Backed By Statistics

(with apologies to Mark Twain and Henry Du Pré Labouchère)

UPDATED 6 October 2014; see below. Original content follows:

This afternoon, I came across a fascinating post by Prem Sichanugrist on the Thoughtbot blog, courtesy of the ginormous informational firehose known as Twitter. In it, he discusses a new feature of Bundler 1.4.0, currently in prerelease, that lets you use multiple cores to install the Gems for your project.

Finally, you exult. One of the slowest (read: most network-bound) tools in the Ruby toolbox is getting at least some help. In his post, Prem recommends "setting the size" (number of concurrent bundle subtasks to run concurrently "to the number of CPU cores on your machine".

Don't do that! At least, not with doing some more exhaustive testing first. Not because the feature doesn't work (it does, splendidly) but because matching the --jobs option to the number of cores may well give you suboptimal performance.

Benchmarks

Following is a summary of the results on my current project on my main development system, a Mid-2011 iMac with a 3.1 GHz Core i5 CPU (4 cores) and 16 GB of RAM under OS X 10.8.4. The command lines used for benchmarking were

for BIJOBS in 1 2 3 4
do
  export T1=`date`
  rm -rf Gemfile.lock vendor/cache/* vendor/ruby/*
  bundle install --jobs $BIJOBS --path vendor
  bundle package --all
  bundle install --binstubs
  echo $T1
  date
end

Time to Install Gems with Different --jobs Settings

Value of	Elapsed	Time Savings from	Time Savings in % from
$BIJOBS	Time	$BIJOBS	$BIJOBS
1	10m 41s	0m 00s	0%
2	6m 21s	4m 20s	40.56%
3	4m 44s	5m 57s	55.69%
4	6m 31s	4m 10s	39.62%

As you can see, the trend was progressing nicely for 2 and 3 cores, at ~40% and ~56% time savings relative to --jobs 1 respectively. When --jobs had the value of 4, however, the savings was less than when using only two cores. Why?

My conjecture is that Bundler suffers a (relative) drop in efficiency if it has to swap out use of one of the CPU cores between bundling Gems and everything else. I suspect that many people will find the advice

Set the value of --jobs to one less than the number of cores in your CPU(s)

to be optimal.

Conclusion

It's good to have our tools use the available hardware resources (CPU cores, RAM, etc) more efficiently. But we should always remember that the obvious "best" configuration is not always the actual best configuration.

Thanks again to Prem Sichanugrist of Thoughtbot, as well as to Kohei Suzuki who Prem credits with the original patch enabling multiple cores in Bundler. Thanks, guys!

---

UPDATE — 6 October 2014: Matthew Rothenberg has published new benchmarks and a discussion for how things have (apparently) changed significantly with the current-as-I-write-this Bundler 1.7.3 and apparently since 1.5.3. I don't have the resources to replicate his test environment (local propaganda is titled "From Third World to First"; we made it halfway) but I don't doubt his results. It will be interesting to see what insights may be drawn from repeating his benchmarks on slower hardware and a far slower Internet connection. I expect things to become I/O-bound far more quickly than in his results, but I'll be benchmarking and reporting again by mid-October.

Thanks again, Matthew!