Showing posts with label rails. Show all posts
Showing posts with label rails. Show all posts

Saturday, 7 June 2014

Rails, Circular Logic and Incremental Big Bang Architecture

(Initially published on my Tumblr, which I'm finding increasingly usable. Thoughts on Tumblr vs Blogger.com?)


Rails is a fantastic solution to a well-understood class of problem: the CRUD-oriented (or REST-oriented) query/report/update Web application, circa 2004–2006. Its philosophy of curation over configuration, despite having been likened to the report-producing languages of old (such as IBM RPG II, means that you really only have two things to worry about: your domain business logic and page transitions (which is another way of saying “your domain business logic, as it applies to presentation”).

And yet, the farther your app gets from that sweet spot,, either by decoupling the UI and using a fancy JavaScript framework; by needing to support user inputs other than typing into form fields and clicking buttons; or even the distance imposed by being nearly ten years on from the “sweet spot” and so many tools and other bits of code have changed around it — the harder it gets to do what you want to do in The Rails Way. Lots of people much smarter than me, from Robert C “Uncle Bob” Martin to Jim Weirich to Sandi Metz to Matt Wynne and a host of others, have spent many person-months thinking about, grappling with and writing about how to mitigate Rails, the knots it ties you into and the practices that much of the rest of our craft has built consensus on as being essential for good practice, that you can’t use in Rails without superhuman effort. Because the apps we’re building now, even the relatively simple ones, aren’t largely analogous to Basecamp (the application from which Rails was originally extracted).

So here’s the problem I’m facing, and the problem you’ve more than likely faced if you’ve done a non-trivial app in Rails.

If you’ve done any medium-to-large-scale development in other systems, especially in a decent OO language, one of your high priorities is going to be separating your domain logic from your “infrastructure” logic, including layering your presentational logic related to your data models separately from the data models themselves. Small is beautiful, and Ruby has some great tools to let you do that, including in particular the whole universe of Gems. Rails, however, makes it bloody hard to build what the rest of the craft would view as modular apps that follow SOLID principles. The retort to that is generally along the lines of “Rails is omakase” — which is fine as long as your app is living within that circa 2004–2006 sweet spot I mentioned earlier. The clock in the upper-right corner of my display here informs me that today is Saturday 7 June 2014 as I write this. Oops.

So we have great ideas like Hexagonal Rails , Architecture: The Lost Years and so on, that showed ways, with varying degrees of universality, to “get a handle on” the application/system you were developing.

But, can you answer what your application does? Not “what is its data model”, “what are the different types of users and permissions”, but a pithy sentence or two that gets across the idea of what makes your application different than every other CRUDtastic app out there, Rails or otherwise. Because if you can’t, none of this Rails-survival lore will help you.

And, equally importantly, as Jim Weirich points out in his Cincy.rb talk [Decoupling from Rails](http://youtu.be/tg5RFeSfBM4), is the question of “has this ever been successfully done before? If so, how? If not, what’s been the stumbling block that we have to stand back and work around using something else?” Because, in all fairness to Uncle Bob, if there aren’t examples of his built-on-Rails architecture in the wild for people to poke at, projector-ware can be inspirational but not terribly practical. For instance, as Jim points out at about 10 minutes into his presentation, Uncle Bob’s architecture shows where Rails controllers and views go, but ActiveRecord “models” don’t seem to be there at all*. Separating domain logic from persistence just happens to be one of the hardest nuts to crack in a Rails app to begin with. Hmmm.

Is the solution more decoupling? Jim spends the rest of that talk isolating and extracting what several teams publish as “interactors”, or support for what I call in my own apps “domain service objects” or DSOs. These help, a lot; having “skinny” controllers that don’t really know anything at all about your app beyond how to hand data off to classes that do lets you refactor your business rules efficiently by putting their control logic all in one place separate from Rails. Similarly, implementing façades or “decorators” with presentational logic around your models, as with Draper and others, helps you write clean, business-logic-free view templates.

But you’ve still got ActiveRecord to deal with in your models, and it leads you down the garden path and over the cliff with things like, oh, relations. When you have, say, a Blog model that has_many :posts and a Post model that belongs_to :blog, what you’ve got is a circular dependency; you can’t make changes to your Posts with certainty that those changes are isolated from your Blog, or vice versa, because each depends on the other. There have been numerous serious attempts to effect such a decoupling; perhaps the most widely known is Ruby Object Manager, whose Version 2 has been in the works for over two years now. It’s led by one of the greats in the Ruby community, Piotr Solnica, but it’s a Hard Problem. Especially when coexisting with Rails. People even resort to uncomfortable hacks like this to Get Things Done. There has to be a better way, and monorail (traditional Rails all-in-one non-modular app) is not that way.

And there are other, seemingly orthogonal issues to deal with. Authentication? Authorisation? Several solutions that are widely used exist for these in the “traditional” Rails “monorail” application style; how will adapting your architecture affect these, and how will you implement them?

Anybody?

Wednesday, 14 May 2014

Why is "doing things right" a "thing"? Whither professionalism?

Over on the Reddit r/programming subreddit, /u/jfalvarez posted a link to Tom Stuart's great presentation at the Scottish Ruby Conference 2014, entitled "The DHH Problem".

If you're involved with Ruby or Web development in any way, it's well worth three and a half minutes of your time.

On the Reddit post where Mr Alvarez posted the link to the video, another Redditor, dventimi, asked

Why is this a thing?

I disagree with the later poster who snarked "People like gossip." It's much, much more important than that.

What follows is the content of my reply to "why is this a thing?" I'd love to have a discussion with you about it, either here or on the original post.


Because people are finally waking up to the fact that the coding style and philosophy espoused by DHH are disastrous in non-toy applications maintained over any significant period of time.

That this should have been obvious to all, but wasn't discussed as such, once Twitter, after having conflated Rails with Ruby as a whole, left Ruby because they couldn't make Rails do what they needed to no matter how many times DHH came onsite and insulted their intelligence, speaks eloquently to how broken things are.

Most master software developers I've met and worked with have been mediocre-at-best promoters, including self-promoters. DHH has the opposite problem: his initial extraction of Rails from Basecamp's codebase "worked", for apps very much like Basecamp at the time. His fixed, what some have called "trolling", attitudes and the personality they describe, prevent him from seeing things from any other perspective or from growing his skills beyond a certain point. The echo chamber of acolytes and zealots his sublime self-promotional talents have allowed him to accrete around himself "ensure" that he never has to.

That's a problem for the rest of us, who have deadlines, work to finish within those deadlines, and paycheques we'd like to be able to cash that we won't get unless we meet those deadlines with that work. Especially for those of us who wish to improve our craft and, in whatever way we can, nudge ourselves and others towards helping the craft of software development more closely approximate a true engineering discipline.

Paying excessive heed to the opinions-masquerading-as-principles brayed by DHH, and twisting ourselves to fit inside The Rails Way, actively impede all of the above tasks. Tom Stuart was absolutely right to thank DHH for producing Rails and promoting it sufficiently that the rest of us get to write Ruby for a living…just before tearing DHH a new one, in the polite, discreet way only a well-educated Brit can, for getting so much of Rails and how software should be written so disastrously wrong.

Thank you, Tom Stuart. An entire industry may well be in your debt.

Tuesday, 1 April 2014

Rails is Rails is NOT Rails. Oh, my aching head.

For a couple of decades, I was into Linux. One of the ways you could tell good distributions and (especially) good documentation for those distributions is that they were (usually) careful to make notes like

This is a new feature as of kernel 2.6, first introduced in QuuxLinux in Version 6.0.

or

That API changed in kernel 2.4 and again in kernel 3.0, first introduced in QuuxLinux in Versions 2.0 and 11.0 respectively.

Now I'm a Ruby and (pro tem) Rails developer, and all that is old is new again, and you don't always know it.

I just spent a day and a half of highly-energetic tail-chasing, working at getting my code and specs for (what is so far) a "toy" demo project to understand Rails routing when namespaces are introduced. Namespaces, which are (as far as I can tell) functionally equivalent to Ruby modules, let you provide an additional level of separation between different parts of your app. They're also practically essential for writing Ruby Gems of any complexity, which is a basic requirement for "Vendor Everything" — an important tool in structuring your apps for maintainability and reusability. So "supporting namespaces" was an essential, if not universally obvious, part of the exploration that I'd embarked upon.

As I write in the project README, I want to "namespace everything". I also write "no code in the absence of a failing spec", so obviously my specs need to work properly with namespaces. Believing in six impossible things before breakfast is trivial in comparison, if you've never done it before.

Well, how better to learn than with a demo project? And how better for my current and future employers to judge my progress and reasoning than by making such (non-proprietary) explorations in the full glare of public scrutiny and ridicule, à la GitHub?

After accomplishing this first tiny step of The Mission, I'm even more convinced than ever that part of what makes Rails development "interesting" is that there's such a mix of information about Rails 2.3, Rails 3.x and Rails 4 out in the wild, treated as though they are slight variations on a common theme, rather than three different products unified mainly by a shared product name.

For those who care, the initial plea for help was on Reddit's /r/ruby sub-Reddit. Initially I'd written up this Gist to document the files I was using and the output I was getting. After reading /u/Nitrodist's response, I made some changes, as reflected in this Gist. Finally, after further prodding from Nitrodist and others, and intensive BFI, I determined the two changes needed to my original code. Documented in this final Gist, and pulling together information from several sources referencing different versions of Rails, this *early step* in the exploration has been completed — with several times the expected effort.

Things have to get better, don't they? Only if you're not aware that Captain Edward A Murphy was a raving optimist.

Wednesday, 19 February 2014

Going Forward; Avoiding the Cliff While Looking for Cliff's Notes

Over on the Ruby sub-Reddit, I got into a discussion (mostly with Jelly_Jim, the OP, and with realnti) about the conflation in far too many minds of Ruby, the programming language, with Ruby on Rails, the famously-"opinionated" Web application framework. While pontificating on Jelly_Jim's original question, I described my frustration-bordering-on-antipathy with the traditional (known as "The Rails Way") application structure; listed a few presentations, books and blog posts that have influenced me greatly, and described my own take on the Kübler-Ross model for Ruby-based (and at least initially, largely Rails-based) Web development.

That discussion, over the course of roughly a day, helped me figure out how our app is going to survive the expected Slashdot effect, transitioning from a pretty traditional, megalithic Rails app to an app that uses Rails components in a sharply different, more recognisably "object-oriented", service-based architecture.

Leaving aside my more prosaic experiential difficulties with Rails (I really loathe ActiveRecord as commonly used), and taking into account the "Uncle" Bob Martin keynote, the various blog posts I referenced in the original Reddit post, and a couple of books I mentioned in a later reply, I think I've possibly hit upon a way to get where I want us to go.

That's the easy part, of course; it only took me never-mind-how-many months. The "interesting" part is going to be getting there — understaffed, on a shoestring budget for the moment even by startup standards (another problem I'm working), and working a schedule for two and a half years that would land a healthy twenty-year-old in hospital by now. Comments/suggestions are, as always, greatly appreciated.


Basic Architectural Principles

  • Loosely-coupled, granular architecture FTW. That means, inter alia,
    • Nearly the entire app to be packaged as Gems used by an otherwise minimal application;
    • Rails mountable engines to package assets (particularly CoffeeScript/JavaScript) and major components;
    • Plan and design for updating and provisioning components which communicate across natural architectural seams.
  • Hexagonal, "ports-and-adapters", or "clean" conceptual architecture; all dependencies point inward. Your domain entities, at the centre of the system, will change less than details like the delivery mechanism (Web UI, phone app, etc) or the database (which the domain shouldn't even be able to prove exists as such).
  • By adopting a heavily job-story-oriented view of the workflows; with the use of online tools like Trello, WebSequenceDiagrams.com; and with supporting code tools like ActiveInteraction and so on, we should be a lot more productive and a lot more Agile than we have been during our forty (calendar time: two) years lost in the wilderness.
  • And oh, yeah, just to keep things interesting: we've got to keep regularly demonstrating forward user-visible progress, for all the obvious reasons and a couple I really can't go into here.

How We Get There, v. 28.0

First, since the major focus of late has been on some unique (and not-so-unique) CoffeeScript code, start there. Separate the existing Script code out into four engine-based Gems, following/adapting Derek Prior's example. These levels are, from conceptual bottom to top:

  1. A "gateway" CoffeeScript class we've written as a poor man's RequireJS, that gives us just enough pseudo-dynamic referencing of data/code without changing anything the Rails asset pipeline assumes;
  2. Our directly DOM-aware Script code, as one or possibly two Gems (one is a major crown jewel; we might decide to split off utility/support code into a separate Gem, so as not to touch one when maintaining the other);
  3. The code proxying our internally-used message queue (decoupling "FTW", remember?) and the Ajax code to communicate with the Rails back end; and
  4. The code implementing various domain/service objects sitting atop the Script stack; one Gem per service.

Next, adapt the relatively-straightforward (as yet) Ruby/Rails code to honour the architectural principles listed earlier. This is where tools like ActiveInteraction pull their weight (and solve several existing pain points in the process).

Just to make sure we really do understand where we're going with this, take our first user story-specific code (including interactor) and package it up as a Gem that should then Just Work as in the previous step. Start thinking about whether we really want traditionally-packaged Gems or unbuilt dependencies that remain part of the single app repository.

Proceed to reimplement and repackage the (again, relatively few) job stories already existing as was done in the preceding step.

Start knocking down the remaining Trello cards as we complete our app.

We've been saying since the Very Beginning that anything that we were building that wasn't inextricably tied to proprietary, commercially-valuable code (or that gives too much of a hint into any unique details it may have) "should be" openly released, but hadn't yet figured out a feasible, cleanly reusable way to do that. If we've done our jobs properly up to this point, we have that way now.

Push the "deploy and launch" Big Green Button. We've bloody well earned it by now.


Any Suggestions or Comments?

Sunday, 3 November 2013

No, You Can't Always Get What You Want...

…but as Mick Jagger and Keith Richards remind us, if you try "sometimes you get what you need". Any software developer lives with that on a daily basis; the implications of that are one of the major things that separate our craft from the profession of engineering. I had several of those "implications" blow up in my face this week as I was working on (what else?) a Rails app.

Or, more properly, an app that uses Rails. Because what the majority of my time has been spent with for the last few weeks has not been Rails, or even Ruby, but trying to get the front-end features that have to be implemented in JavaScript (or CoffeeScript or another language that compiles to JavaScript) (generically here, Script) because they're dependent on user-agent and DOM events. The last time I went down this road, I wound up with an app that had five lines of CoffeeScript for every two of Ruby; this one is well along that path. That's relevant; I suspect that there are far more non-trivial apps that use Rails in production than there are Rails apps; for anything beyond a simple pseudo-blog or relatively straightforward database front end, you sometimes wonder what the early Rails core team was thinking when they established some of these "conventions" that trump "configuration". Granted, there are a lot of people using that "sweet spot" for Rails app development, and more power to them, but I'd bet heavily that there are at least as many who started out trying to do an app that uses Rails and fairly quickly find themselves working around or against things as much as they work with them.

A prime example of this is front-end work. Rails has support for *Script and the asset pipeline as ballyhooed features. Once you start writing more than a few dozen lines of Script code in three or four classes or modules, you quickly come to the conclusion that the people who designed and held court over how Rails works really didn't have much more Script experience than using snippets of jQuery or Prototype.js to "jazz up" a form. Heck, the "unobtrusive JavaScript" feature is essentially useless outside forms. What I'm working with at the moment is an app that intelligently reacts to a user selecting content with the mouse, based on the position of the content, the permissions and authorisation of the user, and other factors. Having a few Script classes (and associated specs) to deal with that introduces issues that may not be readily apparent in the beginning, but will bite your team over time.

The JavaScript community have developed several module loaders to work around the lack of such a feature in the core language (for now, at least); there seems to be something of a transition underway from CommonJS to RequireJS, with several alternatives providing variations on the theme. This is important for several obvious reasons, including the ability that programmers have enjoyed in various languages since at least the 1960s to develop, test and maintain cooperating code modules (objects, classes, etc) relatively independently. By having each such module load, and know about, only the other modules it uses, those modules (should) remain unaffected by other modules in the system, even indirect dependencies that continue to work as their dependents expect. Programming 101 stuff; nothing new for anyone reading this.

But, and it's a very large but, the Web prizes a site's ability to perform as few network accesses as practical to get its resources loaded and start doing useful work. To this end, Rails since version 3.1 has included the asset pipeline, which has the effect (broadly) of concatenating and compressing Script files, CSS stylesheets, and other resources so that they can all be loaded in a single request. This is done using an add-in Gem (component) called Sprockets. This works quite well for its intended purpose and truly is one of the nicest things about working in Rails — until you start writing modular Script code.

If you know that the asset pipeline, used by default, means that everything is loaded by the time any of your Script front-end code starts running, but you want to break your code into modules, the "obvious" choice is to create a lot of "global" objects for your classes and shared data and such. If you're an experienced developer, this feels wrong, almost dirty. You might then try only having a single global that functions as a top-level "namespace" for your application, with artefacts spreading out in a sensibly-organised tree below that. This path leads to names like @com.hudsucker.proxy.activation.foo.bar.baz.joebob.and.his.third.cousin. Maintaining such, or even keeping it straight in your head (and your teammates') quickly becomes an "interesting challenge". You quickly realise that there must be a better way, simply because there have been other developers before you who have remained (apparently) sane.

But to use loaders such as RequireJS from Rails, you need components like requirejs-rails, which is wildly, justifiably "popular". Sane Script dependency management should be in Rails, you start thinking. And I'd agree with you. However…

The Might-As-Well-Be-Official Rails "Solution" to the problem is: don't do it that way. Write your front-end code in a traditional Script framework like Dojo or Node or Ember or JoeBobBriggsAmazingFramework or any of no doubt numerous other excellent choices. Use Rails "merely" as the back-end API server for persistence and auth and so on. And yes, with planning and preparation and an adequately-staffed and -trained team starting well before your deadline, that's a splendid choice. You may even realise that Rails is overkill for your needs and move to something like Sinatra or Padrino or, again, numerous others.

But…what if you don't have that time or that team, but you do have a mass of rigidly-organised, fragile Script code that you just promised would be part of a working app in two weeks' time? Assuming that you agree that resignation and suicide are suboptimal strategies, you could try what I'm in the middle of retrofitting our code to:

  1. Create a single, gateway object that you use to get all your other objects. Let's call it a Gateway object. give it a global location at @ourapp_gateway. Give it two methods:
    1. a register method, that associates a string name with an arbitrary object/class/other collection of bits;
    2. a use method, that takes a string parameter and returns the item associated with that name.
  2. Remove the Sprockets #= require modulename lines from each of your existing Script files, and replace them with immediate calls to @ourapp_gateway.use corresponding to each of the objects you need from Your Outside App, as well as calls to @ourapp_gateway.register to register the object(s) declared in that source file with the Gateway;
  3. Add Sprockets #= require directives to your application.js, so that Sprockets actually loads the suckers;
  4. Assign the return values from @ourapp_gateway.use to variables as would be done in Node or whatever.

Why bother, I hear you thinking. Because, even without a proper module loader, this still delivers at least one major benefit: it abstracts away the relationship between your artefacts, the DOM, and the asset pipeline. It gives you reasonable flexibility in moving things around within the DOM and with respect to source files by having the Gateway instance as an intermediary. And, if you're thoughtful, diligent and very, very lucky, it gives you a leg up when you do get around to using a proper module loader, by letting you slip that in behind your Gateway abstraction.

And if you think I'm getting somewhat jaded/disillusioned with my continuing Rails experience and integrating a Script-using front end that's not a complete app in itself…congratulations; you nailed it.

Anybody have any better ideas that they've actually made work?