Hempology

"the climate of contradiction that Toyota uses to stimulate creativity and problem solving" (bellware)

Aside from being a reasonable tagline for my personality, that line alone has convinced me that I need to do a lot of reading about what people are calling, Lean Software Development.

Sorry for the ultra-short post .. I'm too busy reading to do much writing.

My mother has been telling me this for as long as I can remember, but now Microsoft has joined the ranks of those who believe that I am "certifiable". How do I know? Well, like most of my posts, it's a painfully long story.

Throughout my career I've had opportunities to work directly and indirectly with a number of very impressive developers and developer managers. Of course, along the way there have also been those who were; well, less impressive. One thing that seemed surprising to me at first was that the possession of a certification did not appear in practice to be a good indicator of whether someone would fit best in the very impressive or less impressive category. That realization was my first clue that perhaps this certification business was just that, business, and had little to do with actual development skill or quality of experience.

Skepticism comes easily to me, so that bit of anecdotal evidence was enough to turn me off of certifications entirely. More recently it began to negatively skew my evaluation of job applicants who have multiple certifications due to suspicion that they spend more time reading and memorizing than they do applying what they've read. That suspicion plays off an assumption that certification exams only do one thing: evaluate how well a person has memorized the study material targeted for the exam. Here's the problem, without having ever studied for or taken one of these exams, that was a pretty big assumption to make.

Enter Tech•Ed. As recently mentioned, I was given the opportunity by my employer to attend this year's Microsoft Tech•Ed Developers conference. All in all it was a very enjoyable and personally beneficial experience. What's interesting for this story is that included in the materials emailed to me in preparation for the conference was a coupon to take a Microsoft Certification exam on site for only $50. Normally those exams will set you back $125 so that's a 60% discount. Definitely a good deal, if you're into that kind of thing.

At first I just archived the message (perhaps there's a future post in me on why I've stopped deleting) and ignored it. A few days later, though, as I was trying to think of things I could do on airplanes and in airports while traveling just about as far across the continental United States as you can travel, I remembered that email. "Hey, I could study for an exam on the plane, then take the exam at the conference." But if I thought those certifications were so silly, why bother taking an exam?

Simple, really. I needed to validate my assumptions. It has never sat right with me that I criticize certifications while knowing that I don't have any first hand experience at trying to earn one. From everything I had heard, the tests were quite difficult - and I believed it. So the only thing I needed to validate was that they don't prove anything beyond your ability to memorize (short-term) the specific subject material on the exam. Beyond that, it also gave me a specific goal for my time on the plane which I knew would be needed or I'd fall into my typical trap of reading valueless articles in whatever magazines I happened to bring along (seriously, when am I ever going to build new storage under my stairs? Our house doesn't even have stairs! Thank you, Popular Mechanics.)

Off to Microsoft.com I went to decide which test I should take. Then off to Buy.com to purchase the study kit. I did study on the plane for a few hours (though I underestimated how challenging that would be) and for a few more hours Monday and Tuesday evening after the conference sessions ended. In total, though, I would guess it was no more than 6 hours studying and some of that was likely through osmosis while my head lay prone on the open book.

The test I took (and passed) was 70-536: Microsoft .NET Framework, Application Development Foundation. It took me a little over an hour, and it was fairly challenging. What does passing that test get me? In day-to-day tangible items: absolutely nothing. In grander terms, it indicates that I'm eligible to pursue a variety of MCTS certifications by taking any number of additional exams (e.g. becoming a "MCTS: .NET Framework 3.5, Windows Presentation Foundation Applications" requires that I also take and pass 70-502.) Since I'm not actually certified without taking more exams, when someone asks me "what do you get for passing the test?" My best answer is, "it means I'm certifiable!"

Now that I've actually gone through the experience of studying (sort of), taking, and passing a Microsoft Certification exam, I feel better about applying my biases toward the program. There were some positive takeaways, though, such as the realization that in preparation for the test I studied a very broad array of topics (within the specific subject matter of the exam.) So although I still believe these certifications only prove a person's ability to memorize (short-term) the study material, at least I know that the material has exposed them to a fairly wide range of information. Whether or not I think they can effectively apply that knowledge in a real world environment, however, will remain strictly on a "show me" basis.

If you've never heard the name Helmut Bakaitis before, I'm not surprised. Until a few minutes ago, I hadn't either. In fact it wasn't until I decided to look up the actor who played one of the most pivotal characters in The Matrix trilogy that I stumbled upon his name. I also discovered that even though Helmut is an actor, outside of his performance in The Matrix he's not particularly well known.

This isn't really a post about an obscure actor, though. In fact, it's not even really about his character. Oh, that! I almost forgot. Yes, the pivotal character I mentioned but failed to name before is, of course, "The Architect" who reveals himself to Neo in the second film as the creator of the Matrix. "Creator" - now that word has a nice ring to it, don't you think? I have another related word, though. One which I prefer to apply to my work: designer.

Software architecture in its pure form is a very cerebral exercise. Essentially you are the person responsible for taking the overall goals and coming up with the plans or blue-prints which the implementers will use to build a solution. In that (overly simplistic) model, one person writes no code and another person does nothing but write code. Obviously the reality is something far less delineated in most environments, including mine.

However, I have noticed something which relates back to a post I made recently with regard to my productivity. What I noticed is that I seem to have far less issue with productivity when my work on any given day is balanced between architecture and coding. While in that mode I don't consider myself an architect nor a programmer, but rather a designer. It's that sweet spot wherein I'm able to take some time thinking heavily about my next steps and then immediately move to implement the plan I've devised, all the while identifying areas where the design may be weak or could be improved and quickly able to adjust the plans or completely re-architect them as needed. When I get into these cycles, those are the days when I'm most likely to get a call from Crystal saying "are you coming home tonight, or just sleeping there?" Even more telling is that I have to consider the options before responding...

It's one thing to identify these types of motivators, but I think the harder task is to find ways to use them for gain. For instance, having realized this about myself I've considered that rather than cramming all of my design-oriented work into one or two days, I may be better off to try spreading that work over several days or weeks if the task is large enough. In some cases that won't really be feasible, but I think the majority of the time it should be possible.

Determining whether or not this tactic actually produces a noticeable improvement in productivity would be difficult without some sort of metrics. That, of course, leads right back into my afore-mentioned post and the oh-so-loveable NPT form.

Geek Alert: This is a technical post for software engineers - those of you uninterested in such things, feel free to skip it (and all posts like it).

---

We do a lot of things in Software Engineering that seem to be overcomplication for its own sake. One of those is, perhaps, the principle of separation of responsibilities and concerns, partially described by SRP: The Single Responsibility Principle [PDF]. However, I believe most engineers agree that when used with care the benefits of this type of separation, more often than not, outweigh its costs over the life of an application. Certainly Martin Fowler would agree, and he's a kind of rockstar in the agile development world - which if any camp was going to be "anti-separation" I would expect it to be them.

Unfortunately, though, having strict separation creates a scenario not unlike the one depicted in the book, Harry Potter and the Goblet  of Fire, quoted as the subject of this post.

A Tablet PC application I worked on utilized a common MVC pattern. We had a View (the User Interface, made up of Forms, Dialogs, etc), a Controller (various controllers, actually, each with their own set of concerns), and a Model (in this case, a Business Object Layer (BOL), Business Logic Layer (BLL), and Data Access Layer (DAL)). Controllers manage critical UI behavior (beyond simple form interactions) and mediate between the View and the Model. The View does not directly cause action in the Model nor does the Model directly interact with the View (a possible exception being, well, exceptions, which if not handled at the controller layer will bubble up into the View).

There was an activity, a long-running activity, that happened in the Model layer. It prepared a large amount of data for export to another system by compiling it into a compressed file and, at the same time, updated the state of that data. That all happened within a single Transaction, such that if any part failed, the whole transaction was rolled back.

This all worked well, but a new requirement was introduced. The BLL (part of the Model) now had to determine as part of its processing whether the amount of data is "very large" (exceeds some configurable limit) and may result in a timeout during export. If the data is "very large" then the user is to be notified and given three options:

1. Ignore the warning and continue anyway.
2. Send the data to the local filesystem instead (for manual transmission via SneakerNet).
3. Cancel the export altogether.

The first two options are simple enough to handle. Since an export is going to be executed in either case, we can perform the first part of processing (gather all the export data and update data states), then validate the size of the export data, receive user input if needed, and finally perform the export (either to web service or local filesystem). Since these scenarios involve an entirely serial workflow, we just modify our sequence diagram slightly.

But wait a minute - there was a third option, right? The one that reads, "Cancel the export altogether." No problem - if the user wants to cancel then just don't make the ExecuteExport call, right? right? Not exactly. See - there's a problem. During the preparation step it updated the data states of all the data to be exported. This all happened within a Transaction such that any failure to complete the task would result in a roll-back of those data states. In fact, this same issue arises with the first two options as well, because there is no guarantee that sending the export data is going to succeed (even SneakerNet fails occasionally...)

This creates a bit of a pickle. Easily resolved, of course, by moving the Transaction Root out of the Model and into the Controller - but should a Controller really be managing transactions? In general, it is assumed that the logical place for transaction management is the BLL and it makes sense because that is where the truly "critical" business decisions are made.

If determining which changes are persisted and which are rolled back is a business decision (hint: it is) then transaction management ought to remain firmly planted inside the Model and away from any UI Controllers. And if the Controller can't host transactions, then it certainly shouldn't be responsible for canceling the export and rolling back data states. What to do?!?!?

The solution actually turned out to be fairly simple: fire a synchronous event from the BLL, which is handled by the UI Controller, that returns the user's response back to the BLL within the event arguments. This allows us to maintain isolation of transaction management in the Model while gaining user interactivity in the midst of a long-running process.

Here's the final sequence reflecting the addition of an event and evaluating the result of the user's response.