Agile: Specialist vs Generalist

In his recent blog post, Peter Scheyen began a multi-part response to this comment. The blog post contrasted developer specialization and developer generalization in an agile context. The comment suggested that "... Agile robs us of specialization", and Peter's response described how specialization operates within an Agile team. I think the comment and the response come from different perspectives - one where Agile is not suitable (comment) and one where Agile is suitable (response). So allow me to step a little bit.

I wish to recast "specialist vs. generalist" to "Cog-in-the-machine vs. Member-of-Startup". Many large organizations have a decidedly non-Agile project management approach, and for many it works well. I think that specialization in a non-Agile environment is a side-effect of longevity or survival. If I were hired to be part of a large development organization, I would start off as a faceless cog - something that is interchangeable with any of the other cogs. The longer I stay in the organization, the specialized I will become, whether by accident (I worked on system X since day 1!) or design (I became the go-to guy for System X). Cast in pure evolutionary terms, this is a survival tactic, so the mere fact that I am still there (with the corresponding higher costs and "slowness") is because of something that makes me stand out from the faceless cogs. Specialization becomes inevitable.

The other extreme occurs if I were to be hired to a startup. In a startup, I may have some special skills, but everyone must work to pick up any work that needs to be done. In the startup, there is a chronic lack of staffing because the organization is short on resources. If things get left behind, the whole organization fails, so a weak team member is harder to accommodate. Generalization becomes inevitable.

This brings me back to the post: the ideal Agile team described is closer to a startup than anything else. The ideal team works like a startup, identifying and filling any gaps, regardless of specialty, but still leverages the team members with the greatest knowledge in any area.

I wrote a comment to summarize the above and it got me thinking about Agile organizations and more traditional organizations. The Agile team described in Peter Scheyen's post is ideal - the team has no distinct leader, just a set of developers with various skills and specialties that are engaged as needed. This works fantastically well when the entire development organization is "small" and the developers buy into the model. In the more traditional environments, there were team leaders, managers (at several levels), directors, etcetera, that had two roles: assignment of tasks and responsibility for the project. This is a very top-down model - the "top" decides to do something and the requests filter down to the ultimate workers who make it happen. If something goes wrong, there is a chain of responsibility up to the top that can easily be identified. Contrast this with the Agile team described - it is bottom up, so the team as a whole is responsible for getting a request done, but it has always been a bit murky to me on how a large Agile organization with many teams assigns tasks.

I don't want to make this post too long, so I will save this for my next post(Agile: Organizational Balance): How does an organization find the balance between purely distributed (self-organizing Agile teams) and purely leadership (top-down, waterfall) of large projects?

Contractual obligation?

I guess this is like a contractual obligation post - if I'm going to write every day, I've got to do it before midnight :) Ah well - I guess that's just how the holidays go; I don't really find myself with any kind of schedule. Once the major holiday gatherings are done, then it's chores and errands that don't normally get done. Plus lots of gaming. Got the ol' skates sharpened - 5 pairs this time. New record. My sister and nephew went skating with us before Christmas and I'm shocked she could even stand up properly. There was a huge bit of corrosion or something smack dab in the middle of the blade. Plus I can't imagine they were very sharp. So those are done.

Managed to get something else done I've been thinking about for awhile, which is to get some old stuff off of floppies and on to a CD or something. Specifically my old Turbo Pascal 6 kit. It's held up well so far and I was able to pull everything off the floppies. I'm not exactly sure how I'd install it, but I thought qemu or dosbox or something like that would work, if nothing else. Maybe wine on Linux? The next step would be to find some of the old source code... That may still be around somewhere as well... At any rate, maybe one of my boys will be interested in learning pascal, although my older son has used and enjoyed Scratch a fair bit. Getting the algorithmic mindset in place is more important than the specific language.

Opinions on Programming

Saw an interesting article on Slashdot (here's the full article). Of course I saw it's a great article because it lines up with some opinions I've held for several years. Guess it's more of a personal philosophy, probably shaped by learning how to program in text: you've got to know how things work if want to understand what is going on. It applies to so many different aspects of life too - it's shocking at times the trivial things that people don't know. Like how to change a tire - to understand what to do, you have to have an idea of how the tire is attached to the vehicle. Programming with pictures is all well and good, but there comes a point at which you have to reach deeper understanding of what goes on behind the pictures to ensure that things are operating as you wish.

I've had this discussion with one particular person and they pointed out, quite correctly, that there is a good reason why UML programming never took off. That's because the graphical development environments were always the encoding the wrong thing - the requirements. The "what" rather than the "how". The "what" parts are often contradictory, which is why I have a job. A software development professional is there to turn the "what" into the "how" correctly. Break the paradoxes and force the machine to do what is implied in the requirements, not do them literally.

So as one of the developers quoted in the article, graphical environments are good for learning. Small projects that show how things fit together. That first frustrating few experiences when the larger project is attempted and hours are spent to make two things mesh that won't work together. Then showing the text-based programming languages - the "how" behind it all. From what I've heard, any attempts at doing UML programming actually had two steps: First, design in the graphical environment and then second, tune the generated C source code to make things actually work. Very indicative of things to come.

I also appreciated what Herb Sutter had to say - that bare-metal programming and optimization will come back into vogue in the next 10 years. Waste is waste and graphical environments and elaborate abstractions are waste. And this waste can be translated into environmentally-relevant impacts. Smaller, efficient code will use less power, less space and be better all around. I still hope for the day when every chip-based interface responds instantaneously to my input. Even if it has to tell me that what I just requested will take a long time, that initial response shapes my interaction.

Day 12 - The Pudge Factor

One thing about the holidays is that whole "gaining weight" thing.  I am not and have never been the person with the tiny waist.  However, the last few months have been sitting rather uneasily around my middle.  The main problem is simply not getting to the gym.  It's kinda nice to avoid it right around January 1 as there are people in attendance that won't be in attendance in short order.  I'm gonna have to get back on track with that tomorrow morning.  Playing hockey a couple of times a week keeps things reasonable in terms of energy level and total-body-squishiness, but when I can get to the gym for the other 3 days of the week, that does a body good.  Especially in my business - the sitting down staring straight ahead business.  Like the salesman at "The Vast Waistband" on the Simpson's episode King-Size Homer:

Work, huh?  Let me guess. Computer programmer, computer magazine columnist, something to do with computers?

His speculation of "...it must be all the constant sitting and snacking..." is one of the great banes of the information industry.  Ah well - a job is a good thing - it pays for my blogging habit.

Anyhoo, I'm involved with an issue at work that seems to be gaining some traction vis a vis a root cause of the problem instead of an eternal examination of the symptoms.  The current line of investigation suggests the issue was a small problem that leads to performance issues over time.  This is the type of problem encountered at this stage of the project.  The software development is long done, much testing is over and we are left to sort through the "long tail" of remaining issues.  These are the bugs that arrise from complex or long-term interactions, exactly the problem in this case.  We've been working on this project for quite a long time and I suspect that many developers at my work have either passed by or never knew about the "embedded" aspect of our work.

The devices we work on are embedded, but (fortunately) they aren't the embedded ones I first learned on.  These machines have more computing power, memory and long-term storage than I had access to for the first 15 years of using a computer.  My first PC had 1 MB RAM, a 80286 processor and a 10 MB hard-disk.  These devices have 50 MB RAM for Java (plus much more for the underlying OS), hundreds of MB of Non-volatile memory and 300 MIP processors.  So I can understand how embedded ideals can be overlooked during development.  This is the stage of the project when those embedded design patterns pay dividends, or come back to bite us.

One of the best things that these embedded designs lead to is software that is compact, fast and resource-sparse.  These ideals will be more necessary when this software is more widely deployed and possibly developed upon.  Hopefully I can call some attention to this move away from the "small" before we begin the next project in earnest.

Practice

A colleague at work posted a link to this post by Steven Yegge. It was describing that the best programmers would practice the art and skill of programming. In it he drew a parallel to classical musicians, who have been developing methods of practicing music over hundreds of years. The two disciplines appear related, when approaching programming or musicianship as a craft.

I'll probably write again about this post (it's almost two years old now - must be some kind of blog-record, referring to a two-year-old-post), but I really wanted to say something about one of the comments there. The commenter indicated that he was learning the piano and that he found debate over the usefulness of Hanon or Czerny. These are books that work on technique and I am familiar with the Hanon especially. Hanon describes a series of exercises around each of the scales. From the comment:

seems that practicing techniques are still fairly hotly debated even now; some people maintain that non-musical drills such as those by Hanon and Czerny are worthless; it's better to use, say, Bach Two Part Inventions or Chopin Etudes, because technique should never be divorced from music.

So what is my problem with this? It really gets me when people state that "technique should never be divorced from X" where X is sport, music, programming, etc. There are several parallels to this that I personally have encountered, but I'll stick with the piano references first.

The most galling thing about substituting Bach for Hanon is that it makes Bach into non-music. So many people that Bach is very regimented - that there is one true way and that way proceeds in an orderly fashion. They leave no room for interpretation - they are renouncing that it is music! You might be able to tell that I really liked the Bach that I learned to play, but that is only part of it. Music can be interpreted and presented differently - the musician is part of the process. Bach is very technically demanding, so playing Bach will pay dividends in other music. This is precisely the thing that is being suggested - playing something technically demanding will improve your overall ability. This contradicts the reason to dismiss Hanon. Hanon's exercises are technically demanding, but aren't music (can you say: "Wax on, wax off Daniel-san"?). If they are technically demanding, then using them will improve the part of your playing that doesn't involve the musical interpretation part. Also lets you treat Bach like a piece of music that needs to be worked on and interpreted instead of exercised.

This same argument is made of sports. Let's pick, oh I don't know hockey. The argument illustrated by the commenter translated to hockey would be: don't practice skating, just play scrimmages. Clearly anyone who has tried to play hockey or learn to skate will see the folly in that (or foley). Skating is not something that is done alone while playing hockey - in fact it falls into the background while playing as there are other demands on your attention. However practicing skating in isolation can provide a tremendous boost. The reports I have heard from the power-skating classes that some of my work peers have attended show that they are doing things not related to hockey. Things like jumping over 6 pylons or jumping over pylons while doing a 360 in the air. I rarely find pylons at hockey games, so I'm assuming that the practice of skating and the practice of a game are different.

In sports, this is known as "the fundamentals". There have been times, in hockey, were the fundamentals were not stressed and I believe the professional game suffered. The lack of attention at the lowest levels was a problem. Much the same as the guitar analogies in the referenced material - you can learn to play the guitar without the scales and chords, but there isn't room to grow.

Bringing this back to the original post, I believe that the post is advocating the exercise of technical aspects in isolation. Solving problems, critical thinking, abstract thought, improvisation, pattern recognition - these are all things that help develop programming or software development skills. And I heartily agree.