When I consider the experiences I've had with Agile, I found a very strong personal association with the Agile team and easily identify with roles, responsibilities and activities within an Agile team. I think it is an excellent way to organize a small group of 5-9 people. However that experience was embedded within an organization that had many groups of 5-9 people, all with varying ideas of what a team was responsible for and they should operate. I can't say that this is necessarily a problem, however I do not know what the best strategy the "layer above" could use to enable success for the entire group. Hence my question above - should the organization been structured as a recursive or self-similar entity where each of the "teams" were considered members of a meta-team, each group behaving as an individual developer in a simple "agile team"? Or should there be set of managers that treat the agile teams as members of their group and assign them tasks and mete out punishments?
Perhaps "punishment" is incorrect - consequences is probably better, but it is still something missing in the reading I've done on Agile. How does one handle a poorly performing team? How does one measure that output? From a team perspective, the team measures itself, but the team could be deluded or mistaken in its assumptions. I think I can see the comment postings now: "guide the team back to the right path by assigning a mentor or leader that can steer the team in the right direction." So maybe my initial recollections are not correct with respect to Agile and consequences, but it leads back to the idea leadership/management of many Agile teams: How does one make sure the projects stay on track?
One of the key items of Agile is communication between people. Probably the most critical communication junction is that between the implementors and the consumers. In a purely top-down model, the managers fill the role of "consumer" - the implementors talk to the people above themselves in the hierarchy and that's it. That abstraction can hurt the final product, but that abstraction (like any abstraction) provides flexibility: the manager can make absolute decisions and keep the project on track. I believe there must be a way to bring that flexibility into a large Agile organization without the loss of communication.
A hierarchical organization provides benefits - each layer can be thought of a leader with several "subordinates" that fulfill requests. This hierarchy does not preclude Agile - after all the best teams anticipate problems and act independently as much as possible. So I believe there must be a way to fit aspects of the hierarchy with Agile teams.
Why even bring this up? Consider a purely Agile environment with several (>= 10) teams. How would they organize the work? Each team would have to act as an individual member of a larger team. This larger team would define the overall backlog and each team member would take on work for the next iteration. Another way would be to designate one team to define the backlog and the rest to pull from the master backlog. What happens when disagreements occur? Disagreements can be resolved via communication between the effected teams, but what happens if one team is obstinate? More meetings and communication. The point is, to work in a consensus manner takes time, time which may not be available. Having a someone to that holds a "buck-stops-here" authority cuts down on this time. A decision has to be made and the quicker it is made, the better. Not everyone is going to agree with it, so just move on.
In fact, the Agile methodology is eminently suited for this quick-decision management - the organization must be able to take a new decision later if an old one proved incorrect. The benefit of this flexibility is lost if the initial decisions take too long. Have a single decision maker will speed the initial decision making process.
So what is the appropriate balance? My initial idea is that it requires someone who has authority within the company (as in the teams must heed the decisions and requests from this person), but who is not member of any team. This person would not define what needs to be done but would instead be responsible for the output of several teams. They could act on behalf of the teams in some instances, but would have to listen to the teams in others.
Yes very sketchy, but that is how many Agile things were described to me initially. The balance between authority and flexibility needs to be discovered within an organization. I believe this is a vital piece - one that was missing from the organization where I was first exposed to Agile. Moving to Agile, especially within a larger non-Agile organization, is a very tricky business. I'm just hoping to provoke some thought and comment.