Agile is 15 years old & counting… but what exactly is it..?
The Agile Manifesto is 15 years old now. That’s 15 years… No matter how you look at it, 15 years is a hell of a long time, especially in technology. A lot has happened over the last 15 years in the world of Software, Technology, Information Systems and the Disciplines that support (or supported) these areas.
In February, 2001 (when the Agile Manifesto was published), Google was barely a baby, Windows XP was the new OS kid on the block, Industry had just been through the Y2K hype (look it up), the Dot.Com crash had just happened (look that up, too!), MySpace – the first real, modern, Social Network) was barely an idea, FaceBook and BeBo (don’t bother looking that up) were far in the distant future, as was YouTube and Skype. Twitter, Instagram, SnapChat, Pinterest and WhatsApp were so far into the future of mankind’s imagination that they might as well have been time travel, nanobots or cheap 3D Printing.
It doesn’t matter what your view on the Agile Manifesto actually is and it doesn’t matter whether you think Agile is the solution to all our IT and Software Dev problems (which it isn’t), Agile is – by technology standards – very, very ancient.
Read on…
So, why is it only in recent times that Agile is still becoming more established, widespread and almost ‘conservative’..? Is it because it just works…? No. It certainly doesn’t “just work” and it certainly isn’t right for all situations. In fact, in some situations, an Agile approach can be dangerous.
What I like most about an Agile approach to any project – and this, I believe, is a major contributing factor in its ongoing adaptation and acceptance – is the fact that nothing is set in concrete. The basic philosophy of Agile, for a Business Analyst like me, is all about value and timing: source whatever you need just at the time you need it and then provide whatever is needed to other stakeholders just at the time they need it and to the level of detail it is needed at that time, on the way to providing value in some shape. Value, like beauty, is in the eye of the beholder. For that reason, we Business Analysts have to identify how much is just enough to provide whatever value we can to project participants.
If you only need a collection of 1-line User Stories to define scope in order to discuss what might be an acceptable solution and think about T-Shirt sizes (e.g. X-Small, Small, Medium, Large, X-Large), then that’s all that needs to be made available at that stage. This results in less unnecessary work. You can add detail later – when needed. If you need a fully defined set of integrated interfaces (e.g. for mission-critical software), then that’s what needs to be provided in that case. It still means less unnecessary work.
It’s no secret that Organisations differ in their implementation of the various ‘roles’ that are found in the various Agile methodologies (where multiple ‘roles’ are often undertaken by the same person or where multiple individuals share a given ‘role’, as required). This variation in roles depends on the type of Organisation, the Project and the people involved – in an Agile world, ‘each to their own’ is almost a Universal Constant. From conversations in various BA forums, it’s clear that real-world implementations of Agile within Organisations are as variable as the Organisations themselves.
In reality, almost every Organisation will adopt – and adapt – multiple processes & techniques from multiple methodologies as appropriate to their individual needs. In other words everything is a variant or a hybrid and “if it works, it’s right“. Given this attitude, it seems logical that each Organisation defines the activities that each ‘role’ should undertake in a manner that best suits what they believe to be the best for them. For those of us who move between different businesses on a regular basis, this can lead to confusion and a lack of clarity about who should be doing what. For example, for those who suggest that the Product Owner makes the call on which features are to actually be developed (in other words, a green-light & prioritisation role), then what role does the Product Manager play in these scenarios? Or is a Product Manager required at all? Maybe, maybe not. That depends on the specifics of the situation.
If a Business Analyst takes the role of ‘Customer Advocate’ on a customer-facing project, then surely that BA is the arbiter of what constitutes ‘value’ ..? If that is the case, and if value is the key prioritisation attribute in Agile, should the BA be a gatekeeper to the Sprint? Probably not – but it is not clear why not. To avoid such distinctions, should there be a Product Owner who also acts as BA? Maybe, maybe not.
Similarly, where does the Business Analyst actually ‘sit’ in an Agile scenario? Do you see only one BA (in the Dev Team) or one BA (in the Product team) or do you see two BAs – or more – who collaborate? Where does the Project Manager (or Project Owner) role fit in? Is it essentially the same as a senior Scrum Master or are they supposed to be ‘competing’ roles, with the purpose of creating constructive friction between each other, to make sure that what gets done is actually what needs to get done?
Is it different if the Organisation is a software development business compared to other sector? Almost certainly. In recent years, Agile methodologies have taken root in software development Organisations in particular. Although Agile ‘philosophies’ and concepts are not limited to software Vendors, it does seem as if there are often more ‘mature’ and ‘separated’ Agile-type methodologies in place where the Organisation develops software. This, however, is not as straightforward as it seems. Agile approaches to software development seem to be very much suited to organizations that produce their own software. Those who produce software for other businesses often can’t do so in an Agile manner. This is because of the need for the client to understand what they are getting, how much it will cost and when it will be delivered. Agile is not great for that level of (apparent) certainty.
As just one example of the roles and responsibilities in an Agile environment, the following hypothetical scenario is denvied from various engagements I have been involved in over the last 5-6 years…
NewCo (made-up name, of course!) has developed software for online payments, document archiving and electronic file transfer systems for many years. They are particularly expert in developing functionality for all things privacy-related and there is a huge knowledge of the various rules / regulations that must be complied with. Software is developed using the Scrum method and NewCo is responsible for its own strategic direction, supplying systems and services to a variety of big clients.
There are separate ‘Product’ teams for each of the product areas (or ‘verticals’, to use a business term). Each Product team has one or more BAs assigned to it. The BA is essentially the SME (subject matter expert) of how the product addresses the needs of the business, the customer base or others. These are not necessarily technical people (i.e Coders or Developers), being focused instead on Product features from a Market, Customer or Business perspective.
There is a Product Manager who has the job of understanding the longer term strategy of the organization, in terms of where they want their products to go in future. They will also be aware of plans for future products, clients’ custom requirements, feature enhancements and phasing out. The Product Manager briefs the Product BAs on these matters almost on a need-to-know basis (and what the BAs need to know will vary over the evolution, development and support of the Product).
There is also an experienced Tech Architect. S/he has a very thorough understanding of how all the various systems and subsystems in the organization actually work together to provide solutions. S/he also knows what platforms they are implemented on and what other technologies are common in the marketplace and customer base. The Product Teams (Product Managers & Business Analysts) define the high-level features the Products should ideally have and the rules & regulations the Products must comply with. This team prioritises the features according to the needs of the Market / Sector / Customer but they accept that other factors might affect actual delivery timeframe.
While the BA / Product Team do not devise potential solutions to be implemented (and, in some cases, they are not permitted to), the BAs usually at least have sone idea how something might be achieved – they know what a successful solution ‘looks’ like. This is not arrived at in isolation, however. Instead, the BAs and Product Team have one or more discussions with the Tech Architect and various specialist Tech Leads (such as DBA, Lead Engineer, InfoSys specialist etc) to discuss each requirement and get their view on how they might think about implementing it – if it can, in fact, be implemented.
…and this is often long before any Sprints are planned…
There is also a Project Owner. Their job is to manage the process of getting Epics and User Stories onto the Product Backlog, once they have been started by the Product Team. The Project Owner is happy to take one-liner Stories and Epics onto the Backlog because they won’t go any further. The Project owner then ‘ignores’ these Stories until the BAs and colleagues ensure that any Story they want to implement are adequately defined – to a level of detail required by the Tech Leads or others responsible for Estimation and Implementation.
The Project Owner has the final say on whether a Story is ready to be Estimated or needs further Elaboration (sometimes called Collaboration) but s/he will take the view of the BAs and the Scrum Master when doing so. The Project Owner is also aware of the multiple Development teams available and the various speciality skills available in the Organisation and this helps to identify the best team (and shared resources needed) to allocate Estimation and eventual development of individual features to.
The Estimation Sessions then allow those who will actually develop the solutions for the Stories discuss them, get some clarifications, suggest some solutions, provide some alternatives and so on. Because of this, the relevant Product BA is present in the Estimation Session, as is the Product Owner if possible. The Estimation Session might conclude with some Stories fully understood, fully Estimated and ready to go. Other Stories might be sent back for further investigation because someone has an unanswered question. In other words, if there are outstanding issues, the Story can’t be Estimated since it can’t be implemented as it currently is. Some Stories might simply be pushed back without discussion because of some unforeseen problem (maybe a technical or other impact one that wasn’t foreseen).
Once a Story is Estimated, preferably by the Team or Developer who will actually be implementing it, the Project Owner then eventually allows it onto the Sprint Backlog (which ins not the same as the Product Backlog, since the Sprint Backlog contains fleshed-out, estimated Stories that are ready to be developed). They also monitor the delivery of these features, once the Sprint starts, similar to a project manager.
In this hypothetical software development Organisation, there are a number of different Agile Development teams, some in remote offices and some in the same office as the Product teams. Each team has multiple skill sets: Developers, QAs, Architects etc. Each team also has a Technical BA assigned to it. The Tech BA is not the same role as the Product BA (even if they are the same person). The Tech BA is in daily conversation with the Agile Dev Team and may even be an occasional Developer themselves, or may simply be technically literate enough to help solve issues that raised during Dev.
A Scrum Master oversees multiple teams and collaborates closely with the Project Owner when planning an upcoming Sprint. The Scrum Master manages the daily Stand Up, handles the (re)allocation of tasks within the Sprint Development Team and is responsible for addressing any Dev-related issues that arise, often requiring access to expertise outside the Dev Team. As such, the Scrum Master performs a similar role (in addressing issues) as a Business Analyst or Product Owner but does so at a much more granular level and mostly with issues that arise from User Stories currently being implemented (e.g. In-Sprint, not on the Product Backlog or Sprint Backlog).
There are also shared specialist resources with expertise in areas such as DBA, Security and System Architecture, who are co-opted and queried as and when necessary, by either the Product teams (for Analysis & Elaboration) or the Dev teams (for Estimation and Implementation advice).
After each Sprint, the Scrum Master, the Product Owner and perhaps the BA / Product Team convene to review the Velocity of the previous Sprint, comparing it to the Capacity originally planned for, the pre-Sprint Estimated size of the User Stories developed (using any appropriate metric), any post-Sprint re-Estimate of User Stories ‘actual’ size (to avoid incorrect Estimates of similar stories later) and anything that could be changed (before or during future Sprints) to make future work more effective and efficient.
My personal experience over recent years suggests that a setup like this would work very well, especially once the two collaborative BAs (one in the Dev team and one in the Product team) have come to understand the ‘modus operandi’ of each other. For this reason, the two BAs should be often & repeatedly ‘paired’ over time to develop a shared understanding, thereby minimizing miscommunication and misunderstanding. A particular team should also regularly develop the User Stories defined by a particular Product BA – meaning that it becomes easier over time, due to familiarity, to explain / understand what is required. This makes Estimation more efficient & effective and ensures that the Product Team has a more direct and immediate understanding of what is to be implemented.
Of course, the hypothetical software development organization outlined here is only one possible arrangement. This hypothetical scenario may be completely unworkable in some cases, either due to resource constraints or – perhaps the opposite problem, in a way – scaling Agile for larger Organisations. However, noting that the various ‘roles’ outlined above could actually be carried out by either one person (for smaller, resource limited Organisations) or could be shared horizontally among similarly-skilled individuals (for larger Organisations), the actual structure itself is not difficult to adopt – and adapt – to the needs of most Organisations.
This, of course, is the essence of Agile.
