It’s not uncommon see tweets bouncing around the twittersphere describing how agile methodologies for software development are bad because of [insert problem here] and can be fixed by [insert solution here]. This week, a tweet by Ryan Singer caught my eye:
“Been talking to CTOs and Product Managers about their processes. Themes: low morale among engineers and the feeling they aren’t making meaningful changes to the product. Two-week sprints, Scrum and JIRA contribute to a high-overhead, low-productivity culture of micromanagement.”
Ryan correctly identifies an issue that every engineering manager needs to be cognizant of: engineers need to feel like they are making meaningful changes to the product or team morale will be low. In my opinion, however, his proposed solution misses the mark. In a subsequent tweet, he suggests that teams adopt longer sprints where engineers are free to define tasks and implement them how they see fit within a sprint. He’s proposing a Data Plane fix for a Control Plane issue.
In the world of routing, the Control Plane manages the rules and configuration of the system, including the current known network topology and other information necessary to route packets. The Data Plane forwards packets based on the information that has been provided by the Control Plane. The Data Plane is essentially the execution engine, while the Control Plane is providing the rules with which to execute. The Control Plane and Data Plane operate in separate threads and have very different traffic profiles. The number of requests to the Control Plane is relatively few, so handling requests can take orders of magnitude longer, but it’s important that every request is received and handled correctly. The Data Plane, on the other hand, is typically flooded with requests that must be handled quickly, but it’s less critical that every request is forwarded optimally because errors can often be handled downstream. The integration point between planes is typically the Routing Information Base (RIB), which is written to by the Control Plane and read from by the Data Plane.
In Scrum, the Control Plane is the work that is traditionally done by a Product Owner to generate the Product Backlog as well as some of the work that is typically handled by a Scrum Master to define the parameters under which the process operates (sprint duration, ritual details, etc.). The Data Plane includes all of the rest of the work that is done by the Scrum Master and the Development Team to pull work from the Product Backlog into the Sprint Backlog and execute on that body of work. Like routing, the Control Plane and Data Plane operate on separate logical “threads”; a different cast of characters is usually involved in the decisions made on each plane, and the cadence of delivery is different (eg. writing to the Product Backlog does not follow the Sprint cycle). The integration points between Scrum planes are the the Sprint Planning process, where work is pulled from the Product Backlog into the Sprint Backlog (the equivalent of the routing Data Plane reading from the RIB), and the Sprint Retrospective process, where feedback logically flows back into the Control Plane and can be used by the Scrum Master to change the parameters of the Scrum process.
Based on the context, Ryan’s statement that engineers aren’t “making meaningful changes to the product” could mean one or more of the following:
- So much time is being spent on Scrum rituals that engineers have no time to engineer.
- Project management tools like JIRA take so much time to use that engineers have no time to engineer.
- Engineers have no mechanism to give input into the Product Backlog; the Product Owner is dictating product changes without taking feedback and the Scrum Master is also being over prescriptive about implementation details.
#1 and #2 are Data Plane issues that I have trouble buying. On teams where I’ve run 2-week sprints, I’ve typically held a two hour Sprint Planning meeting, one hour Sprint Review meeting, one hour Sprint Retrospective, and fifteen minute daily Stand-up meetings. That means that <10% of our Sprint time was spent in team rituals. In my experience, the time necessary to keep task status up to date in JIRA or another project management tool once the Sprint has been planned is also trivial in comparison to the amount of time available in a Sprint. I’m not against making Data Plane optimizations to these processes, but there isn’t much to be gained.
#3 is a Control Plane issue that I have seen many times, but it cannot be adquetely solved by changes to the Data Plane. Engineers care deeply about the products that they work on and have unique insight into the details of the product and how it is implemented. It’s critical to inject engineers into the Control Plane by building mechanisms that give them a seat at the Product Owner table. On my team at Amazon, we do this a number of ways, including engaging engineers in our Working Backwards process (eg. having them assist in writing PR/FAQ docs for new features that we’re considering) and by including then in roadmap planning exercises as a key stakeholders and contributors.
If you want to boost team morale, avoid micromanagement, and ensure that team members can meaningfully impact the product, plug engineers into the product planning process instead trying to hack product planning into the middle of a Sprint.