It’s really quite simple to prevent innovation and ruin productivity on software development teams. Companies that send a clear message that initiative and experimentation are punishable and move the decision making away from the people building the software, create organizations where innovation and effectiveness are impossible.
In most companies struggling with these issues leaders say they want the organization to innovate. They very well might, however there is often frustration around why they can’t grow their organization and be as effective as companies like Netflix, Etsy, and Amazon. They don’t recognize how fundamentally different these companies are, both in the way they structure their work, and how they treat their employees. Instead, they assume these companies must hire better engineers.
The more time managers spend communicating and elaborating and being transparent about the job to be done, about the challenges the business is facing and the larger competitive context, the less important policies, approvals, and incentives are.
— Patty McCord, former Chief Talent Officer at Netflix and Author of Powerful; Building a Culture of Freedom and Responsibility
The source of the problem lies in actions of company leadership and the old playbook they are using to manage a software organization. If you strip away the thin veneer of generic company values that always include terms like “innovative” and start looking at the language and actions of management, the processes in place, and the system of work, it becomes obvious that the company is “optimized” for anything but innovation.
These companies take the Taylorist playbook and continue to apply it to creative knowledge work. With the rise of poor Scrum implementations (command-and-control waterfall projects wrapped in Scrum terminology) and Scaled Agile frameworks like SAFe, these problems thrive despite organizational claims about being “agile.”
One of the key attributes of agile teams is that they are staffed with all of the skills and roles necessary to take an idea from formation to delivery. Lean software development calls this taking an idea “from concept to cash.” But companies often want the benefits of agile without putting in the hard work to transform themselves, create the environment necessary for autonomous teams, and upend some entrenched company fiefdoms in the process. This results in doing work the Same Old Way, Command-and-Control waterfall organizations adopting the terminology of agile without any of the substance.
Companies often build what they believe are cross-functional teams, but these teams must take so much external direction that it serves to stymie all attempted progress. This destroys any chance for the team to innovate, and renders them helpless to deliver anything without first:
I regularly hear managers moan about “learned helplessness”—teams that are afraid to try stuff. If you tell the teams what to do, you're telling them not to try stuff. You can’t have it both ways. An org that imposes a company-wide framework is destroying innovation.— Allen Holub (@allenholub) October 1, 2018
The crux of this problem is that decision makers are separated from those who are doing the work.
When teams can decide which tools they use, it contributes to software delivery performance and, in turn, to organizational performance.
— Nicole Forsgren, PhD - Accelerate: Building and Scaling High Performing Technology Organizations
When coaching software development teams, one thing I focus on is helping to identify and minimize waste. One of the biggest sources of waste is the unwavering belief that for companies to grow and operate “at scale” an organizational optimization (and risk reduction technique) is to enforce organizational standards. This is often done by creating standards bodies, review boards, working groups, etc. This commonly results in the creation of several roles in the organization to create and manage these standards. The board members can’t possibly work on every team they are making decisions on behalf of. This separation of the decision makers from the workers creates broken feedback loops that slow down every team delivering software.
The company signals that it values identical teams and organizational sameness over creating empowered, autonomous teams. Consequently, morale plummets when the realization sets in that the teams responsible for doing the actual work are just typists, mercenaries doing as they are instructed to the codebase.
Teams of missionaries are engaged, motivated, have a deep understanding of the business context, and tangible empathy for the customer. Teams of mercenaries feel no real sense of empowerment or accountability, no passion for the problem to be solved, and little real connection with the actual users and customers.
— Marty Cagan - Missionaries vs. Mercenaries
Now this doesn’t mean that every team works in isolation and teams create inconsistency in the product in the name of autonomy. Rather, teams can organize around areas requiring consistency and tools can be built for teams to use that provide consistency (e.g., a logging service to format log messages in a similar fashion across teams). The difference here is that the team understands where consistency is required. They get to decide whether to adopt a tool that is supported for them and when they need to deviate and eliminate the dependency on another team’s tools. I’ll be discussing this nuance in more detail in a future post.
Those who have worked in highly productive learning organizations know that companies that centralize standards, decision making, and process change, crush innovation. It sends a clear message that innovation and initiative, at the team-level, is to be punished not celebrated.
"Punish people for their improvements and the process of ongoing improvement comes to a grinding halt." —Eli Goldratt #tocot— Goldratt Books (@GoldrattBooks) November 1, 2018
If a team needs to convince organizational leadership, a review board, or a team of architects of the merit of an improvement or experiment, they will simply stop trying to improve. There is too much friction involved in affecting change. Over time, these teams become apathetic because they don’t have the time or energy to deliver on their commitments and put in the effort necessary to create change.
One of the many important lessons we’ve learned from lean is that process improvement should come from the practitioners. Mantras like “Move Authority to the Information” exist to remind leaders that, in knowledge work, separating decision making from doing the work is a fool’s errand.
This may seem depressing, but there is hope. The most successful software companies today understand the creative, collaborative, and experimental nature of software development; that it’s a process of discovery. These companies have adopted principles and practices from DevOps, Lean Software Development, Extreme Programming, and Modern Agile. They focus on empowering autonomous teams, technical excellence, and continuous experimentation and learning. Consequently, they avoid the pitfalls of stamping out order-taking cookie-cutter teams, and separating the decision makers from the software creators.
If what I’m saying sounds impossible, I recommend a trio of books.
Mary and Tom Poppendieck’s Lean Software Development describes how centralized standards results in the opposite of their intended affect. The book is teeming with incredible insights and is at the top of my list of recommended reading for software professionals.
Nicole Forsgren (with Jez Humble and Gene Kim) have written Accelerate and are responsible for the annual State of DevOps Report. In it, they describe the research that indicates that among other practices, having a light-weight change review process like light-weight code reviews, pair programming and mob programming (as an extension of pair programming) is much more effective than review boards and centralized standards bodies.
The last book I will recommend is Marty Cagan’s Inspired - How to Create Tech Products Customers Love. In it, he describes how great software companies can maintain an innovative culture as they grow and operate at scale.