Engineering for the Why

Folks Doing Stuff

Several years ago, I remember a particularly wise-for-his-age engineer telling me how members of our team just did things. “They don’t think! They just do!” he said in frustration one day. At the time, I wasn’t sure exactly what he was talking about (let alone how to fix it!) but I did have a general sense that something wasn’t right. At the time, I was a new-ish Engineering manager in a department that was rapidly growing and would have multiple fits and starts (and way too much drama!) over the few years I was there. I had a lot to learn. This complaint would come up over and over, and he would keeping saying members of the team “weren’t doing it right.” Last Fall, we got together for coffee and this observation came up again. Things hadn’t changed. Or maybe he hadn’t changed. But I have.

These days, I have a much better vocabulary to describe this behavior and a lot more experience doing it wrong and seeing it done wrong to truly get what he meant all those years ago. One way to describe the phenomenon is when a team focuses more on activities than on outcomes. I’ve seen this at multiple levels and in different roles from executives to product management down to individual engineers. This incredibly common way that organizations think about work manifests itself in various unhelpful behaviors that hold us back individually and as teams. Whether it is a product owner, engineering leader, or someone that checks in code daily, everyone appears busy. Stuff happens. Red/green/yellow “traffic-light” slide decks are produced. Tickets are closed. Customers may even get new features. Releases come and go. Deals might even close, but something is missing.

Just working is easy, however painful it may seem at times. But working on the right things (and haggling over what is “right”) is much harder. Getting consensus and clarity about what not to work on, and identifying who and when you can safely say “No!” to is even tougher. For some teams, dare I say impossible. It is just how we are naturally wired as humans. We want to please. We are generally conflict-averse. It is easier to take the well-worn path and keep our heads down and just do it.

Unfortunately, overly superficial Agile metrics sometimes help reinforce our attempts to stay busy and focus on activity over outcomes. What does your burndown chart look like? How many story points did the team complete this sprint? What is your say-do ratio? These aren’t particularly helpful in measuring outcomes, but they certainly prove things are getting done. Maybe. Sometimes they even make us feel good about ourselves and what we are accomplishing.

Eyes Wide Open

Being an individual contributor again for a few months now, I feel like I’ve been granted a gift to see things that I couldn’t see before as a manager, that sometimes I couldn’t feel as a leader. As an engineer again, I now have the white spaces on my calendar I could never seem to find when I managed people. Working in a slower-moving organization after years of fire-fighting gives you time to stop, think, and reflect.

Most importantly, joining an organization outside the leadership circle and with no accountability for anyone but yourself provides a very different perspective on what works and doesn’t work on teams and why because you simply don’t have the same skin in the game. Hearing the things IC’s share with each other and what they talk about compared to the discourse between managers, directors, and VP’s has been fascinating and eye-opening. It confirmed some of the suspicions I had when I was a leader that messages are too frequently misinterpreted, misunderstood, or simply tuned-out at the lower levels.

(This is obviously the point of asking lots of questions and doing the hard work listening and driving clarity in your 1:1’s and skip-level’s with your team, but I digress.)

Within high-performing teams, leaders continuously effuse mission, purpose, and context. The why is communicated everywhere and in every interaction. You feel it. You know it. It is repeated. There is focus and clarity in the messaging from managers and leaders. There is a richness that goes beyond sound bites, buzzwords, and corporate platitudes that are so quickly mocked by the rank and file as soon as the quarterly all-hands is over. In organizations that effectively define and communicate their missions, there are ready-made narratives that new (or not so new) team members can easily latch onto when the business responds to internal and external threats or adapts to a changing environment.

The reasons for doing things (whether launching a new product, making a major technology shift or a minor process change, or perhaps something scarier like a lay-off or a reorg) all make much more sense when there is a frame of reference and intentional flow of context. If these narratives are true (well, or at least consistent) they can provide an antidote to the alternative narratives (call them conspiracy theories if you like) that propagate and run rampant when there is conflict among teams, when folks get frustrated or burn out, or when times generally get tough.

Furthermore, if the why is clear enough, it can allow healthy, productive conflict around the what and the how. It also avoids organizational navel-gazing, which can be counterproductive or divisive. Obviously, leaders must believe these narratives. They must be authentic. And yeah, this is much easier said than done. Alignment is really hard.

In the military, there is the concept of “commander’s intent” that I keep falling back to as useful metaphor right balance of delegation that is required in properly mission-focused teams. When troops or front-line NCO’s understand the mission and they are granted the autonomy and flexibility to accomplish the mission using the “ground truth” that only they possess, this is when “empowered execution” happens. This is the opposite of the much-maligned “command-and-control” leadership style that “servant leaders” like to talk about in anti-management strains of Agile.

Growing up in an Army family during the last gasp of Cold War, I remember hearing stories (call it propaganda if you like) of how U.S. soldiers would be able “think for themselves” and would keep on fighting if their leaders were killed or they were “cut off”, unlike what would happen to Soviet soldiers. Sure we were outnumbered, but we were better! We could think for ourselves! We weren’t automatons like the other side.

Obviously, World War III never happened, but 20th Century U.S. military history (especially before 1960) is full of many successful examples where senior leaders provided the strategic and operational context for the mission, but execution was left to those closest to the front. A side effect of this sort of “empowered execution” improved accountability because folks can’t make the “I was just following orders” if the mission fails. They must own it.

Worse than Micro-Management

Working in mid-size product companies, you get a chance to see micromanagement laid bare in a variety of forms, ranging from executive “napkin architectures” to support requests from to individual engineers and everything in between. There is no shortage of “helpful” solutions to problems they have not been fully defined or adequately vetted, so that people immediately jump on them — or want your team to jump on them:

• Just run this SQL query for our data warehouse and don’t ask any questions.
• Your UI MUST be this color. Just because.
• Give me SSH access to this server so I can run this script.
• Don’t use cloud-provider Managed Services so we can run On Prem!
• Can’t you just? How hard could that be?

(You get the point.)

Leading operations and security teams, I would encourage my engineers to push back on these “predetermined solutions” to drive deeper discussions about intent. What problems are you trying to solve? What is the larger context or system? What is the end goal? What are you trying to do here and why Because security and infrastructure teams are often the bottleneck/blocker/gatekeeper for new things that are being built or deployed, they are in a unique position to uncover ill-conceived ideas and initiatives and driver larger WTF discussions across the organization and with the business. This is a blessing and curse.

The TLDR is that engineers want problems they can solve, not solutions they must follow from a script. We are not order-takers. Don’t just do what people want you to. They simply don’t understand the infrastructure, the implementation, or how things work. This anti-pattern can happen at the engineer to engineer (or manager to engineer) level but also between the product management and engineering orgs. In both cases, it must be kept in check. Unfortunately, in the vicious cycle of getting shit done (I’ve heard leaders promote “GSD” as a model for what should be rewarded), we rack up technical debt and build products and features that customers may never use until it all comes tumbling down.

As an engineering leader, you must constantly watch yourself so that you do not abuse your position of authority to stray into the how. This doesn’t mean you don’t think about the how. It doesn’t mean you check out from the solution space, but you have to resist the temptation to tell others how to do the work. This is the only way to have any chance of building a learning organization. One way to do this is to focus on non-leading questions. And if you provide a technical suggestion provide ample caveat’s that what you might be stupid, ill-informed, or generally terrible.

Not the Daily Three Questions

Agile is full of useful “three-word” phrases (start/stop/continue, yesterday/today/blockers, and these “3 Amigos” I recently heard about but have no idea what they are) but something I’ve settled upon over the years for framing the definition work is three simple words: outcomes, assumptions, unknowns. I’m pretty sure I didn’t come up with this on my own. I got help.

Even though these 3 words are compatible with “user stories” or “acceptance criteria,” I’ve found that using Agile terminology can be unhelpful because people think they know how to write user stories well (they don’t!) or can define clear acceptance criteria (they won’t!) and just doing these two rituals often leads to an ill-defined or uncritical definition of done. Lastly, jargon has side-effects that I’ve found turn people off or shut people down. Keeping things simple (even if somewhat more ambiguous) can result in a “mind-open” state that we want on our teams and in ourselves.

Starting with outcomes drives discussion at a higher level of abstraction and delays prematurely getting into the weeds. These three words can be used to define larger initiatives (call them epics if you want) or smaller bits of work (stories or tasks) but can be a helpful antidote to focusing on “the what” instead of “the why.”

In my experience, ticketing systems such as JIRA (yeah, especially JIRA) tend to draw engineers into that trap of prematurely defining “the what” that needs be done without clearly allowing discussion of “the why” or the “why not” to be fully explored. And of course, it discourages the examination of the mountain of assumptions and dependencies that the piece of work depends on — and the unknowns that will result in endless discovery and rework.

Ticketing systems make it very difficult to visualize the work as a whole or in relation to other systems or activities. Tickets can become an unhelpful “todo list” that is too low-level too soon. This prematurely narrows the solution set before clearly mining the problem set. Jumping to the solution before clearly understanding (and debating and communicating) the problem is one of the mortal sins of engineering.

This is why I prefer taking notes in Word Online or a notepad and NOT editing directly in JIRA or Rally. There is something about jumping into a web tool that shuts off people’s brains. Rant over.

In the beginning, product and engineering leaders might have to jump-start this definition of outcomes, but eventually, this should be a back and forth discussion about the target state given the constraints. You want engineers to be thinking about how an end-user’s behavior will change when using a feature, what pain-point it will address, or how the system must be changed to achieve the desired results NOT the way the outcome will be achieved or implemented. This is especially true if the ask seems out of the blue or a radical shift in direction, as is bound to happen. Also, when you are talking about outcomes it is also an ideal time to frame them without the purpose of your organization and re-affirm its mission — or clearly say this is a shift in focus. No need to be heavy-handed, but a little repetition never hurts.

This leads to the second question. What are we assuming with this chunk of work that must be delivered? What is the prior art? What is the tech stack? Does anyone remember how this could actually be wired up? What are the existing conventions to re-use or reject?

The slate is seldom blank, but frequently engineers don’t shine the light on what is assumed and you would be surprised how different the assumptions are across even a small team of 5 or 6 engineers, let alone when a feature touches other teams across specializations. Folks are seldom on the same page, in my experience — especially on distributed teams. A clear definition of the constraints or things that are NOT being tackled is critical. Clearly defining the scope to the best of your ability upfront is never a bad thing, even though it could change. It will change, of course.

Lastly, and most importantly after defining the desired end state, ask what don’t you know now that you probably should? The team should do its best to enumerate questions and areas where there must be discovery work. Channeling Rumsfeld, what are the “known unknowns?” This might be the time to attempt a “premortem” to work backward from projected failure to deliver on time or if we fail to all the desired outcomes.

Smaller features using existing tech or tooling may have very little here, but if the team can’t define any questions about what you don’t know, then perhaps the work isn’t ambitious enough or provide enough opportunity for learning.

For Further Reading and Thinking

This is obviously just a starting point but at over 2000 words it is long enough. This is as good as it gets. I heavily borrowed from the following books I’ve read (or mostly read, and sometimes re-read) over the last few years:

• One Mission: How Leaders Build a Team of Teams, by Chris Fussell , C. W. Goodyear , et al.
• The Advantage: Why Organizational Health Trumps Everything Else In Business, by Patrick M. Lencioni
• Outcomes Over Output: Why customer behavior is the key metric for business success, by Josh Seiden
• Start with Why: How Great Leaders Inspire Everyone to Take Action, by Simon Sinek
• The Product Mindset: Succeed in the Digital Economy by Changing the Way Your Organization Thinks by David H. DeWolf and Jessica S. Hall