Empathy is a critical component in Software Engineering
empathy — noun meaning “the ability to understand and share the feelings of another.”
(This article is 1 of 4 in a series about cultural issues in reliability and software engineering. Be sure to also check out parts 2, 3, and 4)
I was recently told that empathy had no place in engineering decision making, which was a bit surprising, so I asked “Why? What do you believe [inspires innovation] if not empathy?” The response was (paraphrased), “As a libertarian, I believe in rationality and reason. Feelings don’t matter.”
Even if you ignore the past four decades of work in cognitive science, there’s a lot to unpack in that statement. When I mentioned this encounter to a friend, they said, “You got them to say the silent part out loud.” Another friend, while reviewing this article stated, “the way that was phrased makes it sound like a Dalek approach, not a libertarian one.” (they got bonus points for the Doctor Who reference)
While I don’t have sympathy for the political leanings that inspired this response, nor believe people who claim to be “analyzing all the data” when they bury their biases under claims of libertarianism, rationality is an important component of good technical decision making and direction setting. In Thinking, Fast and Slow (2011) Daniel Kahneman showed us emotion is inextricably linked to decision making, and experience has shown a lack of empathy when considering product function or operation, or how to grow an organization and business, will lead you astray in ways that will undermine business objectives.
Daniel Goldman and Paul Ekman defined three types of empathy:
- Cognitive empathy — the ability to understand how a person feels and what they might be thinking.
- Emotional empathy — the ability to share the feelings of another person.
- Compassionate empathy — goes beyond understanding and sharing emotions, and motivates action to help however we can.
Think of the example of an oncall engineer getting paged in the middle of the night; it takes cognitive empathy to know that in that person’s half-awake state they will not be at their sharpest. Emotional empathy weighs in when you can identify with a person who needs to take action, but may not feel prepared and comfortable, or even know how to do it. Compassionate empathy is the backbone of reliability engineering because it compels engineers to:
- Jump in and help if possible,
- Create and test accurate runbooks, or automated remediations for your own or other teams.
- Design automation and software improvements that create systematic improvements to remove the need to wake up in the middle of the night for a page.
I’ve said before(on twitter):
“SRE is the path from emergent to repeatable to best practice. It’s a result of examining a thing from a customer’s quality perspective and willing it to be good enough. Without the examination, or the will, it is useless.”
That statement is a clear stance that both cognitive and compassionate empathy are critical to the practice of reliability engineering. There may be areas of software development where the need for empathy doesn’t apply directly, such as technical implementation of deeply embedded systems. However I can’t think of an area of software engineering practice or leadership where empathy shouldn’t be applied regularly to define requirements and bridge collective understanding of problem spaces.
Four Business Applications of Empathy
“The emotional tail wags the rational dog.”
― Jonathan Haidt
Empathy impacts both end users and organization members, and I’ve identified four business applications of empathy. Each of the three types of empathy play different roles in these business applications. Cognitive and emotional empathy play significant roles in understanding needs, desires, and complaints. While compassionate empathy leads us to take action either proactively or reactively to meet needs or address complaints.
Empathy for Users
Empathy for users is the most common form exercised by organizations. Often they will have values defined such as “Customer Obsession”, “Focus on the user and all else will follow”, or “Make something people love.” Cognitive and emotional empathy are leveraged heavily in the processes used to understand customer requirements or desires, and throughout the process for testing theories of how to meet those needs.
By putting product managers, designers, and engineers in the position of having to use systems as users do, it’s possible to achieve higher quality outcomes because frictions can’t easily be ignored. High performing organizations use their own products, while lower performing organizations have more “blaming” or “throw it over the fence” attitudes.
There are situations and products that are difficult to test as you would a normal end user. There are often regulatory or compliance concerns around data privacy, for example. It’s also important to recognize that not all users come from outside the company, you may have developers on another team that are experiencing frictions from the system as well.
You can tell how successful you are at empathizing with your users through customer loyalty metrics, or more concretely, the dollar amounts you pay back to your customers as fair recompense for their dissatisfaction with your product.
Empathy for Stakeholders
“This is the essence of intuitive heuristics: when faced with a difficult question, we often answer an easier one instead, usually without noticing the substitution.”
― Daniel Kahneman, Thinking, Fast and Slow
Empathy for stakeholders really means understanding their needs, concerns, and motivations. Frequently in larger organizations, work can’t simply be done within a single team, and therefore it’s the work of a management layer to coordinate and align to achieve company objectives. Exercising cognitive and emotional empathy will help understand the underlying motivators (for right or wrong) leading people to ask for effort from other teams. Compassionate empathy leads proper expectations and trustworthy commitments.
It’s important for teams working together to get together and talk during the planning cycle, and throughout implementation to make sure efforts line up. High functioning organizations have clear and pre communicated expectations with regular syncs afterwards to accommodate for any changes. Low functioning organizations traditionally suffer from a number of surprise priority shifts, and broken commitments. Often there will be low trust and a high degree of blame across teams, where individuals are punished.
To avoid these problems, ensure there is time in the planning cycle to truly understand stakeholders needs, and create a shared alignment around business priorities in case adjustments are necessary. To proactively address these problems, dive deeper into requests than you think may be necessary, as often we are biased to solve simpler problems than those we’re being asked to solve. And examine the processes or incentives that lead expectations to be violated in the first place.
You can see the impact of a lack of empathy for your stakeholders through poor alignment or rework which leads to wasted engineering hours or slipped ship dates, or the number of commitments broken over some time period.
Empathy for Implementers
Complex problems have different solutions than complicated problems, versus simple problems. Assuming you agree with this (you’re free not to), then it’s reasonable to say that unknown unknowns happen and are highly variable — versus known unknowns which should be relatively bounded, and standard or best practices which are simply known. So when implementers are wrong because of unknown unknowns (or even sometimes known unknowns), it’s important to understand what went wrong and why rather than punish the individual for not having the psychic foresight to know what was — in that moment — unknowable.
High performing companies — according to the DORA state of devops reports— create continuously learning cultures where the messengers of bad news are not shot, but actively encouraged to bring bad news forward, and use those failures as learning opportunities. (Source: Westrum Culture Model) Low performing companies tend to punish individuals, or create an ever shifting landscape of priorities that prevent engineers from focusing and doing their best work.
It’s important that organizational leaders understand what model of operation they are working in (consider using the Cynefin framework), and account for the fact that estimates or problem scoping are going to be (more) wrong if any aspect of your development cycle is in “Complex” territory (frequently it is — listen for engineers mentioning “unknown unknowns”). It’s equally important that leadership is not allowed to simply blame individuals and not share the accountability for being wrong in these complex problem spaces. True proactive measures to address the issues of solving complex problems include alignment and expectations setting exercises, frequent follow ups for adjusting alignment and expectations, training individuals to raise issues early and often, making it safe to be wrong, but learn and become more right over time.
Failure to do these things will lead to low levels of psychological safety and trust in the organization, which will lead to data hiding, and fear of retaliation. Metrics to watch would be company culture survey data (If you’re a software company, I highly recommend the Westrum Culture Survey questions be worked into company culture surveys), the number of anonymous questions versus named questions at all hands (a measure of psychological safety), and the number of messages flowing through communal (vs private) chat channels on company messaging applications.
Empathy for Leaders
“The affect heuristic simplifies our lives by creating a world that is much tidier than reality.”
― Daniel Kahneman, Thinking, Fast and Slow
I can kind of already hear it, “Empathy for Leaders” — do they really deserve it? I get it, but there’s definitely a need to rise above and offer empathy for people even when they seem to not give it at times. Leadership is made of humans, and usually a lot of competing and conflicting interests. That, at least in my book, makes them empathy worthy. But it’s a two way street. The critical requirement for this to work is empathy from the leader, which doesn’t always exist. Leaders who “walk the floor” build more trust and credibility than their office bound, or upward focused peers. Leaders need to understand all (and more) of the “Empathy for Individuals” section above and act accordingly. Leaders who do will build more psychological safety, and exhibit more humility so they can listen to and solve problems for their organizations therefore earning broader trust and loyalty from their reports than those who exhibit dominating behaviors or favoritism. “Culture eats strategy for breakfast” (Peter Drucker, March 2000) and empathy is the foundation for deeper, more meaningful, and more authentic Individual/Manager engagements which create that culture.
One thing that frequently comes up is leadership having incomplete or no knowledge of the problems that exist below their level. The Iceberg of Ignorance is a real thing. Power distribution and knowledge are frequently unevenly distributed in an organization. The most empathetic thing someone can do with their leadership is assume good intent, and offer objective, honest assessments of the problems they are having or witnessing, and communicate those things in clear, concise, and convincing ways.
All of these issues are intertwined to some degree, and are reflected in the communications and levels of trust within the organization, the level of loyalty of contributors, and the amount of time leaders spend talking versus listening.
Conclusion
This is not a scientific study. It’s based on observation, experience, and conversations with others in the field. Empathy as a critical principle required for any Product or Reliability Engineering organization, or any organization where the service offered impacts the lives or safety of their users.
While critical reasoning skills are important when considering selection of optimal software libraries, applications, or automation to fit specific use cases, the designs and requirements that drive those use cases are deeply rooted in — and are inextricably linked to — empathy for the user or operator that would be using them. Try to remove empathy at your own peril. Doing so will have negative ripple effects throughout your organization and your product will suffer.
