What "Being Wrong" Can Teach Us About Software Development"

| 6 min read
#leadership #engineering


I gave this talk last week at work, based on a few of the books I’d be reading over the last 6 months, centered on Being Wrong by Kathryn Schulz. The goal of the talk was to think through ways that humans consume and understand information so that we could better operate as a team.

When I started reading Being Wrong, I assumed it would be about how humans jump to conclusions and create theories and ideas out of thin air with minimal evidence. I hoped to come away with some tips and tricks to combat my biases. That's not what I got from the book, but didn’t have many expectations beyond that. The book surprised me, bringing together scientific, sociological, and literary ideas into a conception of how humans differentiate right from wrong. It changed my perspective on what "being wrong" is all about.

How Do We Understand What “Wrong” Is?

At first glance, defining "wrongness" seems like a theoretical subject. But Schulz uses illusions and mental health to talk about how certain types of wrongness differ from others. An illusion, something we consciously know is "wrong", differs from madness, something that others might know is "wrong" but we do not. She uses this to talk about the social component of belief. The difference between us being tricked and having a mental health disorder is the recognition of other people. An illusion is something we can share with our friends to trick them; a disorder we cannot. She uses this to argue that one of the key fears humans have about "being wrong" relates to our desire to be connected to the people we care about.

Vocab Part 1

Anosognosia: a medical condition where we don’t recognize that we have an illness. I.e. blind people who believe they can see.

Superior Mirage: A mirage that’s a real “image” that’s been location shifted because of the refraction of light.

We're constantly generating ideas about the way the world works, and when our brains don't

The Sociological Component to Belief

The social networks we're a part of drive our beliefs. Simplified (probably too much). This means that our opinions are more influenced by who we know, than who we know being influenced by our opinions. This is an oversimplified view in the sense that we won't invite people into our social networks based on their opinions, but it also means we're more likely to adopt the opinion of a social network we're already in rather than leave it.

Some of this explains why Board Game Geek has a bias for more complex games, or why so much of the competition on Kickstarter has become a race towards higher production values and gigantic plastic miniatures. Those types of markers (complexity and chunky miniatures) become identity markers within the community, and typically when people have to choose between belief systems and community, they will tend to modify their beliefs to stay with the community they're a part of.

Vocab Part 2

Confabulate: to fill in the gaps in our knowledge with stories

Consistency and Dissent

What does this mean for how we disagree and why we disagree? Both Being Wrong and In Defense of Troublemakers have some stuff to say about this. In Defense of Troublemakers is a short book about how dissent works in social settings and some thoughts on what we can do to better cultivate dissent in our daily practices.

Vocab Part 3

French Resistance Fantasy: the author’s coining for our belief that in times of strife we’d definitely be the ones doing the absolutely right thing

One of the key points both books hit home is that we're wired to agree. In one study where participants were in a room and asked a basic question like "What is the color of the shape on the slide?", they were likely to share the wrong color if another person in the room answered differently than what was on the screen. We like to think of this term as peer pressure, but the roots run far deeper; when asked later about the color, the participants maintained they saw the color they reported. Their brain rewired their understanding of the world to fit the new status quo. But, if there were another person in the room disagreeing with the dominant (incorrect) answer, the tester was more likely to give the correct answer. People willing to speak up and disagree are valuable parts of a community because they prevent that community from defaulting to group think.

Commentators like No Pun Include who articulate the challenge board gaming has with colonialism are a perfect example. You don't need to agree with every point in the article to recognize this genre of game as a dominant strand of thought in the medium. By doing so, they've opened up the possibility (even in a small way) for other types of genres to gain purchase in the industry. Dissenters also don't have to convince everyone else of their belief in order to provide value to a group. By convincingly articulating their point of view, they expand the potential points of view for the entire group.

Error Blindness and Doubt

But what does this mean for us as we navigate the world of game reviews? One challenge to persistent growth is what Schulz calls "error blindness". It's the concept that we can either be wrong or know we're wrong, but not be both at the same time. Because we replace our incorrect beliefs with our new "correct" ones, we don't cultivate a memory of being wrong. For example, I used to play Apples to Apples in high school and enjoyed it. I can remember the feeling of enjoying playing and I know I don't like the game now, but I can't consciously explain the feeling of liking the game and also knowing the reasons I don't like the game. This isn’t a problem on its own. I don’t think anyone is looking for an Apples to Apples apologist, but because we replace beliefs like this means we don’t have a conception of doubt related to our internal beliefs: we hold on to them right until we let them go, and our brain tells us we were correct the entire time. This doesn't mean that all certainty is bad, but we should recognize that it's more a reflection of our emotional connection to our beliefs than a marker of how accurate that belief is. For example, my belief that 2 player games are great communication mechanisms has more to do with my upbringing playing checkers with my grandfather, cribbage with my dad, and magic with my close high school friends than it is a marker of its accuracy.

What does this mean for our relationship to doubt? Schulz talks about the concept of doubt as being a luxury for evolutionarily advanced neurological systems. (Oof that's a mouthful) Doubt is something we have to practice at, whereas certainty is something that feels natural. Uncertainty causes us anxiety and we look to reduce it as fast as possible. This isn't a prescription to disbelief, it's a recognition that doubt is an activity that we have a lot of feelings about. Schulz describes this as the "perverse pleasure of art". We enjoy it because it makes us feel lost and allows us to learn in a safe place. I think this is the power of a strong reviewer or critic (something I aspire to); they're someone who can open the door to an unknown experience by connecting it to something we already know. Go get lost, but know that safety is only an arm length away.

Vocab Part 4

Distal Belief: Any belief that doesn't have a direct ability on our day-to-day life.

Identity and Learning

But what does this mean for our ability to learn and grow? Schulz describes this system as the network of beliefs: we can’t remove or replace one belief without also removing and replacing other beliefs (and potentially social groups) we hold dear. I concluded that in order to be a lifelong learner you need to be comfortable changing your identity. You can't be the same person who happens to know different things.

This concept explained to me one gap between the board gaming community and people who might be interested in games but find it hard to get into the hobby. One strength of the board gaming community is the idea that it's socially valuable to learn, experiment, and play a bunch of different new games. So our identities as board gamers are more tied to learning and experimentation than it is to being Hearts or Bridge players. The gap here, as explained by error blindness, means that it's hard for us to imagine what it's like to anchor to a single game, and that becoming a teacher can be a challenge.

Vocab Part 5

Theory Drive: The human desire to categorize and explain the world around us

What To Do With This

Vocab Part 6

The GI Joe Fallacy: The mistaken belief that knowledge is half the battle.

It's easy to come away from this with an idea that the knowledge has inoculated us against these forms of bias for certainty. But these psychological patterns exist deep in our brains, and it isn't something we can simply think our way out of. Rather, our goals should be to build social groups that encourage dissent by expanding the reviews we read and games we play. We can create implementation intentions (an "if-then" commitment we make to ourselves about how we're going to behave in certain situations) to speak up when we disagree and find gaming groups that support our efforts to expand our horizons.

TL;DR and My Implementation Intent

  • Our Idea of Being Wrong has as much to do with our social groups as it is the facts of the matter

    • Feeling "wrong" about something feels like being disconnected from the people around us
  • Dissent is powerful because it expands the range of viewpoints within a group.

  • Certainty has less to do with correctness and more to do with our attachment to a belief

  • We can't learn without also shifting our identity

My Commitments

  1. I plan to use examples as much as possible to make it clear what the underlying evidence I have so you can draw your own conclusion
  2. Call out any reviewers that I got the initial recommendation from (if I can remember). I filter games from a few different prominent reviewers, and I want to be clear about what's influencing my recommendations.
  3. Use what Being Wrong calls self-subversive thought: couching my language in my personal perspective, limiting it to my vantage point and making it clear I think I'm ok with it changing or being disagreed with.