Saturday, September 23, 2017

Thoughts on Confirmation Bias

This post is part of a series on cognitive biases. You might also be interested in my thoughts on groupthink and the survivor bias.
_____

"People have no trouble turning any information into a coherent narrative."
Jason Dana

Zener Cards

J.B. Rhine, the founder of parapsychology, was convinced that ESP (extrasensory perception, ie receiving information with the bare mind, instead of the known senses) exists. In the 1930s and 1940s he conducted hundreds of experiments to prove his claim. Amongst other things Rhine invented a card deck called Zener Cards. A deck comprised of 25 cards with one of five different symbols on each card. The "sender" (often Rhine) looked at a card, while the "receiver" (a person potentially capable of ESP) had to guess the symbol on the respective card. If people would just randomly guess, they would get an average of five correct answers. When a person scored way above five, Rhine saw this as prove for ESP. Over time, Rhine thought he had found people, who were extraordinary good at ESP, because they consistently scored above five.
Millions of people believed Rhine‘s research, he sold lots of books and got funding after funding. However, later it turned out that his methods and his data were completely screwed. Rhine was so desperate to prove his claim, that he went through his data over and over again, until he finally found what he was looking for. For instance, when a promising "receiver" seemed not to perform well enough in a series of tests, Rhine found what he called "forward displacement". This means that the "receiver" did guess the correct answer, but his/her guesses were "displaced", so that he/she did not guess the actual card, but the upcoming one. Another obscure phenomenon Rhine discovered was what he called the "decline effect". When a receiver started promising, but then failed to continue his/her correct guesses, he/she must have been fatigued or bored (and therefor the results must be discarded). Probably the most absurd claim Rhine made was that whenever people scored above average, they clearly were capable of ESP; and when they scored below average, this too was proof for ESP, because they used their ESP skills to embarrass Rhine!

J. B. Rhine with one of his "receivers", Hubert Pearce. Source: Wikipedia
I‘ve learned about Rhine and the Zener Cards from reading the great book Standard Deviations by Gary Smith. The story is a striking, yet extreme example of confirmation bias, "the tendency to search for, interpret, favour, and recall information in a way that confirms one's preexisting beliefs or hypotheses" (Wikipedia). One interesting thing about Rhine is that there‘s no strong evidence that he intentionally cheated. He thought his methods were totally valid. And he was without any doubt a smart man.
You‘ve probably seen lists of dozens of cognitive biases. While it turns out that some of them are so context-dependent that it‘s questionable how useful they really are (and others could not be replicated in subsequent experiments), it appears to me that there‘s no real doubt that confirmation bias exists and that it seriously impacts our behaviour and decision making. We are all affected by confirmation bias, even if we think we would be better able to deal with it than others. For a vivid and astonishing example (there‘s a magician involved!), watch this video. You will get to know choice blindness, which is very close to confirmation bias IMO.



Why are we affected by confirmation bias?

It‘s not entirely clear why confirmation bias (and similar effects) seem to be hardwired into our brains, but there are some explanations that make sense to me:
  • In this article Jonah Lehrer links confirmation bias (and similar phenomena) to our ability to persuade others, which is a critical skill for humans as social beings. In that sense there‘s a lot value in our brain finding more and more arguments for our point, so that the rest of the group follows or supports us: "the function of reasoning is rooted in communication, in the act of trying to persuade other people that what we believe is true." 
  • David McRaney suggests that confirmation bias might has offered an adaptive advantage to humans as a species He gives the example of a caveman being out in the woods. He has a hunch that there are cheetahs out there, which is obviously dangerous to him. He therefore looks for clues that confirm his hunch. In cases like this it‘s generally much better to go with confirming information, instead of looking for disconfirming information. Or as McRaney puts it: "It‘s better to be wrong than to be dead." (You Are Not So Smart Podcast, episode 103
  • Sarah Gimbel and Jonas Kaplan from the University of California point out that relatedness is extremely valuable for humans. Whenever we change an important belief, we put this relatedness to our peer group at risk. So although it might make sense to change a belief from a rational, individual‘s perspective, it might be to costly from a group perspective.
    Gimbel and Kaplan also explain that important beliefs can become part of our identify, and threatening this beliefs triggers very similar reactions in our brains than eg meeting a bear in the woods.
  • Kaplan adds two more interesting views on this: Firstly, mental models need some level of stability to be useful in the real world. Therefore we need to balance the influence of new information with this need for stability. Secondly, new information ist not stored independently in our brains. We don‘t simply add another piece of information, but instead we connect it to other information we already have. So changing one piece of information could cause a "chain reaction" and might not be worth the effort. (Listen to this excellent episode 093 of the You Are Not So Smart Podcast with Gimbel and Kaplan.) 
Whether or not these theories can be backed up by more research in the future, there‘s strong evidence that humans have a hard time dealing with cognitive dissonance: When our beliefs don‘t match our behaviours, we tend to become psychologically uncomfortable. Therefore we strive for internal consistency, either by changing our behaviour or by justifying it (or avoiding situations in which this inconsistency becomes evident). Changing our behaviour seems extra hard, one reason being that when we do change it, we‘ve created another inconsistency, because our current behaviour deviates from our past behaviour. Therefore oftentimes we justify our behaviour by looking for more and more confirming data and neglecting disconfirming data. Confirmation is not a big problem with minor or relatively unimportant beliefs, but it really kicks in with "cherished beliefs", those beliefs that are linked to our core identity (for many people religion or politics).
It gets weirder: in this blog post David McRaney writes about a study from 1979, which suggests that "even in your memories you fall prey to confirmation bias, recalling those things which support your beliefs, forgetting those things which debunk them."
The internet does not really make things better. As Justin Owings points out in this blog post: "Thanks to Google, we can instantly seek out support for the most bizarre idea imaginable. If our initial search fails to turn up the results we want, we don’t give it a second thought, rather we just try out a different query and search again. [...] People have always been prone to confirmation bias, but the Internet amplifies the phenomenon since we need not look far to confirm our particular bias. It’s always a click away.” That‘s also the reason why it‘s almost impossible to win an argument online - the other party will always find websites which support their opinion.

What can we do about it?

The research on confirmation bias is not new, and most people have heard about at least some of it. Yet I am amazed by how little we talk about the impact of confirmation bias (as well as other biases) in our community. One reason for this might be a phenomenon called Bias blindness, which means that we underestimate the effect that biases have on us compared to others.
If only a small part of the research about confirmation bias is valid, we are wasting tons of money by making flawed decisions in any part of our organization: Should we hire this person? What product should we develop next? Should we invest in a new market? etc.
So what to do about it? Obviously I don‘t have the silver bullet and research shows that it‘s incredibly hard to overcome or even dampen cognitive bias. But I really think we should talk more about these things and share ideas. Here are some of my thoughts on how to fight confirmation bias, please share yours in the comments section):
  1. Whenever we hear someone (our ourselves) say things like: "I knew it! Here‘s yet more evidence that XY is correct!", we should be cautious and try to look out for confirmation bias.
  2. I think it‘s important to institutionalise a process in which we come up with criteria for success and failure (or better: confirming or disconfirming data) in advance, i.e. before we start with a project, initiative, etc. "How will we know we are successful?", "What are signs of failure?", "At what point will we re-evaluate what we‘re doing or stop it all together?" are good questions to discuss (I believe this is called Preregistration in the scientific community). What happens instead (and I am guilty of doing this many times) is this: We are very enthusiastic about a thing and really want it to succeed. So we start a project and invest time and money. After a while the data shows that the desired effect does not occur. But instead of questioning what we are doing, we start looking for other "great" effects we can observe: "Yes, sales did not go up after implementing this feature, but look how much this blogger liked it! So let‘s do more of it." In this case we not only deal with confirmation bias, but most likely also with the sunk cost fallacy.
  3. Writing things down often helps. By grabbing a piece of paper and writing down our assumptions, we can create a commitment, which makes it harder to justify contradicting actions afterwards. The story goes that Warren Buffett admired Charles Darwin‘s approach to confirmation bias: "Charles Darwin used to say that whenever he ran into something that contradicted a conclusion he cherished, he was obliged to write the new finding down within 30 minutes. Otherwise his mind would work to reject the discordant information, much as the body rejects transplants. Man's natural inclination is to cling to his beliefs, particularly if they are reinforced by recent experience--a flaw in our makeup that bears on what happens during secular bull markets and extended periods of stagnation." (source
  4. Have someone take the role of devil‘s advocate is always a good idea, especially with big decisions. In addition, it‘s often beneficial to invite known critics to voice their opinion. Warren Buffett, again, is known for inviting his critic Doug Kass to challenge his investment plans (source). 
  5. On a related note David Rock points out in this webinar that it‘s a good idea to fight bias as a team. Rarely all members of a team need to be involved in actually making a decision. So some of them can focus on observing how the decision is being made, share their observations and trigger a discussion about the biases at work.
  6. It‘s good to keep in mind that often people are not as rational as we think (or wish for). If we wonder why someone is insisting on his/her opinion, despite all the evidence against his/her point, chances are that we won‘t convince this person by adding more arguments. In fact, this could result in this person insisting even more on his/her option. This is called the backfire effect.
  7. If someone seems to behave irrational, we should not assume he/she is an idiot. It‘s almost always a good idea to take a break and let heated discussions cool down. From what I understand the prefrontal cortex plays an important role in what we call rational behaviour. Unfortunately, this part of the brain is easily tired. So give it time to "recharge", before making important decisions.
  8. Taber and Lodge have found evidence for what they call the "Attitude strength effect": Subjects voicing stronger attitudes will be more prone to confirmation bias. (for a summary of the study see this PDF document.) One conclusion from this might be that we should not expect someone to change his/her opinion, when this very person has just talked about this very opinion with utter conviction. In this case it might be a better idea to take a break and continue the conversation in a smaller group.
  9. The way most people (me included) search for information seems to be problematic. When we do research, we often use a search engine to find what we are looking for, or so we think. But knowing a little bit about confirmation bias, I tend to believe that we are rather typing in what we already believe and/or what want to be true. I remember searching for phrases like "Kanban success stories" or "Lean cycle time reduction". And of course I will find what I was looking for. But what if I would search for phrases like "Kanban does not work" or "Lean is is fad"? Wouldn‘t I find completely different results? I now try to make it a habit to search for disconfirming results on Google. But what can I say? It‘s hard...
  10. We all know the term filter bubble and some of its implications. We might call social media a big confirmation bias machine. The first problem is that we are only connected with people who are mostly like us, meaning they probably share most of our cherished beliefs. So I don‘t even get to hear disconfirming data. The next problem is that I get rewarded, whenever we post something that confirms our group‘s beliefs. For this we will be instantly rewarded in the form of likes, retweets and positive comments. So our beliefs are being re-affirmed and strengthened all the time. If we want to change this, we might want to consider following/friending people on social media, who are different from us and have very different opinions/beliefs. I‘ve done that, and it‘s interesting but it can also be very exhausting. 
These are some ideas I have about mitigating confirmation bias. I will soon publish another blog post on debiasing strategies in general.

If you want to dig deeper, you might want to learn about biases that are closely connected to confirmation bias. Here‘s a list I‘ve compiled:

backfire effect, belief bias, belief preservation, belief overkill, choice blindness, desirability bias, frequency illusion, hypothesis locking, morton's demon, motivated reasoning and motivated scepticism, myside bias, non-falsifiable hypothesis, polarization effect, positive bias, selective thinking, Tolstoy syndrome

P.S. Of course the belief that confirmation bias exists is also affected by confirmation bias. I‘ve at least done some googling and searched for phrases like "confirmation bias is not real", "confirmation bias is overrated" etc. I did not find any useful information.

Sunday, August 13, 2017

Thoughts on Survivor Bias

Over the course of the past years I‘ve become more and more interested in cognitive biases, how they affect our work and what could be done to mitigate them. A couple of months ago I‘ve published my thoughts on groupthink. Today I want to share some thoughts about survivor bias. I believe it‘s hard to exaggerate the effect of this bias, but at the same time it seems to be relatively unknown in our community.

Never give up!
For whatever reason Facebook decides to show me these "motivational" pictures with "deep" messages every now and then. Yesterday I saw this gem:





Seems like good advice. It must be good advice to never give up, if this was the secret sauce for Coca-Cola‘s success, right? Of course it‘s not! It‘s pure survivor bias. Just as well you could ask a millionaire what her recipe for success was and then she would reply: "Quit your job, sell your house and your car, go to the casino and bet everything on number 12! That‘s exactly how I got rich." Of course that‘s bad advice, but unfortunately this is exactly how many business books and other guidebooks work.
We‘ll come back to this later, but first let‘s hear about a story from World War II.

Abraham Wald
The Hungarian statistician Abraham Wald was part of a group of geniuses, who helped the US win World War II with weapons of mathematics. The group was called "Applied Mathematics Panel", and there‘s a wonderful article on this group, written by David McRaney. One task the group was assigned to was to figure out how to improve the armor of allied bombers. It was clear they needed extra protection (too many were destroyed in combat), but it was unclear were exactly to put the additional armor, and protecting the whole plane would make it too heavy. So commanders examined the planes that returned from battle and looked at the bullet holes. It was easy to spot patterns (one obvious one was along the wings), so they concluded to add extra armor at exactly these places. Fortunately, Wald was involved in solving this problem, and his clear thinking probably saved hundreds of lives. He concluded, that the exact opposite of this had to be done: improve the armor at the places where no bullet holes could be spotted. His reasoning went like this: The planes we get to see here all have survived the battle, so they have done pretty okay. But all the planes who did not return from combat would have needed extra protection, because they had been hit at all the places we don‘t see at our planes here.

Survivor Bias
Wald‘s story perfectly explains what survivor bias is:

The logical error of concentrating on the people or things that made it past some selection process and overlooking those that did not, typically because of their lack of visibility. (Wikipedia)

It would be easy to smile at this story from World War II and just carry on. But survivor bias has serious implications for almost all areas of our professional and personal lifes. Here are some examples:
  • Ever heard someone say this:
    "The quality of XY nowadays is so crappy. I have a really great 20-year old XY, but they don‘t make these anymore"?
    I‘ve certainly said something like this many times. However, chances are that this statement is not true. In fact, the quality of many things has drastically improved over the last decades. They‘ve made a lot of crappy items in the past as well, but they all have disappeared, so that only the (few) good ones have survived. Or, if you look at it the other way around: They build good ones and crappy ones today, but we haven‘t figured out yet, which ones are the good ones.
  • We tend to believe that the cities of eg the Renaissance must have been really beautiful, and all the art of this time was extraordinary. This must be the case, because all the buildings and all the art from the Renaissance we get to see is very beautiful. But of cause we never get to see all the ugly buildings, paintings, songs etc., because they were torn down, cast away, not passed on and not talked about in history books.
  • Social media is a big survivor bias machine. Whenever I browse through my Facebook stream, I get the impression that all my friends have spectacular lifes, while mine is nothing but boring. Wherever I look: Happy people with a drink in their hands (one with an umbrella!), travelling the world, chilling at the beach in wonderful weather. At least two survivor biases at work here: 1) People only take pictures when the skies are blue and the wine is plenty. 2) People tend to only post spectacular stuff. They rarely post pictures of them being bored on a cloudy day.
    In the case of social media, events "survive" the processes of photography and social media posting only if they are interesting and colorful.
  • I do not understand a lot about mutual funds, but I stumbled on this paper, whose authors claim that "[a]lmost all prior mutual funds studies suffer from survivorship bias".
  • Ever heard of this anecdote? When cats fall from a balcony, they have a bigger chance to survive, when they fall from a greater hight! The data for this conclusion was drawn from the observation that cats that were brought to the vet had greater injuries if they had fallen from 1-6 stories, compared to cats who had fallen from higher than six stories. One explanation given was that whenever cats fall from a greater hight, they have enough time to spread their legs and use their body as some kind of parachute. Sounds amazing, but perhaps the more reasonable explanation for this phenomenon is this: Cats who fall from higher than six stories are rarely taken to the vet, because they did not survive the crash or, if they do, their owners don‘t find them or do not have any hope for cure.
These are just some examples from very different areas, and the list could go on and on. What all these stories have in common is that the conclusions we draw from the data are backward looking: We look at a set of visible data and conclude what must have happened in the past. That seems reasonable, but as Gary Smith claims in his book Standard Deviations: "When we chose a sample today and look backwards, we only see the survivors" and that those studies often suffer from survivor bias. From a scientific standpoint we should have done the following to conduct serious, forward looking research: Take 100 random cats, for each one toss a coin, throw half of them from the 3rd story and the other half from the 8th story and then see which group has the greater injuries. No, I am in no way proposing actually doing this, but you get my point:-)

Great Companies
Now what has all this to do with the Lean/Kanban community? I think one serious implication is best illustrated by Jim Collins‘ best-selling book "Good to Great". Collins compares eleven "great" companies and then finds a set of seven characteristics all these companies share. His conclusion is that he has found the recipe for success and if you use this recipe, your business will be successful as well. Gary Smith points out that Collins‘ approach is pure survivor bias, because it‘s backward looking. Collins only looks at companies that still existed at the time of the writing of his book. So we have no way of telling how many other companies did the exact same thing but went bankrupt anyway. Smith also points out that the eleven "great" companies do not perform that great anymore: Between 2001 and 2012, these companies combined did worse than the overall stock market.

Now if even Collins‘s work (who worked as a Stanford professor) is affected by survivor bias, we can be sure that many, many other articles and books are as well. So we should be very careful, whenever we read something about "the 10 things that all successful managers/teams/organizations do" or the like. Plenty of these writings exist in our community, as well. I am not suggesting that these stories are useless. I find many inspiring ideas in them, many worth trying and adapting. I just disagree with the (implicit or explicit) conclusion that we, too will become successful by doing the same thing.

P.S. I‘ve also written a blog post on confirmation bias you might also enjoy reading.

Tuesday, March 7, 2017

Kanban and the Post Office


This is a translated and modified version of my article "Wie bei Kanban die Post abgeht", which was published online at Projektmagazin in February 2015. If you prefer German, you can find the original German article here.
_____________________


Once I was with a company, that had freshly started using Kanban. Peter, the CEO was happy with what they already had achieved and proudly presented their board to me. Indeed I was impressed with how far they had come in such a short time without having much external expert support! The board roughly looked like this:

Sipping our coffee, we had a chat about Kanban in general and their board in particular. After a while I asked about the problem with queues in the post office. My conversation partner looked in me in disbelief, so I started from the beginning.

"When I was a kid", I said "every post office was organized in a way that each counter had its own queue. If you had a package to ship, you had to decide for one of the queues and line up - of course it always was the wrong choice, because it turned out that all other queues were serviced much faster. Today, post offices are organized differently. Usually, you‘ll only find one central queue for all counters. Only shortly before its your turn, you‘ll be directed to the next available clerk. Why is this? This system offers at least three major advantages:
  1. Predictability improves significantly. In the old system, it happened quite frequently that I had lined up in a specific queue only to discover later, that the person in fron of me had a very complicated issue he wanted to discuss with the clerk. Bad luck! The same thing happened, if my queue was served by a trainee, who (of course) was way slower than his/her colleagues. Certainly these things happen in the new system, as well. But here trainees and complicated requests have different ramifications, because the delays are split amongst all the customers, so to say. This is called "using pooling to buffer variability". Needless to say that this improved predictability and unified wait times lead to higher customer satisfaction.
  2. The new system is way more reliable. Imagine one of the clerks needs to leave his/her counter, because he/she is needed elsewhere. In the old system, this had caused severe disturbance, because the queue now had to be distributed amongst the other queues. But by what mechanism? Should the customers just line up at all the other queues? And wouldn‘t that be unfair, because they had already waited in the disintegrated queue? In the new system, it‘s still annoying, when a counter is being closed. But the impact for the overall system is rather low. Nobody has to be redirected. The wait time for all customers gets a little bit longer, but nobody feels treated unfairly, because the additional wait time will be evenly distributed amongst all customers.
  3. The new system is less stressful for the clerks. In the old system, every clerk had to serve each of his/her customers, before he/she could close the counter and leave work. Everyone had "own" customers and should have felt responsible for them individually. In the new system, all the clerks work as a team and share responsibility for all the customers. If someone is having a rough day (or difficult customers), his/her team mates will compensate for this automatically."


Keeping an eye on the queue as a whole

During my little monolog, Peter had been nodding a lot, so ha seemed to agree with all this. After I had finished, he directed his gaze to the board again and started thinking out loud: 
"In our context, the tickets on our board are the customers in the post office, each with different requests. Our team members are the clerks, who deal with the requests. A finished ticket is the equivalent of a served customer (hopefully a happy one!) At the moment we assign people to tickets right from the beginning, everyone has his/her own queue and is responsible for dealing with it. This means we are working in the old post office system. When I think about it, I have witnessed all the disadvantages you have just described. So for us, too, one joint queue should be a much better solution. But for this our board had to look differently."
He grabbed a marker, went to a nearby flip chart and started to sketch a modified version of the board. 


For the columns "To Do", "Next" and "Done", the swim lanes had disappeared. They now only existed for the activity columns "Dev", "Test" and "Deploy". It seemed like a small change, but I think it was a big step in the right direction towards more flexibility, collaboration and knowledge sharing. 


Holes in the input queue

I then nudged Peter a little bit more and asked: "What about holes in the "Next" column?" Again, he seemed to be puzzled, so I elaborated: "I guess, that the tickets in the "Next" column are prioritized by some sort of mechanism that makes sure the tickets with the highest value (or best value-cost-ratio) are on top of the queue and the less valuable ones are at the bottom, right?" Peter nodded, so I went on: "Now what happens when, let‘s say Stefan is done developing a ticket and wants to pull the next one from the "Next" column? Of course, he should pull the one at the top of the queue." 
I grabbed a marker and wrote "1" on the top ticket, "2" on the next one and so on. "Unfortunately, I am pretty sure that in many cases that will not happen. And the cause for this is specialization. That‘s probably the reason why you have the swim lanes in the first place. So if ticket 1 requires a specific skill, which Stefan does not master, he will leave the ticket where it is and pull ticket 2 instead. Now after a while Inken wants to pull a new ticket. But again, she‘s lacking the required skill for ticket 1. She doesn‘t feel comfortable pulling ticket 3, either, so she pulls ticket 4. Does this scenario sound realistic?" 
Peter nodded his head thoughtfully. He knew where I was getting at and changed his sketch of the board again. At this point it looked like this:


Before I could go on, Peter started speaking, and he said the exact same thing I was about to say: "This is a disaster! From a business perspective, these "holes" are an absolut no-go, because they mean we are delaying the most important tasks/projects, while we are working on less important ones. We cannot have this!" 
While he paused, I explained, what I had seen many times before: "Yes, I agree. But here comes the catch: If you want to make sure the most important tickets are always being worked on, the company has to invest in this new way of working. And depending on the degree of specialization, the technology you use, your code base etc. it might be a quite high investment. The board helps, because if designed appropriately, it will show you how far you‘ve come at any time. But of course, the sole visualization will not be enough. You will probably have to train your staff and collaboratively develop policies that make sure you gradually improve."
"Of course, I understand", Peter said (and he looked a little bit depressed). "I will start working on this tomorrow. Do you have any other advice on how the board can support us in this effort?" 


Some more ideas for improvement

I paused for a moment and thought about this. Then I replied: "In my experience it‘s mostly unfavorable to have swim lanes (or columns for that matter) named after specific people, because it tends to perpetuate the current division of work. It might be a good idea to have the specialization written at your swim lanes instead. The specialization might be a technology, a project, a product etc. And if you decide to do so, it might be a good idea to have avatars for each team member to indicate who‘s working on which item. The advantage of avatars compared to swim lanes is that they are more fluid. It‘s easier to move avatars from one item to another. And probably even more important in your case: you can easily have two or even three avatars on the same ticket. And that‘s exactly what you want: several people working on the same task, so that the specialist knowledge can spread." 
Again, Peter started to change the layout of the board, and now it looked like this:


I finished my thoughts by throwing a couple of more ideas at Peter: "In order to make more explicit, what you want to achieve, you could introduce a policy that goes like this: In every column there have to be at least two different avatars at any time. You could easily track your improvements by counting the number of tickets on which two or more people have collaborated. It might also make sense to limit the number of avatars per person, to encourage collaboration across specializations...But I feel we get carried away. So before going any further, I think you should gather your team and explain to them what you want to achieve and why. I am sure they have plenty of ideas how to get there themselves!"



Reflecting

One of the biggest benefits of Kanban is that it focuses on queues. Through the board we can gain visibility of the queues. And by limiting work in progress, we actively manage queues and thereby improve flow. Managing queues is a powerful, yet relatively simple way to decrease lead time dramatically. Of course the board can only show us improvement opportunities, if we are willing to look at them and if we know where to look. So the board should be designed in a way that it shows queues immediately. Ideally, this visualization is accompanied by metrics, which show the impact of (often growing) queues over time. 
In my experience it‘s always a good idea to have a closer look at the division of queues and ask questions like: How many different queues do we see? Why are they separated? What would be the advantages and disadvantages of merging several queues into one major one? How high would the investment be? 
I am not saying it‘s always a good idea to have one common queue (the modern post office system). It‘s mainly a tradeoff decision, which should be answered considering different factors like risk, short-term costs, customer and employee satisfaction etc. In fact, many post offices still do have a separate queue for banking issues. This probably makes sense, because it would be very costly to train every clerk in banking tasks.
Another interesting point is to look at this problem at different levels. In this post I‘ve discussed the issue of having individual specialists, whose work is divided by separate queues. Merging these queues might cause more collaboration and spreading knowledge. The same logic applies at a team or even department level. For instance, in product development it‘s worth asking: Do our development teams work on separate queues or do they pull items from a common queue? And again: What are the advantages and disadvantages? What would it take to change this? At which cost? And then at an even higher level: How do the queues look like for our departments and business units? Would it make sense to have a common queue here, as well (at least for some kind of work)? To get a better understanding of these different levels, I find the concept of Kanban Flight Levels as developed by Klaus Leopold very useful. 


What are your experiences with separate vs. common queues (and the post office)? Please leave a comment!


_______________________

Like this post? Then you should check out my post Utilization as a proxy and my more recent post Keep the Ball rollin´

Monday, February 20, 2017

Seriously, what is a Pull System?

Almost 5 years ago, I‘ve published a blog post called What the F*** is Pull? The distinction between Pull System and Pull Behavior, that we‘d come up with earlier at the Kanban Leadership Retreat still makes a lot of sense to me. Yet I keep seeing a lot of confusion around the concept of pull, and I myself often had troubles explaining it in a crisp, comprehensive way. A couple of months ago, I was fed-up, freed up some time and thought about it a little bit more. After a little bit of scribbling and googling, I wrote down a short definition, which I am quite happy with. And like with most things, I did not invent this definition, it had all been written down before. It‘s just that I did not read the right resources before and that the wording of many texts did not convince me. Often, the definitions are too detailed for my taste and too focused on manufacturing systems. So here's how I define a Pull System as opposed to a Push System in a context of Kanban - maybe it‘s useful for others, as well.

Definition of Push vs. Pull

In a Push System, new input is determined by a plan or event. Output has to be adjusted accordingly.




In a Pull System, new input is determined by the system‘s capacity/capability. Input has to be adjusted to the output.

Pull Systems and WIP Limits

Now the connection between Pull and WIP limitation becomes evident. As Don puts it: "WIP limits are inherent to Pull Systems." If the input is to be determined by the system‘s capacity/capability, we 1) have to know this capacity/capability (therefore Lean‘s notion of studying the system and Understanding as one of Kanban‘s core values); and 2) we have to make sure that we never load the system beyond its capacity/capability. The easiest way of doing this is to only allow a new work item to enter the system, after another one has been finished. We have to "read" our system from right to left - just as we should "read" our Kanban board from right to left - hence the slogan Stop Starting, Start Finishing! 


Pure Pull Systems?

It‘s worth mentioning that pure Pull Systems probably do not exist. As Don Reinertsen points out in his brilliant presentation The Science of WIP Constraints, even the leanest system has a push-pull-boundary, meaning that the pull mechanism only starts after a certain process step. Before this step, work is pushed into the system. Even at Toyota, there is a minimum of planning and buffering involved - they don‘t melt new steel for every new car.
What‘s probably more important to knowledge work is the fact that want to achieve as much pull as possible in our system, but we also want our system to be able to absorb some push. Sounds strange, but it enables us to cope with major unforeseen events. In Kanban lingo, most expedite tickets will be pushed into the system. Ideally, our system is under-utilized, so that it provides spare capacity to deal with this extra work. But even if it does not, we might be willing to accept the push, because the cost of waiting for a free slot would be much higher than the cost of temporally overburdening the system. But that should be discussed further in a separate blog post...

_______________________

Like this post? Then you should check out my previous post Keep the Ball Rollin‘


Tuesday, February 7, 2017

Keep the Ball Rollin‘


This is a translated (and slightly modified) version of my article "Der Ball muss rollen!", which was published online at Projektmagazin in May 2014. If you prefer German, you can find the original article here.


Yet another sports analogy

Once I was with a company, where every conference room was soccer-themed. Not only were the rooms named after famous soccer clubs, but they were also decorated with "devotional objects" like jerseys, balls, pennants, etc. Of cours,e you would also find those funny quotes like "Soccer is like chess, only without the dice" all over the place.
I recall one meeting, where people were vividly discussing how effective software development teams should be set up. Probably influenced by the environment, I started thinking about soccer teams and what we could learn from them. Okay, sports analogies are not really new in software development, but let‘s give it a shot...


How (not) to manage a soccer team

Let‘s start with the composition of a soccer team. We‘ll find a goalie, defending players, midfielders, and of course the strikers. Looks trivial at first glance, because in order to win a match, there are "tasks" that need to be done in each of these areas. And obviously, a goalie needs completely different skills than a striker.
But wait a minute! If we think about it a little bit harder, we‘ll find an incredible waste of resources here! Even if midfielders support the defense every now and then and strikers fall back occasionally, it‘s very clear that all players are extremely poorly utilized! What‘s the ball possession of an average striker? Two to three minutes? And if we look at the goalie, it‘s even worse! It looks like he‘s just standing there and waiting for something to happen at least 99.9% of the time. Now think about the ridiculously high salary level of professional soccer players and you‘ll probably be close to a nervous breakdown.
Needless to say, that we as experienced project manager instantly understand the disastrous management we‘re dealing with here! If a striker is only needed, let‘s say, 30% of the time (we also calculate things like zone defence here), then why not have him play three matches in parallel? We would still have a 10% buffer if something goes wrong. Looking at the goalie, we‘ll find even more to optimize, because he‘s "needed" even less often. So he could easily play ten matches simultaneously. That‘s good, because our amateur teams are desperately looking for a better goalie...
Now let‘s take a quick look at the substitution bench! Here we‘ll find plenty of great resources that are not utilized at all. They are paid for doing nothing! So let‘s reduce the number of substitutes dramatically. How many of them do we really need? Three should be more than enough. All the others could be used way better outside of the bench: They could play in other teams, train our junior teams, give out autographs at the mall, etc.


How (not) to manage a project

All this is, obviously, nonsense! Nobody would even consider optimizing a soccer team in the way just described. But why not? Why exactly is it common in professional soccer teams to under-utilize extremely expensive "resources", while it scares us to death to do the same thing in knowledge work? One major difference lies in the fact that in soccer it‘s really easy to see the damage that‘s done, when a player is not at the right spot at the right time: the other team scores! In professional soccer, the difference between scoring a goal and allowing the other team to score, is probably worth a 6- or 7-digit number. Given this order of magnitude, who cares if a player is not fully utilized?
And a second point comes into play: In soccer, it‘s clear to everyone, that what the coach can do is to train the team and provide them with a viable strategy. What he can not do, though, is to come up with a detailed plan for the whole match - or even the first ten minutes. If this would work, we could indeed create plans for every individual player, and we could even have them play several matches simultanously. The plans then would read something like: "At 15:53 pass the ball to player 8...At  15:57 prevent the ball from being lost on field 3...At 15:59 score a goal on field 2..." Of course it‘s ridiculous to even try this, because we know a soccer match is way too complex to even try to plan at this level of detail (1). And we all know that not everything goes according to our plan in a soccer match, and we have no chance of predicting what the opposing players will do at any time.


Comparing projects to soccer?

Let‘s summarize what we‘ve got so far: When it comes to professional soccer, we‘ve long accepted the fact that the high degree of uncertainty makes it useless to come up with detailed plans upfront. In addition, it‘s relatively easy to access the risk of a player not being at the right spot at the right time. This enables us to make reasonable trade-off decisions: How high are the costs of under-utilized players compared to the cost of a delay, because the team has to wait for a player, who‘s not ready to take-over the ball or block an opponent? In soccer, this cost of delay is so enormous, that dramatic under-utilization is accepted even for players who earn millions of Euros.

If we keep this in mind, perhaps the comparison to project work is not that absurd after all! Just as in soccer, in many projects we have to deal with great uncertainty, that often renders our beautiful plans useless. Also, in project work cost of under-utilization and cost of delay are factors that should be taken into consideration (2).


A fresh view on resource planning

Just to be clear: under-utilization of people and machines does matter, because it can lead to lost opportunities: When a highly skilled expert is idle, she might do something of great value elsewhere instead. That‘s the reason why it seems totally normal to us to come up with plans that make sure this person wil never be idle. What we ignore, though, is the fact that costs also occur when a project (or even a supposedly small milestone) is blocked, due to an expert, who is not instantly available.
If we would have more clarity on these two different types of costs, we would certainly make different decisions and our resource planning would appear in a different light. For instance, in some contexts it now might make a lot of sense to build and keep stable, cross-functional product teams, consisting of developers, testers, analysts and designers. There might be times when a designer or a tester is not fully utilized. But when she is needed, she will be there to help the team immediately (just like the soccer player when a ball is passed to him). This is a major advantage and solves a lot of problems we witness in our daily project work: low quality due to frequent context switches; rework due to long feedback loops; poor transparency on our project‘s progress, because all work packages are "80% done", just to name a few.
It‘s true in project work as much as in soccer: We must keep the ball rolling (3)! If we manage to do this, it‘s way less relevant, how many players are moving at which pace. This, by the way, is the difference between resource efficiency and flow efficiency: When we focus on resource efficiency, we make sure that everyone is busy; when we focus on flow efficiency, we make sure that we make progress on the most important tasks at any time. For several decades now, we only took resource efficiency into account. It‘s time to give flow efficiency priority now (4)!

P.S. I‘ve just learned that there‘s an old song called "Keep the Ball Rollin‘" by a band called Jay & the Techniques. Looking at the lyrics, I don‘t think the song has much in common with this blog post;-)

(1) Funny enough, I‘ve just finished reading the book Team of Teams by General Stanley McChrystal. To illustrate one of his points, McChrystal describes, how the coach of a fictional basketball team tries exactly this level of detailed planning and fails miserably, despite the fact that the team comprises of the world‘s finest athletes.
(2) For more details on cost of delay, look into see the brilliant analyses of Don Reinertsen.
(3) The idea is not new, neither is explaining it with sports metaphors:-) Years ago, Don Reinersen  coined the phrase: "Watch the baton, not the runner!"
(4) Niklas Modig brilliantly illustrates the difference between resource efficiency and flow efficiency in his book This is Lean: Resolving the Efficiency Paradox and in this Ted Talk.

_______________________

Like this post? Then you should check out my post Utilization as a proxy and my more recent post Seriously, what is a Pull System?

Sunday, November 20, 2016

Thoughts on Groupthink

During the past couple of months I‘ve done quite some reading on cognitive biases. Wikipedia states that “[a] cognitive bias refers to a systematic pattern of deviation from norm or rationality in judgment, whereby inferences about other people and situations may be drawn in an illogical fashion.” As I read more about biases and how our brain works, I had to realize that humans are incredibly bad in making rational decisions. We often believe that we analyze a situation carefully, then evaluate the pros and cons of a decision and decide for the option with the best cost-benefit-ratio. In fact, that‘s not true. And even worse: Our brain tricks us into thinking that it is true! If we can believe the many findings of neuroscience and social psychology, our decisions are heavily influenced by factors that we are not even aware of.


What is groupthink?

One of these factors is the very strong human need of relatedness. When people feel that their belonging to a group is in danger, it‘s very threatening to them. In fact, our brain works pretty much in the same way it did 10,000 years ago. And back then, it actually was a death threat when someone was excluded from a group. The thing is: When we feel threatened, a couple of very archaic mechanisms kick in, which tend to overrule all rational reasoning.
So humans have a very strong tendency to conform with a group. This leads to groupthink, where “the desire for harmony or conformity in the group results in an irrational or dysfunctional decision-making outcome. Group members try to minimize conflict and reach a consensus decision without critical evaluation of alternative viewpoints, by actively suppressing dissenting viewpoints, and by isolating themselves from outside influences."
Groupthink is also called the Bay-of-pigs-effect, because the disaster of the invasion to Cuba is believed to be caused by groupthink: People self-censored their doubts about the plan, because they felt it was not appropriate to contradict the predominant opinion in the group. Fortunately, Kennedy learned his lesson and - less than one year later - had mechanisms in place to avoid groupthink and make much smarter decisions during the Cuban Missile Crisis. Other really severe incidents like the explosion of the Challenger Space Shuttle have also been linked to poor decision making based on groupthink.
Some people say that World War III has been avoided during the Cuban Missile Crisis, because Kennedy had learned how to avoid groupthink and embrace dissent. Image credit: doe-oakridge on flickr
Now think about a group in the context of knowledge work - could be a team at a planning meeting or retrospective; could be a management group talking about strategy; could be a committee organizing the next christmas party. When you use the same approach I used to use in my role as a moderator, chances are that the decisions of these groups are flawed. Why is that? Common moderation techniques rely on writing topics on cards, clustering them and eventually having the group dot voting the topic they want to tackle. Nothing wrong with this, but the devil‘s in the details!


A real-life example

Let‘s look at a real-life example of a workshop I was observing. The group had been talking about areas for improvement and how good they thought they already were (from 1=excellent to 5=very bad). You can clearly see big clusters and very little variation.
When the moderator saw this, he said something like: “Excellent, you are all very aligned here. Let‘s go on with the topic you‘ve rated worst.”
The problem was that people dot-voted one after the other or in small groups. So the later a person voted, the more votes were already visible. In such a situation, according to neuroscience, it becomes more and more difficult to give a completely different vote. In this context it‘s extremely interesting to look at Asch‘s Conformity Experiment, where people gave obviously wrong answers to a simple task, because they wanted to conform with the group.
When I think back to the moderations I did, I did not take this much into consideration (and probably others don‘t, as well). So what can we do about it? I think first it‘s important to acknowledge that groupthink (like all cognitive biases) is not a bad thing in itself. It serves an extremely important purpose. But in many of our modern contexts, it becomes an obstacle.

What can we do about it?

Because of their evolutionary importance, biases seem to be hard-wired into our brains. Therefore it‘s of little help to tell people not to groupthink. Here‘re some things I think are more useful to reduce the effects of groupthink:
  • Let people vote simultaneously, if possible.
  • Let people think about the topic individually or in small groups. Let them write down their opinions/votes and only after that collect all the individual votes. This gives them the space to think about the issue with very little influence from the big group. Also, if people have written down their preference on a piece of paper one minute ago, it‘s quite hard for them to deviate from this preference shortly after.
  • When you decide to do a circle, where every group member voices his/her opinion, make sure that the boss and obvious opinion leaders speak last.
  • Try to make the group as diverse as possible. Diverse groups have proved to be less vulnerable to groupthink.
  • Try to add people to the group, who are likely to have a different opinion than the majority of the group.
  • Invite an outside expert to the group. Outsiders are less vulnerable to groupthink (especially when they know they are only temporarily a member of this group). This is one big advantage of consultants, especially when you tell them they are paid, because they are less likely to conform with the group.
  • Observe closely, which members are quiet during a workshop or meeting and explicitly encourage them to voice their opinion, even if it‘s controversial. Usually we assume that the quiet ones agree with the group. But what if they are the ones who disagree but are afraid of isolating themselves from the group? Think about what would help those people to gain confidence and openly disagree.
  • Explain the role of devil‘s advocate to the group and assign this role to one person. The explicit role makes it much easier to speak up, because people know it‘s “the role” speaking, not necessarily the person (and people might dislike the role, but they won‘t dislike the person who disagrees). Actually I think the “10th Man Doctrine” was the one cool thing in the movie World War Z. It says: “whenever 9 people looking at the same information come to the same conclusion, it's the 10th's duty to disagree and actively look for evidence to the contrary.” Although this doctrine is not a real thing, I was surprised to learn that the Israeli military seems to have a devil‘s advocate office, which task it is to ensure that “intelligence assessments are creative and do not fall prey to group think.” (see this thread on Quora).
  • Think about rules and processes that help dealing better with groupthink. One such rule could be: “If everyone in the group agrees, we assume we have overlooked something and we should actively look for different angles.”
  • Be very aware of signs of pressuring dissenters. Phrases like “Are you with us or not?”, “Are you a team player or not?”, “You‘re either in or out!” should get all alarm bells ringing.
  • Brief someone to wear the clown suit. I stole this metaphor from this blog post about lonely dissent. The author states that “[l]onely dissent doesn't feel like going to school dressed in black. It feels like going to school wearing a clown suit.” Modifications of Asch‘s experiment have shown that chances to disagree with the group grow dramatically as soon as at least one other person has disagreed before. This first person is the one with the clown suit. Look for someone with enough confidence and standing to wear the clown suit. And if you consider yourself a leader, think about wearing the suit yourself.
  • As a leader, reward deviating from the group‘s opinion. I am not talking about monetary incentives here. I think it can be extremely powerful to praise someone in front of the group for wearing the clown suit.
What are your experiences with groupthink? I am happy to read your comments below!

P.S. If you find cognitive biases interesting, you might like my posts Thoughts on survivor bias and Thoughts on confirmation bias

Tuesday, April 19, 2016

An Alternative View on Company Structures

For years my thinking about company structures went like this: “The more structure a company has, the more it sucks.” So I was arguing against putting new structures in place, whenever I heard of such ideas. Of course I knew, that no company could exist without any sort of structures. So my credo was: Let‘s only have structures, where it is absolutely necessary. And in my mind it only became necessary when we were growing and the new size made it necessary to come up with new structures, because otherwise things break down..
Potential downsides of rigid org structures are well known: Less freedom for the workforce, and hence less engagement and less innovation; Dilbertesque policies that might make sense for some part of the organization but not for the rest of it; single points of failure due to hierarchical pyramid structures etc.

Back to my credo: “Let‘s only have structures, where it is absolutely necessary”. I still think it‘s valid, but here comes the catch: I‘ve realized that there are other things than sheer growth in headcount that can make it necessary to add more structure! Here are three things I think are worth taking into consideration: Diversity, Fairness and Health.

Diversity
There‘s plenty of research (see this article for more resources) that shows how more diverse groups make better decisions. When people think about diversity, they mostly think about gender. While gender is very important and there is a lot to improve, especially in the tech industry, we should also think of diversity in terms of age, race, cultural and economic background, sexual orientation, political preferences, family situation, etc.
When I talk about diversity with my colleagues, we very often end up with the idea that we should have more structures. There are two reasons for this:
  1. If we want to have a more diverse workforce, we have to change the way we recruit and hire people. For a long time the way I did job interviews went something like: Let‘s have a coffee together and afterwards we do a thumb vote if we want to work with this person or not. If you use a process like this, you can be 100% sure, that your decision is affected by all sorts of cognitive biases and that you are biased against diversity. Like one of my colleagues put it nicely: “Having more diversity in a group feels like a grain of sand in the gear.” It feels uncomfortable. Humans are hard-wired to prefer being with people who are like them. And this is exactly what we want to avoid when we talk about diversity. One countermeasure for this are structured job interviews. Google does this rigidly, as Lazlo Bock (Head of People Operations at Google) presents in his book and this article. What are structures interviews? Not only are the questions for a job  interview formulated beforehand, but there‘s also a definition of the types of answers that are considered to be good/mediocre/bad. Or, as Bock puts it, a structured interview is “a consistent set of questions with clear criteria to assess the quality of responses”. And Google even goes one step further: The hiring decision is not made by the people, who did the interview, but by an separated committee. Sounds like a lot of structure, right? It certainly does to me and I was terrified, when I heard this for the first time. But if we take diversity and de-biasing seriously, this might be the (or at least one) way out.
  2. As soon as we become more diverse in our workforce, we might also need more structure. My colleague Boris just shared his thoughts on this with me: “If we were all clones of each other, we wouldn‘t need any structures. Everyone knew exactly what the others think, how things work and what the next steps are. But this would be zero diversity. If we have people with different backgrounds, we need more explicit structures, otherwise people get lost.” This totally makes sense to me. If all your employees are 30-year old, left-winged white male surfer dudes without kids, you probably don‘t need much structure, because their thinking might be very aligned. And if they face a problem, they will find a way to work around it easily, because every evening they‘re drinking beer together. So this scenario is very comfortable, and although it‘s intentionally exaggerated, I think a lot of start-ups work in a similar way. It might be okay, or even necessary for a small company to operate in such a way, but if the company is growing, and especially if it‘s critical to improve the quality of decisions, increasing diversity becomes very important. And that increased diversity comes with the need for more structures.
Fairness
Every organization has implicit and explicit structures. And probably it‘s a good heuristic to say that the less explicit structures you have, the more important the implicit structures become. That can be considered unfair, because it favours those who are good in navigating through blurry structures. Often the best (or even the only) way to get things done in such a context might be having good personal relationships with the most influential people in the company. For new people it can be really hard to join this game, because the rules are...implicit. And it gets even worse, as soon as you get more diversity. Imagine that, for the first time, a single mother joins the company. She probably does not have the time nor the interest to hang out every other evening with her colleagues.
In this classic feminist article from the 1970s, Jo Freeman argues that so-called structure-less groups are undemocratic, because they tend to be dominated by elites, who are not accountable to the larger group: “For everyone to have the opportunity to be involved in a given group and to participate in its activities the structure must be explicit, not implicit.”
I am not very familiar with the feminist movement, and the groups Freeman talks about are political groups, not companies. Still the argument makes a lot of sense to me and made me think.

Health
While organizations with little (explicit) structure provide a lot of opportunities, they also might make it easy for people to jeopardize their health. The reason for both, the good and the bad aspect of little structures, is what I would call “anything goes”. If responsibilities, decision-making-mechanisms, team structures, career paths etc. are unclear, the organization might be able to exploit the advantages of fast decision-making (“if it‘s not clear, who makes this decision, let‘s just decide in this group - right now”) and fluid teams (“let‘s team up and build this thing”). On the other hand, the same context might encourage people to work in an unhealthy way. By this I not only mean the sheer amount of hours they work, but also the effect of over-commitment and mental overload. Because it‘s unclear, where my responsibility ends and what the company expects from me, I might take on everything I find interesting or important. Of course I can only do so many things, but I am very ambitious, and nobody stops me from starting all these exciting things. So maybe I should start working a little bit longer every day and think about all the interesting problems at the weekends and during my holidays?

What now?
I think it‘s important to realize that structures in itself are neither good nor bad. And I am not making the case for excessive structures. Like with many things, it‘s a permanent trade-off decision we have to make. Instead of falling in the trap of binary thinking (“all structures are good/bad”) we should think about the pros and cons of adding more structure and then find a healthy balance for our context. My impression is that in the Agile/Lean community we have very much focused on the downsides of structures in the past. Maybe it‘s time now to take the upsides into consideration a little bit more.

_______________________

Like this post? Then you should check out my previous post Radical Transparency? and one of my newer posts Seriously, what is a Pull System?