Reading Notes: 'NOISE 組織はなぜ判断を誤るのか?' (NOISE: Why Organizations Make Wrong Decisions) by Daniel Kahneman
I read both volumes of 『NOISE 組織はなぜ判断を誤るのか?』ダニエル・カーネマン(著) (NOISE: Why Organizations Make Wrong Decisions by Daniel Kahneman), so I’ll share the insights I gained from this book.
I read this book after Shintaro Yamada introduced it on his blog and Yo Shibata introduced it on the START/FM podcast.
Below are quotes and notes from sections that left an impression on me.
Let's talk about noise in sentencing: "Studies show there are significant differences in sentence severity for the same crimes. This is unfair. It's clearly wrong that sentences differ depending on which judge is assigned." "The severity of punishment shouldn't be influenced by a judge's mood or whether the day is hot or cold." "Sentencing guidelines are one way to address this problem. But many people dislike guidelines. They say it narrows the discretionary space needed to make appropriate judgments. Certainly, you could argue that each case is different."
Let's talk about noise in insurance companies: "In insurance companies, the quality of professional judgment is extremely important. We thought judgments would be nearly the same regardless of who was in charge, but this premise was wrong." "System noise was five times more than expected. That is, five times the acceptable level. We wouldn't have noticed without noise auditing. The noise audit shattered the illusion of agreement." "System noise is a serious problem. Losses would amount to hundreds of millions of dollars." "Where there is judgment, there is noise. And more than we think."
Let's talk about one-off judgments: "This is indeed a rare event, but the current approach seems likely to create a lot of noise." "One-off judgments are just cases where repetitive judgments happen to occur only once. Don't forget this." "Is the past experience you're basing your judgment on relevant to the current decision?"
Let's talk about the error equation: "Reducing bias and reducing noise by the same amount seem to have the same effect on accuracy." "Reducing noise in predictive judgments is very effective. Whether there's a lot or little bias doesn't matter." "The ratio of predictions above and below the true value was 84:16, so there was considerable bias. Still, if it's a normal distribution, there would be as much noise as bias." "Every decision involves predictive judgment. In predictive judgment, accuracy should be the only goal. So keep your personal values separate from facts."
Let's talk about noise analysis: "When judges differ in their level of severity, level noise exists. When judges disagree on whether to be harsh or lenient with a particular defendant, pattern noise exists. Part of pattern noise is occasion noise. That is, the same judge makes different decisions on different occasions." "In a perfect world, defendants would receive justice. But in reality, they're at the mercy of a noisy system."
? You are not always the same person.
Many other accidental factors cause occasion noise. Stress and fatigue can be sources of external factors that shouldn't influence professional judgment.
? NOISE: Stress and fatigue
Candidates: High evaluations hit the mark by fluke in cases where one item is exceptionally good, so you overlook that others are terribly inferior. Personnel evaluation: Rating someone as a special "superstar" is usually wrong.
? Ranking rather than scoring.
? The first case tends to become the reference point. Ranking judgment.
That’s all from the Gemba about wanting to reduce NOISE from all judgments.