The Costly Coordination Mechanism of Common Knowledge

Prisoners’ Dilemmas vs. Coordination Problems

  • The prisoners’ dilemma
    • Setup:
      • Two people are in a situation where they cannot coordinate
      • They may choose to cooperate or they may choose to defect
      • If they both cooperate, they both gain a moderate amount
      • If one cooperates and the other defects, the cooperator gains, while the defector loses
      • If both defect, they both lose a moderate amount

      • Payoff matrix:

                    Cooperate Defect
  Cooperate   3/3       4/0
  Defect      0/4       1/1
    • Examples:
      • Nuclear disarmament - no country wants to use nuclear weapons or have nuclear weapons used on it, but countries maintain nuclear arsenals to defend against the other countries who have them
      • Doping in athletics - no athlete wants to dope, but athletes dope because otherwise they risk falling behind the defectors who do dope
    • The scaled up version of the prisoner’s dilemma problem is the free rider problem
  • Coordination problems

    • With the prisoner’s dilemma in mind, let’s look at a slightly different payoff matrix

                    Cooperate   Defect
  Cooperate     3/3         0/0
  Defect        0/0         1/1
    • Note that this differs from the prisoner’s dilemma problem in that there’s no reward for defecting
    • There are two Nash equilibria here, not one
    • However, no player can unilaterally move from defect/defect to cooperate/cooperate, even though everyone would be better off with the cooperate/cooperate option
  • Prisoners’ dilemmas can be solved by changing incentive structures to make defecting more costly
  • Coordination problems can be solved by looking at information flows and common knowledge
  • Common knowledge is the confidence that someone else will act in the way that you expect them to act, after a small exchange of information (like a knowing look)
  • Creating common knowledge is a powerful way of making coordinated action easier, and many of our social rituals are designed to do just that
    • News
    • Advertising
    • Education
    • Corporate mission statements
  • The production of common knowledge at scale is very expensive
  • However, most people’s instinct’s for the cost of generating common knowledge are calibrated to groups of Dunbar’s Number
  • This is why when many people encounter a coordination problem, their response is exasperation – they don’t understand how hard it is to generate the common knowledge that’s necessary to overcome the problem
  • When attempting to solve coordination problems at scale, it’s much more fruitful to pay attention to information flows and incentive structures than it is to become indignant

It’s Not What It Looks Like

  • The representativeness heuristic is when we think that something is what it looks like
    • Note: Duncan is wrong. He’s referring to correspondence bias not the representativeness heuristic. However, in order to match the original text, I’m continuing to use “representativeness heuristic”
  • We’ve learned in certain situations (race, gender, etc) that the representativeness heuristic can lead us astray
  • However, we’ve implemented these exceptions as a blacklist, not as a whitelist
  • We think that representativeness works in most situations, and only fails in certain situations, when maybe we should think of it as working in only certain situations and failing in most situations
  • Example: posting party pictures on Facebook
    • People caution against posting party pictures on Facebook
    • They make it clear that while they don’t have any objection to these pictures, they’re worried about others with power (namely, employers) who might have an objection
    • But do employers really think that employees don’t party on weekends?
  • So how does this dynamic get started?
    • Someone posts a photo of them doing something really egregious while inebriated
    • They get fired because of it
    • This tells all the cautious people that they can get fired because of things they post on Facebook
    • As a result, those people clean up their Facebook profiles
    • This leads to a ratchet effect that eventually leaves only the most egregious partiers posting party photos on Facebook
    • At this point employers are fully justified in using party photos as a negative signal, since the fact that someone is posting party photos on Facebook means that they’re more likely to be a poor employee
  • Other examples:
    • Parenting
      • There is a constant ratchet for parents to be ever more attentive to their kids
      • This is not just because the parents want to pay more attention to their kids, but also because they’re afraid of the social approbation that will result if they don’t pay attention to their kids
    • Using shorthand from mystic and mythic traditions to teach rationality
      • These traditions often do have wisdom
      • However, because they sound like what crazy mystics say, one has to spend half their time adding disclaimers in order to prove that one is not a crazy mystic
  • Split and commit
    • We can deal with the representativeness heuristic with a technique called split and commit
    • Split your impression into two parts
      • Is the situation mostly what it looks like
      • Is the situation mostly not what it looks like
    • Commit to a different course of action in each scenario
    • Split and commit leads to a world where there’s more critical thinking at critical moments

Common Knowledge and Miasma

  • There are three layers to “common knowledge”
    • Everyone knows the information
    • Everyone knows that everyone else knows the information
    • Everyone knows that everyone else knows that everyone else knows the information
  • This third level is important, because if it’s not clear that everyone knows that that everyone else knows something, it’s possible for people to feel uncertain in their common knowledge
  • This lack of common knowledge can lead to people getting called out for behaviors that no one in the group has any objection to, but which people think that other people might find objectionable
  • Miasma
    • People are able to track about three levels of “meta”, but they unconsciously track many more
    • These other layers get compressed into snap judgments
    • When we talk about people reacting “for no reason”, we’re usually talking about the effects of miasma
  • How do we fight miasma
    • Ask ourselves whether we can name who would be bothered by a particular action
    • Then ask ourselves whether we can take specific actions to address concerns, rather than just not doing the thing

Unrolling Social Metacognition: Three Levels of Meta Are Not Enough

  • Conceptual introduction by example
    • Alex leaves the milk out
    • Bailey observes Alex leave the milk out and thinks that it’s bad
    • Alex feels judged by Bailey’s observation
    • Alex reflects on being judged, doesn’t like it, and concludes that Bailey is a “downer”
  • Higher levels of metacognition get distilled into “is a downer” or “is welcoming”
    • These levels track reality, but are often oversimplifications
  • People don’t usually unroll their metacognition when it comes to social events – why?
    • People aren’t aware that this is something that they can do
    • Requires work – usually a shared Google doc or whiteboard
  • Why can’t this be replaced by a simple analogy
    • Sometimes you want the other person to feel judged
    • Sometimes saying you’re sorry comes off as a platitude
  • How generalizable is this unrolling technique
    • Has been used with multiple people
    • Works with people who don’t have a background in symbolic reasoning
  • Relation to miasma and hype
    • “Miasma” can be viewed as negative ungrounded social metacognition
    • “Hype” can be viewed as positive ungrounded social metacognition