Inadequacy and Modesty

  • When should you believe that you can do something unusually well?
  • How do you distinguish between a bad idea an insufficiently explored good idea?
  • Modest epistemology
    • Eliezer noticed that the Bank of Japan’s monetary policy was not creating enough money
    • Why should he be confident in his conclusion?
    • Surely the Bank of Japan is staffed by experts and they know better
  • Why shouldn’t we go with conventional wisdom?
  • Conventional wisdom only applies in a limited set of circumstances
    • Lots of data
    • Fast feedback loop
    • Exploiting an inefficiency corrects that inefficiency
    • People who exploit inefficiencies are rewarded
  • The problem wasn’t that no one knew that the Bank of Japan’s monetary policy was broken, the problem was that no one had any incentives to correct it
  • We encounter situations in which conventional wisdom is wrong in everyday life
    • Eliezer noticed that an effective cure for Seasonal Affective Disorder was simply more light
    • But why should he have believed that more light ought to work?
    • There were no studies showing that more light was beneficial
    • Nevertheless he had a hunch that more light would work, so he spent $600 on LED bulbs and turned his entire apartment into a lightbox
  • Given that there was a fairly simple remedy, why weren’t there studies examining the use of more light as a cure for Seasonal Affective Disorder?
  • Humanity isn’t being maximally opportunistic about finding treatments for Seasonal Affective Disorder
  • If you want to outperform society, you have to find places where society isn’t being maximally opportunistic

An Equilibrium of No Free Energy

  • We need to distinguish between three concepts
    • Efficiency
    • Exploitability
    • Adequacy
  • Efficiency
    • Efficiency merely means that you can’t predict price movements in a market
    • Has nothing to do with justice
    • The efficiency of a market is relative to the participants - a market might be effcient for an average trader, but inefficient from the perspective of an hedge fund manager or computer-driven algorithm
  • Exploitability
    • Not all things involving money are efficient
    • Housing market has way more money in it than the stock market, but is far less efficient
    • It’s not easy to make money from correcting inefficiencies in the housing market - no way to profit from house prices going down
    • The housing market is inefficient, and the inefficiency persists because it is not exploitable - no way to arbitrage across the inefficiency to make money
  • Adequacy
    • Have all of the easy solutions been exploited?
    • Example: seasonal affective disorder
      • Eliezer found that hanging a bunch of LED lightbulbs was an easy solution
      • So why isn’t this a standard solution?
    • How does inadequacy arise?
      • Inadequacy arises when the incentives in a system aren’t what you think they are
      • It’s not sufficient for one part of a system to be broken - there have to two parts with misaligned incentives so that a single individual can’t fix it without having the resources to change the entire system
    • Inadequate systems are like inefficient markets in that even though there are easy solutions available, the existing incentive structures make it imposible for people to exploit those solutions
    • When Eliezer found that hanging up lots of LEDs was an effective treatment for SAD, he didn’t go around telling SAD researchers about it - it’s an obvious enough idea that if the incentives had been in place to study it, it would have been studied
  • It’s easy to come up with a priori arguments for why a system is inadequate
  • Before you conclude that the system is inadequate, do some research and see if the problem has been studied

Moloch’s Toolbox

  • A primary cause of inadequacy is being stuck in “inferior Nash equilibria”
  • An inferior Nash equilibrium is a Nash equilibrium that is strictly worse than other Nash equilibria in the same system
  • Once a system has reached a Nash equilibrium, it takes a lot of energy to break out
  • Example: medical system
    • The US medical system is broken in a lot of ways, but to make it better off would require coordinated changes in two or more components
    • Because there is no external coordinator this will never happen
  • Inferior Nash equilibria become really bad when they cascade
  • For example: the reason the government can’t step in as the external coordinator for the medical system is because the government itself is stuck in an inferior Nash equilibrium that prevents it from doing so
  • Even if everyone recognizes the bad Nash equilibrium, the fact that there are so many interlocking inferior Nash equilibria means that no one has the resources to make things better on a global scale
  • The best we can hope for are local changes that incrementally make individual lives better by bypassing parts of the broken system