Meditations on Moloch

  • Alan Ginsberg’s poem on Moloch conceives of civilization as a distinct entity - Moloch
  • Moloch is the answer to the question: “Why can’t we have nice things?”
  • Thinking about systems as agents shows us all the ways in these systems are not agents
  • Systems endure because we don’t have good coordination mechanisms to get rid of them
  • The reason we don’t have good coordination mechanisms is because we are stuck in multipolar traps
  • Examples of multipolar traps
    • Prisoner’s Dilemma - the best outcome is {cooperate, cooperate}, but the stable outcome is {defect, defect}
    • Dollar auctions - if you set up auction rules correctly, you can make someone pay $10 for a $1 bill
    • Tragedy of the commons - everyone would like the common resource to be preserved and harvested sustainably, but people’s individual incentives are to maximize exploitation, leading to the degradation of the shared resource
    • Malthusian traps - any group that sacrifices a value (such as art, or leisure) in order to maximize reproductive advantage will have a slight advantage over groups that don’t
    • Capitalism - capitalism can be seen as a Malthusian trap for organizations rather than individuals
    • “Two income” trap - if every household has both parents working, then no one is relatively better off, and everyone is absolutely worse off
    • Agriculture - there is a substantial body of evidence that hunter/gatherers had more enjoyable lives, but societies with agriculture were able to support more people, even though those people were not as happy
    • Arms races - countries have to spend a substantial amount of money on defense, otherwise they’ll be overrun by countries that spend even more
    • Cancer - in the absence of an external coordination authority (such as an immune system), cells will “defect” and form cancers, improving their own reproductive success at the cost of the health of the overall organism
    • “Race to the bottom” - in an effort to attract investment, politicians will reduce regulation - this leads to every jurisdiction lowering regulations, and no one being relatively better off while everyone becomes absolutely worse off
  • Once one agent becomes more competitive by sacrificing some value, then all agents have a choice: either sacrifice the same value yourself, or be outcompeted
  • Even when we have external forces that prevent these processes from reaching their most extreme state, we can get some pretty bad outcomes
    • Education - it’s easy to think of a better education system, it’s hard to change the incentives to allow that education system to actually exist
    • Research - scientists are stuck with using obsolete statistical methods because there is no “science czar” who has the ability to impose changes wholesale
    • Corporate welfare - even though no one wants to spend millions of dollars on incentives for corporations, people feel like they have to in order to compete with other jurisdictions
    • Congress
      • Has 7% popularity as a body
      • But individual representatives have a 62% approval rating, on average
      • Representatives have an incentive to sell out the interests of the country as a whole in favor or the interests of their district
  • There is a common pattern to all of the multipolar traps described above
    • Whenever a group in competition for X has an opportunity to throw out some other value Y in return for improved ability to get X, it will do so
    • Then other groups can either do the same thing, or lose access to X
    • The worst case of this is the Malthusian trap, which reduces everyone to subsistence
    • So why do we persist with these traps? Why don’t we unilaterally change the system to make things better?
    • The incentive to change the system is itself an incentive - you have to create a sort of meta-incentive that will make it possible for people to change object-level incentives
  • Our current civilization was not designed, by humans or by anyone else
    • Civilization is the result of people following incentive gradients
    • People follow incentive gradients just like water follows a gravitational gradient and heat follows a thermal gradient
    • In its own way, Las Vegas is as inevitable as the Grand Canyon
  • So why haven’t things degraded “all the way” - why do we still have some “nice things” like art and culture?
    • Excess resources - we have excess resources which allow us to (temporarily) escape the Malthusian trap
    • Physical limits
      • People, due to biological limitations, need a certain amount of rest and leisure
      • The only reason we don’t have a Malthusian explosion is because women can have babies only so quickly
    • Utility maximization
      • Many intense competitions in the marketplace have the side-effect of maximizing human values rather than minimizing them
      • Corporations have to maintain a certain minimum level of safety and quality, otherwise they’ll lose their customers
      • Corporations have to maintain a minimum level of working conditions otherwise they’ll lose their workers
      • But this is a fragile equilibrium - if something allows corporations to decouple economic growth, they’ll do so
      • Democracy has the same problem as capitalism here - just replace corporations with candidates
    • Coordination
      • The opposite of a jungle is a garden
      • Garden implies gardener - agent imposing rules and solving coordination problems “from above”
      • This is government, but it is also many other organizations (guilds, unions, social clubs, etc)
  • The problem with technological advancement is that it weakens all of the above constraints on multi-polar traps
    • Excess resources can be found and exploited faster than ever
    • Physical limits can be overcome (i.e. ems who don’t need leisure or rest)
    • Utility maximization suffers as technology (robots) break the link between maximizing human values and success in the marketplace
    • Coordination - right now technology is used as much to come up with new ways of defeating coordinating institutions (e.g. cryptocurrency) as it is to come up with improved coordinating institutions
  • This process ends in one of two ways: either we build a UFAI that wipes us out or we have a civilization of ems who only fulfill narrow market-driven niches (perhaps without even requiring consciousness)
  • Both outcomes involve the loss of humanity in some sense
  • There are those who would work with Moloch
    • Nick Land talks of “Gnon”
      • Calls Gnon the god of “natural law”
      • Argues that civilizations that diverge too much from Gnon/Moloch’s laws decline and fall
    • However, fetishizing Moloch gets you nothing - your civilization loses all of its values just the same, only by choice instead of by force
    • Warg Franklin talks about walling off Moloch
      • Argues that authoritarian government and conservative social institutions can ward off multipolar traps
      • But what makes a society survive isn’t what makes a society good
      • Moreover, even if this could prevent Moloch from eating society from the inside out, the society can still get eaten from the outside in
      • Someone else maximizes reproduction, meme virulence, firepower, resource exploitation, and this society is outcompeted
  • So what is to be done?
    • You cannot bargain with Moloch
    • Submitting to Moloch brings no benefit
    • Moloch cannot be walled out
    • But maybe… Moloch can be killed
  • Our civilization stands on the threshold of making superintelligent AI
    • Such a machine could be the ultimate coordinator and could rewrite incentive structures to preserve human values rather than erode them
    • We’re only going to get one chance at this
    • We’re going to lift something to Heaven - we should make sure that thing shares our values and is not a blind optimizer

The Toxoplasma Of Rage

  • PETA vs. Vegan Outreach
    • PETA uses outrageous tactics and everyone knows about them
    • Vegan Outreach, which does much the same work, uses much less provocative tactics, but hardly anyone has heard of them
    • PETA doesn’t continue with its tactics because they’re stupid or evil
    • They continue with their tactics because their incentive structure pushes them to be as provocative as possible in order to generate maximum publicity
  • Highly publicized rape cases
    • Only 2-8% of of rape allegations are false
    • Yet the proportion of rape cases that turn out to be false on social media turns out to be an order of magnitude higher
    • These cases are being promoted by feminists, so you’d think the ones that would have the most publicity would be the strongest cases, not the weakest ones
  • It turns out that both PETA’s tactics and the disproportionate discrediting of viral rape accusations arise from the same root cause
    • Something that is true is universally acknowledged to be true, and people stop talking about it soon after
    • Something that is controversial sparks debate and stays in the spotlight for days or weeks
    • Double bind: you can either get people to admit that something is true and ignore it or you can get people to debate you and spread controversy
  • This dynamic also comes up in race relations
    • When polled about the Michael Brown shooting, 73% of white people sided with the officer
    • When polled about the Eric Garner choking, 63% sided with the black victim
    • Yet the Eric Garner killing did not go viral, even though it took place prior to the Michael Brown killing, and there was more agreement over its injustice
    • And while the Michael Brown controversy did real and lasting damage to race relations, it had a barely measurable impact on the one concrete policy proposal that resulted from the controversy - usage of body cams
    • Both whites and minorities are overwhelmingly in support of body cameras and this support barely budged as a result of the Michael Brown controversy
  • The most highly viewed posts on Slate Star Codex are also the most useless
    • Race, gender, politics posts get an order of magnitude more votes than posts about charity, or transhumanism
    • If Scott were making a living off his blog, he’d write non-stop about controversial topics
  • Toxoplasma
    • Toxoplasma is a parasite with two forms
    • In its cat form, it breeds in cats, and is excreted by them in their feces
    • It then gets into the water supply where it enters the brains of mice
    • It makes mice behave in a more risk-taking manner, which causes them to be eaten by cats, thus spreading the parasite
  • What would an idea look like, if it had the same life-cycle as toxoplasma?
    • Jihad/War on Terror
    • Islamic fundamentalists come up with ever more effective and provocative ways of killing American soldiers
    • Americans come up with ever more effective and provocative ways of striking back
    • Both sides feed off each other, coming up with more and more effective ways to provoke the other, spreading the meme more and more efficiently
  • Tumblr
    • Tumblr doesn’t allow you to post comments on others’ blogposts
    • Instead you reblog them - you copy the blog post into your own and create a new blog post consisting of the original blog post plus your comments
    • This turbocharges the spread of controversy, since, in order to reply to a provocative post, you have to spread the provocation to your own followers
  • In any complex multi-polar system, the system acts according to its own incentives, not necessarily what would produce the best outcomes for its participants
  • The emergence of controversial topics in the “national conversation” is the outcome of an uncontrolled emergent natural process
    • The “media” is not a conscious entity capable of goal-directed decision-making
    • This means that the cases that receive the most publicity are the most controversial
    • Cases are controversial because they are weak - if the situation was unambiguous, then everyone would agree and stop talking about it
  • Under this uncontrolled natural process, everyone is incentivized to keep arguing about the things that divide us instead of writing and speaking about topics on which we have common ground
  • This process has only accelerated with the shift to online media