Dreams of AI Design

  • Artificial intelligence is fundamentally about reducing the mental to the non-mental
  • Reductionism is not easy to perform
  • Very easy to fall into a traps where you make empathic inferences without realizing it
  • Our brains are very good at modeling other human beings, so it’s easy to model things as human beings
  • Modeling something as an aspect of human beings is fake reductionism - human mental states are complex so it’s not necessarily a reduction in complexity to analogize something to a human mental state
  • Any AI design that operates by analogy to human mental states is flawed
  • You have to be able to explain how your AI behaves without resorting to analogies

The Failures of Eld Science

  • Too much science is about finding applications for existing knowledge rather than discovering new knowledge
  • Science can move very quickly if there’s a firm goal and an urgent need to reach that goal
  • Scientists are not trained in how to deal with open problems
  • The scientists who do solve the open problems become famous precisely because they’re able to do what so few other scientists are able to do

No Safe Defense, Not Even Science

  • Rationalists all have a common theme of doubting the rationality of people around them
  • Inoculation against social pressure allows rationalists to look at evidence on its merits
  • Traditional scientific process isn’t rigorous enough
  • There is no process that you can follow that will ensure that you’re right in all situations
  • You’re not really a rationalist until you’ve followed best practices, had them fail, and had to discover new best practices in order to solve your problem

Einstein’s Speed

  • Scientists must call upon their individual rationality in order to come up with hypotheses and decide which hypotheses to test
  • Sometimes new hypotheses can be generated just by looking at existing data, but not always
  • Einstein invented General Relativity with almost no data - only the precession of Mercury
  • Einstein didn’t think about astronomical observations, he thought about how the universe ought to work given the already confirmed laws of physics
  • Instead of extrapolating from data, Einstein went meta and extrapolated from other laws of physics
  • What Einstein did was not easier or more reliable than “ordinary” science
  • We celebrate people like Einstein for taking shortcuts only when the shortcuts work - if Relativity had not been confirmed by experiment, no one would be talking about Einstein today
  • The sort of reasoning that Einstein performed to get Relativity is not the reasoning that we want to use in normal cases
  • Sometimes, however, we need to reason on scanty evidence, because gathering more evidence is very expensive or impossible
  • For those cases, learning to reason like Einstein may allow us to derive the true nature of reality much more quickly

That Alien Message

  • There is a limit to how much you can deduce about the world from a single piece of information, but that limit is very high
  • 3 days for a human can be the equivalent of 500 million years for an AI on fast hardware
  • Don’t underestimate the amount of impact something that’s both smarter and thinks faster can have in a very short period of time