- Artificial intelligence is fundamentally about reducing the mental to the non-mental
- Reductionism is not easy to perform
- Very easy to fall into a traps where you make empathic inferences without realizing it
- Our brains are very good at modeling other human beings, so it’s easy to model things as human beings
- Modeling something as an aspect of human beings is fake reductionism - human mental states are complex so it’s not necessarily a reduction in complexity to analogize something to a human mental state
- Any AI design that operates by analogy to human mental states is flawed
- You have to be able to explain how your AI behaves without resorting to analogies
- Too much science is about finding applications for existing knowledge rather than discovering new knowledge
- Science can move very quickly if there’s a firm goal and an urgent need to reach that goal
- Scientists are not trained in how to deal with open problems
- The scientists who do solve the open problems become famous precisely because they’re able to do what so few other scientists are able to do
- Rationalists all have a common theme of doubting the rationality of people around them
- Inoculation against social pressure allows rationalists to look at evidence on its merits
- Traditional scientific process isn’t rigorous enough
- There is no process that you can follow that will ensure that you’re right in all situations
- You’re not really a rationalist until you’ve followed best practices, had them fail, and had to discover new best practices in order to solve your problem
- Scientists must call upon their individual rationality in order to come up with hypotheses and decide which hypotheses to test
- Sometimes new hypotheses can be generated just by looking at existing data, but not always
- Einstein invented General Relativity with almost no data - only the precession of Mercury
- Einstein didn’t think about astronomical observations, he thought about how the universe ought to work given the already confirmed laws of physics
- Instead of extrapolating from data, Einstein went meta and extrapolated from other laws of physics
- What Einstein did was not easier or more reliable than “ordinary” science
- We celebrate people like Einstein for taking shortcuts only when the shortcuts work - if Relativity had not been confirmed by experiment, no one would be talking about Einstein today
- The sort of reasoning that Einstein performed to get Relativity is not the reasoning that we want to use in normal cases
- Sometimes, however, we need to reason on scanty evidence, because gathering more evidence is very expensive or impossible
- For those cases, learning to reason like Einstein may allow us to derive the true nature of reality much more quickly
- There is a limit to how much you can deduce about the world from a single piece of information, but that limit is very high
- 3 days for a human can be the equivalent of 500 million years for an AI on fast hardware
- Don’t underestimate the amount of impact something that’s both smarter and thinks faster can have in a very short period of time