- Ethical injunctions are things you shouldn’t do even if you believe they’re the right thing to do
- Never try to deceive yourself or believes something other than what is probably true. It’s more likely that you’ve made a mistake than found the one case in which a false belief is more helpful than a true belief
- Believing something false is a black-swan bet. Even if it works out most of the time, the one time it doesn’t work out can undo all the gains you got from winning the bet
- Your brain is corrupted hardware - you will always believe that your calculations are correct, even if they’re way off
- So why compound the errors by choosing to believe something that you know is probably not true?
- Spreading falsehoods for the greater good can backfire really badly
- Truth is always easier to make consistent than falsehood
- When people “rationally” abandon ethical principles, it always seems to be that they go towards greater ethical leninence, rather than greater ethical strictness
- Rationality must have purpose, otherwise it collapses into infinite recursion
- Rationality is about being more effective at achieving your goals
- “Truth” is too abstract a goal - too easy to claim that you’re defending truth when you’re defending superstition
- Requiring beliefs to yield predictions and then testing those predictions brings us much closer to truth than trying to reason out what’s true on our own
- Rationality is not about choosing only methods that are guaranteed to work
- Rationality is about embracing the hard math of probability and being willing to at least look at the conclusion of that math, even if it’s considered “morally repugnant”
- Part of the aesthetic of rationality is subordinating rationality to a higher purpose - treat rationality as a means, rather than an end
- If you don’t know the probability of something, don’t make up a number for the sake of having a number
- Everything has a probability, but that probability may be too difficult to compute
- In those case, we should avoid making up numbers to give ourselves a feeling of having more information than we actually do
- If you find an inconsistency in your model of the world, patching your model to be consistent doesn’t necessarily make it more accurate
- Example numbers may be helpful, but don’t confuse those example numbers with true probabilities
- Counterpoint from Slate Star Codex: If It’s Worth Doing, It’s Worth Doing With Made Up Statistics
- Setup:
- Box A: Transparent, contains $1000
- Box B: Opaque, may contain $1,000,000 or nothing
- Superintelligent being Omega offers you a choice:
- Box B only (“one-box”)
- Box A and Box B (“two-box”)
- Omega chooses the boxes before you’re offered the choice - neither you nor Omega can alter the contents of the boxes
- Everyone who has chosen the two-box strategy has found that Box B is empty
- Everyone who has chosen the one-box strategy has found Box B to contain a million dollars
- Which box should you choose?
- Causal decision theory says that you should two-box, even though everyone who has taken the two-box strategy has had a worse outcome
- “It’s not your fault that Omega chooses to reward irrationality”
- However, rationality is about winning
- Omega doesn’t care the reasoning by which you come to your decision
- You don’t have to have a “good reason” for choosing Box B, you only need to choose Box B
- Instead of starting with a “rational” decision theory and then saying that Omega rewards irrationality, we should start by looking at the outcomes and then come up with a decision theory that ensures that we’ll choose the best outcomes
- Rationalists should never envy the choices that nonrationalists make - there is nothing stopping us from making the same choices
- The important thing is to make progress
- Never take pride in your ignorance; take pride in the work that you do to remedy your ignorance