My Kind of Reflection

  • When Eliezer is talking about ethics, he’s talking about ethics as implemented in an AI system
  • Reflective consistency is not necessarily a good end goal for a system of epistemics
    • 5 maps of a real city will all be consistent with one another
    • 5 maps of fictional city can also be consistent with one another
  • Distinguish between the “defensive” posture that you use to justify your approach to others and the “aggressive” posture that you use to get as close to the truth as possible
  • When you examine the foundations of your beliefs, you should improve them
  • However, your improved foundations should still result in “normality” - you should still be able to explain everything that you’ve observed until now with your new foundations
  • Distinguish between “Why does it work,” and “Does it work?”
  • Worry about making your philosophy correct before you worry about making it interesting
  • Going through a loop of justifications in meta-reasoning isn’t the same as circular logic

The Genetic Fallacy

  • The genetic fallacy is the fallacy of attacking a belief because of the reason that someone holds the belief
  • However, knowing the reason that someone holds a belief can give us evidence as to whether we should update towards that belief or not
  • The genetic fallacy is a fallacy because the reason that someone holds the belief may not be the best justification for the belief - the belief may be sound, even though the reasoning is silly
  • This fallacy is much less of a fallacy among actual humans than it is among ideal Bayesians
    • People change their minds much less often than they think
    • If you realize that one of your sources is flawed, then you have to forcibly clear your mind of all the beliefs that were supported by that source
    • This is very difficult
    • You should be extremely suspicious of ideas that came from a flawed source but still ended up being correct
  • On the other hand, once you have empirical evidence for or against a belief, it doesn’t matter where the belief came from
    • Ex: Once you have experimental evidence for the structure of benzene, it doesn’t matter that Kekule originally saw the structure in a dream
  • In the absence of clear-cut experimental evidence, you need to take into account the trustworthiness of your source
  • Good rules of thumb
    • Be wary of leveling genetic accusations against ideas that you dislike
    • Once there’s experimental evidence on the table, it gets priority - you can no longer resort to criticizing the idea based upon the original reasoning for it

Fundamental Doubts

  • Because humans are not perfect Bayesians, the genetic fallacy is not entirely a fallacy
  • When suspicion is cast upon one of your sources, you must immediately cast doubt upon all of the conclusions you have drawn from the source
  • This is one of the most difficult techniques in all of rationality
  • However, there are some things you can’t doubt, things which are built into your mental hardware
  • We must do the best we can, with the brains we have
  • By learning about and recognizing the flaws in our cognition, we can work around them to some extent

Rebelling Within Nature

  • You can rebel against a facet of nature, but your rebellion is still within nature as a whole
  • You and your brain are a result of nature; everything you do is a result of some natural force
  • You can’t escape the processes that made your mind the way it is - there’s no way to turn your brain into the empty mind of an ideal philosopher
  • So, what can we do?
  • Even though we can’t escape the fact that we are a result of evolution, we can question it
  • The fact that something was designed by evolution is not a sufficient reason to reject it
  • Judge emotions for what they are, not as the result of an evolutionary process
  • Use the full power of your morality and your rationality, without worrying that it has been somehow tainted by being a product of evolution

Probability Is Subjectively Objective

  • E.T. Jaynes described himself as a “subejctive-objective” Bayesian - probabilities exist in our minds, but that doesn’t mean we can use whatever priors we want
  • How can something be “objective”, but still exist only in our minds?
  • If you can change it by thinking differently, it’s subjective; if it remains the same no matter how you think of it, it’s objective
  • Even though every thought you take takes place within your brain, not every thought you have is about your brain
  • So is probability subjective or objective?
  • We assign probability distributions to events, but we only see individual events, not the probability distribution
  • The uncertainty about an event that causes us to assign a probability to it resides in our minds, not in the world
  • Jaynes recommends that we never use unconditional probabilities - every probability is conditional upon our background priors
  • This reinforces the point that probabilities only exist in minds - there is no such thing as a truly unconditional probability
  • But, just because priors reside in our mind, it doesn’t mean that all priors are equally good
  • Changing your beliefs about probabilities by editing your brain doesn’t actually change whether that event will occur in the world