Reading Notes

Desire Is The Direction, Rationality Is The Magnitude - Nate Soares

  • Sterotype: rationality is about removing “hot”, emotional reasoning and replacing it with “cool”, calculated reasoning
  • Makes people uncomfortable
  • Emotional reasoning is a core part of humanity and people feel like they’re giving up a portion of their humanity by being asked to give up emotional reasoning
  • The reality is that rationality focuses on the reasoning, rather than the kind of reasoning
  • The core aim of rationality is making you better at acheiving your goals, rather than telling you what your goals should be

What Sort of Thing a Brain Is - Nate Soares

  • A brain is a specialized device that takes in information from the outside world and changes its internal state in response
  • But that’s not the only thing it does:
    • Planning
    • Prediction
    • Consciousness
  • The brain is far from a perfect information processing machine
  • There is no homonculus sitting inside the brain - the brain is all there is
  • Rationality is about making the brain better at building up an internal representation of the outside world

Ephemeral Correspondence - Nate Soares

  • Sometimes the brain updates its model of the world automatically, but sometimes it allows the mind to choose how the model gets updated
  • This is deeply troubling
    • Why should the part of the brain responsible for making decisions based upon the model of the world have any say at all in how the model is built?
  • Sometimes it takes conscious effort to acknowledge evidence and update your world-model
  • Rationality is about putting in that effort to make your mental model of the world have a closer correspondence to how the world is

The Brain/Mind Distinction - Nate Soares

  • The brain is not the mind
  • The mind is implemented by the brain
  • The mind is part of the brain, but not the only part
  • Changes in the focus of our awareness are the result of changes in the brain’s model of the world

The Way of the Rationalist

  • On the path to rationality, there are no beliefs, only probabilities
    • Believing the hypothesis that has the balance of probability in its favor leaves you open to being proven “wrong” by chance
  • Stop thinking in terms of what will/won’t happen, but instead think of what will probably happen
  • Try to keep multiple probable outcomes in your mind simultaneously
  • This is difficult, which is why we should practice it
  • Replace qualitative intuition with quantitative intuition

Discussion Notes

Lightning Talks

Rohit - Colors and How We See Them

  • Nate states that perceiving the sky as blue is one of the instances where the brain updates its model automatically, free of outside influences
  • This is false - even the perception of something as fundamental as color is culturally influenced
  • 3 examples
    • Ancient Greeks - never used the word blue
      • Homer describes people has having violet eyes
      • “Wine-dark seas”
    • Himba Tribe of Namibia
      • Don’t have separate words for green and blue
      • Both colors are described the same word
      • Have trouble distinguishing between blue and green swatches - study by Roberson, Davidoff et. al.
    • Davidoff’s daughter
      • Professor Davidoff carefully avoids teaching his daughter that the sky is blue
      • Points at sky and asks her what color it is
      • Response: “white”
  • The presence of categories enhances our ability to see distinctions between objects in different categories
  • Categories are historically and culturally determined
    • For color, the invention of these categories appears to correspond with the invention of dyes - “blue” is a relatively young category because blue dye is more difficult to make
  • Himba, ancient Greeks and Davidoff’s daughter all have different categories for color than the Western mainstream, so in a sense they do see colors differently
  • Same light hitting the retinas is interpreted differently by the brain

Eric - How Sailboats Work

  • All sailing is an exercise in applied Newtonian mechanics
  • Wind pushes against sail
  • Sail pulls on boat
  • Water pushes against boat in the opposite direction
  • Sailboats are able to sail at an angle to the wind by exploiting Newton’s third law
    • Mount sail on a pivot
    • Rotate sail such that it’s perpendicular to the wind, but at an angle to the boat
    • Wind pushes on sail
    • Equal and opposite force from water pushing on boat
    • Boat moves relative to water at a different speed and direction that it moves relative to the wind
      • This is fine - you don’t care about crossing a wind, but you do care about crossing the ocean
  • The role of the keel in a boat is to keep the boat from sliding sideways through the water
  • Triangular sails are especially handy, since they are easier to rotate - easier to match changes in the wind, easier to reposition for changes in desired course

Ben - Brainstorming

  • To effectively brainstorm, set two timers
  • Write down ideas and explore problem space in first timers
  • Think of solutions and prune ideas in second timer
  • Helps prevent you from getting fixated on a particular idea or solution

Reading Discussion

Desire Is The Direction, Rationality Is The Magnitude

  • Rationality is a means, not an end
  • Yudkowsky’s definition of rationality, which Soares is using, doesn’t disentangle intelligence and rationality
  • Is intelligence a multiplier for rationality?

What Sort of Thing a Brain Is

  • Purpose of this essay to show why we need rationality
  • Brain isn’t a special, different thing
  • Part of the body and subject to many of the same flaws of biology and evolution
  • Human brains are weird, compared to animal brains
    • Can hold a much broader set of relatively arbitrary information
    • Allows for memes to move around between brains - semi-separate memetic ecosystem

Ephemeral Correspondence

  • Effort needed to make your mind correspond to reality - this is not something that happens automatically
  • Agree with conclusions of essay, but think that the argument could be improved
  • Don’t agree with Nate’s use of the word “executive” - ascribes executive function to many things that we don’t have conscious control over

The Brain/Mind Distinction

  • Largely a restatement of What Sort of Thing a Brain Is
  • Consciousness is a part of the brain, not king of the brain

The Way of the Rationalist

  • Can we actually think in terms of probabilities at a System 1 level - Kahneman doesn’t seem to think so
  • Perhaps the best we can do is be aware of situations where our intutions towards certainty aren’t good approximations and prime ourselves to use System 2 in those situations
  • Assigning probabilities of 0 and 1 may not be mathematically accurate, but is a useful mental shortcut for computing approximations