For Signalling (Part 1)

  • Signalling is about showing off
  • The whole point of signalling is to have costs
  • Wearing an embarrassing T-shirt is a refusal to signal
  • Signalling attempts to ensure honest communication
    • Signalling is meant to be costly for liars
    • Drivers’ licenses - signal that you’re qualified to drive
    • Job market - potential employees have to signal that they’re qualified for the position
    • Clothing - signals personality and interests
    • Doing nice things for friends
  • The world is full of people pouring wealth into things whose only purpose is to signal that one has great wealth
  • Is it really beneficial for society for everyone to see who is rich, who is poor, who is socially competent, who is smart, etc?
  • You don’t serve society by failing to signal because signalling well is part of winning
  • Distinguish signalling “respectability” from signalling wealth

There’s No Fire Alarm For Artificial General Intelligence

  • The purpose of a fire alarm is to make it socially acceptable to acknowledge a fire
  • People will remain in smoky conditions if there’s no alarm making it socially acceptable to evacuate
  • People are bad at knowing what they believe so they allow social pressure to override their better judgement
  • If AGI seems far away, is it even worth doing research into AGI alignment?
  • If we got 30 years’ warning about aliens coming, we would start discussing what to do today, we wouldn’t wait until the aliens were six months away to start thinking about the problem
  • History shows that key technological developments seem far away until they happen
    • Powered flight
    • Nuclear power
    • Both predictable with hindsight bias, yet both were surprising to people at the time
  • Progress is driven by peak knoweldge, not average knowledge - the field is always farther ahead than you think because it takes time for knowledge to percolate out
  • The future has different tools and can easily do things considered difficult today
    • We think that AGI is far away for the following reasons
      • Don’t know how to get AGI with present technology
      • Even getting the impressive results that we do have is really hard
      • Current AI systems are really dumb in a lot of ways
    • In machine learning, there’s a short gap between “possible” and “easy”
    • The experience of AI researchers on the cutting edge blinds them to how solutions become widely available once discovered
  • Most of the discourse on AGI isn’t geing driven by actual models, just rough intuitions
  • The signs of imminent AGI will be subtle and debateable
  • Experts will only believe AGI is imminent if
    • They can personally see how AGI could be built
    • Their personal jobs give them a sense that AGI is feasible to build
    • They are impressed by a potential AGI being smart in a way that feels “magical”
  • The problem is that any system that satisfies these preconditions will be an AGI, at which point it’ll be too late
  • The choice to delay action until a future alarm is reckless enough that it invokes the law of continued failure
    • Any civilization competent enough to deal with AGI once an alarm is sounded wouldn’t have waited until the alarm to stop preparing
  • If we were serious about addressing AGI, we’d be reviewing the state of the art in AI research every six months to see if we were on the threshold of AGI

The Order of the Soul

  • Higher cognitive functions have two modes
    • Bias world towards certain outcomes
    • Appreciation of the structural symmetry in the universe
  • Bias and symmetry
    • Bias: active force bending things to our will
    • Symmetry: fundamentally reflective tendency to make our minds reflect the true state of the universe
  • Too much bias leads to either freaking out or deluding yourself that everything’s okay
  • Too much symmetry leads to inaction
  • Bias in social settings helps you be charismatic and get people to see things your way (“reality distortion field”)
  • Mindfulness meditation works by lowering bias tendencies
  • The order of the soul
    • The Bhagavad Gita talks about 3 tendencies that everyone has
      • Sattva - wisdom, harmony, purity
      • Rajas - activity, ambition
      • Tamas - ignorance, chaos
    • This tripartite model is also present in western philosophy, in both Plato and Freud
    • Everyone agrees that the bottom is sensual appetites and the middle is self-assertion
    • However Freud differs from Plato and the Bhagavad Gita by saying that the top is the internalized voice of authority figures
    • In contrast, sattva or Plato’s logos can be seen as thinking at the meta-level (thinking about different strategies vs. executing strategies)
  • Good at school
    • Schools have poisoned people’s experiences with all sorts of subjects
    • This is because formal education has a curriculum and doesn’t tolerate people’s desire to learn about topics at their own pace in their own order
    • Marshmallow test is a test of the desire to pass tests, not of innate willpower
    • The primary thing that schools teach is obedience
  • Precepts and concepts
    • Most students in school aren’t thinking conceptually
    • They’re trying to figure out what’s necessary to pass the test
    • But life isn’t a test

The People In My Head Who Make Me Do Things

  • It can be helpful to cluster your motivations and assign a persona to each cluster
  • Recognize that each of your motivations has a role and purpose
  • Might be helpful to be more explicit about giving different parts of yourself a chance to be at the forefront

Guided By The Beauty of Our Weapons

  • Tim Harford: The Problem With Facts
    • Argues that people are mostly impervious to facts and logic
    • “Backfire effect” - telling people facts that contradict strongly held beliefs causes them to hold that belief even more strongly
    • Agnotology - deliberate production of ignorance
  • Backfire effect - doesn’t replicate
  • The subtext to Harford’s article is that the ingroup acknowledges facts and evidence, but the outgroup does not
  • There is no tribe of “fact immune troglodytes” out there
  • Focus on transmission is part of the problem - assumes that people will take your facts at face value and will be convinced the moment they hear your facts
  • The problem is that there’s no real debate happening at all
  • What constitutes “real debate”
    • Bilateral communication - two people are communicating with each other
    • Both people have chosen to enter the debate
    • Spirit of mutual respect and truth-seeking
    • Outside of a high-pressure “point-scoring” environment
    • Single topic and try to stick to the topic at hand
  • The closest thing Scott has seen to good debate is psychotherapy
  • There’s no single moment of blinding revelation that will make people change their minds - changing minds is a process
  • Is there anything the media can do?
    • Treat disagreement as a need to collaborate to investigate the issue further
    • Assume good faith of the other side
    • Engage in adversarial collaboration - choose someone from the other side and offer to explore questions with a mutually-agreed-upon methodology
  • Why should we engage in logical debate
    • Logical debate is an asymmetric weapon - helps those on the side of truth more than the rest
    • Rhetoric and violence are both symmetrical - doesn’t matter if you’re on the side of truth or not
  • Improving the quality of debate is a painful process, but we have to start sometime
  • The only path forward is to raise the sanity waterline