Expert Trustworthiness

Consider these questions:

  • If you haven’t spent at least 1,000 hours studying [extremely complicated subject], do you think your views on the subject are likely to be carefully formed from first principles?
  • If you haven’t spent at least 1,000 hours studying medicine, do you think you are qualified to give medical advice on complex illnesses?
  • If you haven’t spent at least 1,000 hours studying nuclear safety, do you think others should blindly trust your views on nuclear safety policy?
  • If you haven’t spent at least 1,000 hours studying AI risks, do you think you are likely to have well-reasoned perspectives on AI safety?

Generally the best we can do in these cases is to trust the experts. Choosing which experts to trust is a skill, which unfortunately often requires expertise itself. But hints on how to do it:

  • Look for a long track record of accomplishment in their core field. Ideally look for accomplishment across other fields, as well. 
  • Look for a long track record of predictions or bets they publicly made which came true or they won. Compare the hits and misses. Ideally look for a good Brier score and lots of money won.
  • Look for evidence they pass the holistic understanding test (“to have an informed opinion on something, you must be able to identify all of the key perspectives relevant to it, including the pros and cons for each perspective”). This is similar to the ideological Turing test.
  • Analyze the soundness, validity, and persuasiveness of their arguments.
  • Analyze their character and competence. Try to disabuse yourself of heuristics and biases when doing this. Be careful with hype and groupthink.
If that doesn’t work, you may have to generate your own views without deep study. That is very hard to do and would take a book* to explain, not a short post.
 

*Perhaps the best book on that is Rationality: AI to Zombies, especially on the epistemic rationality level. Or the Upgrade Program Corpus helps, especially on the instrumental rationality level.