“What’s important in ‘AI for epistemics’?” by Lukas Finnveden

This is a link post. Summary This post gives my personal take on “AI for epistemics” and how important it might be to work on. Some background context: AI capabilities are advancing rapidly and I think it's important to think ahead and prepare for the possible development of AI that could automate almost all economically relevant tasks that humans can do.[1] That kind of AI would have a huge impact on key epistemic processes in our society. (I.e.: It would have a huge impact on how new facts get found, how new research gets done, how new forecasts get made, and how all kinds of information spread through society.) I think it's very important for our society to have excellent epistemic processes. (I.e.: For important decisions in our society to be made by people or AI systems who have informed and unbiased beliefs that take into account as much [...] ---Outline:(00:07) Summary(04:38) Structure of the post(05:35) Previous work(07:31) Why work on AI for epistemics?(07:36) Summary(09:40) Importance(12:17) Path-dependence(14:47) To be more concrete(15:39) Good norms and practices for AI-as-knowledge-producers(17:14) Good norms and practices for AI-as-communicators(19:32) Differentially high epistemic capabilities(27:15) Heuristics for good interventions(28:03) Direct vs. indirect strategies(29:27) Indirect value generation(33:33) On long-lasting differential capability improvements(36:13) Painting a picture of the future(41:06) Concrete projects for differentially advancing epistemic capabilities(43:50) Evals/benchmarks for forecasting (or other ambitious epistemic assistance)(44:30) Automate forecasting question-generation and -resolution(46:02) Logistics of past-casting(50:24) Start efforts inside of AI companies for AI forecasting or other ambitious epistemic assistance(51:37) Scalable oversight / weak-to-strong-generalization / ELK(53:56) Experiments on what type of arguments and AI interactions tend to lead humans toward truth vs. mislead them(56:14) Concluding thoughtsThe original text contained 22 footnotes which were omitted from this narration. --- First published: August 24th, 2024 Source: https://forum.effectivealtruism.org/posts/jPKoNFRowKJwGgGyy/what-s-important-in-ai-for-epistemics --- Narrated by TYPE III AUDIO.