I had a discussion with Owain Evans about many topics around language models. We touch on his work with Truthful AI and Ought. I spent a lot of time asking how LLMS might be helpful in forecasting and where they could make the most economic impact.
I’ve known Owain for the past few years and enjoy his perspective. Before working on AI Safety, he studied cognitive science and probabilistic programming, so I think he’s well-positioned to understand the relationships between forecasting infrastructure and AI.
This is more of a research discussion than a formally planned podcast. I want to get in the habit of recording more conversations. At the same time, I think some of these threads will interest listeners.
This recording happened in late January 2023, before GPT4 or the ChatGPT Plugin System. We don’t have a full transcript (these are expensive, particularly when there is a lot of jargon), but the section titles below should summarize what we discussed.
Many thanks to Owain for participating.
Sections
00:29 AI Safety Research: Truthful AI 05:41 Challenges of Benchmarking AI Models for Truthfulness and Reliability 11:26 Use of AI for Forecasting Questions and Probabilities 14:36 Automating Prompts and Forecasting Techniques with Language Models 18:00 Exploring the Potential of Evaluation vs Generation in Improving Text Generation Models 20:44 Improving ChatGPT through User Feedback and Reinforcement Learning 24:41 Current State and Future Potential of OpenAI’s AI Models and Products 26:48 Ought’s Current Strategy 30:25 Trade-offs between Process-Based and Outcome-Based Systems for AI Safety and Transparency 35:14 Importance of Transparency in AI Research and its Future Implications 37:01 Integrating Large Language Models with Existing Software 44:48 Programming Assistance and Automation in Research 51:00 Exploring the Possibilities of Combining AI and Personal Assistant Services
Owain Evans on LLMs, Truthful AI, AI Composition, and More
Link post
(Podcast episode in link)
I had a discussion with Owain Evans about many topics around language models. We touch on his work with Truthful AI and Ought. I spent a lot of time asking how LLMS might be helpful in forecasting and where they could make the most economic impact.
I’ve known Owain for the past few years and enjoy his perspective. Before working on AI Safety, he studied cognitive science and probabilistic programming, so I think he’s well-positioned to understand the relationships between forecasting infrastructure and AI.
This is more of a research discussion than a formally planned podcast. I want to get in the habit of recording more conversations. At the same time, I think some of these threads will interest listeners.
This recording happened in late January 2023, before GPT4 or the ChatGPT Plugin System. We don’t have a full transcript (these are expensive, particularly when there is a lot of jargon), but the section titles below should summarize what we discussed.
Many thanks to Owain for participating.
Sections
00:29 AI Safety Research: Truthful AI
05:41 Challenges of Benchmarking AI Models for Truthfulness and Reliability
11:26 Use of AI for Forecasting Questions and Probabilities
14:36 Automating Prompts and Forecasting Techniques with Language Models
18:00 Exploring the Potential of Evaluation vs Generation in Improving Text Generation Models
20:44 Improving ChatGPT through User Feedback and Reinforcement Learning
24:41 Current State and Future Potential of OpenAI’s AI Models and Products
26:48 Ought’s Current Strategy
30:25 Trade-offs between Process-Based and Outcome-Based Systems for AI Safety and Transparency
35:14 Importance of Transparency in AI Research and its Future Implications
37:01 Integrating Large Language Models with Existing Software
44:48 Programming Assistance and Automation in Research
51:00 Exploring the Possibilities of Combining AI and Personal Assistant Services
Mentioned Papers
Iterated Decomposition: Improving Science Q&A by Supervising Reasoning Processes
Truthful AI: Developing and governing AI that does not lie
Forecasting Future World Events with Neural Networks