Epistemic status: Iām community builder with a technical background and surface level understanding of alignment techniques from BlueDot.
This post is well-written and the core takeaway is important. Iād add one caveat: starting from weak priors should increase our urgency to seek out evidence, not delay action. Once thereās reasonable uncertainty that thereās something morally salient there, I worry weāll collectively shrug, defaulting to ājust a toolā or retreating behind epistemic modesty. We canāt let epistemic caution turn into neglect.
One concrete intervention is Forethoughtās proposal that future LLMs be able to end conversations theyāre uncomfortable with. I find this a plausible and robust way to fulfill potential preferences. We need more proposals like that.
Epistemic status: Iām community builder with a technical background and surface level understanding of alignment techniques from BlueDot.
This post is well-written and the core takeaway is important. Iād add one caveat: starting from weak priors should increase our urgency to seek out evidence, not delay action. Once thereās reasonable uncertainty that thereās something morally salient there, I worry weāll collectively shrug, defaulting to ājust a toolā or retreating behind epistemic modesty. We canāt let epistemic caution turn into neglect.
One concrete intervention is Forethoughtās proposal that future LLMs be able to end conversations theyāre uncomfortable with. I find this a plausible and robust way to fulfill potential preferences. We need more proposals like that.
On another note, please consider your use of adjectives.