Moreover, people generally prefer life to freedom (they wouldn’t choose the death penalty as a punishment over prison or even modern slavery), and if you go out and talk to people in LMICs, you’ll largely find they prioritize the lives and health of their kids and themselves + having a job & escaping poverty over abstract notions of freedom. If you want to centre people’s preferences, you have to take that into account.
I don’t see how this example shows anything. The reason governments have to expend effort enforcing their paternalistic regulations is because these regulations violate the preferences of consumers. Now maybe you think consumer preferences are wrong, but forcibly limiting people’s abilities to take an option that they prefer is clearly not “centering people’s preferences”, unless you think your methodology is somehow better placed to figure out what people’s preferences are than their actual choices.
Plausibly Excel has obtained superintelligence but is, as per the aestivation hypothesis, biding its time and storing energy until the universe cools.