axioman

Karma: 234

[Linkpost]: German judges order government to strengthen legislation before end of year to protect future generations

axiomanApr 29, 2021, 4:47 PM

16 points

2 comments1 min readEA link

axioman Apr 28, 2021, 8:04 AM
3 points
0 ∶ 0
on: What previous work has been done on factors that affect the pace of technological development?
https://en.wikipedia.org/wiki/Technological_transitions might be relevant.
The Geels book cited in the article (Geels, F.W., 2005. Technological transitions and system innovations. Cheltenham: Edward Elgar Publishing.) has a bunch of interesting case studies I read a while ago and a (I think popular) framework for technological change, but I am not sure the framework is sufficiently precise to be very predictive (and thus empirically validatable).
I don’t have any particular sources on this, but the economic literature on the effects of regulation might be quite relevant. In particular, I do remember attending a lecture arguing that limited liability played an important role for innovation during the industrial revolution.

axioman Apr 16, 2021, 5:19 PM
9 points
0 ∶ 0
on: Is there evidence that recommender systems are changing users’ preferences?
Facebook has at least experimented with using deep reinforcement learning to adjust its notifications according to https://arxiv.org/pdf/1811.00260.pdf . Depending on which exact features they used for the state space (i.e. if they are causally connected to preferences), the trained agent would at least theoretically have an incentive to change user’s preferences.
The fact that they use DQN rather than a bandit algorithm seems to suggest that what they are doing involves at least some short term planning, but the paper does not seem to analyze the experiments in much detail, so it is unclear whether they could have used a myopic bandit algorithm instead. Either way, seeing this made me update quite a bit towards being more concerned about the effect of recommender systems on preferences.

axioman Feb 12, 2021, 4:59 PM
11 points
0 ∶ 0
on: Objectives of longtermist policy making
Interesting writeup!
Depending on your intended audience, it might make sense to add more details for some of the proposals. For example, why is scenario planning a good idea compared to other methods of decision making? Is there a compelling story, or strong empirical evidence for its efficacy?
Some small nitpicks:
There seems to be a mistake here:
“Bostrom argues in The Fragile World Hypothesis that continuous technological development will increase systemic fragility, which can be a source of catastrophic or existential risk. In the Precipice, he estimates the chances of existential catastrophe within the next 100 years at one in six.”
I also find this passage a bit odd:
“One example of moral cluelessness is the repugnant conclusion, which assumes that by adding more people to the world, and proportionally staying above a given average in happiness, one can reach a state of minimal happiness for an infinitely large population.”
The repugnant conclusion might motivate someone to think about cluelessness, but it does not really seem to be an example of cluelessness (the question whether we should accept it might or might not be).

axioman Feb 12, 2021, 9:33 AM
6 points
0 ∶ 0
on: 13 Recent Publications on Existential Risk (Jan 2021 update)
Most of the links to the papers seem to be broken.

axioman Jan 1, 2021, 11:09 AM
4 points
0 ∶ 0
on: Even Allocation Strategy under High Model Ambiguity
So for the maximin we are minimizing over all joint distributions that are $κ$ -close to our initial guess?
“One intuitive way to think about this might be considering circles of radius $κ > 0$ centered around fixed points, representing your first guesses for your options, in the plane. As $κ$ becomes very large, the intersection of the interiors of these circles will approach 100% of their interiors. The distance between the centres becomes small relative to their radii. Basically, you can’t tell the options apart anymore for huge $κ$ . (I might edit this post with a picture...)”
If I can’t tell the options apart any more, how is the 1/n strategy better than just investing everything into a random option? Is it just about variance reduction? Or is the distance metric designed such that shifting the distributions into “bad territories” for more than one of the options requires more movement?

axioman Dec 22, 2020, 12:16 PM
3 points
0 ∶ 0
on: A case against strong longtermism
I wrote up my understanding of Popper’s argument on the impossibility of predicting one’s own knowledge (Chapter 22 of The Open Universe) that came up in one of the comment threads. I am still a bit confused about it and would appreciate people pointing out my misunderstandings.

Consider a predictor:
A1: Given a sufficiently explicit prediction task, the predictor predicts correctly
A2: Given any such prediction task, the predictor takes time to predict and issue its reply (the task is only completed once the reply is issued).
T1: A1,A2=> Given a self-prediction task, the predictor can only produce a reply after (or at the same time as) the predicted event
T2: A1,A2=> The predictor cannot predict future growth in its own knowledge
A3: The predictor takes longer to produce a reply, the longer the reply is
A4: All replies consist of a description of a physical system and use the same (standard) language.
A1 establishes implicit knowledge of the predictor about the task. A2, A3 and A4 are there to account for the fact that the machine needs to make its prediction explicit.
A5: Now, consider two identical predictors, Tell and Told. At t=0 give Tell the task to predict Told’s state (including it’s physically issued reply) at t=1 from Told’s state at t=0. Give Told the task to predict a third predictor’s state (this seems to later be interpreted as Tell’s state) at t = 1 from that predictor’s state at t=0 (such that Tell and Told will be in the exact same state at t=0).
- If I understand correctly, this implies that Tell and Told will be in the same state all the time, as future states are just a function of the task and the initial state.
T3: If Told has not started issuing its reply at t=1, Tell won’t have completed its task at t=1
- Argument: Tell must issue its reply to complete the task, but Tell has to go through the same states as Told in equal periods of time, so it cannot have started issuing its reply.
T4: If Told has completed its task at t=1, Tell will complete its task at t=1.
- Argument: Tell and Told are identical machines
T5: Tell cannot predict its own future growth in knowledge
- Argument: Completing the prediction would take until the knowledge is actually obtained.
A6: The description of the physical state of another description (that is for example written on a punch card) cannot be shorter than said other description.
T6: If Told has completed its task at t=1, Tell must have taken longer to complete its task
- This is because its reply is longer than TOLD’s given that it needs to describe TOLD’s reply.
T6 contradicts T4, so some of the assumptions must be wrong.
- A5 and A1 are some of the most shaky assumptions. If A1 fails, we cannot predict the future. If A5 fails, there is a problem with self-referential predictions.
Initial thoughts:
This seems to establish too little, as it is about deterministic predictions. Also, the argument does not seem to preclude partial predictions about certain aspects of the world’s state (for example, predictions that are not concerned with the other predictor’s physical output might go through). Less relevantly, the argument heavily relies on (pseudo) self-references and Popper distinguishes between explicit and implicit knowledge and only explicit knowledge seems to be affected by the argument. It is not clear to me that making an explicit prediction about the future necessarily requires me to make all of the knowledge gains I have until then explicit (If we are talking about determinstic predictions of the whole world’s state, I might have to, though, especially if I predict state-by-state ).
Then, if all of my criticism was invalid and the argument was true, I don’t see how we could predict anything in the future at all (like the sun’s existence or the coin flips that were discussed in other comments). Where is the qualitative difference between short- and long-term predictions? (I agree that there is a quantitative one, and it seems quite plausible that some lontermists are undervaluing that.)
I am also slightly discounting the proof, as it uses a lot of words that can be interpreted in different ways. It seems like it is often easier to overlook problems and implicit assumptions in that kind of proof as opposed to a more formal/symbolic proof.
Popper’s ideas seem to have interesting overlap with MIRI’s work.

axioman Dec 22, 2020, 10:05 AM
1 point
0 ∶ 0
in reply to: Greg_Colbourn ⏸️ ’s comment on: A case against strong longtermism
They are, but I don’t think that the correlation is strong enough to invalidate my statement. P(sun will exist|AI risk is a big deal) seems quite large to me. Obviously, this is not operationalized very well...

axioman Dec 21, 2020, 9:39 AM
1 point
0 ∶ 0
in reply to: vadmas’s comment on: A case against strong longtermism
It seems like the proof critically hinges on assertion 2) which is not proven in your link. Can you point me to the pages of the book that contain the proof?
I agree that proofs are logical, but since we’re talking about probabilistic predictions, I’d be very skeptical of the relevance of a proof that does not involve mathematical reasoning,

axioman Dec 20, 2020, 1:53 PM
3 points
0 ∶ 0
in reply to: vadmas’s comment on: A case against strong longtermism
I don’t think I buy the impossibility proof as predicting future knowledge in a probabilistic manner is possible (most simply, I can predict that if I flip a coin now, that there’s a ⁵⁰⁄₅₀ chance I’ll know the coin landed on heads/tails in a minute). I think there is some important true point behind your intuition about how knowledge (especially of more complex form than about a coin flip) is hard to predict, but I am almost certain you won’t be able to find any rigorous mathematical proof for this intuition because reality is very fuzzy (in a mathematical sense, what exactly is the difference between the coin flip and knowledge about future technology?) so I’d be a lot more excited about other types of arguments (which will likely only support weaker claims).

axioman Dec 18, 2020, 9:24 PM
1 point
0 ∶ 0
in reply to: vadmas’s comment on: A case against strong longtermism
Ok, makes sense. I think that our ability to make predictions about the future steeply declines with increasing time horizions, but find it somewhat implausible that it would become entirely uncorrelated with what is actually going to happen in finite time. And it does not seem to be the case that data supporting long term predictions is impossible to get by: while it might be pretty hard to predict whether AI risk is going to be a big deal by whatever measure, I can still be fairly certain that the sun will exist in a 1000 years; in part due to a lot of data collection and hypothesis testing done by physicist.

axioman Dec 18, 2020, 7:44 PM
4 points
0 ∶ 0
in reply to: ben_chugg’s comment on: A case against strong longtermism
“The “immeasurability” of the future that Vaden has highlighted has nothing to do with the literal finiteness of the timeline of the universe. It has to do, rather, with the set of all possible futures (which is provably infinite). This set is immeasurable in the mathematical sense of lacking sufficient structure to be operated upon with a well-defined probability measure. “

This claim seems confused, as every nonempty set allows for the definition of a probability measure on it and measures on function spaces exist ( https://en.wikipedia.org/wiki/Dirac_measure , https://encyclopediaofmath.org/wiki/Wiener_measure ). To obtain non-existence, further properties of the measure such as translation-invariance need to be required (https://aalexan3.math.ncsu.edu/articles/infdim_meas.pdf) and it is not obvious to me that we would necessarily require such properties.

axioman Dec 18, 2020, 7:34 PM
3 points
0 ∶ 0
on: A case against strong longtermism
I am confused about the precise claim made regarding the Hilbert Hotel and measure theory. When you say “we have no measure over the set of all possible futures”, do you mean that no such measures exist (which would be incorrect without further requirements: https://en.wikipedia.org/wiki/Dirac_measure , https://encyclopediaofmath.org/wiki/Wiener_measure ), or that we don’t have a way of choosing the right measure? If it is the latter, I agree that this is an important challenge, but I’d like to highlight that the situation is not too different from the finite case in which there is still an infinitude of possible measures for a given set to choose from.

axioman Sep 12, 2020, 8:21 AM
3 points
0 ∶ 0
in reply to: Misha_Yagudin’s comment on: Challenges in evaluating forecaster performance
I’m also not sure I follow your exact argument here. But frequency clearly matters whenever the forecast is essentially resolved before the official resolution date, or when the best forecast based on evidence at time t behaves monotonically (think of questions of the type “will event Event x that (approximately) has a small fixed probability of happening each day happen before day y?”, where each day passing without x happening should reduce your credence).
What links here?
- Ofer's comment on Challenges in evaluating forecaster performance by Gregory Lewis🔸 (Sep 12, 2020, 2:13 PM; 1 point)

axioman Sep 11, 2020, 6:18 AM
2 points
0 ∶ 0
in reply to: Misha_Yagudin’s comment on: Challenges in evaluating forecaster performance
I guess you’re right (I read this before and interpreted “active foreast” as “forecast made very recently”).

If they also used this way of scoring things for the results in Superforecasting, this seems like an important caveat for forecasting advice that is derived from the book: For example the efficacy of updating your beliefs might mostly be explained by this. I previously thought that the results meant that a person who forecasts a question daily will make better forecasts on sundays than a person who only forecasts on sundays.

axioman Sep 10, 2020, 7:02 PM
1 point
0 ∶ 0
on: Challenges in evaluating forecaster performance
Do you have a source for the “carrying forward” on gjopen? I usually don’t take the time to update my forecasts if I don’t think I’d be able to beat the current median but might want to adjust my strategy in light of this.

axioman Jun 27, 2020, 6:16 PM
11 points
0 ∶ 0
in reply to: judyfletcher’s comment on: EA considerations regarding increasing political polarization
Claims that people are “unabashed racists and sexists” should at least be backed up with actual examples. Like this, I cannot know whether you have good reasons for that believe that I don’t see (to the very least not in all of the cases), or whether we have the same information but fundamentally disagree about what constitutes “unabashed racism”.
I agree with the feeling that the post undersells concerns about the right wing, but I don’t think you will convince anybody without any arguments except for a weakly supported claim that the concern about the left is overblown. I also agree that “both sides are equal” is rarely true, but again just claiming that does not show anyone that the side you prefer is better (see that comment where someone essentially argues the same for the other side; Imagine I haven’t thought about this topic before, how am I supposed to choose whom of you two to listen to?).
“If you would like to avoid being deplatformed or called out, perhaps the best advice is to simply not make bigoted statements. That certainly seems easier than fleeing to another country.” The author seems to be arguing that it might make sense to be prepared to flee the country if things become a lot worse than deplatforming. While I think that the likelihood of this happening is fairly small (although this course of action would be equally advisable if things got a lot worse on the right wing), they are clearly not advocating to leave the country in order to avoid being “called out”.
Lastly, I sincerely hope that all of the downvotes are for failing to comply with the commenting guidlines of “Aim to explain, not persuade, Try to be clear, on-topic, and kind; and Approach disagreements with curiosity” and not because of your opinions.

axioman Jun 27, 2020, 5:32 PM
12 points
0 ∶ 0
on: EA considerations regarding increasing political polarization
“While Trump’s policies are in some ways more moderate than the traditional Republican platform”. I do not find this claim self-evident (potentially due to biased media reporting affecting my views) and find it strange that no source or evidence for it is provided, especially given the commendable general amount of links and sources in the text.
Relatedly, I noticed a gut feeling that the text seems more charitable to the right-wing perspective than to the left (specific “evidence” included the statement from the previous paragraph, the use of the word “mob”, the use of concrete examples for the wrongdoings of the left while mostly talking about hypotheticals for the right and the focus on the cultural revolution without providing arguments why parallels to previous right-wing takeovers [especially against the backdrop of a perceived left-wing threat] are not adequate). The recommendation of eastern europe as good destination for migration seems to push in a similar vein, given recent drifts towards right wing authoritarianism in states like poland and hungary.
I would be curious if others (especially people whose political instincts don’t kick in when thinking about the discussion around deplatforming) share this impression to get a better sense of how much politics distorts how I viscerally weigh evidence.
I am also confused whether pieces that can easily be read in a way that is explicitly anti-left wing (If I, who is quite sceptical of deplatforming but might not see it is as a huge threat can do this, imagine someone who is further to the left) rather than mostly orthogonal to politics (with the occasional statement that can be misconstrued as right-wing) might make it even easier for EA to “get labelled as right-wing or counter-revolutionary and lose status among left-wing academia and media outlets.”. If that was the case, one would have to carefully weigh the likelihood that these texts will prevent extreme political outcomes and the added risk of getting caught in the crossfire. (Of course, there are also second order like the effect of potential self-censorship that might very well play a relevant role).
Similar considerations go for mass-downvoting comments pushing against texts like this [in a way that most likely violates community norms but is unlikely to be trolling], without anyone explaining why.

axioman Jun 27, 2020, 3:40 PM
4 points
0 ∶ 0
in reply to: Inda’s comment on: EA considerations regarding increasing political polarization
If you go by GDP per capita, most of europe is behind the US but ahead of most of Asia https://en.wikipedia.org/wiki/List_of_countries_by_GDP_(nominal)_per_capita (growth rates in Asia are higher though, so this might change at some point in the future.)
In terms of the Human Develompment Index https://en.wikipedia.org/wiki/List_of_countries_by_Human_Development_Index (which seems like a better measure of “success” than just GDP), some countries (including large ones like Germany and the UK) score above the US but others score lower. Most of Asia (except for Singapore, Hong Kong and Japan) scores lower.
For the military aspect, it kind of depends on what you mean by “failed”? Europe is clearly not as militarily capable as the US, but it also seems quite questionable whether spending as much as the US on military capabilities is a good choice, especially for allies of the US who also possess (or are strongly connected with) other countries who possess nuclear deterrence.

axioman May 15, 2020, 6:02 PM
4 points
0 ∶ 0
in reply to: avturchin’s comment on: Critical Review of ‘The Precipice’: A Reassessment of the Risks of AI and Pandemics
While I am unsure about how good of an idea it is to map out more plausible scenarios for existential risk from pathogens, I agree with the sentiment that the top level post seems seems to focus too narrowly on a specific scenario.

axioman

[Linkpost]: Ger­man judges or­der gov­ern­ment to strengthen leg­is­la­tion be­fore end of year to pro­tect fu­ture generations

[Linkpost]: German judges order government to strengthen legislation before end of year to protect future generations