DanielFilan

Karma: 1,081

DanielFilan 19 Sep 2023 21:31 UTC
6 points
0 ∶ 0
in reply to: Gerald Monroe’s comment on: AI Pause Will Likely Backfire

we don’t use superintelligent singletons and probably won’t, I hope. We instead create context limited model instances of a larger model and tell it only about our task and the model doesn’t retain information.

FYI, current cutting-edge large language models are trained on a massive amount of text on the internet (in the case of GPT-4, likely approximately all the text OpenAI could get their hands on). So they certainly have tons of information about stuff other than the task at hand.

DanielFilan 18 Sep 2023 17:19 UTC
5 points
0 ∶ 0
in reply to: DanielFilan’s comment on: AI Pause Will Likely Backfire
I asked Alex “no chance you can comment on whether you think assistance games are mostly irrelevant to modern deep learning?”

His response was “i think it’s mostly irrelevant, yeah, with moderate confidence”. He then told me he’d lost his EA forum credentials and said I should feel free to cross-post his message here.

(For what it’s worth, as people may have guessed, I disagree with him—I think you can totally do CIRL-type stuff with modern deep learning, to the extent you can do anything with modern deep learning.)

DanielFilan 16 Sep 2023 17:58 UTC
63 points
17 ∶ 2
on: AI Pause Will Likely Backfire
The core argument of Nick Bostrom’s bestselling book Superintelligence has also aged quite poorly: In brief, the book mostly assumed we will manually program a set of values into an AGI, and argued that since human values are complex, our value specification will likely be wrong, and will cause a catastrophe when optimized by a superintelligence. But most researchers now recognize that this argument is not applicable to modern ML systems which learn values, along with everything else, from vast amounts of human-generated data.

For what it’s worth, the book does discuss value learning as a way of an AI acquiring values—you can see chapter 13 as being basically about this.

I would describe the core argument of the book as the following (going off of my notes of chapter 8, “Is the default outcome doom?”):
- It is possible to build AI that’s much smarter than humans.
- This process could loop in on itself, leading to takeoff that could be slow or fast.
- A superintelligence could gain a decisive strategic advantage and form a singleton.
- Due to the orthogonality thesis, this superintelligence would not necessarily be aligned with human interests.
- Due to instrumental convergence, an unaligned superintelligence would likely take over the world.
- Because of the possibility of a treacherous turn, we cannot reliably check the safety of an AI on a training set.
There are things to complain about in this argument (a lot of “could”s that don’t necessarily cash out to high probabilities), but I don’t think it (or the book) assumes that we will manually program a set of values into an AGI.

DanielFilan 16 Sep 2023 17:39 UTC
15 points
3 ∶ 0
on: AI Pause Will Likely Backfire

Stuart Russell’s “assistance game” research agenda, started in 2016, is now widely seen as mostly irrelevant to modern deep learning— see former student Rohin Shah’s review here, as well as Alex Turner’s comments here.

The second link just takes me to Alex Turner’s shortform page on LW, where ctrl+f-ing “assistance” doesn’t get me any results. I do find this comment when searching for “CIRL”, which criticizes the CIRL/assistance games research program, but does not claim that it is irrelevant to modern deep learning. For what it’s worth, I think it’s plausible that Alex Turner thinks that assistance games is mostly irrelevant to modern deep learning (and plausible that he doesn’t think that) - I merely object that the link provided doesn’t provide good evidence of that claim.

The first link is to Rohin Shah’s reviews of Human Compatible and some assistance games / CIRL research papers. ctrl+f-ing “deep” gets me two irrelevant results, plus one description of a paper “which is inspired by [the CIRL] paper and does a similar thing with deep RL”. It would be hard to write such a paper if CIRL (aka assistance games) was mostly irrelevant to modern deep learning. The closest thing I can find is in the summary of Human Compatible, which says “You might worry that the proposed solution [of making AI via CIRL / assistance games] is quite challenging: after all, it requires a shift in the entire way we do AI.”. This doesn’t make assistance games irrelevant to modern deep learning—in 2016, it would have been true to say that moving the main thrust of AI research to language modelling so as to produce helpful chatbots required a shift in the entire way we did AI, but research into deeply learned large language models was not irrelevant to deep learning as of 2016 - in fact, it sprung out of 2016-era deep learning.

DanielFilan 12 Sep 2023 18:17 UTC
24 points
6 ∶ 0
in reply to: Joel Becker’s comment on: Sharing Information About Nonlinear
Sorry, maybe this is addressed elsewhere, but what relationship have you had with Nonlinear?

DanielFilan 10 Sep 2023 5:36 UTC
12 points
4 ∶ 0
in reply to: Joel Becker’s comment on: Sharing Information About Nonlinear
I suppose it’s relevant if you want to get a sense of the chances of ending up in a situation reminiscent of the one depicted in this post if you work for Nonlinear.

DanielFilan 10 Sep 2023 5:33 UTC
3 points
1 ∶ 1
in reply to: Linch’s comment on: Sharing Information About Nonlinear
FWIW my intuition is that even if it’s permissible to illegally transport life-saving medicines, you shouldn’t pressure your employee to do so. Anyway I’ve set up a twitter poll, so we’ll see what others think.

DanielFilan 10 Sep 2023 5:24 UTC
16 points
7 ∶ 1
on: An Incomplete List of Things I Think EAs Probably Shouldn’t Do

I believe there is a reasonable risk should EAs… [d]ate coworkers, especially when there is a power differential and especially when there is a direct report relationship

I think you’re right that there’s some risk in these situations. But also: work is one of the main places where one is able to meet people, including potential romantic partners. Norms against dating co-workers therefore seem quite costly in lost romance, which I think is a big deal! I think it’s probably worth having norms against the cases you single out as especially risky, but otherwise, I’d rather our norms be laissez-faire.

DanielFilan 10 Sep 2023 5:03 UTC
3 points
4 ∶ 0
in reply to: Linch’s comment on: Sharing Information About Nonlinear

For example some countries ban homosexuality, but your typical American would not consider it blameworthy to be gay.

I would object to my employer asking me to be homosexual.

DanielFilan 10 Sep 2023 4:58 UTC
12 points
0 ∶ 0
in reply to: Aaron Bergman’s comment on: aaronb50′s Shortform
Are you factoring in that CEA pays a few hundred bucks per attendee? I’d have a high-ish bar to pay that much for someone to go to a conference myself. Altho I don’t have a good sense of what the marginal attendee/rejectee looks like.

DanielFilan 8 Sep 2023 19:15 UTC
1 point
1 ∶ 0
in reply to: JoshuaBlake’s comment on: Sharing Information About Nonlinear
Definitely more plausible, but as a rule, “whenever you engage in some risky activity, you should do it to the standards of the top organizations who do it” doesn’t seem a priori plausible.

DanielFilan 7 Sep 2023 23:18 UTC
11 points
6 ∶ 0
in reply to: DanielFilan’s comment on: Sharing Information About Nonlinear

Would this post meet the standards of investigative journalism that’s typically published in mainstream news outlets such as the New York Times, the Washington Post, or the Economist?

I’m not exactly sure what this means, not being aware of what those standards are. It does strike me that IIUC those venues typically attempt to cover issues of national or international importance (or in the case of the NYT and WaPo, issues of importance to New York City or Washington, DC), and that’s probably the wrong bar for importance for whether someone should publish something on the EA forum or LessWrong.

Anyway, hope these responses satisfy your curiosity!

DanielFilan 7 Sep 2023 23:16 UTC
4 points
1 ∶ 0
in reply to: DanielFilan’s comment on: Sharing Information About Nonlinear

Does the piece conform to accepted journalist standards in terms of truth, balance, open-mindedness, context-sensitivity, newsworthiness, credibility of sources, and avoidance of libel? (Or is it a biased article that presupposed its negative conclusions, aka a ‘hit piece’, ‘takedown’, or ‘hatchet job’).

As a consumer of journalism, it strikes me that different venues have different such standards, so I’m not really sure what your first question is supposed to mean. Regarding your parenthetical, I think presupposing negative (or positive!) conclusions is to be avoided, and I endorse negatively judging pieces that do that.

DanielFilan 7 Sep 2023 23:13 UTC
7 points
3 ∶ 1
in reply to: DanielFilan’s comment on: Sharing Information About Nonlinear

Does the piece offer a coherent narrative that’s clearly organized according to a timeline of events, interactions, claims, counter-claims, and outcomes?

I think organization is a virtue, but not a must for a piece to be accurate or worth reading.

Does the piece show ‘scope-sensitivity’ in accurately judging the relative badness of different actions by different people and organizations, in terms of which things are actually trivial, which may have been unethical but not illegal, and which would be prosecutable in a court of law?

This strikes me as a good standard.

DanielFilan 7 Sep 2023 23:12 UTC
4 points
2 ∶ 1
in reply to: DanielFilan’s comment on: Sharing Information About Nonlinear

Did the author give the key targets of their negative coverage sufficient time and opportunity to respond to their allegations, and were their responses fully incorporated into the resulting piece, such that the overall content and tone of the coverage was fair and balanced?

Given the prominence of the comments sections in the venues where this piece has been published, I’d say allowing the targets to comment satisfies the value expressed by this. At any rate, I do think it’s good to incorporate responses from the targets of the coverage (as was done here), and I think that the overall tone of the coverage should be fair. I don’t know what “balance” is supposed to convey beyond fairness: I think that responses from the targets would ideally be reported where relevant and accurate, but otherwise I don’t think that e.g. half the piece should have to be praising the targets.

DanielFilan 7 Sep 2023 23:07 UTC
9 points
5 ∶ 0
in reply to: DanielFilan’s comment on: Sharing Information About Nonlinear

Were the anonymous sources credible? Did they have any personal or professional incentives to make false allegations? Are they mentally healthy, stable, and responsible?

I think the first two questions make sense as good criteria (altho criteria that are hard to judge externally). As for the last question, I think somebody could be depressed and routinely show up late to events while still being a good anonymous source, altho for some kinds of mental unhealth, instability, and irresponsibility, I see how they could be disqualifying.

Does the author have significant experience judging the relative merits of contradictory claims by different sources with different degrees of credibility and conflicts of interest?

I think most of us have been in situations where different people have told us different things about some topic, and those different people have had different degrees of credibility and conflict of interest? At any rate, I’m more interested in whether the piece is right than whether the author has had experience.

DanielFilan 7 Sep 2023 23:02 UTC
13 points
3 ∶ 0
in reply to: DanielFilan’s comment on: Sharing Information About Nonlinear

Does the author have any personal relationship to any of their key sources? Any personal or professional conflicts of interest? Any personal agenda? Was their payment of money to anonymous sources appropriate and ethical?

These seem like reasonable questions to ask. I whole-heartedly agree that such amateur journalists should only make payments that are appropriate and ethical—in fact, this strikes me as tautological.

DanielFilan 7 Sep 2023 23:00 UTC
15 points
6 ∶ 0
in reply to: DanielFilan’s comment on: Sharing Information About Nonlinear

Did they ‘run it by legal’, in terms of checking for potential libel issues?

If Ben wants to assume liability for libel lawsuits, I don’t see why he should be prevented from doing so. In the domain of professional investigative journalism, I can see why a company would have this standard, since the company may not want to be held liable for things an individual journalist rashly said, but that strikes me as inapplicable in this case.

(Incidentally, it seems like this is probably a standard of professional investigative journalism that I don’t think amateur investigative journalism should attempt to adhere to)

DanielFilan 7 Sep 2023 22:58 UTC
5 points
4 ∶ 1
in reply to: DanielFilan’s comment on: Sharing Information About Nonlinear

Did the author have any appropriate oversight, in terms of an editor ensuring that they were fair and balanced, or a fact-checking team that reached out independently to verify empirical claims, quotes, and background context?

I think it’s fine to attempt to do these sorts of things yourself, as long as you don’t make serious errors, and as long as you correct errors that pop up along the way.

DanielFilan 7 Sep 2023 22:55 UTC
8 points
3 ∶ 0
in reply to: Geoffrey Miller’s comment on: Sharing Information About Nonlinear

So, you don’t think amateur investigative journalism should even try to adhere to the standards of professional investigative journalism?

That’s not what I said. I said “I think it’s fine to investigate something and write up your conclusions without having training as an investigative journalist” in response to the first thing you proposed as a way to evaluate the piece: “Does the author have any training, experience, or accountability as an investigative journalist, so they can avoid the most common pitfalls, in terms of journalist ethics, due diligence, appropriate degrees of skepticism about what sources say, etc?”

I don’t know what the standards of professional investigative journalism are, so I’m unable to say whether amateur investigative journalism should try to adhere to them.

[EDIT: I can say what I think about the standards you propose in replies to this comment]