Hey! I am Mart, I learned about EA a few years back through LessWrong. Currently, I am pursuing a PhD in the theory of quantum technologies and learning more about doing good better in the EA Ulm local group and the EA Math and Physics professional group.
Mart_Korz
Fascinating! I would appreciate an essay arguing for this rather strong claim
My conclusion is that if something is expressed only in writing it cannot reach the absolute majority of the population, any more than a particularly well-written verse in French can permeate the Anglosphere.
I have read weaker versions of how hard successful communication is, such as Double Illusion of Transparency and You Have About Five Words – but I think that your example is even stronger than this and an interesting addition.
Personally, I think I also belong to the group of 2nd-order-illiterate people in that I need to push my concentration a lot in order to read with sufficient care. My default way of reading is nowhere near enough and I need to read a text several times until I feel that it doesn’t contain ‘new thoughts’ even if it is well-written. I do profit a lot from podcasts and lectures, even if it is just by ‘watching a person think about the topic’ and the content is the same as in a text book.
I also found Social Movement Lessons from the Fair Trade Movement interesting to read
Regarding “Pascal’s Mugging”:
I am not the author, so I might well be mistaken. But I think I can relate to the intended meaning more closely than “vaguely shady”One paragraph is
EA may not in fact be a form of Pascal’s Mugging or fanaticism, but if you take certain presentations of longtermism and X-risk seriously, the demands are sufficiently large that it certainly pattern-matches pretty well to these.
which I read as: “Pascal’s mugging” describes a rhetorical move that introduces huge moral stakes into the world-view in order to push people into drastically altering their actions and priorities. I think that this in itself need not be problematic (there can be huge stakes which warrant change in behaviour), but if there is social pressure involved in forcing people to accept the premise of huge moral stakes, things become problematic.
One example is the “child drowning in a pond” thought experiment. It does introduce large moral stakes (the resources you use for conveniences in everyday life could in fact be used to help people in urgent need; and in the thought experiment itself you would decide that the latter is more important) and can be used to imply significant behavioural changes (putting a large fraction of one’s resources to helping worse-off people).
If this argument is presented with strong social pressure to not voice objections, that would be a situation which fits under Pascal-mugging in my understanding.If people are used to this type of rhetorical move, they will become wary as soon as anything along the lines of “there are huge moral stakes which you are currently ignoring and you should completely change your life-goals” is mentioned to them. Assuming this, I think the worry that
[...] the demands are sufficiently large that it certainly pattern-matches pretty well to these.
makes a lot of sense.
I am not sure what the amount of useful training text would be, but the transcripts of the 80k podcast could be useful as a source of ‘spoken language’ EA thinking.
Regarding ”??% rationality” (Scout Mindset is a great choice), my impression is that these did significantly influence some fraction of EAs, but not nearly all .
For HPMOR I think that there are a few arguments against including it: For one, I could imagine that the fictional setting can let the model give unexpected answers that refer to a fictional world if the input accidentally resembles discussions in HPMOR too much (I am not familiar enough with Transformers to say whether this would actually be a problem or not, but it could be very confusing if it starts mentioning Transfiguration as a cure to Alzheimer’s).
Also, some characters in there are explicitly malevolent or highly cynical about humanity – I do not think that it would push EA GPT in a good direction to be trained on these.For a nice selection of rationality texts, the LessWrong books might be a good choice as they contain texts from many different writers, and which were chosen by the LW community to be exemplary.
Stories of this nature are sobering to hear; thank you for posting this—each post like this gets people in the community mentally closer to seeing the base rate of success in the EA community for what it is.
Your writing is enjoyable to read as well—I would read more of it.
I agree. And now I wonder whether someone already did write more about this? And if not, maybe this could be a great project?
I found the ‘personal EA stories’ in Doing Good Better (Greg Lewis) and Strangers Drowning (well, many of these are not quite about EA, but there are many similarities) very helpful for clarifying what my expectations should or could be.
A book where, say, each chapter follows the EA path of one person with their personal successes, struggles, uncertainties and failures could span the different experiences that people can have with EA. Similarly to how many people found semicycle’s story valuable, I could imagine that such a book could be very helpful for actually internalizing that EA is very much a community project where doing the right thing often means that individuals will fail at many of their efforts.
If this book already exists, I would be very happy to know about it :)
Sorry, I don’t understand the trickiness in ’[...] in a way that makes “importance” tricky’
In my mind, I would basically think about the expected improvement from thinking through my decisions in more detail to be the scale of importance. Here, realizing that I could avoid potential harms by not accelerating AI capabilities could be a large win.
This seems to treat harms and benefits symmetrically. Where does the asymmetry enter? (maybe I am thinking of a different importance scale?)
I think that a part of this perception is created by him actively framing his actions in a way compatible with a ‘press secretary model of the human mind’ (cf. “Why everyone (else) is a hypocrite”).
My impression is that he does consciously notice a mistake, and is shaken to some degree. In distancing himself from the aspects of his thinking which led to this mistake, he treats his motivations as more clear-cut than they truly were and pushes against them.
If that story were the full truth, he would not have given these answers which are basically the opposite of “locally and situationally optimal in terms of presenting himself”.
I think that we would make a mistake in thinking “well, clearly SBF was a bad person all along, so I and other EAs will not end up making structurally similar mistakes anyway” (I am not trying to imply that this is what you said/think and only add this for completeness). Regarding lessons on a community level, I think that much of the discussion on the Forum in the recent days makes a lot of sense.
This is kind of a detail, but if we already assume methods which would allow for unbounded value in the absence of vacuum decay, it should not be certain that the presence of vacuum collapse creates a bound.
I would expect that it basically creates a finite (expected) lifetime to any single causally connected bubble of the universe, but this could be counteracted by sufficient rates of inflation creating and disconnecting ever more bubbles at a higher rate.This point aside, I had not realized that there actually are theoretical expectations for the Higgs ground state(s). I only learned about the toy models in my lectures and never looked up how they related to the full standard model.
Thanks!
Are there considerations on whether naturally occurring things would have triggered decay already if it were (sufficiently) “easy to trigger”?
My expectation would be that e.g. neutron star+black hole merging events create quite extreme conditions and might rule out some possible ways/parameter regimes of vacuum decay?
I have the slight suspicion that the author did not set a clickable link to reduce self-promotion.
I hope it is thus okay if I add it here in the comments https://www.amazon.com/dp/B0BSXHJRBQ
For anyone interested: A Forum post with more background info about the novel is I’ve written a Fantasy Novel to Promote Effective Altruism
The library is well-visited today and there currently are no additional empty chairs left – if you join (you can recognize us by the copy of the book Doing Good Better) we can just switch to a more suitable part of the library.
If we move, I’ll correspondingly note this here as a comment.
Hi Felix, I was involved in many of the discussions and will try to answer your questions.
you did a speed giving game? It’s about giving people the opportunity to choose for themselves, and there may also be some rebels in your target group. ;)
Yes, many of the students initially liked the idea of playpumps and chose them as preliminary favourite and some actually directly put their coin into the playpump-box[1]. After we provided them the more detailed info, most changed their decision and for one of them it just felt more proper to treat their putting their coin in the playpump-box as final, even though they would have made a different decision with their updated knowledge. For the second person, “rebel” actually kind of fits as a description at least for this one interaction with us :)
So with 1 € per person, this would make 29 players. How much time and people did you invest in the uni forum? How was the ratio with people getting a flyer / speaking with you and not participating in the giving game to those who did (had you more impressions on people than 29)?
Hmm, preparation was at maybe 30 h total (where most of this time was specifying what exactly we intended to do, reading about the experiences and guides from other groups, and also collecting and adopting the printed resources. If we were to repeat this in a few weeks or next semester, the preparation would be a lot faster) and we were two people who were present during the event itself, maybe 5 h each.
We made a lot of use of the giving game being a neat way to engage people without them feeling pressured or committing to anything in the future, so that I would say that the majority of people we reached also participated in the giving game. Unfortunately, I don’t think I can give a more more precise estimate.
Have fun with your intro meeting in six days. :)
Thanks!
- ↩︎
we were using large glasses, but ‘box’ feels like a better description of their purpose
- ↩︎
I have little experience on quantifying value, so I don’t feel that I have a relevant opinion about approaches to this topic. But improving our conceptual tools for this clearly seems valuable :)
I feel like a commonly occurring result would be that the comparison tables include contradictory properties. If someone asked me about my relative preferences for apples, bananas and cherries, I think the chance would be significant that I give a ‘contradictory’ answer like “5 cherries = 1 apple or 1 banana, but also 1 apple = 2 bananas”. Using distributions might help, but I think a corresponding property of “tension between ratios” should still appear quite frequently.
It feels like a confusing property that one could get different results by converting to the final units, WELLBYs for example, in two steps instead of using a direct conversion. First translating everything to QALYs and then to WELLBYs would usually give different results than the direct path.[1]
Would the solution be to quantify all actions using their native units and then convert to the unit of interest without intermediate steps? I can see a case for this. If we are highly uncertain, avoiding unnecessary mental steps is a good idea.
Possibly something like this is the best we can do as long as we cannot define an explicit utility function. Still, I would be interested whether relative value functions could be a tool that helps us resolve confusions in what we value?
- ↩︎
I think that this is actually the additional information which having such a table adds compared to using a single central unit of comparison. If there were no path dependency, the table would be redundant and could be replaced by a single central unit (= any single line of the table). This makes me extra curious about the question of what this “extra information” really means?
- ↩︎
Audio matters
Are there by any chance plans to collect the audio in a podcast feed?
I would :)
I think you might not quite yet grok the main benefits of relative values
Thanks for your reply, you are probably right. Let my share my second attempt of understanding relative values after going through the web app.
‘strict’ relative values
If I did not overlook some part in the code, the tables created in the web app are fully compatible with having a single unit.
For every single table, one could use a single line of the table to generate the rest of the table. Knowing for all , we can use to construct arbitrary entries.
Between the different tables, one would need to add a single translation factor which one could then use to merge the tables to a big single table.
Without such a translation factor, the tables would remain disconnected (there could be a single unit for all tables, but it is not specified). Still, the tables could still be used to make meaningful decisions inside of the scope of each table.
If this is the intent of how relative values are meant to be used, my impression of their advantages is:
they are, in principle, compatible with a single value/utility function. One does not need to change one’s philosophy at all when switching over from using a single unit for measuring value.
they allow for a more natural thought process when exploring the value of interventions
one can use crisply defined units at each step of one’s research: “Person in city x of income y gets $1” can be distinguished from “Person in city x of income y gets $5″ as necessary.
throughout the process, one will tend to work ‘bottom-up’ or ‘top-down’, that is for bottom-up, start out with very specific value-measures and expand their connections (via relative values / translation factors) to more and more abstract/general values (such as maybe WELLBYs)
If one feels that there is an unbridgeable gap between two currently non-connected groups of values, one can keep them as separate value tables and decide to add the connection some time in the future
thanks to using distributions, one can also decide to add a connection and use a very high uncertainty instead.
This version of relative values (let’s call it “strictly coherent relative values according to Mart’s understanding v2” or “strict relative values” for short) feels quite intuitive to me and also seems significantly similar to how givewell’s current cost-effectiveness analyses are done (except that they do not create a value table with all-to-all translations and there being no/fewer distributions[1].)
Your link to the usage of relative values in Finance seems to me to be compatible with this definition of relative values.
Beyond ‘strict’ relative values
But, from reading your OP (and the recommended section of the video), my impression is that relative values are intended to be used to describe situations more general than my “strict relative values”.
Your
and also David Johnston’s comments seem to refer to a much more general case.
For this more general version my ‘strictness’ equation would typically not be valid. Translated into David’s notation, the ‘strictness’ equation would be where is the reference value, and are the relative values comparing and .
David’s
Note that, under this interpretation, we should not expect i=j$. This is because items have different values in different contexts.
is clearly not compatible with ‘strictness’ [2].
In such a generalized case, I think that the philosophical status of what entries mean is much more complicated. I do not have a grasp on what the added degrees of freedom do and why it is good to have them. In my last comment, I kind of assumed that any deviation from strictness would be “irrational inconsistency” by definition. But maybe I am just missing the relevant background and this really does capture something important?
- ↩︎
This impression is based on the 2023 spreadsheet. This might well be a mistaken impression
- ↩︎
Proof: Insert and into the ‘strictness equation’ and see that the results are the reciprocals of each other
Ooh, that makes sense. Thanks!
So my idea of ‘strict relative values’ turns out to be an illusory edge case if we use distributions and not numbers, and in practice we’ll usually be in the ‘generalized case’ anyway.
I fear, my not-grokking the implications remains. But at least, I don’t mistakenly think I fully understood the concept any more.
It is probably not worth the effort for you to teach me all about the approach, but I’ll still summarize some of my remaining questions. Possibly my confusions will be shared by others who try to understand/apply relative value functions in the future
If someone hands me a table with distributions drawn on it, what exactly do I learn? What decisions do a make based in the table?
Is the meaning of each entry “How many times more value is there in than in ? (Provide a distribution)”?
Would one only use ‘direct steps’ in decision-making? How is “path dependency” interpreted?
what is the necessary knowledge for people who want to use relative value functions? Can I do worse compared to using a single unit by using relative values naively?
- ↩︎
As you write, this is not really well-defined as one would need correlations to combine the distributions perfectly. But there should still be some bounds one could get on the outcome distribution.
- ↩︎
For example, it might totally happen that I feel comfortable with giving precise monetary values to some things I enjoy, but feel much less certain if I try to compare them directly
Thanks! I’ll reply in separate comments
Is the meaning of each entry “How many times more value is there in item1 than in item2? (Provide a distribution)”?
Yep, that’s basically it.
Okay, so maybe relative values are a more straightforward concept than I thought/feared :)
Would one only use ‘direct steps’ in decision-making? How is “path dependency” interpreted?
I’m not sure what you are referring to here. I would flag that the relative value type specification is very narrow—it just states how valuable things are, not the “path of impact” or anything like that.
After talking to GPT about this[1], I think that my concern is actually already covered by your
If people were doing it by hand, there could be contradictory properties, as you mention. But with programming, which we likely want anyway, it’s often trivial or straightforward to make consistent tables.
and could be addressed to a large degree with a few automatized checks and a user interface (one could even auto-fill the table given the first line of entries by assuming ~maximum resulting uncertainty for the unknown correlations). I feel like this could be really helpful for reflecting on one’s values if done right, or overwhelming if done wrong.
- ↩︎
[GPTs answer includes a summary, so I’ll leave out my prompt:] I see your point now. You’re referring to a potential inconsistency in the triangle inequality context for relative values. Let’s use a concrete example to explore this. Suppose the relative value of ‘chocolate’ to ‘vanilla’ is given by a 90% credibility interval from 0.8 to 1.2, and ‘banana’ to ‘vanilla’ is from 1.0 to 1.5. This would imply that, based on your preferences for chocolate and banana ice cream relative to vanilla, ‘banana’ should have a relative value compared to ‘chocolate’ that lies within a certain range. If, however, you then provide a wide 90% credibility interval for ‘chocolate’ to ‘banana’ (e.g., 0.2 to 5), it may seem inconsistent with the narrower ranges implied by the comparisons with ‘vanilla’. In this case, it may be that you need to revisit your estimates for ‘chocolate’ to ‘vanilla’ and ‘banana’ to ‘vanilla’. If you feel a lot of uncertainty about ‘chocolate’ vs. ‘banana’, perhaps you also should feel more uncertainty about these options compared to ‘vanilla’ than you initially thought. You may have overestimated your confidence in these comparisons. Alternatively, if you’re confident in your estimates of ‘chocolate’ to ‘vanilla’ and ‘banana’ to ‘vanilla’, you may want to narrow down your 90% credibility interval for ‘chocolate’ vs. ‘banana’. In any case, it’s a good point to keep in mind when building a relative value table. You want to be consistent in your uncertainty estimates across different comparisons. If there seems to be a contradiction, it’s a sign that you may need to rethink some of your estimates.
- ↩︎
Regarding the adoption of a fully vegan diet, I can understand the argument.
Could you say something on your position about a ‘vegan-friendly’ position?
In my experience, diet and taste are strongly formed by habits and expectations, and given habit-forming over a few years it should be easy to substantially reduce animal products in one’s diet (compared to the average western diet) without needing to put in much money, time or mental effort. For example one might decide to not actively seek vegan food it it comes at any relevant cost to oneself, but treat it as a serious option otherwise. I think that even when including the danger of trivial inconveniences into this consideration, it should be possible for most people to slowly accumulate some easily accessible vegan foods that they enjoy into their diet without bearing nearly the full costs of becoming vegan.
Would you agree that such an approach to vegan food would increase the altruistic cost:benefit ratio?