There’s a lot about this idea that I agree with. It seems important to get effective altruists (and everyone) to share more information about what they’re trying to achieve, whether it worked or didn’t, and why. In particular, a lot of us are trying radical things like applying for finance jobs or starting companies, which have high variance, making it hard to infer whether these are good decisions.
I think this idea can grow a lot. It would be good to have everyone we know pooling together information about when they started a company, for example, so that we could infer whether it’s better to start more companies or work for somebody else. In the long run, it would be ideal situation would be to have thousands of people making their decision in similar ways to you, and to have a recommender system that can give you suggestions by giving extra weighting to the experiences of people similar to you.
However, I do have a bunch of feedback and questions:
1 Is a registry the best way of sharing information about what activities are working well and which aren’t?
2 Is there some reason to focus just on explicit experiments rather than lots of activities with uncertain payoff? Are there enough explicit experiments to support this project? Don’t we want to reduce reporting bias in general, rather than just publication bias? i.e. we want people to report the successes and failures of activities with uncertain payoff, rather than just explicit experiments. So it might be better to call it a Reporting Bias Registry.
3 On second thoughts, shouldn’t it be named by what it’s trying to achieve rather than what it’s trying to avoid? e.g. the ‘charity project performance registry’, or ‘social impact registry’, or something like that.
4 I think it’s important to be clear about what exactly you want people to do. At the top or bottom of the page, you could write in bold that what you want is for people to write information about their projects on the wiki.
5 It’s not clear that a wiki is the best way to implement this. Few people don’t use the wiki, and some of the important experiments that people run might be private? Perhaps it would be better to make a Google form, and to assure people that any information that is sufficiently specific to identify them or their project will not be disclosed? Or perhaps privacy would only work if there are at least dozens of projects. It’s at least worth thinking about.
6 Is this substantially different from the EA Survey? Is it substantially different from the EA Profiles? Is it substantially different from GWWC’s donation registry? CFAR’s alumni community? 80k’s alumni group? Can it be integrated with any of these things?
Maybe we should also get everyone to report their progress on the projects that we already know about. This is kind-of different, because the reports will be somewhat biased but it still seems worthwhile.
So I would be interested in how you would respond to some of these challenges and how we could plan around them in order to make the project more likely to be the big success that the idea deserves.
There are plenty of valuable thoughts here. I also like Giles’ idea and think it’s worth giving a go.
A few specific comments:
It’s not clear that a wiki is the best way to implement this.
I think a wiki’s fine for now, and pretty simple. I don’t think there’s yet reason to worry about privacy. In general I think Giles’ setup is a decent minimum viable product.
Maybe we should also get everyone to report their progress on the projects that we already know about. This is kind-of different, because the reports will be somewhat biased but it still seems worthwhile.
I believe .impact was set up partly for this purpose, and its projects page is a pretty good place to report this sort of thing.
There’s a lot about this idea that I agree with. It seems important to get effective altruists (and everyone) to share more information about what they’re trying to achieve, whether it worked or didn’t, and why. In particular, a lot of us are trying radical things like applying for finance jobs or starting companies, which have high variance, making it hard to infer whether these are good decisions.
I think this idea can grow a lot. It would be good to have everyone we know pooling together information about when they started a company, for example, so that we could infer whether it’s better to start more companies or work for somebody else. In the long run, it would be ideal situation would be to have thousands of people making their decision in similar ways to you, and to have a recommender system that can give you suggestions by giving extra weighting to the experiences of people similar to you.
However, I do have a bunch of feedback and questions:
1 Is a registry the best way of sharing information about what activities are working well and which aren’t?
2 Is there some reason to focus just on explicit experiments rather than lots of activities with uncertain payoff? Are there enough explicit experiments to support this project? Don’t we want to reduce reporting bias in general, rather than just publication bias? i.e. we want people to report the successes and failures of activities with uncertain payoff, rather than just explicit experiments. So it might be better to call it a Reporting Bias Registry.
3 On second thoughts, shouldn’t it be named by what it’s trying to achieve rather than what it’s trying to avoid? e.g. the ‘charity project performance registry’, or ‘social impact registry’, or something like that.
4 I think it’s important to be clear about what exactly you want people to do. At the top or bottom of the page, you could write in bold that what you want is for people to write information about their projects on the wiki.
5 It’s not clear that a wiki is the best way to implement this. Few people don’t use the wiki, and some of the important experiments that people run might be private? Perhaps it would be better to make a Google form, and to assure people that any information that is sufficiently specific to identify them or their project will not be disclosed? Or perhaps privacy would only work if there are at least dozens of projects. It’s at least worth thinking about.
6 Is this substantially different from the EA Survey? Is it substantially different from the EA Profiles? Is it substantially different from GWWC’s donation registry? CFAR’s alumni community? 80k’s alumni group? Can it be integrated with any of these things?
Maybe we should also get everyone to report their progress on the projects that we already know about. This is kind-of different, because the reports will be somewhat biased but it still seems worthwhile.
So I would be interested in how you would respond to some of these challenges and how we could plan around them in order to make the project more likely to be the big success that the idea deserves.
There are plenty of valuable thoughts here. I also like Giles’ idea and think it’s worth giving a go.
A few specific comments:
I think a wiki’s fine for now, and pretty simple. I don’t think there’s yet reason to worry about privacy. In general I think Giles’ setup is a decent minimum viable product.
I believe .impact was set up partly for this purpose, and its projects page is a pretty good place to report this sort of thing.