Eddie Liu

Karma: 121

Eddie Liu 9 Feb 2024 1:51 UTC
4 points
0 ∶ 0
in reply to: huw’s comment on: Self-guided mental health apps aren’t cost-effective… yet
Ratio of Installs to DAU: Hmm, that’s an interesting metric...the way I think about retention is like a layered cake, kind of like the baumkuchen I just ate for breakfast, but linear instead of round. Anyways, there’s time on the X axis and users on the Y axis. For any given day, there’s a sizeable layer of cream at the top which are the Day 0 users. And then right below that, a smaller layer of Day 1 users, etc. etc. Ultimately there are hundreds of layers of users from older daily cohorts. You can track each daily cohort through time and it’ll start big and then shrink rapidly, following the retention curves, until eventually it flatlines at some point (ideally above 0).
So you could look at overall installs to DAU, but that gives an advantage to new apps because they don’t have a lot of old installs from years-old cohorts that have left. Or you could compare daily installs to DAU, but that gives an advantage to old apps because they’ll have a lot of users from old cohorts.
A better metric could be DAU / MAU ratio, which measures like out of all of your active users, how many of them use the app every day. Here ~25% would be exceptional with an average of probably around 10%. But that’s also biased based on how many new users you’re bringing in each day.
By the way, the only peer group benchmarks that apple provides are Conversion Rate, Proceeds per Paying User, Crash Rate, Day 1/7/28 Retention. https://developer.apple.com/app-store/peer-group-benchmarks/ . But they might be announcing more in March thanks to the EU’s DMA. https://developer.apple.com/support/dma-and-apps-in-the-eu/#app-analytics
CPI: Yes, those numbers are from Facebook / Instagram / TikTok ads.
In terms of organic traffic, it’s also a measure of time. Say for example you’re bringing in 1000 organic users a day. After a year, that’s 365k users. After 5 years that’s 1.8M users. Of course, the app still has to remain good to continue getting organic downloads. Since the definition of good is always improving, the app would need to be consistently updated.
I’d say estimate around ²⁄₃ of our lifetime installs are organic, but it really depends on the app. I speculate that Daylio might be closer to 100% organic while Breeze is probably closer to 0%.
What links here?
- Therapy without a therapist: Why unguided self-help might be a better intervention than guided by huw (8 Apr 2024 6:01 UTC; 67 points)

Eddie Liu 8 Feb 2024 6:47 UTC
24 points
1 ∶ 0
on: Self-guided mental health apps aren’t cost-effective… yet
Great overview! I’m Eddie, long-time EA and co-creator of the mental health app Clarity https://apps.apple.com/us/app/clarity-cbt-thought-diary/id1010391170
I pretty much agree with your broad points. Some quick thoughts on each section from an “insider” point of view:
Short-term impacts of treatment: The problem with these studies is that they study “apps” as a category, but apps are 1. extremely different from one another and vary drastically in quality and 2. are constantly improving. The best apps are probably 10-100x better than the worst mental health apps. Also, functionality of the best apps today are very different than the best apps from a few years ago when these studies were run. In the future, they’ll probably be even more different. I don’t think we’re close to peak mental health app effectiveness.
Self-guided app adherence: Apple actually has made benchmarks available! I can grab them for you. For health and fitness apps which rely on a subscription business model, here are the 25th / 50th / 75th percentiles.
Day 1: ~16% / ~25% / ~34%
Day 7: ~3% / ~7% / ~13%
Day 28: ~0.75% / ~2.5% / ~6%
Note: The exact percentages vary by a little bit depending on which week is selected.
User acquisition costs: I can confirm there are huge differences in cost per install between US iOS (in the range of $1-2 per install) vs low-income country Android ($0.02 - $0.10). Although of course, ads aren’t the only way to get app downloads. Organic app downloads are $0.00!
Development costs: If you were to put us on your comparison, we’d be at 2 employees (my wife and I) @ ~8,000,000 downloads across all of our apps. I can confirm that service and hosting costs are basically negligible.
How to improve: I agree that marketing to low-CPI regions could be a great impact opportunity. I’m most excited about improving drop-off rates / effect sizes (two sides of the same coin). I can absolutely imagine a world where a self-guided mental health app is 10x more effective than the best one available now.
Again, great overview! Let me know if I can be helpful!
What links here?
- Therapy without a therapist: Why unguided self-help might be a better intervention than guided by huw (8 Apr 2024 6:01 UTC; 67 points)

Eddie Liu 12 May 2023 20:34 UTC
6 points
1 ∶ 0
on: Simple charitable donation app idea
I would lean towards making the first version a website instead so you wouldn’t have to go through the hassle of following Apple / Google’s rules. For example in the App Store Review Guidelines there are extra procedures to follow for such apps—https://developer.apple.com/app-store/review/guidelines/ it explicitly states:
3.2.1 Acceptable: (vi) Approved nonprofits may fundraise directly within their own apps or third-party apps, provided those fundraising campaigns adhere to all App Review Guidelines and offer Apple Pay support. These apps must disclose how the funds will be used, abide by all required local and federal laws, and ensure appropriate tax receipts are available to donors. Additional information shall be provided to App Review upon request. Nonprofit platforms that connect donors to other nonprofits must ensure that every nonprofit listed in the app has also gone through the nonprofit approval process. Learn more about becoming an approved nonprofit.
3.2.2 Unacceptable: (iv) Unless you are an approved nonprofit or otherwise permitted under Section 3.2.1 (vi) above, collecting funds within the app for charities and fundraisers. Apps that seek to raise money for such causes must be free on the App Store and may only collect funds outside of the app, such as via Safari or SMS.

Eddie Liu 9 Nov 2022 21:24 UTC
21 points
0 ∶ 0
on: Money Stuff: FTX Had a Death Spiral
https://archive.ph/6AwJo

Eddie Liu 20 Aug 2022 2:57 UTC
17 points
0 ∶ 0
on: “Call off the EAs”: Too Much Advertising?
Part of the ad overload problem you’re pointing out comes from the nature of both elections and book releases. They’re both things that happen once and benefit from gaining a lot of momentum in a short period of time (winning the election, getting on best-seller lists). As a consequence:
- Ad frequency (how often the same user sees an ad) becomes really high in a short period of time.
- There’s not a whole lot of time to do creative / copy testing and optimization, so users are often hit with the same or similar ads.
Ads that are offered on a more ongoing basis can reach users more gradually, making it more likely that they’re not targeted again until 1. a lot of time has passed and 2. the ad content / copy has changed due to optimization over time.
So one takeaway could be finding ways to advertise effectively over time instead of in bursts!

Eddie Liu 3 Aug 2022 5:07 UTC
22 points
0 ∶ 0
on: Some updates in EA communications
LOVE the new intro article!!
Feedback
At least for me, it was hard to tell the hierarchy of the content. I wonder if a table of contents might be helpful?
I think the issue stems from H3 and H4 tags being hard for me to tell apart, so a little confusing to subconsciously keep track of where I was in the document. Another problem could be the “What values unite effective altruism?” and “What are some examples of effective altruism in practice?” are H3 but “How can you take action?” and “FAQ” are H2 but in my mind they should all have been at the same level? Maybe just promoting the first two headers to H2 would be good enough to solve most of my confusion.
Also, the preview image when sending the link to someone on LinkedIn strikes me as a little odd and might hinder virality when it’s time to share it on social media.
Ideas for Iteration
If the intro article takes off and becomes the “top of the funnel” of effective altruism for a lot of people, optimizing the “conversion rate” of this article could have big downstream effects.
I would definitely encourage collecting 1 on 1 feedback by having people new to EA read the content in person and speak their thoughts out loud.
Qualitative feedback can also be gathered more quickly with a tool like Intercom to directly chat with people while they’re reading it and hear their thoughts / answer their questions.
It might also be a good idea to get some quantitative feedback with a tool like hotjar to see how far people scroll.
If the goal of the article is to get people intrigued with EA and diving deeper, perhaps emphasizing the “How can you take action” or the other parts in the “What’s next?” section with special graphics or banners kind of like the newsletter box would be helpful. Then you can A/B test different iterations to see what gets people to tap more.
Speaking of A/B tests, you might be able to squeeze out a few more percentage of engagement by experimenting with the order of the examples, the number of examples, which examples are shown, the actual words in the content itself, the preview image, etc.

Eddie Liu 25 Apr 2022 23:51 UTC
2 points
0 ∶ 0
on: What are the key claims of EA?
I was just thinking about this the other day. In terms of pitching effective altruism, I think it’s best to keep things simple instead of overwhelming people with different concepts. I think we can boil down your moral claims to essentially 3 core beliefs of EA:
1. Doing good is good. (Defining good)
2. It is more good to do more good. (Maximization)
3. Therefore, we ought to do more good. (Moral obligation)
If you buy these three beliefs, great! You can probably consider yourself an effective altruist or at least aligned with effective altruism. Everything else is downstream of these 3 beliefs and up for debate (and EAs excel at debating!).

Eddie Liu 17 Apr 2022 21:18 UTC
7 points
0 ∶ 0
on: Is working for Meta a good or bad option?
I’m not confident in the belief that Meta as a company causes more harm than good. If you look closely at most of the criticism, it’s generally overblown.
For example, the headline of the 2021 Facebook Leak is “Instagram Harms Teenagers!” but the reality is more complicated—https://www.npr.org/2021/10/06/1043138622/facebook-instagram-teens-mental-health . Cambridge Analytica too: https://mobiledevmemo.com/cambridge-analytica-was-a-false-panic-its-time-to-move-on/ .
I’m more convinced of the overall harms of social media (esp. in regards to teenage mental health), but is Meta is worse than other social media companies out there like TikTok? Probably not?
In terms of benefits to the world, they enable a lot of small businesses to exist through personalized advertising and they’re pushing the field of virtual reality forward faster than it would otherwise develop.
If I had to guess, Meta’s impact on the world (compared to a world without Meta) is net neutral or maybe slightly positive (/negative), instead of significant negative impact. So your impact of joining Meta as 1 of 70k+ employees is likely a few orders of magnitude lower than the impact you’d have through earning to give.
Other factors to consider—What team would you be on? Can you get into other FAANGs? How long would you even work at Meta before you jump jobs? Is there a significant reputational risk of working at Meta, regardless of the facts?

Eddie Liu 17 Jan 2022 20:58 UTC
4 points
0 ∶ 0
in reply to: Chris Leong’s comment on: Introducing Effective Self-Help
Depending on the study design, it can actually be relatively cheap! Just use Amazon Mechanical Turk or Positly to get participants. Of course, some study designs would be 20x harder than others. I imagine that testing different self help interventions would be rather hard but smaller surveys would be really easy.

Eddie Liu 12 Jan 2022 22:44 UTC
8 points
0 ∶ 0
in reply to: IanDavidMoss’s comment on: Momentum 2022 updates (we’re hiring)
Seriously! This has gotta be one of the best update/recruiting posts I’ve ever seen on EA Forums. Wishing you guys the best!

Eddie Liu 5 Nov 2021 17:29 UTC
12 points
0 ∶ 0
on: Who are your role models?
Personally, I found this article to be full of inspiring people for me—https://www.nytimes.com/2021/04/27/magazine/global-life-span.html
Here are my notes I took on the article at the time:
- Pattern is that science discovers something important, a long time passes, then a crusader spreads it to the masses.
- Variolation (an early form of vaccination): Invented in 1000s, 700 years later, popularized by Mary Montagu, an aristocrat in 1716. IMPACT!
- In 1858, a JOURNALIST published an expose denouncing MILK as a liquid poison. Pasteur invented pasteurization by 1865. 30 years later. Nathan Straus in 1892 took up the cause, ran an experiment, published it, started an extended advocacy campaign. IMPACT!
- John Leal, a sanitary adviser, quietly added chlorine to the city water. It was a success. It spread from city to city! Triumph of applied chemistry. GLORIOUS!
- Peniccillin: Fleming invented it very early, but 2 Oxford scientists 10 years later did experiments and spread it. GO OXFORD!
- Oral Rehydration Therapy: A johns-hopkins educated physician & researcher pioneered the use case, went against standard of care at the time, and crushed it! Started an education campaign (advocacy) letting amateurs into the act. IMPACT!
PS. If you want to read the article but can’t, try this extension https://chrome.google.com/webstore/detail/disable-javascript/jfpdlihdedhlmhlbgooailmfhahieoem?hl=en-US

Eddie Liu 26 Oct 2021 5:19 UTC
6 points
0 ∶ 0
on: Many Undergrads Should Take Light Courseloads
I agree with this advice in general. I’d add a caveat that it might be a good idea to prioritize taking courses where you can make smart, like-minded friends. For example, a Computer Science class with a lab component where you work in small groups to get assignments done.