COVID-19 Risk Assessment App Idea for Vetting and Discussion
Please note: These opinions are my own. I’m not an expert, but I have a Q&A below with Sukrit Silas, an expert on infectious disease who supports these ideas and thinks they merit further discussion.
Below I discuss some worst-case scenarios for COVID-19, and my preliminary ideas for an app that could help slow its spread. Like anyone, I don’t know how bad things will get; we just have to wait and see. But obviously, if you want to be ready for a deadly pandemic, however unlikely, you have to start well in advance. This is particularly true for developing an app, which requires time and resources.
I’m hoping for some EA feedback and comments.
Introduction
COVID-19 has a relatively long infectious incubation period, roughly 5 and up to 14 days, during which there may be asymptomatic transmission. This allows it to spread more quickly than many other viruses do. It also makes contact tracing — to keep infections under control — harder, slower, and less scalable.
However, it is now easy to get GPS data for individual smartphone users. In fact, mapping apps like Google Maps and MapQuest already collect this data, by default, for their users.
This GPS data could perform a rapid, automatic, and probabilistic form of contact tracing. It could also alert users who might be exposed to the estimated risks associated with a given area.
The app could also provide individualized recommendations, based on risk calculations from their shared GPS data. These recommendations would reduce uncertainty and could help people better understand their own risks, make more informed decisions to avoid personal exposure, and slow the spread of the virus. Ultimately, this could save lives.
We could also provide users with a heat map, showing the risk levels of surrounding areas. This map could be more fine-grained than Johns Hopkins, while still coarse-grained enough to protect users’ privacy. Crucially, users would not be able to identify other users, based on the heat map information.
I want to have an app like this, to use myself. And I want other people to have it, and use it. My goal with this post is to increase the chance that someone can, and will, build such an app. Doing so could have a measurable impact on slowing the spread of COVID-19.
Privacy Focused
China has developed its own coronavirus app, but it worried me. I’m a libertarian, and I started thinking about whether an alternative privacy-respecting version could work. I understand China’s doing everything they can to fight the virus, but the level of privacy violation is huge. So, to be clear, that is not what I’m proposing.
I believe that level of invasiveness is also completely unnecessary for the app’s effectiveness. I think the highest impact information that individuals could give is a few answered questions and their GPS data — which most people share with many private companies already. For the average person, the additional privacy burden could be minimal to non-existent, if the app is built by someone who cares about privacy.
The app could be built by a not-for-profit group collecting only the (limited) information that would have the highest impact on the spread of the virus. Theoretically, the app’s database could still be hacked and abused by governments. So the app would need good security, and perhaps even to be blocked in countries where citizens’ rights are most likely to be compromised.
This could be balanced with considerations regarding how likely it is to prevent even worse future incursions on your privacy that could follow a societal breakdown, city or community quarantines, a totalitarian government app, forced compliance measures, military involvement, or other things no one wants.
Of course, there’s still a risk that this app wouldn’t be accepted because of incorrect pattern matching and association with China’s app. Quick adoption will be a hard challenge, even without additional PR challenges. But the actual privacy harms seem like they can be significantly mitigated.
The Incentives Align
The most promising thing I want to communicate about this app is that the incentives to use it align much better than one might assume.
It’s in your interest to install the app, if only to see the heat map and monitor the risk levels in your own area. It’s in your interest to turn on your GPS and keep it on (or connect it to the info Google already has for most people’s phones), because the app couldn’t calculate your risk score without that. And it’s in your interest to report that you have cold symptoms — whether you think it’s COVID-19 or not — because then you can see how it changes your risk profile, and better learn what precautions to take.
We should also find ways to incentivize those who already have a confirmed case of COVID-19 to put that into the app. Also, preventing hoaxes (at least early on) is important. With any luck, if someone making this app can make the right connections, they can ask the CDC to give people with confirmed cases (or their close contact or family member) a code for their phone and strongly suggest they put in the code to confirm, and to help everyone else out. Most people listen when the government suggests something even if it’s not a requirement. These are people who would also already be subject to CDC contact tracing, so sharing their confirmed status / GPS is less invasive than what you’re already subject to at that point.
Also, if things got really bad and there were a lot of cases, we could then relax the constraints for needing a code because at some point we’d have enough data that hoaxes would have little effect.
Reasoning About Scenarios
(I’m imagining one of the worst case scenarios here, and I’m not making claims about what’s likely to happen now.)
In some reports, many of the lethal and non-lethal cases of COVID-19 seem like they ended up needing extended intensive care that only a hospital could provide well—for example, 1-2 weeks of supplemental oxygen. If 1⁄3 − 2⁄3 of the world eventually get COVID-19, the case fatality ratio is 2%, and the transmission rates are not reduced enough because of the roughly 5 and up to 14 day infectious incubation period, many hospitals would be overwhelmed and the fatalities will be considerably higher than they might have been if the spread of the virus had been slower.
In this scenario, by participating in the app, you’ll get information quicker about possible exposure. And, if you do get the virus, you’re more likely to get treated effectively at the hospital if there are fewer other people using the same hospital resources at the same time. In this scenario, what helps everyone also helps you and we all get a slower moving pandemic where hospitals don’t get as overwhelmed.
There is also a possibility in early scenarios where the virus hasn’t taken hold in a community, if enough people participate in the app, in addition to the measures already being implemented, that the app could not just slow, but help to stop the virus. There lots of caveats here about stopping the virus though. It’s theoretically possible in some scenarios on the margins, but we’re unlikely to be in exactly those scenarios.
This could also help slow down the spread of the virus anywhere people have smartphones. Even if the virus is contained in the United States, in other countries it may not be contained.
And, even if the severity of the situation increases, providing good guidance based on clear information also helps to prevent panic. Panic makes things worse for everyone and it takes lives too. The safest world is the one that slows or stops the spread of the virus while continuing to provide normal services as much as possible. Therefore, to save lives, it is important both (1) for people who are at lower levels of risk to continue to participate and provide normal services and (2) for people who learn they are at higher levels of risk to stay home and not contribute to the spread.
Fortunately, we could all help fight the virus even in these scenarios, saving lives while also rationally protecting ourselves, just using our current technology.
If COVID-19 Stays with Us
Take a similar scenario where the virus also comes around every year like the common cold, but continues to have a relatively high fatality rate (however unlikely). Even if we have imperfect adoption of the app, the first year COVID-19 comes around, a lot of people who didn’t use the app or otherwise prepare would be likelier to get infected. The second year it comes around, the outbreak is considerably less because now many of those people might still have immunity. And the rest say, “Ok, here we go again” and turn on the app. And the second time around, it’s more likely to be shorter, or stopped.
Perhaps only a tiny fraction of people use it. In that case, you could think of it more like early Google traffic updates. Still a little useful. And if it works well, and guidelines tell you to stay home given a level of risk, perhaps it’s like a bad snow day, except in more localized areas, and more like a snow week-or-two.
Also, for those who start prepping or considering self-quarantine for some period of time, the question might be asked—what’s your trigger for going out into the world again? Seeing the heat map settle down nearby you or seeing your score go down would be very helpful information for making this decision.
Example App Questionnaire
* Do you have confirmed COVID-19? If so, include code. Perhaps options for lab / CT diagnosis.
* Have you been to any of these countries in the past two weeks / month?
* Have you been informed that you might have been exposed? If so, when?
* Have you been told by any authorities to self-isolate or that you should have self-isolated for a given period? If so, when and for how long? (Maybe to what extent would you say you were able to comply during that period? 0-100%? Not necessarily your fault, perhaps you were informed late / or had some other emergency.)
* Do you have any upcoming travel plans? Are you traveling through any of the following airports?
* Have you had cold or flu symptoms recently? (Even if you’re quite sure it’s not COVID-19) If so, when?
* Are you currently waiting on test results for the virus?
You could easily go back to the questionnaire to update your answers at any time as they change or you need to make a correction. You could get pinged once in a while and asked for updates. You could also get pinged if your level of risk increases.
I think asking people on the questionnaire whether they just have cold or flu symptoms is possibly really useful information. For example, right now, if you come down with a cold for the foreseeable future in the U.S., it’s very unlikely it’s COVID-19. Lots of the usual stuff is going around. However, let’s say you are living in an area where the heat map is lighting up. This gives the app very useful information about whether you might be exposed, not necessarily for direct health advice, but for estimating risk for yourself and others, considering where you may have gone out to (even only briefly) recently.
Q&A With Sukrit Silas
In 2019, Sukrit Silas was listed in “Forbes 30 Under 30: Healthcare.” Working in the lab of Nobel laureate Andrew Fire, Silas discovered how bacteria use CRISPR, which is part of their immune system, to learn to attack viruses that are made of the genetic material RNA. His company, BillionToOne, raised $15M in 2019 for developing diagnostics for beta thalassemia and Down syndrome. He’s currently a postdoctoral fellow in the the Laboratory of Prof. Jonathan S. Weissman (UCSF).
Q: Do you think this app would be useful for slowing the spread of a virus?
A: It doesn’t make as much sense to do for something general, like the flu, because you’re not asymptomatic for long before you’re infectious. But if there is a real threat where the virus either stays active on dry surfaces for a long time, or like with this coronavirus, where there is a long infectious incubation period, this would be very useful.
Q: Might an app like this be broadly applicable for public health in the long term?
A: Might be a good excuse to build the app, for the next thing… I would definitely build it to be generalizable though. The threat might not be viral. Pan-Antibiotic resistant bacterial infections are probably going to be a much nastier fight. And something like this could really help there.
Q: Do you think showing people a heat map would be useful? Some kind of geographical risk estimate?
A: I think an integrative geographical risk estimate could be useful, and has the advantage that if you have lots of data (even *bad* data), your predictions will still be useful.
Q: Where do you see this being most helpful? And what risks do you foresee?
A: I’m quite interested in the potential for this to be useful for infections that actually perdure, like antibiotic resistant infections. I’m generally less worried about privacy/rights issues as I am about the possibility of starting a panic/hurting people’s livelihoods.
Q: Anything else you would like people to know?
A: I think it makes sense to post to EA and gauge interest/explore ideas. I just have a few ideas:
(1) I agree with what you say completely: The success will depend on the “fidelity” of the self-reporting more than anything else. Especially in places where people might be afraid of their governments, I’d think a way to reassure people that their responses will not jeopardise their safety would be important. For example, I trust the LastPass folks, but not Facebook.
(2) Perhaps a way that it would be impossible to centrally trace which users are infected.
(3) I do think that something like this could be helpful in the long term, in a more general fight against infectious disease.
(4) It could be useful to get someone at CDC or something to look at an early prototype and get a sense of tweaks that might be helpful given the ground reality. I think they’d be interested to chat, if a working model were available.
Envisioning A Prototype
This is what I imagine an implementation as a standalone app might look like.
Name of app
COVID-19 Risk App
What is the app for?
Guidance and GPS based personal risk assessment for COVID-19
About the app
People are getting anxious about coronavirus. People want to know how to understand their personal risk, how to protect themselves and prepare given their level of risk, and how to help.
This is a COVID-19 risk assessment app that would calculate a risk level for you personally. To calculate your score, install the app and answer a questionnaire (anonymous) about your possible exposure to the virus (currently have it, close contacts, when they had it, recent travel, etc.) and allow access to your GPS location and your GPS location history (if available) from Google/Apple/Other mapping apps you already have on your phone from the past 1 month.
The app will show you a heat map with risk levels for different areas nearby. Also, once we have a few days of GPS data, you will get a risk score. You will see the risk score immediately if you already have an app with sufficient map history on your phone. The app will give you advice about what to do based on your current level of risk. Your risk level updates over time.
When a user updates the app that they have the virus (confirmed with a code from the CDC) and we have the past ~2 weeks of their GPS data, we can quickly and easily get updated scores and recommendations to self-isolate or stay home to other users who have shared their GPS data and have some likelihood of having been near an infected person during their asymptomatic infectious period.
The effectiveness of the app for assessing your risk depends on (1) whether you fill in and update the questionnaire as appropriate and (2) how often and for how long you’ve kept your smartphone near you with your GPS on. Your questionnaire and GPS location sharing are the foundation of the app, allowing for an improved evaluation of exposure and risk with as minimal impact on your privacy as possible to give you an estimate of your risk.
The apps also asks you to agree to confirm that you have the virus on your phone (or ask a contact to do so) if you become ill and are confirmed to have the virus.
Also, this app is for personal recommendations for people who are not otherwise receiving recommendations. If local authorities make other recommendations, they take precedence.
Why Should I Care?
I think it meets GiveWell’s “Importance, Tractability & Neglectedness.”
* Great in scale (it affects many people’s lives, by a great amount)
* Highly solvable (additional resources will do a great deal to address it), and
* Highly neglected (few other people are working on addressing the problem).
I acknowledge that I have limited insight about how neglected this is, or how many people are working on this. It’s possible that the CDC has a big project on this and we don’t know. It’s also possible they’re really busy right now and new ideas aren’t the priority.
What Can I Do Now?
Right now, I’m hoping for some EA feedback and comments.
I also think a small, well-funded, agile, and motivated group could do this best, who coordinates with multiple large orgs like Google Maps, Johns Hopkins, CDC, WHO, etc. But I could be convinced an existing org with the right set of people could do this too. In either case, it could be useful to get someone working on a demo right now because people (perhaps at the CDC or Google Maps) are more likely to talk to you, and be convinced if they can see a working example.
I imagine you also wouldn’t want to bother the CDC much when they’ve got important things going on right now. Doing this as a separate group while doing some minimal amount of pinging one or two people at the CDC to check would not be too disruptive. In case the app ends up not working well, at least we didn’t waste anyone’s time much. Ideally, you get people working with you who are not otherwise working on the problem.
It seems possible that Google is already doing something like this or thinking about it, but if not, an EA at Google could advocate for this to happen. Google Maps would be very well positioned to help implement a big part of this.
My hope is also that this reaches someone with connections who can do this and form a team. I’m not an expert, but I wrote up a short plan that might be worth skimming over quickly—including more possible useful questions, example recommendations, a 1-7 risk scale, and an overview of the people you would need to form an initial team for the problem. If someone is interested, contact me if you think you can contribute.
Should I Build My Own Rogue App?
No. My sense from talking to experts is that it would be really easy even for well-intentioned people to cause harm. People are inclined to panic in a pandemic, and incorrect or poorly communicated information could cause people to act in ways that make things worse. Bad scores, maps, or advice that make people panic or act too soon (or too late!) would cause more harm than good. You also have to worry about hoaxes.
If you’re considering building a tool like this, collaborating with the health security community is essential. You should probably be working with someone who has relevant expertise and connections in this space. Ultimately, I think the only version that really works without likely causing much more harm than good is one that has some support from authorities like the CDC.
Author
Tina White
I’m a PhD Candidate at Stanford doing applied machine learning research for aerodynamic design. I’ve also done a couple personal web/mobile projects, so I have some sense of what the app might involve, and if there was time, I could make a demo. I started going to Stanford EA Meetups about a year ago because they had consistent and high quality AI Safety discussions.
Website: www.tinarwhite.com
Email: crwhite@stanford.edu
Acknowledgements
I’m grateful to Chris Olah and Joshua Zader for writing feedback, and I appreciate the comments and critical feedback I got from Howie Lempel and the Effective Altruism Coronavirus Discussion Group.
This app idea is focused on privacy, but not for the reasons people may assume. It isn’t just an abstract ideal. It’s privacy-focused because it would need to be. The app would rely on people voluntarily sharing data. In many countries, people do not feel safe sharing certain kinds of information because of their government. And if it’s not safe to share your information, people won’t. Making this as safe as possible would be crucial, and that’s what the privacy focus is about.
We have a website now too. Please check out our collaborate page if you’d like to help: https://www.covid19risk.com/
I know of one related effort: https://twitter.com/webdevMason/status/1234216664113135616
http://www.coepi.org/
CoEpi is a great team too! We’re helping each other out now with a lot of cross-communication between our slack channels.
Your sections on incentive design is very persuasive, and seems like a great starting point.
And before I start with my concerns, Western governments are already kind of doing a similar thing: They identify contacts of infected people, including people who dined at the same restaurant at a similar time, in order to test them.
However, some concerns/questions that were partly raised in the fb group already:
0)Even basic questions about the virus and how it spreads are still unanswered, like how infectious one is during the incubation period. This makes more advanced questions regarding a risk score difficult to answer.
1) How likely are you to catch the virus at all just by being in the same area/frequenting the same shops as somebody infected? My impression from the Western cases so far was that it infections occurred generally with close contacts; this risk changes obviously when more infected people are around, but still should be estimated to decide whether such an app would be worth it.
2) Regarding the computation of the risk score: If you only use confirmed cases with voluntary sign up, you might not get enough data; if you use suspected cases by symptoms, you will get a lot of false positives due to worried people with the flu. In the absence of data on how to properly account for that, this is a very difficult problem.
3) You mention that Google traffic data is still useful, even when few people use it. I am not familiar with that part of the app, but if it involves some form of prediction, it is important to note that Google has had years to get this right. With a pandemic, you have at best months(!), and on top of that the situation changes constantly.
Given what I assume is a dearth of good historical data on every aspect of this , I think that calculating both an accurate risk score AND communicating it without doing more harm than good, is a very tough problem.
A dumbed down (but maybe not useful anymore) version of the app could just do without the risk score. It would essentially be a more convenient approach for governments to identify people who have been to certain areas to self-isolate or watch their symptoms. The location tracking and notification could still be done by the app, but the decision making would lie with health offices.
Thank you for raising the concerns that you did. I appreciate the opportunity to explain more about how I’ve been thinking about these concerns. This is the kind of feedback I was hoping for from posting on EA Forum. I’ll try to address each one.
This is called contact tracing. In the Introduction, I refer to contact tracing, but I think I missed an opportunity to define what it is, so thank you for pointing this out. What I’m proposing, like you say, is similar. It’s like an automatic, probabilistic form of contact tracing aided by a lot of GPS data.
Just to be clear, to me the fact that Western government are already doing it is a positive point in favor of your proposal, since it is evidence for utility of contact tracing in containing the virus.
I agree. And there are a lot of resources being put into research on this right now, so in time I hope we have better answers. But even imperfect information could be helpful. See the Q&A with Sukrit Silas. At first I imagined the app could only give a risk score that was very coarse. Just levels 1-7. I’ve commented separately, header Example Score Levels, with an example of what I mean, which I didn’t put in the post because I have no confidence in what it should be. But you might be able to show a decimal point too. I think it’s good not to start out being committed to any kind of risk scoring system until you have a sense of what’s possible.
Yes. This is another reason why working with someone like Google Maps or some other mapping app could be crucial, because they have accumulated domain / tribal knowledge that no one else might have.
[Edit: I’ve received some private feedback that neither of us might be right here. The calculation for both (density of traffic and density of “infectiousness”) may be quite straightforward. The reason traffic updates got so much better might just have been more data.]
These are significant challenges and I talk a bit about how they can be addressed in The Incentives Align and at the end of the section Example App Questionnaire. I imagine there would always be more confidence put in a confirmed case with a code than someone who just answers yes to having cold or flu symptoms recently.
Also, for confirmed cases, as part of contact tracing, the CDC sometimes identifies a site of concern where a patient might recall that something particularly infectious happened before they were aware they were sick. For example: “Oh. I remember that a few days ago, I sneezed quite forcefully and unexpectedly at my favorite buffet, so I couldn’t cover my nose. Oh, and then again on the way home on the BART! I’m so sorry.” Tracing multiple paths backwards, you might get a lot of data from a single event.
I imagine the CDC could already have some estimates for this, which they might use in contact tracing. And it might turn out that contact tracing is enough to solve the problem. It seems to be working well right now in the U.S.
But if not, a not-so-educated guess for a general outline of the calculation of risk for the app might be (1) Close contact. You were in the same location where a possibly infectious person spent some time within maybe 10 minutes of them. This is higher risk. (2) Semi-Close Contact. If the virus might live on surfaces for a few hours, the close contact risk distribution tapers off over a few hours and (3) Infectiousness. This isn’t binary so the infectious person’s distribution also peaks sometime around their first symptoms, and tapers off over a few days up until the tail reaches some maximum.
A heat map that changes with time could reflect this information. And any individual’s risk could be calculated by integrating over it. And, if the local situation is suddenly found to have had multiple people in it in the last few weeks, and the level of precision was possible, you could do multiple iterations of this given each user’s (small) risk of contracting the virus from their interactions too, given an estimate of what we think the time might be between when a person contracts the virus to when they become infectious. This is harder, but if everything else works, it’s possible.
Writing this out in words is long, but the actual summation/integral is not too complicated. It’s a combination of science and hack-y guesses. But this seems true to me of almost all engineering.
Potentially useful to build on, I have some colleagues that have done something similar:
https://www.lshtm.ac.uk/newsevents/news/2018/bbc-experiment-co-developed-lshtm-generates-dream-dataset-flu-researchers
It doesn’t include the user feedback element (i.e., telling them when they are at risk), but seems to have many of the other elements.
Wow, what a great dataset! If you have some colleagues that might be interested, please link them here to the forum. I’ve also made a couple public Facebook posts about it looking for collaborators in COVID-19 discussion groups and publicly on on my personal page here and here with more information about who can help.
Example Score Levels
I didn’t put this in the original post nor have I formatted this well because I have no idea what the levels should look like and I’m not committed to them. These are just example score levels for reference:
7 - severe—you have confirmed COVID-19
6 - very high—you are in Wuhan right now, the cruise ship in Japan, or your family member or roommate has had the virus
5 - high—you are in another outbreak in China right now or live near a different community outbreak or in a quarantined rescue from one of these places
4 - medium—you have significantly crossed paths with someone who had the virus and was infectious or you live somewhat near a community outbreak
3 - low—you have crossed paths with someone who may have had the virus or you live or work in an area with more risk of this
2 - very low—you may have crossed paths with someone who may have the virus, but it’s still very unlikely you are infected
1 - negligible—there is no one near you with the virus and you are in a lowest risk area
And maybe
0 - you already had confirmed COVID-19 and recovered
I don’t know this, but I think most people in the U.S. probably have the lowest level 1 - negligible risk right now. It would also be reassuring to see reflected in a score. They’d only need to listen to general advice (wash your hands more and pay attention to the CDC’s recommendations and updates).
These are pretty coarse levels so they aren’t too hard to calculate. And if you are at levels 5-7, you probably already know. If you’re at level 4, you’ve probably already gotten a call from the CDC (or equivalent). But anyone at a lower level has no idea where they are at. Hence, the app.
I also want to point out that at high levels, things like quarantines, contact tracing and requesting that some people self-quarantine are known to be quite effective. At lower levels, we’re given guidance like washing your hands. It wouldn’t be too surprising that an app that assesses situations in between using this sort of probabilistic contact tracing and gives recommendations for things like individual self-quarantine could also be somewhat effective without being too disruptive.