When To Find More Information: A Short Explanation

What do you need to know to make a de­ci­sion, and when should you try to find out more?

I think this is a re­ally im­por­tant topic, but when you’re mak­ing a de­ci­sion, it’s one that de­serves very lit­tle at­ten­tion—it’s just rather im­por­tant that it gets that bit of at­ten­tion.

I wrote my dis­ser­ta­tion on the topic, and would ar­gue that ba­si­cally all ques­tions that mat­ter should be ap­proached us­ing Value-of-In­for­ma­tion (VoI). And there are com­plex quan­ti­ta­tive tools that are use­ful for de­ci­sions about this. In do­ing my dis­ser­ta­tion, I tried to ac­tu­ally use some of these meth­ods. I came to the con­clu­sion that you will all live far hap­pier lives if you never ac­tu­ally use them, or even read a dis­ser­ta­tion about them. In­stead, there are some sim­ple ap­proaches and heuris­tics that get you al­most the whole way there, so I’m writ­ing this post.

Hope­fully, this post can help get most of the value at a very low cost. But it will help even more if you use it as an ex­er­cise for a spe­cific de­ci­sion you are mak­ing. If pos­si­ble, pick some per­sonal de­ci­sion you need to make now—even if it doesn’t help, it will be use­ful prac­tice.


VoI for Smart People

You have a de­ci­sion to make. The fact that you are try­ing to make a de­ci­sion im­plies that the an­swer isn’t ob­vi­ous—you’re un­sure about some­thing. Given that, the first step is to spend 5 sec­onds think­ing about whether get­ting more in­for­ma­tion would help you make the de­ci­sion. At this point, you al­most always should be able to think of some­thing that you would want to know. (If the an­swer is that you don’t need any, you prob­a­bly want to think again—or read the above linked post.)

If you think of in­for­ma­tion you can get eas­ily and at very low cost, get it. After­wards, if the de­ci­sion isn’t made, come back to the post and start over—is there any­thing more?

If you dis­cover there are un­cer­tain­ties you’d like to un­der­stand, but it’s un­clear what you need to know, where to find out more, or how hard it will be to get the in­for­ma­tion, you should now spend an­other five min­utes, by the clock, re­ally think­ing about what you might want to know. But be­fore you start, there are a few ideas to con­sider.

For that five min­utes of think­ing, you need to think about what you are try­ing to de­cide and how in­for­ma­tion could help. Here’s a set of ques­tions to ask your­self dur­ing those five min­utes:

1) What de­ci­sion(s) are you mak­ing?

2) What do you not know, speci­fi­cally, which is im­por­tant for the de­ci­sion?

3) If you knew more, would you change your de­ci­sion? (If not, it’s not use­ful for the de­ci­sion.)

4) Is there a way to be­come less un­cer­tain? What could you find out that might help?

5) Is the in­for­ma­tion that would change your mind worth the cost of gath­er­ing it? (This might be tricky, but see be­low.)

For the last ques­tion, usu­ally the an­swer is ob­vi­ously yes or no. Some­times, how­ever, it’s un­clear, and you need to think a bit more quan­ti­ta­tively about the value of the in­for­ma­tion. If you want to see the math for how VoI is used in prac­tice, here are some ex­am­ples, and some more, of how to do the ba­sic quan­ti­ta­tive work.

Some­times, how­ever, you will re­al­ize that you need to spend time think­ing about the ques­tions and con­sult­ing with oth­ers be­fore you can start putting num­bers to­gether. That’s a bit more com­plex, and if you want to read about how I recom­mend do­ing this in great depth, feel free to read the last chap­ter of the dis­ser­ta­tion.

(1) Jus­tify­ing this would re­quire a much more in-depth dis­cus­sion than this post war­rants, so feel free to ig­nore the claim.

(2) Speci­fi­cally Bayesian Markov Chain Monte Carlo simu­la­tion, and Discrete Bayesian Net­works. It’s all in the dis­ser­ta­tion—which you shouldn’t read (see be­low.)

(3) If you want to put your­self to sleep, feel free to read any­thing other than the first 5 pages of the fifth chap­ter of the dis­ser­ta­tion. (4)

(4) Really, only pages 159-164, here. That lit­tle bit will provide 90% of the value. That’s the part where I point out that do­ing the ba­sics is a bet­ter use of your time than quan­ti­ta­tive mod­el­ing. Trust me - the rest of the in­for­ma­tion in the dis­ser­ta­tion is low-value, and for al­most ev­ery­one, it’s not worth the cost of read­ing.