I’m not sure if it’d be remotely compelling to people who have a very different perspective overall (I’m already fairly sympathetic; see e.g. my “y-axis” post).
Still, pulling out a few parts I appreciated:
[…] For Want of Constrained Hypothesis Space
One of the reasons mature fields can rely so much on summary statistics is that they have a constrained hypothesis space. […] in pre-paradigmatic fields we don’t know what hypotheses to consider! A physicist in the 1800s coming up with explanations for energy production in the Sun would have entirely missed nuclear physics.
So the goal of science in pre-paradigmatic fields is to first figure out what hypotheses we should be considering! And this means working in a rather different way…
[…] Why are summary statistics so popular? One reason involves the ecosystem of interfaces for scientific research. We don’t often think of things this way, but scientific research implicitly involves interfaces for thinking about data. For example equations, line plots, and terminology are all interfaces.
[…]
Rigor and The Signal of Structure
How do we know if qualitative results are “real”? What does rigorous qualitative research look like?
We don’t have tools like “statistical significance” to fall back on, and so it’s easy for this research to seem non-rigorous. […] We suspect that one of the most reliable ways to know that a qualitative result is trustworthy is what we’ll call the signal of structure:
The signal of structure is any structure in one’s qualitative observations which cannot be an artifact of measurement or have come from another source, but instead must reflect some kind of structure in the object of inquiry, even if we don’t understand it.
You might think of this as the informal, “unsupervised” version of statistical significance. Whereas statistical significance tests a particular (hopefully pre-registered) hypothesis against a null hypothesis, the signal of structure observes an unpredicted high-dimensional pattern and rejects the hypothesis it was noise or an artifact, typically because the structure is so compelling and complex that it’s clearly orders of magnitude past the bar.
[…] It’s worth noting that it’s also exceedingly easy to fool oneself with qualitative research. (This is likely another reason why scientists are often skeptical of it!) …
Some signs of good qualitative work include:
The Signal of Structure.
A principled understanding of what you’re seeing and why it makes sense to look at. A magnifying glass makes things bigger; the weights of a neural network are the fundamental computational substrate of them.
Sort of related: “Research as a Stochastic Decision Process” — my mental motto version of the post is something like “try to be greedy about the rate at which you’re gaining/producing information/clarity”
(TBH part of the reason I’m posting this here and right now is that I made a hacky commitment to post a few things tonight and this is the lowest-friction option; I’d already posted a version of this in Slack and this was vaguely relevant...)
When I’m writing I sometimes get stuck trying to read what’s been written on the topic before.
I think often that’s driven by an earnest interest/hope/curiosity (looking to learn or read something exciting), or wanting to scope down the thing I’m writing by linking to past stuff (or just driven by remembering a piece that I really appreciated and then getting nerd-sniped along the way). Sometimes it’s more directed (noticing a crux, trying to see if I can look that up) (in general I want to do more of this one, relative to the rest). I still think e.g. that linking is often useful, but overall IMO I do the above too much. (I think actually maybe there’s a form of planning fallacy at play, where I often feel like this will be a 5-minute task but actually it too frequently turns into a 2-hour detour...[1])
There are two patterns that seem more harmful, at least in my case:
(a) a defensive move; wanting to check if what is actuallynew or completely accurate or somehow silly.… , or (b) just following some sort of “path of least resistance” gradient when I”m trying to think about something complicated and it’s easier to go and read about something vaguely related to the topic (or about what the hell is going on with diffusion models or whatever’s on my mind).
I want to think more about (b) later. For now I just noticed myself wanting to post a quick note re the defensive stuff in (a) — the “is this really contributing something new?” point in particular. IMO:
Very little is “actually new” the way people look for
Relatedly IMO we have a pretty bad sense for what is and isn’t “the frontier”. For one thing, some of the most useful pieces people can write are about ideas that feelobvious to them (because they’ve figured them out, it’s clicked, etc.), but which the people around them totally don’t get (i.e. which are on some relevant frontier). (Semi-related.)
It’s often really useful to say things that are really not new, or say things more than once (related)
(especially if you are in conversation with a group of people who share some interests/… with you, and the reason you’re bringing up something “not actually new” is the fact that you usefully noticed that this is relevant here[2])
I also think orienting towards “help people around me (and myself) understand this thing they’re missing (even if that process doesn’t look like ‘push the frontier’!)” is often just a really useful way to prompt oneself when doing research
More generally, IMO orienting (especially right away!) towards outputs that are higher status in various ways seems pretty unhelpful. At least personally, this moves me out of a natural/free/“real” mode of trying to deeply understand and explain — imagining talking to someone I can imagine — and towards a abstract/ far- /fake? mode in which I tend to retreat more to more immediately defensible claims, start writing with “academic distance”, etc.
(This view can be taken too far—“reverse all advice you hear” etc. But still, I think “novelty is often overrated, and maybe that overrating often happens in large part inside people’s heads—unsure.)
Anyway, it’s funny- I’ve personally told people that they shouldn’t worry too much about novelty before, and I endorse what I’ve said there. I think people overrate it, I think writing the same thing multiple times is often useful, etc… But I still notice myself thinking “eh but surely someone has said this, IDK if this would be that useful” (and then spending more time trying to check a minor thing than I would have just writing it if I went at it with a no-apologies attitude)[3]
What gives? Imposter syndrome? (Or thinking too much about how things make me look instead of about what people will read, maybe? Something like low-status-behavior intuitions for women?) I don’t know. I suppose I’ll leave it at that for now, in part as a public reminder or commitment that I don’t have to do this. And I’ll try to resist linking more than I already have.[4] 🙃
(
A flip side of / complementary pattern to the above is making sure we sometimes get into a mindset (state?) where we can:
Notice confusion / disagreement / etc. → find a question
Make deliberate progress on this
Build deep understanding of some area to get a bunch of surface area / intuitions
This makes me remember a recent thread comparing different AI systems’ cyber capabilities (finding vulnerabilities in a codebase, IIRC), where the open-source models had (unlike the others) been given the places with vulnerabilities and asked to figure out how to exploit them instead of having to locate things themselves. Someone compared this to something like giving someone a needle instead of a haystack-with-a-needle
ANyway, IMO noticing that X is relevant for your life / context is really useful, and underrated (vs “coming up with” X)
(and I’m pretty sure I apologize for lack of novelty when I post stuff , which probably worsens the norms here—in fact I might have done that in the previous comment I posted—I should do less of that!)
Althuogh I’m sort of using this as a low-setup-cost / MVP version of a personal wiki, and I had collected some links in an old slack message. So, in the interest of not actually following the links right now...
[I don’t want to spam the Quick Takes section, so I’ll use this thread as a place to jot down some meta-ish notes on research and writing. (I might find a better place for this stuff later.)]
...
Some reasons I poke at words (i.e. when I ask “what do you mean by...?”[1])
I’m just caught up in pedantry / unnecessary nuance-finding
(I’m noticing a distinction that could be made, or some vagueness at the corners of this term, or a pathological example that doesn’t fit neatly into the category they’re trying to use ---- but it’s not actually that relevant or important)
I would personally value clarification / feel blocked by my uncertainty
I am just not following things / can’t picture what you mean / don’t know if you’re using this as some kind of jargon; I’m having trouble engaging with the rest without clarification here.
-> A variant of this makes it hard for me to engage with IMO-half-operationalized exercises or questions
like “what are your timelines” (to what?) or “rate this 1-10 on impact” (what do you mean by impact exactly?) or “how much influence do you think altruists/idealists have?” (...)
I’m trying to flag that a particular group of people will probably interpret the term in problematic ways
(i.e. not how you’re trying to use it)
I’m quickly reporting a bit of unprocessed “user feedback” about a reaction I notice in myself
Even if I can work through this myself, the doc can probably be improved here
I’ve noticed I feel confused and suspect that might be tracking a real issue with the terminology or with your argument
(or maybe I’m misinterpreting)
I can’t picture what this is meant to point to and don’t know if the (provisional?) use of the term is fake or not;
you may want to consider tabooing it, and I would probably personally appreciate it
[Probably others, may add later]
Tbc often the background generator of a comment like this (especially before I’ve written it out-- when I’m at the point of considering saying or writing something) is not an explicit thought, and certainly not something that cleanly maps into one of the above. Often it’s just a fuzzy intuition that something is off; things aren’t matching up/ they’re of the wrong type, or I’m struggling to keep a picture in mind (I’m just lost and can’t even start putting it together or I’m failing to make one that doesn’t look absurd, …), or I’m feeling something like triggered by the word (maybe because I’ve seen people talk past each other with it in frustrating ways, or because I have sometimes felt like it’s circular in the bad way), … This list was a quick attempt at noticing the differences later; I think I should train myself to notice more easily what’s going on, and in some cases I should process things more before posting or saying anything (and what to do will depend on the case I’m in).
==
(For “practical advice on threading the needle between pedantry and just vibing”, I remember liking this post btw, but I don’t remember exactly how relevant it was. Also stuff on inferential distances, maybe)
Also relevant (sort of[1]) — this cool post from Chris Olah & Adam Jermyn (from a while back): Reflections on Qualitative Research
I’m not sure if it’d be remotely compelling to people who have a very different perspective overall (I’m already fairly sympathetic; see e.g. my “y-axis” post).
Still, pulling out a few parts I appreciated:
Related: What’s So Bad About Ad-Hoc Mathematical Definitions? (discussion here IIRC)
Sort of related: “Research as a Stochastic Decision Process” — my mental motto version of the post is something like “try to be greedy about the rate at which you’re gaining/producing information/clarity”
Also from Chris Olah (and Shan Carter), and IMO great: Distillation and research debt
(TBH part of the reason I’m posting this here and right now is that I made a hacky commitment to post a few things tonight and this is the lowest-friction option; I’d already posted a version of this in Slack and this was vaguely relevant...)
[This is an unedited babble]
When I’m writing I sometimes get stuck trying to read what’s been written on the topic before.
I think often that’s driven by an earnest interest/hope/curiosity (looking to learn or read something exciting), or wanting to scope down the thing I’m writing by linking to past stuff (or just driven by remembering a piece that I really appreciated and then getting nerd-sniped along the way). Sometimes it’s more directed (noticing a crux, trying to see if I can look that up) (in general I want to do more of this one, relative to the rest). I still think e.g. that linking is often useful, but overall IMO I do the above too much. (I think actually maybe there’s a form of planning fallacy at play, where I often feel like this will be a 5-minute task but actually it too frequently turns into a 2-hour detour...[1])
There are two patterns that seem more harmful, at least in my case:
(a) a defensive move; wanting to check if what is actually new or completely accurate or somehow silly.… , or (b) just following some sort of “path of least resistance” gradient when I”m trying to think about something complicated and it’s easier to go and read about something vaguely related to the topic (or about what the hell is going on with diffusion models or whatever’s on my mind).
I want to think more about (b) later. For now I just noticed myself wanting to post a quick note re the defensive stuff in (a) — the “is this really contributing something new?” point in particular. IMO:
Very little is “actually new” the way people look for
Relatedly IMO we have a pretty bad sense for what is and isn’t “the frontier”. For one thing, some of the most useful pieces people can write are about ideas that feel obvious to them (because they’ve figured them out, it’s clicked, etc.), but which the people around them totally don’t get (i.e. which are on some relevant frontier). (Semi-related.)
It’s often really useful to say things that are really not new, or say things more than once (related)
(especially if you are in conversation with a group of people who share some interests/… with you, and the reason you’re bringing up something “not actually new” is the fact that you usefully noticed that this is relevant here[2])
I also think orienting towards “help people around me (and myself) understand this thing they’re missing (even if that process doesn’t look like ‘push the frontier’!)” is often just a really useful way to prompt oneself when doing research
More generally, IMO orienting (especially right away!) towards outputs that are higher status in various ways seems pretty unhelpful. At least personally, this moves me out of a natural/free/“real” mode of trying to deeply understand and explain — imagining talking to someone I can imagine — and towards a abstract/ far- /fake? mode in which I tend to retreat more to more immediately defensible claims, start writing with “academic distance”, etc.
(This view can be taken too far—“reverse all advice you hear” etc. But still, I think “novelty is often overrated, and maybe that overrating often happens in large part inside people’s heads—unsure.)
Anyway, it’s funny- I’ve personally told people that they shouldn’t worry too much about novelty before, and I endorse what I’ve said there. I think people overrate it, I think writing the same thing multiple times is often useful, etc… But I still notice myself thinking “eh but surely someone has said this, IDK if this would be that useful” (and then spending more time trying to check a minor thing than I would have just writing it if I went at it with a no-apologies attitude)[3]
What gives? Imposter syndrome? (Or thinking too much about how things make me look instead of about what people will read, maybe? Something like low-status-behavior intuitions for women?) I don’t know. I suppose I’ll leave it at that for now, in part as a public reminder or commitment that I don’t have to do this. And I’ll try to resist linking more than I already have.[4] 🙃
(
A flip side of / complementary pattern to the above is making sure we sometimes get into a mindset (state?) where we can:
Notice confusion / disagreement / etc. → find a question
Make deliberate progress on this
Build deep understanding of some area to get a bunch of surface area / intuitions
...
)
Plus just having a somewhat poor memory, and being distractible. And not being better at “form factors” / scoping.
This makes me remember a recent thread comparing different AI systems’ cyber capabilities (finding vulnerabilities in a codebase, IIRC), where the open-source models had (unlike the others) been given the places with vulnerabilities and asked to figure out how to exploit them instead of having to locate things themselves. Someone compared this to something like giving someone a needle instead of a haystack-with-a-needle
ANyway, IMO noticing that X is relevant for your life / context is really useful, and underrated (vs “coming up with” X)
(and I’m pretty sure I apologize for lack of novelty when I post stuff , which probably worsens the norms here—in fact I might have done that in the previous comment I posted—I should do less of that!)
Althuogh I’m sort of using this as a low-setup-cost / MVP version of a personal wiki, and I had collected some links in an old slack message. So, in the interest of not actually following the links right now...
https://www.henrikkarlsson.xyz/archive?sort=new
https://maggieappleton.com/garden-history
https://michaelnotebook.com/qtr/index.html
https://scienceplusplus.org/cusc/index.html
https://www.lesswrong.com/posts/hjMy4ZxS5ogA9cTYK/how-i-think-about-my-research-process-explore-understand
[I don’t want to spam the Quick Takes section, so I’ll use this thread as a place to jot down some meta-ish notes on research and writing. (I might find a better place for this stuff later.)]
...
Some reasons I poke at words (i.e. when I ask “what do you mean by...?”[1])
I’m just caught up in pedantry / unnecessary nuance-finding
(I’m noticing a distinction that could be made, or some vagueness at the corners of this term, or a pathological example that doesn’t fit neatly into the category they’re trying to use ---- but it’s not actually that relevant or important)
I would personally value clarification / feel blocked by my uncertainty
I am just not following things / can’t picture what you mean / don’t know if you’re using this as some kind of jargon; I’m having trouble engaging with the rest without clarification here.
-> A variant of this makes it hard for me to engage with IMO-half-operationalized exercises or questions
like “what are your timelines” (to what?) or “rate this 1-10 on impact” (what do you mean by impact exactly?) or “how much influence do you think altruists/idealists have?” (...)
I’m trying to flag that a particular group of people will probably interpret the term in problematic ways
(i.e. not how you’re trying to use it)
I’m quickly reporting a bit of unprocessed “user feedback” about a reaction I notice in myself
Even if I can work through this myself, the doc can probably be improved here
I’ve noticed I feel confused and suspect that might be tracking a real issue with the terminology or with your argument
(or maybe I’m misinterpreting)
I can’t picture what this is meant to point to and don’t know if the (provisional?) use of the term is fake or not;
you may want to consider tabooing it, and I would probably personally appreciate it
[Probably others, may add later]
Tbc often the background generator of a comment like this (especially before I’ve written it out-- when I’m at the point of considering saying or writing something) is not an explicit thought, and certainly not something that cleanly maps into one of the above. Often it’s just a fuzzy intuition that something is off; things aren’t matching up/ they’re of the wrong type, or I’m struggling to keep a picture in mind (I’m just lost and can’t even start putting it together or I’m failing to make one that doesn’t look absurd, …), or I’m feeling something like triggered by the word (maybe because I’ve seen people talk past each other with it in frustrating ways, or because I have sometimes felt like it’s circular in the bad way), … This list was a quick attempt at noticing the differences later; I think I should train myself to notice more easily what’s going on, and in some cases I should process things more before posting or saying anything (and what to do will depend on the case I’m in).
==
(For “practical advice on threading the needle between pedantry and just vibing”, I remember liking this post btw, but I don’t remember exactly how relevant it was. Also stuff on inferential distances, maybe)
e.g.
AGI/ …
“[the US] would want to … ; who are you thinking of here?”
[various bits of jargon]