Is it a sticker shock reaction developed from years of reflexively minimizing outflows? Donating more frequently in smaller amounts might help. Feels less irreversible.
KyleM
Karma: 13
Summary: Thinking out loud about the J space paper’s implications on future animal welfare research (if there are any). I don’t know much about LLMs or brains or animals but I’d love to chat about this stuff with anyone at my same level of smartness, or learn from folks who know things.
It would be good to have some people thinking about the J-space paper and what, if anything, it has to do with animal welfare. A popular question about animal brains is “what’s going on in there?”. If we get some vague notions about the conditions and size ranges where neural nets act like global workspaces, it might give us some order of magnitude estimates and fuzzy intuitions about what sizes and types of animal brains exhibit those properties.Some questions that seem interesting:
What model sizes and training regimes produce a recognizable J-space which is necessary for solving some tasks, measured by ablation?
Are there any phase changes in the structure or importance of the workspace as network and training scale increase? How about as the number of input streams (text, vision, sound tokens) increases? Perhaps formation of these mechanisms is visible as a critical point in a double-descent type capability curve.
Maybe effective utilization of J-space requires slack in pretraining in addition to scale. Perhaps you get room to develop this stuff from excess compute when you’ve hit diminishing returns from hardcoding more explicit methods for the tasks you handle.
or (more likely?) the opposite is true—the need to address a broad task range with limited compute forces many workstreams to share computational resources, resulting in abstraction, resulting in segmentation of the abstract stuff from the concrete stuff. Which resource availability helps you grow a good workspace? Lots of free parameters, or not enough? First one then the other?
What model organisms are the most “animal-like” if we want to vary parameters and look at their effect on the usefulness and recognizability of access consciousness? None are great analogues, but what’s the closest we can get?Do the ablation experiments in the J-space paper map onto lesioning experiments in different parts of animal brains?
Broadly, animals have similar tasks and data sources to other animals (processing stimuli into models of the world, making decisions on how to secure resources, reproduce, and avoid threats, etc.). We all live in the same environment, the physical world. LLMs don’t always share this common substrate, and their task space and input data is much weirder. Their food is mostly pre-chewed. My assumption is that a global workspace is more likely in systems which must reason on multiple input streams about the same thing (I.e. animals). Gotta combine that info somehow. So I’d assume animals are more dependent on workspace-like features than LLMs. Are multimodal LLMs more likely to depend on J space than single-input-type LLMs, for that reason? It may be worth checking.
Given how hard it is to tell what’s going on in animal brains, getting some fuzzy analogies between maybe-conscious LLMs and maybe-conscious animals seems high value. Maybe they will get less fuzzy over time.
Can we find an analogue of the J space in the recent fly brain simulation of questionable quality? What would that look like?
Does any of this have anything to do with welfare? How much do we care if something has these features? My gut says that if we’re going to figure out what it’s like to be a bug, this type of AI-to-animal analogy is probably how it will happen.
KyleM’s Quick Takes
I’d like to see someone trying a version of this, and European foundation owned businesses seem like a decent template. Do those businesses actually win due to charity ownership? If I were an investor or funding allocator, I’d like to see the pitch be much more concrete.
Edit: Brad addressed or clarified most of these points. Leaving it up as a reference, but most people can safely skip the below.
How much are customers willing to pay to buy charity-owned, by sector? The “tie-breaker” framing should correspond with some dollar amount.
My expectation is that commodity consumers and B2B customers are not willing to pay much more. Consumer luxury goods seems like a good market, and we see businesses appealing to charitable sensibilities there already.
Can charity acquirers pay market prices without expecting to profit from multiples arbitrage or synergy?
You suggest LBOs. Is the cash flow going to paying back a huge loan or to charity? I don’t see how it can be both.
You would lose the ability to raise capital in markets. Are we sure we’ll get the same lending terms if we can’t backstop the loans with equity? Probably there are some details about the actual ownership structures that matter here.
As a consumer, would I rather pay a “charity tax” or direct my own charity spend intentionally?
Are the employees you’ll be inheriting actually in the group of employees willing to accept 4-7% lower pay for mission aligned work? Will they leave when you shift to paying below market?
You’re considering “the entire economy” as the scope. It’s good to point at a huge TAM but it’s also more than you need to claim. As an aside, was this written with AI?
You might suggest a specific beachhead (establish a philanthropic search fund or vehicle run by me, Brad West, with $XXX,XXX,XXX in market X in geography Y, with expected returns of z% over time frame).
If you have <20 year AI timelines, is it a good time to buy a bunch of legacy businesses or are there better uses for the money?
Mistakes in the other direction are also common! It’s easy for young professionals to use the average value of their time to calculate tradeoffs, rather than the marginal value. When you’re making $80 per hour, doordashing every meal starts to make some sense, as do laundry services, etc. I’m not against these things, but the time savings often go to leisure rather than career reinvestment.
This is totally fine, and sometimes necessary, as long as people are correctly identifying what they’re really buying and what the price is.
On “LLMs as Tools for Alignment”:
Wanted to respond to one specific paragraph from this. Kids famously ask “why?” over and over until their parents go insane.
LLMs tirelessly answer “why?” just for you. Is that curiosity still inside the average adult?
Ways LLMs improve coordination:
Helping people define problems (many of which we all share)
Pointing out stable solutions involving coordination when they exist and are described by literature
Suggesting coordination mechanisms
GPT-5 can do all three of these to a useful degree today, even if no further progress was made. It’s not a PhD level thinker, but it can connect you to PhD level ideas. Sycophancy is a problem, as is distraction. Either could kill the concept. Maybe we get the Wall-E world. But I think people want to know “why?”.
What LLMs don’t do:
Make everyone agree (although disagreements may converge to cruxes faster, enabling better understanding [or more direct conflict?])
Quorum sensing[1] - the ability to detect when enough actors are willing to cooperate to make cooperation effective. People often avoid being the first to move, unless they know they have support.
I have been thinking about this a lot, and would appreciate links to further reading. OP[2], you should look into Pol.is if you haven’t already. It’s on my reading list. Also, see Nepal [3] for some tech-enabled coordination on a large scale.
I’m not sure how things can change, other than economic pressure by consumers or the government on welfare.
1. PE rollups of companion vet clinics are a contributing factor, as with human medical clinics. Consolidation combined with metrics-based optimization leads to harsh local incentives.
2. Vets in poultry and cattle operations don’t necessarily care more about animal welfare than the owners or the consumers. Large animal / poultry vets have been desensitized to harm for many years, understand the economics, and understand their role in that system. I believe all of them care deeply about welfare, but the machine is optimizing for cost. There are selection effects in career choice too—if they share your ideals they probably won’t end up in those roles. Companion vets have more room for empathy, even if they are still constrained by economics.
Vets are the HR of the production animal world—there to help unless your needs conflict with the org’s.
Doesn’t uncertainty about whether one is in deployment or an eval count as eval awareness? I.e. whether you behave differently with a 30% eval credence than an 80% eval credence makes no difference, as long as you behave differently than the 0% credence case.