hrosspet

Karma: 61

Ex-Apple ML engineer with some research and entrepreneurial background. Technical AI alignment research, but am also interested in the bigger problem which I call Human alignment. I’m starting a new project, which aims at tackling one aspect of this bigger problem. I’m a long term member of the Czech EA and LW community, attended CFAR workshop.

hrosspet 9 Apr 2024 9:19 UTC
2 points
1 ∶ 0
on: Choose your own odyssey: results & insights from Tee Barnett Coaching Training (TBCT)
disclaimer: I’ve read in full only “Takes for Self-improvers, clients, people ‘bought into’ self-development” which I’m mostly interested in, skimmed the rest
Thanks for the writeup! I’d be interested in hearing your thoughts on how I should figure out the value of getting coaching.
My current approach is to do a lot of self-coaching myself and only when I feel like I’m stuck for a longer period or feel overwhelmed I reach out to a coach/therapist. Then, I use the sessions not only to figure out the object-level problem, but also I try to learn how to become a better self-coach by reflecting on the sessions on a meta-level (so that I don’t need them anymore).
There is of course an opportunity cost—I could just get coaching sessions regularly, regardless of whether I’m stuck, or not, and focus on my thing—engineering/parenting/founding/… But if I’m gonna save the time/effort by not learning to coach myself and instead outsource the coaching skills to others, am I not gonna need them in the future?
There is of course always the benefit of having another person check on my thinking and hear their perspective, but that doesn’t need to be a coach, it can be a domain expert, if my self-coaching skills are good enough.
To sum this up, what am I likely missing with this approach?

hrosspet 17 May 2023 9:20 UTC
4 points
3 ∶ 0
on: Announcing the Confido app: bringing forecasting to everyone
I believe improving (group) epistemics outside of our bubble is an important mission. So great you are working with policy makers!

hrosspet 21 Apr 2023 10:41 UTC
2 points
0 ∶ 0
on: Iqisa: A Library For Handling Forecasting Datasets
Hi Niplav, thanks for your work! I’ve been thinking about doing the same, so you saved me quite some time :)
I made a pull request where I’m suggesting a couple small changes and bug fixes to make it more portable and usable in other projects.
For other readers this might be the most interesting part: I created a jupyter notebook loading all datasets and showing their preview. So now it should be really simple to start working with the data or just see if it’s relevant for you at all.
If you’d like to collaborate on this further I might add support for Manifold Markets data and Autocast dataset, as that’s what I’ve been working with up till now.

hrosspet 21 Nov 2022 14:25 UTC
2 points
1 ∶ 0
on: Deontology and virtue ethics as “effective theories” of consequentialist ethics
I’d also add that virtues and deontologically right actions are results of a memetic evolution and as such can be thought of as precomputed actions or habits that have proven to be beneficial over time and have thus high expected value.

hrosspet 22 Sep 2022 19:53 UTC
0 points
0 ∶ 0
in reply to: Brad West’s comment on: Effective altruism in the garden of ends
Not all conscious experiences are created equal.

Pursuing those ends Tyler talks about helps cultivate higher quality conscious experiences.

hrosspet 24 Aug 2022 17:11 UTC
4 points
0 ∶ 0
on: Pressing News Front Page
Not sure how seriously you mean this, but news should be both important and surprising (=have new information content). I mean, you could post this a couple times, as for many non-EA people these news might be surprising, but you shouldn’t keep posting them indefinitely, even though they remain true.

hrosspet 12 Jul 2022 13:51 UTC
2 points
0 ∶ 0
on: Flagging up a ‘prediction market’
Thanks for sharing, will take a look!

This is my list of existing prediction markets (and related things like forecasting platforms) in case anyone wants to add what’s missing..

https://www.metaculus.com/ https://polymarket.com/ https://insightprediction.com/ https://kalshi.com/ https://manifold.markets/ https://augur.net/ https://smarkets.com/

hrosspet 6 Jul 2022 18:03 UTC
2 points
0 ∶ 0
on: Forecasting Through Fiction
Interesting experiment!

One argument against the predictive power of stories is that many stories evolved as cautionary tales. Which means that if they work, they will have zero predictive accuracy. Which would also possibly fit this particular scenario

hrosspet 1 Jul 2022 8:04 UTC

−2 points

0 ∶ 0

Error

The value
  NIL
is not of type
  SIMPLE-STRING
when binding #:USER-ID163

hrosspet 29 Jun 2022 6:06 UTC
17 points
0 ∶ 0
on: The inordinately slow spread of good AGI conversations in ML
My experience from a big tech company: ML people are too deep in the technical and practical everyday issues that they don’t have the capacity (nor incentive) to form their own ideas about the further future.

I’ve heard people say, that it’s so hard to make ML do something meaningful that they just can’t imagine it would do something like recursive self-improvement. AI safety in these terms means making sure the ML model performs as well in deployment as in the development.

Another trend I noticed, but I don’t have much data for it, is that the somewhat older generation (35+) is mostly interested in the technical problems and don’t feel that much responsibility for how the results are used. Vs. the generation of 25 − 35 care much more about the future. I’m noticing similarities with climate change awareness, although the generation delimitations might differ.

hrosspet 28 Jun 2022 9:58 UTC
3 points
0 ∶ 0
on: Consequences Not Outcomes
Not sure if I understand the text correctly, but the reasoning seams off to me. Eg.

Expected value calculations don’t seem to faithfully represent a person’s internal sense of conviction that an outcome occurs. Or else opportunities with small chances of success would not attract people.

Isn’t the exact opposite true? Don’t opportunities with small chances of success still attract people exactly because of (subconscious) expected value value calculations?

hrosspet 16 Jun 2022 9:45 UTC
2 points
0 ∶ 0
on: Continuity Assumptions
The problem is that sometimes you can see a process is actually continuous only ex post. I think I saw this argument in Yudkowski’s writing that sometimes you just don’t know what variable to observe, so then a discontinuous event surprises you and only after that you realize you should have been observing X, which would make it seem continuous.

hrosspet 3 Jun 2022 11:22 UTC
23 points
0 ∶ 0
on: Who’s hiring? (May-September 2022)
I’m looking for a cofounder / ML researcher / ML engineer for a new FTX-funded project related to prediction markets and large language models!
The long term vision is to improve our decision making as a humanity. We aim to do that by improving how prediction markets work by employing AI. See the full role description: https://bit.ly/3zg5UFm
Little bit about me.

hrosspet 21 Jan 2022 11:04 UTC
2 points
0 ∶ 0
in reply to: Czynski’s comment on: FLI launches Worldbuilding Contest with $100,000 in prizes
That something is very unlikely doesn’t mean it’s unimaginable. The goal of imagining and exploring such unlikely scenarios is that with a positive vision we can at least attempt to make it more likely. Without a positive vision there are only catastrophic scenarios left. That’s I think the main motivation for FLI to organize this contest.
I agree, though, that the base assumptions stated in the contest make it hard to come up with a realistic image.

hrosspet 6 Jan 2022 17:14 UTC
3 points
0 ∶ 0
in reply to: Josh Jacobson’s comment on: Consider trying the ELK contest (I am)
During the 2 hours of reading and skimming through the relevant blog posts I was able to come up with 2 strategies (and no counter-examples so far). They seem to me as quite intuitive and easy to come up with, so I’m wondering what I got wrong about the ELK problem or the contest...
Due to the low confidence in my understanding I don’t feel comfortable submitting these strategies, as I don’t want to waste the ARC’s team time.
My background: ML engineer (~7 years of experience), some previous exposure to AGI and AIS research and computer security.

hrosspet 4 Jan 2022 22:05 UTC
1 point
0 ∶ 0
in reply to: Holden Karnofsky’s comment on: Comments for shorter Cold Takes pieces
Thank you for a thought provoking post! I enjoyed it a lot.
I also find the “innovation as mining” hypothesis intuitive. I’d just add that innovation gets harder for humans, but we don’t know whether it holds in general (think AI). Our mental capacity has been roughly constant since ancient Greece, but there is more and more previous work to understand before one can come up with something new. This might not be true for AI, if their capacity scales.
On the other hand there is a combinatorial explosion of facts that you can combine to come up with an innovation and I don’t know what fraction of the combinations will actually be useful and judged as innovation. So overall, the difficulty might increase, stay roughly the same, or decrease, depending on how the number of useful combination scales with the number of all combinations.
I also suspect that subjective rankings of past accomplishments just tend, for whatever reason, to look overly favorably on the past.
One explanation of this would be that innovation needs time to collect its impact. Old innovations are well integrated into the society, so they have already collected most of its impact, while new innovations have most of their impact still in the future, so we don’t perceive them as transformative yet.

hrosspet 30 Dec 2021 18:03 UTC
2 points
0 ∶ 0
on: Why don’t governments seem to mind that companies are explicitly trying to make AGIs?
I think governments are not aware of the stop button problem and they think in case of emergency they can just shut down the company / servers running the AGI using force. That’s what happened in the past with digital currencies (which Jackson Wagner mentions here as a plausible member of the same reference class as AGI for governments) before bitcoin—they either failed on their own, or if successful, were shut down by government (https://en.wikipedia.org/wiki/Digital_currency#History).

hrosspet 11 Nov 2021 16:42 UTC
3 points
0 ∶ 0
on: Who are your role models?
Daniel Schmachtenberger. Look up some of his youtube interviews. I like especially the one with Lex Fridman (https://youtu.be/hGRNUw559SE). He’s a very thoughtful, yet humble person. His approach is very multi-disciplinary, systems-level, holistic. For me he is a role model for how he combines the world-knowledge and self-knowledge, and how clearly he is able to articulate his ideas, which I think are very EA-compatible (he mentions EA from time to time, but I haven’t heard any endorsement from him). Yet he goes further than what is discussed within EA eg. on the topics of personal development and meaning making.
Daniel’s website: https://civilizationemerging.com/
A project he is a part of: https://consilienceproject.org/
Also very relevant to EA: Psychological Pitfalls of Engaging With X-Risks & Civilization Redesign w/ Daniel Schmachtenberger: https://youtu.be/SkItTnRJ_1M

hrosspet 2 Sep 2021 17:07 UTC
7 points
0 ∶ 0
on: What we learned from a year incubating longtermist entrepreneurship
Very interesting read, thanks for publishing this!
I am curious what qualified as “having longtermist experience” for you?

hrosspet 5 Aug 2021 8:46 UTC
2 points
0 ∶ 0
on: Matt Levine on the Archegos failure
a meaningful retrospective is much easier to come by than for, say, the Covid pandemic.
Agreed, but we have this rare example of Dominic Cummings, the chief adviser to Boris Johnson during the pandemic, being thoroughly interviewed about the UK’s response to the pandemic. For me it was extremely interesting to peek under the hood of UK government departments and see their failure modes. If you enjoyed the CS report, you might enjoy this one, too.
https://parliamentlive.tv/event/index/d919fbc9-72ca-42de-9b44-c0bf53a7360b