Regarding the questions about feedforward networks, a really short answer is that regression is a very limited form of inference-time computation that e.g. rules out using memory. (Of course, as you point out, this doesn’t apply to other 2020 algorithms beyond MLPs.) Sorry about the lack of clarity—I didn’t want to take up too much space in this piece going into the details of the linked papers, but hopefully I’ll be able to do a better job explaining it in a review of those papers that I’ll post on LW/AF next week.
(I also want to reply to your top-level comments about the evolutionary anchor, but am a bit short on time to do it right now (since for those questions I don’t have cached technical answers and will have to remind myself about the context). But I’ll definitely get to it next week.)
Thanks for the responses, they give a lot more useful context.
(I also want to reply to your top-level comments about the evolutionary anchor, but am a bit short on time to do it right now (since for those questions I don’t have cached technical answers and will have to remind myself about the context). But I’ll definitely get to it next week.)
If it frees up your time, I don’t think you need to write the above, unless you specifically want to. It seems reasonable to interpret that point on “evolutionary anchors” as a larger difference on the premise, and that is not fully in scope of the post. This difference and its phrasing is more disagreeable/overbearing to answer, so it’s also less worthy of a response.
Regarding the questions about feedforward networks, a really short answer is that regression is a very limited form of inference-time computation that e.g. rules out using memory. (Of course, as you point out, this doesn’t apply to other 2020 algorithms beyond MLPs.) Sorry about the lack of clarity—I didn’t want to take up too much space in this piece going into the details of the linked papers, but hopefully I’ll be able to do a better job explaining it in a review of those papers that I’ll post on LW/AF next week.
(I also want to reply to your top-level comments about the evolutionary anchor, but am a bit short on time to do it right now (since for those questions I don’t have cached technical answers and will have to remind myself about the context). But I’ll definitely get to it next week.)
Thanks for the responses, they give a lot more useful context.
If it frees up your time, I don’t think you need to write the above, unless you specifically want to. It seems reasonable to interpret that point on “evolutionary anchors” as a larger difference on the premise, and that is not fully in scope of the post. This difference and its phrasing is more disagreeable/overbearing to answer, so it’s also less worthy of a response.
Thanks for writing your ideas.