In blackjack, a competent player wins against the house ~49.5% of the time, and the house wins ~50.5% of the time. If I was to record a string of 0s and 1s, where a 1 represents a win by a competent player and a 0 represents a win for the house, my string would look almost exactly like noise. If I sit and record 20 games, that’s 2^20 = 1048576 possible strings I could record. So you might naively think that there’s no opportunity for useful predictions here. But in fact, the house edge means that on expectation, the house is going to win money and a competent player (absent card counting) is going to lose it.
In the same way, it’s not the stochasticity of the system that matters so much as whether we can make forecasts. Blackjack has loads of stochasticity, but the ultimate financial outcome can still be usefully forecasted. Weather is also very stochastic and may exhibit chaotic properties (see butterfly example), but weather forecasts are still pretty useful. Etc. The issue for EA is that we are trying to make forecasts in domains where there isn’t necessarily a history of successful forecasting like there is for the weather. This is a hard problem to deal with, but I don’t think it’s completely intractable. I suspect the set of skills needed is similar to the ones you need to be a successful investor or run a successful hedge fund.
In blackjack, a competent player wins against the house ~49.5% of the time, and the house wins ~50.5% of the time. If I was to record a string of 0s and 1s, where a 1 represents a win by a competent player and a 0 represents a win for the house, my string would look almost exactly like noise. If I sit and record 20 games, that’s 2^20 = 1048576 possible strings I could record. So you might naively think that there’s no opportunity for useful predictions here. But in fact, the house edge means that on expectation, the house is going to win money and a competent player (absent card counting) is going to lose it.
In the same way, it’s not the stochasticity of the system that matters so much as whether we can make forecasts. Blackjack has loads of stochasticity, but the ultimate financial outcome can still be usefully forecasted. Weather is also very stochastic and may exhibit chaotic properties (see butterfly example), but weather forecasts are still pretty useful. Etc. The issue for EA is that we are trying to make forecasts in domains where there isn’t necessarily a history of successful forecasting like there is for the weather. This is a hard problem to deal with, but I don’t think it’s completely intractable. I suspect the set of skills needed is similar to the ones you need to be a successful investor or run a successful hedge fund.