There’s no comparison to prior full-press Diplomacy agents, but if I’m reading the prior-work cites right, this is because basically none of them work—not only do they not beat humans, they apparently don’t even always improve over themselves playing the game as if it was no-press Diplomacy (ie not using dialogue at all). That gives an idea how big a jump this is for full-press Diplomacy.
I watched the commentated video you and Lawrence shared, and it still wasn’t clear to me from seeing the gameplay how much the press-component was actually helping the Diplomacy agents. (e.g. I wasn’t sure if the bots were cooperating/backstabbing or if they were just always set on playing the moves that they did regardless of what was being said in the Press.) In a game with just one human and the rest bots obviously the human wouldn’t have an advantage of the bots all behaved like No Press bots. I think a mixed game with multiple humans and multiple bots would provide more insightful.
gwern on /r/machinelearning:
Helpful, thanks!
I watched the commentated video you and Lawrence shared, and it still wasn’t clear to me from seeing the gameplay how much the press-component was actually helping the Diplomacy agents. (e.g. I wasn’t sure if the bots were cooperating/backstabbing or if they were just always set on playing the moves that they did regardless of what was being said in the Press.) In a game with just one human and the rest bots obviously the human wouldn’t have an advantage of the bots all behaved like No Press bots. I think a mixed game with multiple humans and multiple bots would provide more insightful.