Thanks everyone, this is very interesting and well worth having a look through the attached Gitbook.
Around the intuitive interpretation:
Perhaps giving people more options makes them indecisive. They may be particularly reluctant to choose a “relatively ambitious giving pledge” if a less ambitious option is highlighted.
It’s possible that this is the reason, but there’s an alternative interpretation based around the fact that GWWC is already quite well-known and referenced as ‘the place you go to donate 10% of your income’. So if a lot of people are coming onto your page with that goal in mind, then it would make sense that the layouts that centre that option and make it as frictionless as possible will do better. Which is what we see here—the option centring a different option does much worse, but the one that does best is the one that most highlights the 10% pledge, not the one that contextualises it next to an even higher level pledge given equal space.
My own experience of using the site was very similar—I came on, looked around a bit for the 10% option I’d already decided on (in the original setup), then signed up. Things like favouring the middle option and the effects of anchoring are more relevant in a situation where someone has decided to buy, say, a broadband package but hasn’t chosen which one; the lack of effect from them here might indicate that relatively fewer people are coming onto the page unsure how much to give.
You could try testing the 10% pledge next to the further pledge without the 1% pledge, but the really key thing feels like a post-pledge survey. ‘Did you already know what you would pledge when you went on our website?’ ‘If so, did you consider giving at a different level when you saw the options?’ etc. I’m sure you’d get a good response rate as people would be motivated to ensure others completed the pledge. Or if you already have this information, it would be really useful to see it!
GWWC is already quite well-known and referenced as ‘the place you go to donate 10% of your income’. So if a lot of people are coming onto your page with that goal in mind, then it would make sense that the layouts that centre that option and make it as frictionless as possible will do better
Thanks, that makes sense to me in a general senses.
I was thinking in this direction too but having a hard time putting in words ‘why seeing options other than the expected one would make me less likely to follow through’.
Can you dive a little deeper into what the actual ‘friction is’ or ‘what about seeing pledges other than the one I was planning to do would make me less likely to continue?’ I guess my thought was that the mechanism would be indecision, need to take more time to think about this, or maybe a sort of ‘hey, I am over-achieving here, do I really need to signal that I’m a 10-percenter when I could much more easily be a 1-percenter’ … but then I need to think about it more so I don’t decide in the moment.
You could try testing the 10% pledge next to the further pledge without the 1% pledge,
This would be interesting, I agree.
Really key thing feels like a post-pledge survey. ‘Did you already know what you would pledge when you went on our website?’ ‘If so, did you consider giving at a different level when you saw the options?’ etc. I’m sure you’d get a good response rate as people would be motivated to ensure others completed the pledge. Or if you already have this information, it would be really useful to see it!
I think we would get some information from this. (AFAIK we don’t have it but I could ask.) I’m not convinced it would be ‘fully informative’, because of the usual caveats about selection bias and people not always knowing/remembering what was in their minds. But still, it seems worth doing!
‘why seeing options other than the expected one would make me less likely to follow through’
I think the key is that ‘following through’ can mean several things that are similar from the perspective of GWWC but quite different from the perspective of the person pledging.
In my case I’d already been giving >10% for quite a while but thought it might be nice to formalise it. If I hadn’t filled in the pledge it wouldn’t have made any difference to my giving. So the value of the pledge to me was relatively low. If the website had been confusing or offputting I might have given up.
There are others who will already have decided to give 10% but haven’t yet started. The pledge then would have a bit more value if there’s a chance it could prevent backsliding but assuming the person had fully committed to giving at this level already, the GWWC pledge wouldn’t be crucial to the potential pledger.
Finally, there are people who for whatever reason come across the website without yet having decided to give 10% (or even 1%) and make a decision to sign up when they’re there. This is where the more standard marketing theory comes into play.
For the first two groups, the non-conversion is something like ‘I can’t even see what I’m meant to be signing up for. Never mind, it’s not going to affect how I’ll actually give anyway.’ Friction in this case is anything that makes it harder to identify what the 10% pledge is and how to sign up to it. I spent a couple of seconds looking between the three options but it was ultimately pretty easy to work out which one was the one I wanted. This would be even easier if it was the one main option.
For the third, it could well be ‘There’s too much choice, maybe I don’t want to do it.’ At any rate, it will be much different from people who had already committed to giving 10%.
The ‘loss’ to GWWC for all three looks the same but there’s only a substantial loss to the wider world with the third group.
I know people not always remembering what’s in their minds can be an issue but I doubt it would be a problem on something like ‘did you intend to give 10% when you arrived on the GWWC website?’ and certainly not on ‘have you already been giving 10%?’ There’s such a difference between the groups it would be really helpful to at least get an indication how they split out.
Thanks everyone, this is very interesting and well worth having a look through the attached Gitbook.
Around the intuitive interpretation:
It’s possible that this is the reason, but there’s an alternative interpretation based around the fact that GWWC is already quite well-known and referenced as ‘the place you go to donate 10% of your income’. So if a lot of people are coming onto your page with that goal in mind, then it would make sense that the layouts that centre that option and make it as frictionless as possible will do better. Which is what we see here—the option centring a different option does much worse, but the one that does best is the one that most highlights the 10% pledge, not the one that contextualises it next to an even higher level pledge given equal space.
My own experience of using the site was very similar—I came on, looked around a bit for the 10% option I’d already decided on (in the original setup), then signed up. Things like favouring the middle option and the effects of anchoring are more relevant in a situation where someone has decided to buy, say, a broadband package but hasn’t chosen which one; the lack of effect from them here might indicate that relatively fewer people are coming onto the page unsure how much to give.
You could try testing the 10% pledge next to the further pledge without the 1% pledge, but the really key thing feels like a post-pledge survey. ‘Did you already know what you would pledge when you went on our website?’ ‘If so, did you consider giving at a different level when you saw the options?’ etc. I’m sure you’d get a good response rate as people would be motivated to ensure others completed the pledge. Or if you already have this information, it would be really useful to see it!
Thanks, that makes sense to me in a general senses.
I was thinking in this direction too but having a hard time putting in words ‘why seeing options other than the expected one would make me less likely to follow through’.
Can you dive a little deeper into what the actual ‘friction is’ or ‘what about seeing pledges other than the one I was planning to do would make me less likely to continue?’ I guess my thought was that the mechanism would be indecision, need to take more time to think about this, or maybe a sort of ‘hey, I am over-achieving here, do I really need to signal that I’m a 10-percenter when I could much more easily be a 1-percenter’ … but then I need to think about it more so I don’t decide in the moment.
This would be interesting, I agree.
I think we would get some information from this. (AFAIK we don’t have it but I could ask.) I’m not convinced it would be ‘fully informative’, because of the usual caveats about selection bias and people not always knowing/remembering what was in their minds. But still, it seems worth doing!
I think the key is that ‘following through’ can mean several things that are similar from the perspective of GWWC but quite different from the perspective of the person pledging.
In my case I’d already been giving >10% for quite a while but thought it might be nice to formalise it. If I hadn’t filled in the pledge it wouldn’t have made any difference to my giving. So the value of the pledge to me was relatively low. If the website had been confusing or offputting I might have given up.
There are others who will already have decided to give 10% but haven’t yet started. The pledge then would have a bit more value if there’s a chance it could prevent backsliding but assuming the person had fully committed to giving at this level already, the GWWC pledge wouldn’t be crucial to the potential pledger.
Finally, there are people who for whatever reason come across the website without yet having decided to give 10% (or even 1%) and make a decision to sign up when they’re there. This is where the more standard marketing theory comes into play.
For the first two groups, the non-conversion is something like ‘I can’t even see what I’m meant to be signing up for. Never mind, it’s not going to affect how I’ll actually give anyway.’ Friction in this case is anything that makes it harder to identify what the 10% pledge is and how to sign up to it. I spent a couple of seconds looking between the three options but it was ultimately pretty easy to work out which one was the one I wanted. This would be even easier if it was the one main option.
For the third, it could well be ‘There’s too much choice, maybe I don’t want to do it.’ At any rate, it will be much different from people who had already committed to giving 10%.
The ‘loss’ to GWWC for all three looks the same but there’s only a substantial loss to the wider world with the third group.
I know people not always remembering what’s in their minds can be an issue but I doubt it would be a problem on something like ‘did you intend to give 10% when you arrived on the GWWC website?’ and certainly not on ‘have you already been giving 10%?’ There’s such a difference between the groups it would be really helpful to at least get an indication how they split out.