We don’t do significance testing or much other statistical analysis in this analysis… Our sense was that this approach was the best one for our dataset, where the n on any question was at maximum 217 (the number of respondents) and often lower, though we’re open to suggestions about ways to apply statistical analysis that might help us learn more.
Because you have so much within-subjects data (i.e. multiple datapoints from the same respondent), you will actually be much better powered than you might expect with ~200 respondents. For example, if you asked each respondent to rate 10 things (and for some questions you actually asked them to rate many more) you’d have 2000 datapoints and be better able to account for individual differences.
You might, separately, be concerned about the small sample size meaning that your sample is not representative of the population you are interested in: but as you observe here, it looks like you actually managed to sample a very large proportion of the population you were interested in (it was just a small population).
Because you have so much within-subjects data (i.e. multiple datapoints from the same respondent), you will actually be much better powered than you might expect with ~200 respondents. For example, if you asked each respondent to rate 10 things (and for some questions you actually asked them to rate many more) you’d have 2000 datapoints and be better able to account for individual differences.
You might, separately, be concerned about the small sample size meaning that your sample is not representative of the population you are interested in: but as you observe here, it looks like you actually managed to sample a very large proportion of the population you were interested in (it was just a small population).