There’s an interesting new report out from the Minderoo Centre for Technology and Democracy at Cambridge University on the impact of AI on the novel, authored by Dr Clementine Collett. There’s much that I completely agree with and that accords with my experience of working with book trade on AI. But I have some constructive thoughts about the framing/communication.

The report is a mixed methods study combining focus groups, a survey, case studies, interviews and a forum. That’s a great approach for exploratory and descriptive research, but it’s potentially less suitable for population-level inference. The methodology section of the report doesn’t give any detail on recruitment and sampling. Dr Collett was very responsive to a question and confirmed that recruitment included robust steps to confirm the validity of responses. But the survey data comes from a convenience sample of 258 novelists.

The size of that sample isn’t the issue so much. It’s hard to know population size: some industry estimates suggest 70-80,000 authors in the UK, and the Society of Authors has around 12,500 members, though both include non-fiction authors as well. One might estimate 5-10k novelists as a ballpark. A 2-5% sample size is by no means inappropriate for social research—but only with a random sample. As a non-probability sample, it is indicative, but it’s harder to generalise out to a population.

For clarity, the report is always clear about its data. But the dissemination of the research—by Cambridge, some of the researchers involved, and the national media—uses headlines like “half of published novelists say…”, with the nuances somewhere below the fold. From a communications perspective, I absolutely understand why people don’t lead with “half of 258 survey respondents say.” But those are different things.

Regardless, the research captures important lived experience and attitudes, which may indeed be broadly representative. But it’s possibly more nuanced than some of the communication and reporting suggest, particularly if readers don’t get past the headlines.

What I really hope this report does is demonstrate that this is an important subject for further research: I’d love to see a larger, generalisable study, and also to understand the perspective of non-fiction writers (reminder: not in scope of this, so not a criticism of this research).