The regulator had been trialling a new set of questions focused on the nature and severity of harms experienced as a result of someone’s own gambling or someone else’s habits. Also new in the pilot survey were a set of questions about suicide.

The questions were tested on the Commission’s tracker survey on a quarterly basis, before being refined and tested again. These were then reviewed by the NatCen Questionnaire Development and Testing Hub prior to the pilot commencing in January 2022.

Pilot data was analysed by academics at the University of Glasgow to assess performance and make recommendations for future development, while two external experts in Robert Williams and Rachel Volberg, were asked to review the process and provide advice on how the questions could be modified or the approach strengthened.

The Commission said the external review and analysis of the pilot data found that while the question development process to be “robust” and the questions “clear and unambiguous”, some improvements were suggested.

These included ensuring a broader range of harm issues within the questions, with the pilot set being heavily weighted towards financial harm, as well as to change the ordering of harms questions so that items relating to less severe harms are presented first.

Another recommendations was to change the scaled response options to have more equally spaced responses. This would mean changing the lowest level of response from “a little” to a stronger term, to ensure that those measured as experiencing harm were actually experiencing harm, rather than the potential for harm.

Other suggestion was to consider how statistics from scaled answer options are reported and defined, and also that any future question wording be further reviewed by questionnaire development experts.

It was also recommended that the ‘harms from others’ questions should be revised as the current question appears to under-report the number of people who know people close to them who gamble, which could in turn lead to underestimation of harms from others.

In addition, the Commission was recommended to continue collecting data on both suicide ideation and attempted suicide, saying reference to this in the pilot set of questions was encouraging.

Continued progress

Since this review, the Commission has been working with NatCen to broaden the range of harms covered in the questions, developed an experiment to determine whether a binary response or a scaled option works best when asking about experiences of harm, and also revised the filter question into the ‘harms from others’ section.

Other changes include a review of both the ordering and wording of the harms questions by experts at NatCen, as well as the retention of questions on suicide ideation and attempted suicide.

The Commission said it now intends to review the results of the experimental phase and robustly evaluate which approach works best, as well as develop its approach to analysing data and engaging with stakeholders on how headline findings are communicated.

“Our aim is for the harms questions to be asked alongside core questions on participation and problem gambling in 2023, and become part of our suite of official statistics,” the Commission said.

“We are not aiming to develop a headline score or scale of gambling-related harms, or measure the cost of gambling harms to society. The Commission has a role to play in utilising its existing surveys to add to the wider evidence base. 

“However, there is still a need for a range of partners to be involved in the funding and delivery of research to measure and understand gambling harms and the impact that they have on individuals, families, communities, and society.”