Work capability assessments: making them more consistent
On behalf of social security institutions, psychiatrists assess to what extent people with mental health problems are still able to work. However, the work capability assessments tend to be far too dissimilar. A new training course has helped to reduce the differences. This confirms a study conducted by researchers at the University of Basel and the University Hospital of Basel and financed by the Swiss National Science Foundation.
03 July 2019
People who cannot work full-time anymore because of mental health problems are entitled to a benefit. The amount they receive depends on how fit for work psychiatrists consider them to be. Ideally, psychiatrists who undertake work capability assessments should recommend similar work quotas for similar cases, but the reality still lags a long way behind. Sometimes assessments differ by one hundred per cent. Needless to say, this is an unsatisfactory situation for lawyers, judges, insurances and psychiatrists alike.
With the aid of a new type of evaluation, the functional assessment of work capability, and special training for psychiatrists, researchers have been able to achieve more consistent assessments of the remaining work capability. The project was able to reduce the statistical dispersion by more than a fifth. However, practitioners want to see substantially less deviation. On the other hand, both the applicants and the psychiatrists involved consider the new functional assessment process to be fair and transparent. "This is important, because the assessments decide the fate of individuals," says the study leader Regina Kunz from the University and the University Hospital Basel. The study was published in BMC Psychiatry (*).
Focusing on work
In the new evaluation process, psychiatrists focused on work as soon as the conversation is started, rather than on the illness. Their approach was solution-focused. For example, they asked what activities the applicant's last job involved, what they were still capable of doing and what might be helpful. Finally, they had to grade 13 work-related abilities that are often compromised in people with mental health problems. Based on this, they estimated the amount of work someone could do.
The study included comparing assessments by 35 psychiatrists. They assessed the work capability of 40 applicants. The conversations were recorded on video and reviewed independently by three further psychiatrists. In the end, four different assessments were available for each applicant. The differences between the assessments were compared with a previous round of assessments, in which 19 experts had assessed the work capability of 30 applicants. This round had been preceded by a training course for the psychiatrists which had been considerably shorter and had moreover taken place more than a year before the assessment.
Sights set too high?
The researchers counted how many times two assessments differed by a maximum of 25 percentage points in their estimation of work capability: in the control group 39 per cent of the comparisons between two assessments were above this threshold. After the training, the share was reduced to 26 per cent. A statistically significant effect.
In order to gauge the tolerance limit for deviations between two assessments of the same person in practice, the researchers had already conducted a survey among 700 experts in Switzerland: psychiatrists, representatives of social security institutions, lawyers and judges. An analysis of the survey results suggested that a difference of 25 percentage points was regarded as just about acceptable for a process that ought to be as fair as possible (**). "Of course we never expected our approach to solve all of the problems," says study leader Kunz. "But we are nevertheless disappointed that the assessments could not be made more consistent."
Kunz still feels the project has been worthwhile on the whole: "The applicants and psychiatrists involved were satisfied with the new process, so insurers and courts can now work with clearer and more transparent assessments." In a follow-up study, she and her colleagues would like to test whether further improvement is possible if the psychiatrists receive even more intensive training.
However, the results are not likely to fulfil the expectations of the stakeholder groups. The problem of wide variation is endemic to the western world, as the researchers showed in a previously published systematic review (***). "Doctors are not experts on work, as they will willingly admit," says Kunz. "Training alone will not be enough." It is necessary to change tack, for example by using completely new evaluation systems: "In the Netherlands, psychiatrists focus on therapy, the assessments are done by specially trained experts."
The project was funded by the Swiss National Science Foundation (SNSF), the Federal Social Insurance Office and the Swiss Accident Insurance Fund (suva) accident insurance.
Original source
(*) R. Kunz et al.: The reproducibility of psychiatric evaluations of work disability: Two reliability and agreement studies.
BMC Psychiatry (2019). DOI: 10.1186/s12888-019-2171-y
(**) S. Schandelmaier et al.: Attitudes towards evaluation of psychiatric disability claims: a survey of Swiss stakeholders.
Swiss Medical Weekly (2015). DOI: 10.4414/smw.2015.14160
(***) J. Barth et al.: Inter-rater agreement in evaluation of disability: systematic review of reproducibility studies.
BMJ (2017). DOI: 10.1136/bmj.j14
Further information
Prof. Dr. Regina Kunz, University of Basel / University Hospital Basel, Department of Clinical Research, phone: +41 61 328 75 67, email: regina.kunz@usb.ch
