Publications European Commission: EXPH (EXpert Panel on effective ways of investing in Health), Opinion on Assessing the impact of digital transformation of health services
Authors' abstract
The impact of digitalisation of health services has been profound and is expected to be even more profound in the future. Like for other services, it can be important to evaluate the impact of such digital health services. This should be done at the level of the service. Decisions to adopt, use or reimburse new digital health services, at different levels of the health care system, are ideally based on evidence regarding their performance in light of health system goals. In order to evaluate this, a broad perspective should be taken in evaluations of digital health services. Attainment of the broad health system goals, including quality, efficiency and equity, are objectives against which to judge new digital health services. These goals are unaltered by the process of digitalisation. Evaluations should be designed and tailored in such a way to capture all relevant changes in an adequate way. We do not provide a full evaluation framework in this opinion, but we do reflect on important elements. Monitoring can complement evaluations by observing general trends in how health systems evolve, also in light of digitalisation.
Many different categorisations of digital health services can be used in the context of their evaluation. We distinguish between interventions for clients, health care providers, for health systems or resource managers, and data services. Moreover, we distinguish between centralised and decentralised decision making. We advise to start any evaluation with a full description of the relevant digital technology, its use and aims, addressing elements like the ones above to give a full overview of the technology, its intended use, costs and consequences, and its most relevant comparator, in order to be able to select an appropriate evaluation strategy and key parameters to include. Important frameworks and practical guides for the evaluation of digital health services have been proposed. We especially highlight the recent Jasehn and WHO frameworks. These can serve as a starting point both for practical evaluation studies and for further development of evaluation frameworks. In evaluations, the development phase of the digital health service as well as implementation of it, are crucial elements. Combinations of different evaluation types may be required to provide relevant information to decision makers at different moments. Careful selection and justification of applied methods is warranted. Further investment in development of methodologies and a European repository for methods for and evidence on evaluations of digital health services are encouraged.
When evaluating digital health services many specific aspects need to be considered. We illustrate some of the specificities of evaluating digital health services, including creating a suitable policy context, rules for setting HTA priorities, and using appropriate outcome measures. Governments could play a more active role in the further optimisation both of the process of decision making (both at the central and decentral level) and the related outcomes. They need to find a balance between centralised and decentralised activity. Moreover, the broader preparation of the health care system to be able to deal with digitalisation, from education, through financial and regulatory preconditions, to implementation of monitoring systems to monitor its effects on health system performance, remains important. We discuss data sources, broader considerations (including cybersecurity, privacy and market power), provide recommendations for dealing with the digital transformation and finally answer the questions of the mandate.
