NICE AI Multi Agency Service Evaluation

Description

We are looking for an evaluation partner with experience evaluating national programmes of a similar scale and complexity, and where the scope could potentially evolve, to deliver a formative evaluation and hand-over a robust summative evaluation strategy (including proposed metrics and data sources) by March 2023. The supplier should also have the capability to deliver emerging learning throughout the project The Programme is for evaluation of the AI Multi-Agency Advice Service (MAAS) project this is a 2.75 year project, funded by the NHSx AI Lab. The aim of the project is to collaboratively research, test and develop a multi-agency advice service that will offer support, information and advice on the regulation and health technology assessment pathways for artificial intelligence (AI) in health and care, and potentially other data-driven technologies. The purpose of the formative evaluation is to support the MAAS to be as effective in implementation as possible. This will be done through activities such as analysing baseline data in order to make recommendations on focus of the service, and undertaking process evaluation, particularly with regard to the project's interdependencies within the wider innovation ecosystem. As this is a complex intervention which will necessarily evolve, we expect the evaluation partner to be comfortable in an ambiguous environment, and providing us with emergent messages on what they see is working and not working and why. The summative evaluation strategy, once implemented, will be used by MAAS collaborators and commissioners in order to make future decisions about funding for this and other similar initiatives. It will also allow NHSX to report back to HM Treasury on the impact and benefits realised by the MAAS. It must be co-developed with the developer and the MAAS collaborators. It should focus on a range of outputs, outcomes and benefits including near-term (e.g. satisfaction of developers / adopters using the service) to long-term impacts (e.g. patients and citizens benefitting from AI and data-driven technologies). It should be underpinned by a logical and plausible theory of change or logic model. It should also have clearly defined metrics which are prioritised and have recommendations for gathering data. Some outputs from the evaluation may be used for external purposes such as communications; however, this is not the primary purpose. These activities will require contact with the MAAS partners at predetermined time points, and potentially with wider stakeholders involved in the project (TBD), which the AI MAAS secretariat will support Full details in the attached documents