Machine Learning Service Configuration - HEE National Data Service

Description

Summary of the work HEE is seeking a partner with expertise in Data Science / Machine Learning service design to advise and assist the organisation on investment in the design, implementation and management of a Data Science and Machine Learning services infrastructure in our Microsoft Azure environment. Expected Contract Length Minimum of two months and no longer than three months. Latest start date Tuesday 1 February 2022 Budget Range £99,000 (£99k) including VAT. We want suppliers to submit returns which include VAT. Why the Work is Being Done Previous work has demonstrated that viable Machine Learning (ML) use cases exist and a working prototype of an ML system has been delivered using Azure services. Additionally, significant data processing, data science and ML use cases exist in the workforce modelling space that HEE are looking to investigate. Other Teams have also raised interest in how ML activities could be taken forwards. As a prerequisite to making further progress on individual projects, the HEE Board have identified a strategic requirement to create a suite of enterprise data science and ML services, underwritten by data discovery / management and analytical services. HEE would like to develop a platform, services, tools, methods and data/analysis artefacts that can be shared across the business, eliminating potential silo working and providing a foundation upon which to build and expand Data Science and ML insights. Work must be commenced by the 1st February 2022. Problem to Be Solved HEE would like to be supported and guided on how best to invest in / utilise our Azure cloud environment to support enterprise wide Data Science and ML activities. We would like to be directed and assisted to develop infrastructure to support Data Science and ML activity which links to our existing data solutions and enables the development of previous ML projects into usable analytical resources (this will be a separate Digital Marketplace tender developed following this tender for infrastructure / service creation). We envision that services would be deployed and configured using managed code (ML / Dev Ops), link to existing data sources, enable the creation and persistence of intermediate data artefacts, provide cloud based tools to enable ML practitioners to discover data, set up and run ML experiments and establish a platform for the deployment and use of developed ML services / analytics models. Services would be integrated into our current Azure environment. Governance of the service, including implementation of appropriate data ethics frameworks is an important consideration. We would also like to implement / extend the supporting infrastructure required to underwrite a successful service (data discovery, data management etc.). Who Are the Users As a product owner I need to know that the service model implemented delivers business requirements to underwrite enterprise Data Science and ML activities So that the business can carry out Data Science and ML activities efficiently and effectively As a data science / ML analyst I need to be able to access, create and share Data Science / ML resources in the cloud So that I can efficiently produce analyses for the business and the wider NHS As an IG Manager I need to be assured that Data Science and ML activity taking place across the business conforms to ethical guidance and IG and data protection legislation So that we are compliant with the law As an Information manager I need to ensure that data resources used for ML and Data Science are discoverable, accessible and reusable across the business So that effort and resources are not duplicated and there is consistency in analytical outputs As a director of IT at HEE I want to know that any Data Science / ML technical resources deployed integrate with existing HEE enterprise technology stack and are robust maintainable, and secure In order to reduce technical risks to the business Work Already Done Earlier this year, HEE procured support for two ML projects via the Digital Marketplace: Machine Learning for Improved Insight (Discovery & Alpha) https://www.digitalmarketplace.service.gov.uk/digital-outcomes-and-specialists/opportunities/11213 Using Machine Learning for predictive analytics across the NHS trainee pipeline (Alpha) https://www.digitalmarketplace.service.gov.uk/digital-outcomes-and-specialists/opportunities/13735 We currently have a data management infrastructure built in Azure and deployed and managed via Azure DevOp (SQL server managed instance, Azure data Factory). We are also looking into shared access and use of metadata management / data cataloguing tools with NHS Digital. Existing Team The HEE National Data Service senior management and technical teams (skilled with SQL, SQL server and Analysis services) HEE Enterprise Architect HEE Information Governance Manager HEE Regional Business Intelligence stakeholders (some of whom have training and certification in Azure ML). We will be using an Agile approach to project management. No other suppliers will be involved, although previous work conducted in ML may be referenced / reused (see hyperlinks above). Current Phase Beta Skills & Experience • Have experience providing consultancy on enterprise Data Science and ML strategy • Have experience with developing enterprise machine learning solutions architectures and identifying resourcing requirements (staff, infrastructure, technology and tools) • Have experience designing and deploying Data Science and ML infrastructure and services in Azure using ‘infrastructure as code’ approaches • Have experience working on and delivering Data Science and ML projects • Be able to work in partnership with HEE and provide training so that knowledge and skills can be shared to support any alpha services that are developed • Be able to provide specific and relevant examples of previous work • Have experience delivery projects using an Agile approach • Be able to utilise Azure Machine Learning for model development Nice to Haves • Have knowledge of / experience with NHS data and data governance • Have experience designing and deploying ML services within the NHS • Knowledge of NHS strategies including Topol Review, Long Term Plan, Interim People Plan • Knowledge of GDS Service Standard • Knowledge of / experience with GDPR • Knowledge of NHS Digital Guidelines inc. Code of conduct for data-driven health and care technology Work Location The work can be undertaken remotely. If there is a need to meet in person this would be at the London office, address below: Health Education England 3rd Floor Stewart House 32 Russell Square London WC1B 5DN Working Arrangments We would like the team to be available during weekday working hours. No. of Suppliers to Evaluate 4 Proposal Criteria • Technical solution • Approach and methodology • Team structure (inc. no sub-contractors) • How the approach or solution meets our goals • How the approach or solution meets user needs • How they’ve identified risks and dependencies and offered approaches to manage them • How the approach can be flexible in the event of COVID disruption Cultural Fit Criteria • Work as a team with our organisation and other suppliers • Be transparent and collaborative when making decisions • Have a no-blame culture and encourage people to learn from their mistakes • Take responsibility for their work • Work in close collaboration to share knowledge and experience with other team members • Can work with clients with low technical expertise Payment Approach Fixed price Assessment Method • Work history • Presentation Evaluation Weighting Technical competence 70% Cultural fit 10% Price 20% Questions from Suppliers 1. Nice-to-have skills and experience: The description is non-compliant as per DOS5 guidelines and discriminatory due to prior public sector experience..What is in the NHS sector that you need the suppliers to prove ? We strive to be inclusive in our identification and selection of a supplier, hence NHS sector experience being identified as a ‘nice to have’. Should a supplier not have NHS sector experience consider what other comparable experience (e.g. Local Government, National Government, Charity, Wider Public Sector, Private etc.) you could identify in your application. 2. Could you please indicate who supplied the Discovery, who supplied the Alpha, and if either are eligible to bid? For either company to be eligible to bid, they must be a Digital Outcomes and Specialist (DOS) 5 supplier. 3. Please can we obtain clarification on the following:Is this a continuation from the Discovery and Alpha phases? Is HEE looking to productionise the models from the Alpha phase.What lessons did HEE learn from the Discovery/Alpha phases which may impact on the Beta phase?Many thanks 1) The intention of this work is to establish a service that will enable the Discovery / Alpha phases to be implemented in subsequent projects using an Enterprise solution. The key deliverable is to design and establish a service model rather than to implement previous work. This includes both technical implementation and project governance models.2) Lessons learned:There are viable machine learning use cases in the areas investigated (medical trainee management).The Microsoft technology stack used in the Alpha phase integrates with existing HEE systemsAn enterprise service model is required to support coherent development across the business 4. Is the core delivery of the model component to be on Azure Machine Learning or can it be, for example, custom Python on Azure Functions? (all other Azure components would function the same) There is no explicit requirement for any specific implementation, aside from being hosted within our Azure environment, although the use of industry standard approaches would be preferable. 5. What access to subject expertise do you have? ML techniques are mostly forms of optimisation based on input features, so understanding the problems is still critical for success. We have extensive subject matter expertise in all the data domains under consideration and will be able to provide clear guidance on input feature structuring. 6. Can you advise what types of data and how is data currently stored and managed? Most data under the project will be stored in Azure SQL (Managed instance or standalone databases). Data is onboarded using a variety of approaches (SSIS, data factory). 7. Can you advise how many stakeholders there are? Stakeholders will consist of representatives from a number of HEE internal teams (Regional Business intelligence teams, workforce modelling, technology enhanced learning the National Data Service, Enterprise architecture etc). I would expect around half a dozen separate stakeholder representatives with an interest in direct ML use and a similar number of internal stakeholders on the support / governance / technical side. 8. Is there an orchestration framework already in place (Data Factory)? We currently have a data factory instance associated with our SQL managed instance. Infrastructure is deployed using Azure DevOps / ARM templates / PowerShell / Bicep. The data factory instance is hosted on an Azure VM and also support an SSIS runtime.