Natural Language Processing Investigation for ONS website

Description

Summary of the work ONS.gov.uk website was designed over six years ago; the organisation has since transformed and our users, content and customer journey have subsequently changed significantly. ONS require expertise to assist with understanding the use of features such as Natural Language Processing in order to improve findability of content within the website. Expected Contract Length 4 months Latest start date Friday 10 December 2021 Why the Work is Being Done The Digital Publishing team at the ONS is responsible for the ONS dissemination service. The ONS website is the main channel for disseminating statistics and analysis and hosting corporate information. The ONS.gov.uk website was designed for the organisation over six years ago. Since then, the organisation has transformed and our users, content and customer journey have subsequently changed significantly. A clear user need to improve site wide search functionality on the ONS website https://www.ons.gov.uk/ has been identified and a team has been working on this for several months. ONS requirement timetable Milestone 1: Analyse existing search data, document how to produce a robust and user friendly service, produce a backlog of proposed development in order to agree and prioritise prototype(s) development plan. Within 6 weeks of contract start. Milestone 2: Deliver initial prototype(s) as agreed with ONS Product Manager and agree future progress on those or other products. Within 3 months of contract start. Milestone 3: Complete prototype(s), ensure all deliverables documented and shared to standards agreed with Digital Publishing team. Within 4 months of contract start. Completion date: 31st March 2022 Problem to Be Solved ONS require expertise to assist us to understand our use of features such as Natural Language Processing (NLP) in order to improve findability of content within the website. Outcomes Perform analysis of the current data types (27, but these could be grouped), understand and document how one would continue to provide a robust as well as a user friendly experience. Consideration should be given to how different data types will compete for being scored higher and hence be returned in the top results Datasets - limited content Articles/Bulletins etc. - lots of content (NLP should help users find the right content) Geographical areas (likely based on boundary files/geojson) Develop prototypes that integrate NLP into existing services/architecture, ensuring performance is maintained. Proving the benefits of NLP for the different data types. Understanding where NLP is improving findability and when it may become a hindrance - “ideas or solutions are not always guaranteed success”. Ensure the ability to monitor/test search engine functionality - so ONS and digital publishing can be confident the search engine does not degrade. ONS requests suppliers to register on the e-sourcing portal in preparation for the second stage if successful; https://in-tendhost.co.uk/ons/aspx/Home Who Are the Users ONS website users There is a need to be able to easily search for content in my own words So that I can effectively find all relevant/related content available from ONS. Work Already Done Previous alpha work was completed in incorporating Natural Language Processing into Elasticsearch queries using fastText and word vectors, however this work had outstanding query performance issues that have not been addressed. Existing Team The Digital Publishing Team (responsible for the ONS website and content management system) is an independent multidisciplinary team of frontend engineers, backend engineers, and full stack engineers, interaction designers, user researchers, tech leads and product and delivery managers. Collaboration with the Project Management team and individuals from other suppliers who are embedded within the wider Digital Publishing Team will be required. Current Phase Alpha Skills & Experience • An example of a contract successfully completed involving a significant scale user-focused alpha delivery of an Elasticsearch model (wide user-range, multiple data-types) completed in last 3 years. • An example of a contract successfully completed involving an integration into an existing system utilising a Micro-services model. • An example of a contract successfully completed involving creating performant searches including optimising the efficiency of machine learning models. • An example of a contract successfully completed involving working with data models for interpreting textual input such as Natural Language Processing or alternatives. • An example of a contract successfully completed involving balancing accuracy and performance requirements to optimise the performance of a machine learning model. • Proven experience of running a discovery process to review AS-IS analysis of multiple data types/capabilities and developing approaches to successfully transform them using modern practices/solutions through to completion. • Proven experience of setting out the user needs and business requirements for a public facing web service. Nice to Haves Experience of developing productionised systems in Go. Work Location Work to follow Covid-19 guidelines with remote working. ONS Newport and Titchfield offices are the main locations. Working Arrangments Remote working but with a need to come to offices when required for collaboration purposes. Expenses must be agreed in advance on a case-by-case basis in line with the ONS Travel and subsistence policy. Security Clearance BPSS - clearance required. Additional T&Cs SLA’s Agile approach –Collaborate Team on all aspects of work undertaken. Partake in daily stand-ups, perform regular (every 3 weeks) show&tells and engage in software peer reviews as part of Digital Publishing coding standards Meeting service development standards Services developed align with latest versions of internal libraries in-line with latest standards/conventions. Technology stack align with existing programming languages (Go) and Elasticsearch. If not feasible, a strong argument/justification to be provided for using other technologies. Documentation/knowledge transfer – Document services developed and worked on, allowing knowledge transfer to ONS. Include how NLP integrates with existing search pipelines, designs, schemas/specifications where necessary. No. of Suppliers to Evaluate 3 Proposal Criteria • Please confirm your methodology for delivering a prototype to integrate Natural Language Processing into existing services/architecture taking an agile approach to data analysis, development, and testing. • Confirmation that you will meet the deadline for completion of the contract in accordance with Milestone/Deliverables. • Provision of a delivery plan to illustrate your programme for delivery, including confirmation of dates and path to completion of Milestones/Deliverables. • Explanation of how the proposed method would be able to deliver the best product for the end-users. • Explanation of the expected involvement and accountabilities of ONS. • Please illustrate how the approach will support our ‘Statistics for the Public Good’ strategy. • How will you ensure that the scope includes reference to all our user personas in our aim to reach an inclusive audience? • How will you incorporate consideration of accessibility requirements into your prototype design? Cultural Fit Criteria • How will the people put forward by the supplier ensure that they collaborate effectively with other team members, and the ONS as appropriate. • Please describe how the team will communicate effectively with all relevant stakeholders including up the management chain within the ONS. • Please describe how the team will deliver specialist technical information to an experienced technical end-user and; • Please describe how the team will deliver specialist technical information to an end-user assuming that the end-user has no technical knowledge. Payment Approach Fixed price Evaluation Weighting Technical competence 70% Cultural fit 10% Price 20% Questions from Suppliers 1. Team size: There is not much time available to build a prototype. Please could we clarify the budget/number of people? As allowed in the DOS framework, ONS has refrained from providing a budget for this requirement 2. Please could you give us some insight on what the prototype would look like and which kind of search you want to productionise while working on the ML efficiency?Part 1 "The prototype will need to demonstrate the NLP effectiveness in combination with Elasticsearch in returning relevant results written using ONS language for queries made in a user's own words. The exact form of this prototype is flexible. Test scripts, an API or a web search page could all be acceptable provided that the prototype will accept various search queries that have been shown through user research or analytics to be ineffective in the current implementation in order to demonstrate the effectiveness of the solution. 3. Please could you give us some insight on what the prototype would look like and which kind of search you want to productionise while working on the ML efficiency?Part 2 As our search API is written in Go, we would ideally wish for the prototype to utilise Go for the Elasticsearch queries and the interface, but we recognise that another language may be needed for the NLP aspects. The prototype should also ideally include a form of quantitative measure to clearly indicate the superiority of results returned to a user by the search prototype vs the current search implementation. 4. Please could you give us some insight on what the prototype would look like and which kind of search you want to productionise while working on the ML efficiency?Part 3 Regards to the productionising aspect of work, ONS are mostly interested in the search performance is maintained as close to the current search speed as possible and that the resource usage of the models can be optimised to ensure that running the models at scale will not be resource or cost prohibitive.ONS also expecting demonstration of performance meets the expectations for production and either a PoC or at minimum a systems design showing how the NLP could be implemented within the current search. For reference the search implementation that the solution will need to be integrated with is: https://github.com/ONSdigital/dp-search-api. 5. Please could we have more information around the methodologies used in your Alpha product (and in particular about the word vectors), the models used, and which performance issues you faced? It would be useful to know which kind of specs you use to run (and train) the models.Part 1 "All code and some documentation from the previous alpha is publicly available and can be viewed via the following github repositories:* https://github.com/ONSdigital/dp-search* https://github.com/ONSdigital/dp-fasttext* https://github.com/ONSdigital/dp-conceptual-searchUnfortunately, as the individual who completed this work has now left us we do not have the specs of the instances used to train the models, but the resources allocated to running the models can be seen the *.nomad files of the above repos. All of our services run on AWS, so we can certainly experiment with different instance types. 6. Please could we have more information around the methodologies used in your Alpha product (and in particular about the word vectors), the models used, and which performance issues you faced? It would be useful to know which kind of specs you use to run (and train) the models.Part 2 The most critical performance issue faced previously was the search speed. The NLP implementation was too slow to be tenable. In order for this prototype to be effective it needs to demonstrate search speeds as close to the existing search as possible. Secondary to the search speed was the exceptionally high memory usage of the previous implementation, so a strong solution would be expected to be more efficient with its resource usage. 7. Is there an incumbent and can you please advise on the allocated budget for this project? There is no incumbent. 8. Given this project is going into Alpha can you share who delivered the Discovery phase (incumbent) and is there any additional documentation that can be shared? "There is no incumbent. All code and some documentation from the previous alpha is publicly available and can be viewed via the following github repositories:* https://github.com/ONSdigital/dp-search* https://github.com/ONSdigital/dp-fasttext* https://github.com/ONSdigital/dp-conceptual-search" 9. Given this is a fixed price payment approach, are you able to share what the budget thresholds are likely to be? As allowed in the DOS framework, ONS has refrained from providing a budget for this requirement 10. What is the expected budget for this project? As allowed in the DOS framework, ONS has refrained from providing a budget for this requirement