Search Alpha/Beta - Empowering users to search The National Archives catalogue and website

Description

Summary of the work With our new website’s Private Beta nearly complete, we now require a partner with skill and expertise in developing powerful search engines using Elasticsearch to build a new service for the next www.nationalarchives.gov.uk website. Expected Contract Length 4 months Latest start date Friday 1 October 2021 Budget Range Up to £100,000 Why the Work is Being Done For our new website we have focused on a ‘browsing’ experience, providing alternative ways to explore our collections.  But many users, especially more experienced and confident ones, are reliant on search for specific and often complex research queries. We want to build a new and futureproof search experience that meets and exceeds their expectations. The search will combine data from our catalogue and other archives, our website and media files to identify, organise and display all available published content that a user can access. The search Beta is a key service in our new website’s Public Beta, launching early 2022. Problem to Be Solved We have over 24 million records in Discovery, our catalogue, and tens of thousands of published webpages and media files in WordPress and legacy systems. We believe that search should not just be about finding something that matches a keyword, but about guiding users through a frictionless user experience, eliminating confusion, aiding understanding of the complexity of our content and enabling serendipitous exploration. Leveraging the potential of Elasticsearch we want to set a new standard for what searching an archive can be. Who Are the Users We primarily focused on two user groups during our website’s Alpha phase - potential users and first-time users. Now we are engaging with other audiences to create a full understanding of user needs for the new platform. These include existing users such as family historians, academics, researchers and other professionals engaged in personal or historical research using our digital and physical services. Users who start with our existing catalogue search to conduct their research, often over long periods of time and in conjunction with other sources of historical or archival data. Early Market Engagement We are nearing the completion of our website’s Private Beta phase. Our new platform, known as Etna (Explore the nation’s archives), has been built on technologies including Python, Wagtail CMS, Collections Information Integration Middleware (CIIM), AWS and Elasticsearch. We have some in-house skills and experience in Elasticsearch, Python, AWS technologies/hosting, Wagtail and CIIM. We are looking for a partner with a proven track record of developing intuitive, powerful faceted search experiences, utilising user-centered techniques such as search-as-you-type, auto-suggest and auto-complete. We have published a series of articles covering both the Alpha and Beta phases of our website on our Medium blog at https://medium.com/the-national-archives-digital. Work Already Done An initial Search Discovery phase is underway with the following deliverables: Collated user research of known pain points and usability issues with our current search interface and other issues relating to searching our catalogue Literature review and competitor analysis Detailed reports of the current usage of our search, from analytics and log data A set of business rules for our search, agreed with stakeholders and project sponsors A draft proposal for the ‘ideal’ search service. Existing Team Our Digital Services team will lead on delivery. You will lead the Elasticsearch aspect, pairing with our back-end team on Python development and contributing to front-end search UI. Back-end team - 3 Senior Developers (specialists .NET, some Python) UX led by our UX Manager. Our lead front-end developer will be technical product lead, with x1 Designer/Developer (UI design and front-end development). Metrics/Analytics led by our specialist. We've some data analyst and business analyst capability. You may supply a user researcher (collaborating with our UX Manager). We have subject matter experts, with in-depth knowledge of our collection and existing catalogue search. Current Phase Discovery Skills & Experience • Expertise in cloud-based Elasticsearch, Enterprise Search and Elastic Stack (Kibana), and optimising Elasticsearch using filters for faster searching. • Expertise in design, build and configuration of Elasticsearch clusters using infrastructure as code tools such as Terraform. • Expertise in the design and development of user-centered search UIs and faceted search, including implementing search-as-you-type, auto-suggest and auto-complete. • Experience of using Kubernetes in a cloud environment, preferably EKS on AWS. • Expertise in web application development using Python or . NET and delivering production code in line with the GDS Service Standard and Technology Code of Practice. • An understanding of progressive enhancement and accessibility legislation and how to design search experiences compliant with WCAG 2.1. Nice to Haves • Understanding of archives and archival practice • Experience of using data aggregation services or middleware • Experience of Wagtail CMS and Wagtail API • A user researcher on staff who can support this work as required. Work Location The Digital Services team is based at The National Archives, Bessant Drive, Richmond, Surrey TW9 4DU . Remote working by supplier as agreed. Working Arrangments Given the situation with COVID-19, remote working will be the main arrangement for this project. Some work may be conducted at The National Archives, Bessant Drive, Richmond, Surrey TW9 4DU, but only when it is appropriate and safe to do so. Security Clearance Baseline check (blue) security clearance will be required for access to The National Archives. No. of Suppliers to Evaluate 3 Proposal Criteria • Demonstrate the successful development and delivery of cloud-based search engines built using Elasticsearch and/or Elastic Stack and optimisations in Elasticsearch using filters for faster searching. • Demonstrate expertise in design, build and configuration of Elasticsearch clusters using infrastructure as code tools such as Terraform. • Demonstrate examples of modern search UIs, faceted searches and implementations of search-as-you-type, auto-suggest and auto-complete features. • Demonstrate experience of using Kubernetes in a cloud environment, preferably EKS on AWS. • Demonstrate proficiency building web application in Python or .NET and delivering production code in line with the GDS Service Standard and Technology Code of Practice. • Demonstrate ability to create effective, user centered search experiences in non-conventional search landscapes, such as archives. • Demonstrate an understanding of progressive enhancement and accessibility legislation and how to design compliant search experiences. Cultural Fit Criteria • Collaborative and flexible working approach, e.g. working with in-house technical and other digital specialists • Approach to supporting teams to adopt new technologies • Examples of delivering transition, knowledge transfer and handover of code • An appreciation for the importance of technical documentation as a means of ensuring ongoing maintainability of systems • An appreciation for the importance of creating inclusive and accessible digital services for all. Payment Approach Capped time and materials Assessment Method • Case study • Work history • Reference • Presentation Evaluation Weighting Technical competence 50% Cultural fit 20% Price 30% Questions from Suppliers No questions have been answered yet.