UK SBS IT18160 UKRI Ada Lovelace Centre Software Development

Description

Summary of the work For many National Facilities user communities, the computing infrastructure remain very difficult to use. The Ada Lovelace Centre is developing a set of tools to make the computing infrastructure easier to use. Expected Contract Length Seven months (to the end of March 2019) with the option to extend for a further 6 months. Latest start date Wednesday 5 September 2018 Budget Range The estimated value of this opportunity is up to £600k (ex VAT) including any options to extend, although there is no commitment to spend up to this amount. The estimated value of the initial contract term is up to £520k (ex VAT). Why the Work is Being Done The Ada Lovelace Centre (ALC), is being established as an integrated, cross-disciplinary data intensive science centre, for better exploitation of research carried out at our large scale National Facilities including the Diamond Light Source (DLS), the ISIS Neutron and Muon Facility, the Central Laser Facility (CLF) and the Culham Centre for Fusion Energy (CCFE). The Centre has the potential to transform research at the Facilities through a multidisciplinary approach to data processing, computer simulation and data analytics. The impact will be felt across the many science disciplines and communities these facilities support, including industry and academia. Problem to Be Solved 1. Develop options for Peta to Exa-scale database/data archive. 2. Develop GridFTP plugin for Data Management System to replicate ACL and posix permissions for 3rd party transfers. 3. Develop configuration for VMs on commercial cloud platforms to access STFC science data/resources. 4. Develop existing and new analysis environments to support STFC Facility science groups. 5. Develop UKAEA access to STFC computing resources (IRIS), including containerising HPC codes, deploying Spark cluster, and developing ML codes for anomaly detection on MAST data Who Are the Users As a National Facilities user and academic researcher I need to access my experimental data on appropriate compute resources so that I can process and analyse my experimental data to make more effective world-class scientific research discoveries for the benefits of society. Work Already Done The hardware infrastructure is in place and work has been carried out with the National Facilities to gather their requirements. There is a design and prototype of the Data Management system and a preliminary design for the virtual machine management. Existing Team The Software Engineering Group from the Scientific Computing Department is in place at the Rutherford Appleton Laboratory. The supplier’s team will be working with this team – and will also have occasional interactions with the National Facilities teams. STFC will also be recruiting internal staff who are expected to arrive later in the year. The supplier’s team will be expected to work closely with the recruited internal staff, and to perform a complete and thorough handover to them. Current Phase Beta Skills & Experience • Demonstrate how you will apply your expertise in the following areas to ensure the successful delivery of the respective projects (See “Problem to be solved”): • Project1. Large scale databases, data archives and catalogues. • Project2. C, GRIDFTP, x509 certificate authentication. • Project3. Ansible, Python, Networks, security and virtual private networks. • Project4. Ansible, Linux, compilation expertise, python/bash scripting. • Project5. C, C++, or Fortran 90, UNIX, Python, HPC applications, MPI libraries, containerisation, ML techniques - Tensorflow or Caffe Work Location Rutherford Appleton Laboratory, Chilton, Oxfordshire OX11 0QX and Culham Centre for Fusion Energy, Culham Science Centre, Abingdon, Oxfordshire OX14 3DB. Working Arrangments Mon-Fri 9-5 and occasional remote working. Security Clearance Baseline Personnel Security Standard (BPSS) and Disclosure Scotland Additional T&Cs T&S as per UKRI policy. No. of Suppliers to Evaluate 3 Proposal Criteria • Technical solution • Approach and methodology • Estimated timeframes for the work • How they’ve identified risks and dependencies and offered approaches to manage them Cultural Fit Criteria • Demonstrate how you will work with other people throughout this project • Demonstrate how you will solve problems throughout this project • Demonstrate how you will share knowledge and expertise throughout this project Payment Approach Capped time and materials Evaluation Weighting Technical competence 60% Cultural fit 10% Price 30% Questions from Suppliers 1. How do you envisage using Ansible? Along-side Docker or another container tool? Project 3 & 4: We already use Ansible to configure virtual machines running on a OpenStack cloud. Additional configuration will need to be added to existing Ansible playbooks for these two projects.Project 5: We envisage using Ansible for building Docker images containing HPC software. In addition to providing a reliable and flexible way of building images it would allow us to be consistent with others in the EUROfusion community, where Ansible Container is emerging as a standard for building Docker images. 2. Can you expand upon the use of Tensorflow and Caffe and any requirement for deep learning techniques? Part 1: AI and Machine Learning are not currently extensively deployed in Fusion science or Engineering, yet there is an increasing awareness of the benefits these maturing technologies have to offer, whether it be for anomaly detection of complex systems (e.g. nuclear plant), for image analysis (e.g. feature recognition and classification of plasma Magneto Hydro Dynamic instabilities), disruption mitigation (http://images.nvidia.com/content/pdf/princeton-iter--fusion-energy-success-story.pdf), or for steering experimental operations (https://www.theguardian.com/environment/2017/jul/25/google-enters-race-for-nuclear-fusion-technology). 3. Can you expand upon the use of Tensorflow and Caffe and any requirement for deep learning techniques? Part 2: UKAEA has a growing number of projects in the area of AI and ML, including work with the STFC funded UCL Data Intensive Science CDT and as part of our Robotics Centre RACE (http://www.race.ukaea.uk). We also have a growing relationship with the Alan Turing Institute where we intend to make large volumes of data available to the UK AI/ML community to train the next generation of AI/ML specialists. We are now at the point of needing scale out, high throughput processing capability for Deep Learning. 4. Can you expand upon the use of Tensorflow and Caffe and any requirement for deep learning techniques? Part 3: Work as part of the ALC will concentrate upon designing and building this platform (as a baseline, built upon TensorFlow and/or Caffe). The contractor will work closely with our scientists and engineers to design a system that can be deployed on IRIS hardware and connect to UKAEA data systems. If GPUs are needed after assessing initial performance, this work will steer the procurement of new hardware as well as requirements capture for federating in GPUs from across the IRIS network (e.g. from DiRAC). 5. Can you expand upon the use of Tensorflow and Caffe and any requirement for deep learning techniques? Part 4: Data management, APIs and visualisation may also be areas where the contractor will help to develop solutions resulting in a slick, easy to use system. For those workflows not currently using TensorFlow or Caffe, the contractor will work with our users to rebuild their software for the new system and related IRIS infrastructure.