Testing activities for the Office for the Internal Market (OIM) Gateway

Description

Summary of the work The Competition and Markets Authority aims to engage a Supplier to perform testing for the OIM Gateway project which goes live 1st September 2021. This comprises of a public facing web form and a secure portal (TBC). Compliance with GDS design principles and Technology Code of Practice. Expected Contract Length The contract is expected to run through to 31st March 2022 Latest start date Monday 26 July 2021 Budget Range £50,000 Why the Work is Being Done Following EU Exit a new function called the Office for the Internal Market (OIM) has been set up within the CMA. The OIM will monitor the running of the UK Internal Market. Its primary role will be to provide a set of independent advisory, monitoring and reporting functions to support the development and effective operation of the UK Internal Market. Problem to Be Solved To enable the OIM to perform its role there is a requirement to develop and operate an online OIM Gateway to facilitate interaction with stakeholders and gather insight and intelligence from businesses, consumers and other 3rd parties. The following activities are required from the supplier: • Creation of a high-level test plan. • Creation of test scripts based on acceptance criteria defined in the user stories. • Functional testing (including cross platform and browser compatibility testing). • Recording defects and bugs, liaising with developers to explain defects and bugs and performing regression tests. • Performance and capacity testing (to be determined once further analysis on expected transaction volumes has been completed). • Accessibility testing. • Security testing. • Planning and co-ordination of user acceptance testing. • Production of a test summary report. Who Are the Users The main users of the OIM Gateway will be the OIM staff, UK businesses, consumers and (potentially) the Devolved Administrations and UK Government departments. UK businesses and consumers will need to be able to report any potential issues and regulatory distortions within the UK Internal Market. Potentially, the Devolved Administrations and UK Government departments to submit and request advice and reports from the OIM. The OIM staff will need to be able to receive and review submissions from UK businesses, consumers and other stakeholders. Early Market Engagement N/A Work Already Done During the Discovery phase the CMA worked with the suppliers for the original Online State Aid System (OSAS) development who produced a high-level design based on the technologies used to build OSAS. The supplier also produced initial wireframes that will be reviewed and refined during the project. By the time the test resources are in place the CMA expects user needs to have been validated and documented by a user researcher. Existing Team The Supplier will be working with a team that currently consists of a Product Manager, Project Manager, Business Analyst, Service Transition Lead and representatives from the CMA Web Publishing Team. In addition, by the time the test resource starts the team will have been supplemented by a user researcher and developers supplied by other 3rd parties. Current Phase Discovery Skills & Experience • Demonstrable and recent (within the last 2 years) experience of testing services to meet the Government’s Service Standard (5%). • Demonstrable and recent (within the last 2 years) experience of performing accessibility testing for digital solutions to ensure they meet Level AA of Web Content Accessibility Guidelines (WCAG) 2.1 (5%). • Demonstrable and recent (within the last 2 years) experience of testing services that meet both user needs and business objectives (5%). • Give examples where the Supplier has performed testing aligned to agile methodologies (5%). • Give examples where the Supplier has worked on Central Government projects where the solution has had to comply with legislative requirements (5%). • Provide examples where the Supplier has provided test resources that have integrated with the client’s project team (5%). Nice to Haves • Provide examples where the Supplier has provided test resources that have worked with UX and development resources from other suppliers • Give examples where the Supplier has worked on projects that have delivered solutions with data capture and workflow components. Work Location Due to the COVID-19 pandemic the project team and OIM staff are currently working remotely with meetings taking place via Microsoft Teams. Any supplier resources must also be able to work remotely. From September 2021 it is expected that the project team and OIM staff will spend at least three days a week at the CMA’s offices at The Cabot, 25 Cabot Square, London, E14 4QZ. From that point supplier resources may also be co-located onsite. Working Arrangments Client-side delivery and engagement with the Supplier will be provided by the CMA Project Manager who will provide the Supplier interface to key stakeholders within the CMA, OIM, BEIS and GDS. Security Clearance BPSS check completed by contract start date. Additional T&Cs All vendors are obliged to provide sufficient guarantees to implement appropriate technical and organisational measures so that the processing meets the requirements of GDPR and ensures the protection of the rights of data subjects. For further information please see the Information Commissioner's Office website https://ico.org.uk/for-organisations/data-protection-reform/overview-of-the-gdpr/ * The 3 Shortlisted Suppliers will be required to complete a GDPR/Data handling questionnaire prior to the 2nd phase of evaluation, as part of their proposal. This will be provided at notification stage. No. of Suppliers to Evaluate 3 Proposal Criteria • Understanding of the requirement – examples showing that the supplier has understood the goals and deliverables required (10%). • Approach and methodology examples that demonstrate strong understanding of functional, performance, security and accessibility testing concepts and how these have been applied to successfully complete testing on agile projects (5%). • Resourcing structure to deliver, and evidence (including with the use of relevant case studies as appropriate) the successful resilience of the proposed resource pool (5%). • Demonstrate and evidence (including with the use of relevant case studies as appropriate) how best practice in Agile Governance has been used to test successful digital services (5%). • Fully evidence comprehensive and detailed capped time/materials pricing and methodology for all resource/roles rates, materials and expenses, both off and on-rate card, and how it provides value for money (5%). • It is a requirement, if applicable that the Potential Provider is required to assist the Authority in determining their Employment Status for Tax in accordance with the HMRC IR35 regulation. Cultural Fit Criteria • Have a culture of sharing and learning (2%). • Be transparent and collaborative when making decisions (2%). • Be comfortable challenging non-technical and technical decisions where they see value in an alternative approach (1%). Payment Approach Capped time and materials Evaluation Weighting Technical competence 60% Cultural fit 5% Price 35% Questions from Suppliers 1. Do you require testing with users as part of this programme of activities? We will be engaging with a separate supplier for the provision of a User Researcher resource that will define and test user needs with users of the OIM Gateway during the project. The resources provided under this advert will predominantly focus on functional testing, regression testing, accessibility testing and security testing. However, we would also want them to help co-ordinate User Acceptance Testing (UAT), which will predominantly be with business stakeholders within the OIM but may also include some external users. 2. With a Go-Live date 1st September 2021 is the resource expected to remain available until 31st March 2022? Our current plan is for the solution to go live in September and then have a period of enhancements and development of non-MVP functionality through to early December 2021. As such we would expect the test resource to be available during that period. We have suggested the contract runs to 31st March 2022 in case if additional requirements are identified and the project needs to extend into Q1 2022. In that case we would look to potentially procure additional resource via a contract variation. 3. Hi, Does this opportunity fall within IR35 or outside of IR35? The Services required by the Authority may attract trading individuals to submit proposals for this opportunity. The HMRC off-payroll working rules may apply, if an individual provides their services through their own Personal Service Company or another type of intermediary. As such the Authority is required to determine if the off-payroll working rules are applicable. Therefore, any individuals trading through their own Personal Service Company or another type of intermediary who submit their proposal for this opportunity is required to assist the authority by providing any information required to make this determination. 4. Is there an existing supplier providing testing services No, there is currently no existing supplier that is providing testing services. 5. Can you please confirm the timeline – With a start date of end of July/Early August that only leaves one month before the Go Live date of 1st Sept. Do you expect all testing to be completed before Go Live or just elements – if so which ones? We would look for functional testing, accessibility testing and UAT to be complete by go-live. However, that is just for MVP which is the online service/web form and underlying database. We do not currently expect the online service to be particularly complicated. Further testing of enhancements and non-MVP functionality is currently scheduled to take place September to November 2021. 6. Do you anticipate functional testing to be carried out manually or is there an appetite for automation or an existing framework for automation? On previous projects at the CMA testing has been performed manually and the CMA does not have access to any automated test tools in its environments. However, if the supplier has access to such tools and can automate any tests then we would be happy to discuss that option. 7. Do you have any preferred test tools for either performance testing or functional automation? The CMA does not currently use any test tools in any of its environments. As such we would look to the supplier to propose any suitable test tools that they have access to and could use to automate tests. 8. Have the user stories been written, and if so how many user stories are there? User stories have been written. There are currently 66 in the product backlog, of which 28 have been identified as required for MVP. 9. How many browsers do you expect to be tested eg Chrome/Firefox/Edge/Safari.... Given the online service can be accessed by anyone with online access we would expect it to be tested with all major browsers in order to pass through the service assessment. 10. What platforms need to be tested against eg – ipad/mobile/laptop/desktop, how many operating systems (for example just the latest OS or three versions back). Given the online service can be accessed by anyone with online access we would expect it to be tested across all platforms and main operating systems. Backward compatibility with previous versions should also be tested, but we would look to the supplier to recommend the number of versions and rationale based on industry best practice and the Digital Service Standard. 11. Please confirm that the £50K budget is for the testing from September to early December 2021 only. And that if additional testing resources are required beyond early December, it will be with additional budget over and above the £50K. The budget of £50K is for test activities from late July through to early December 2021. If the project extends into 2022 then further resource will be procured if required. 12. Please confirm that the £50K budget is for the testing from September to early December 2021 only. And that if additional testing resources are required beyond early December, it will be with additional budget over and above the £50K. The budget of £50K is for test activities from late July through to early December 2021. If the project extends into 2022 then further resource will be procured if required. 13. We have recently responded to a requirement for Development Activities for the OIM. Can you please confirm if these Testing Activities will form part of the same programme and could be delivered by the same supplier? The testing activities form part of the same programme as the OIM Development opportunity previously published on the Digital Marketplace. Whilst we would no rule out using the same supplier, we would need to see clear evidence of a clear differentiation between the development and test teams to be sure that testing is completely independent and objective. 14. Regarding the requirement for Functional testing cross platform and browser compatibility testing – can you please say how many/which browsers and how many/which platforms. Also do you require testing for just the latest version/operating system or for more eg last three versions. Given the online service can be accessed by anyone with online access we would expect it to be tested with all major browsers in order to pass through the service assessment. We also expect it to be tested across all platforms and main operating systems. Backward compatibility with previous versions should also be tested, but we would look to the supplier to recommend the number of versions and rationale based on industry best practice and the Digital Service Standard. 15. Can you please confirm if testing activities can be delivered offshore? Whilst we don’t preclude the option of testing being performed offshore, a key element of the test resource requirement is for them to integrate with the delivery team that will include CMA resources and 3rd party supplier resources based in the UK. As the project is being delivered using an agile approach, we would expect the testers to participate in daily stand ups and end of sprint showcases, so any time zone differences would need to be accommodated. 16. Can you share any further details regarding the tech stack this solution is being developed on? The codebase may be a consideration in Security Testing. The technical solution is still to be finalised as the development resources are being procured via a separate contract. However, it is expected that the solution will use existing CMA technologies and be a cloud-based solution using Azure, SharePoint online and bespoke HTML web forms. 17. There is no specific mention of post-deployment support requirements under the “Problem to be Solved” section, though it is alluded to in others. Once the site has gone live will there be any need for penetration testing, or will that need be met by another supplier and/or outside of the scope of this budget and tender? The CMA has an existing contract with another supplier for penetration testing and it is expected that they will be used to provided that service. 18. You mention using a separate supplier for research. Will that supplier be responsible for Usability Testing as well? Further, under “Essential Skills and Experience” there is a request for “demonstrable and recent experience of testing services that meet both user needs and business objectives”, which suggests Usability Testing experience; but there is no mention of Usability Testing in the “Problem to be Solved” section, which outlines requisite activities within the scope of this engagement. Can you clarify? A user researcher resource is being sourced via a separate procurement. They will do some initial usability testing based on wireframes and the technical solution. We will expect the provider of the test resources to perform functional testing, regression testing and accessibility testing. This should ensure that the solution works across multiple browsers and platforms and meets accessibility standards.