ISO/IEC TR 29119-11:2020
Software and systems engineering - Software testing - Part 11: Guidelines on the testing of AI-based systems
Abstract
This document provides an introduction to AI-based systems. These systems are typically complex (e.g. deep neural nets), are sometimes based on big data, can be poorly specified and can be non-deterministic, which creates new challenges and opportunities for testing them.
This document explains those characteristics which are specific to AI-based systems and explains the corresponding difficulties of specifying the acceptance criteria for such systems.
This document presents the challenges of testing AI-based systems, the main challenge being the test oracle problem, whereby testers find it difficult to determine expected results for testing and therefore whether tests have passed or failed. It covers testing of these systems across the life cycle and gives guidelines on how AI-based systems in general can be tested using black-box approaches and introduces white-box testing specifically for neural networks. It describes options for the test environments and test scenarios used for testing AI-based systems.
In this document an AI-based system is a system that includes at least one AI component.
This document explains those characteristics which are specific to AI-based systems and explains the corresponding difficulties of specifying the acceptance criteria for such systems.
This document presents the challenges of testing AI-based systems, the main challenge being the test oracle problem, whereby testers find it difficult to determine expected results for testing and therefore whether tests have passed or failed. It covers testing of these systems across the life cycle and gives guidelines on how AI-based systems in general can be tested using black-box approaches and introduces white-box testing specifically for neural networks. It describes options for the test environments and test scenarios used for testing AI-based systems.
In this document an AI-based system is a system that includes at least one AI component.
Additional information
Publication type | Technical Report |
---|---|
Publication date | 2020-11-27 |
Edition | 1.0 |
Available language(s) | English |
TC/SC | ISO/IEC JTC 1/SC 7 - Software and systems engineeringrss |
ICS | 35.080 - Software |
Pages | 52 |
File size | 2613 KB |
The following test report forms are related:
Share your publications
Learn how to share your publications with your colleagues, using networking options.
Payment information
Our prices are in Swiss francs (CHF). We accept all major credit cards (American Express, Mastercard and Visa, JCB and CUP), PayPal and bank transfers as form of payment.
Keep in touch
Keep up to date with new publication releases and announcements with our free IEC Just Published email newsletter.
Contact customer services
Please send your enquiry by email or call us on +41 22 919 02 11 between 09:00 – 16:00 CET Monday to Friday.