Systems Engineering process as outlined in the ISO/IEC 15288 standard and in the INCOSE SE Handbook includes the triad of processes:
– Verification – basically testing the system against its requirements
– Transfer – delivering the system to its actual operating environment
– Validation – testing the fitness of the system to provide the intended value to its users in the real-life operating conditions.
The Verify-Transfer-Validate pattern may be considered conspiracy of the Systems Engineers against the customers and the users:
1. The Verification usually happens under the controlled conditions – in the lab or in carefully orchestrated field tests, while the Validation happens in the real-life environment.
2. The Verification is performed by the professionals – systems engineers, T&E engineers, while the Validation is performed by the users – may be with the support of some engineers, may be not.
3. The Verification is performed when the system is fresh from its assembly and the engineers that perform it have the access to the insiders’ information and to the inner working of the systems, while in the Validation the systems is almost perfectly “black-boxed” and only its inputs and outputs are visible to the evaluators.
4. The Transfer (usually accompanied by the significant payment to the contractor) ensures the interests of the contractor.
Why so many systems that are verified fail their validation. There may be several reasons:
1. The Verification is against requirements that reflect the predicted use of the system and not its actual use. More so – the conditions that the system is used in change over time -both during the system’s development cycle and especially during the system’s long period of service.
2. The Verification is done against the simulations of the systems environment and not against the real one.
3. And the most interesting of all – when the system is put to use, there is no more System – Environment boundary but actually the ongoing change in the network of artifacts and people that try together to perform their missions, disturbed by the introduction of new set of artifacts.
When the System-Environment boundary is set in the beginning of the development cycle, the whole world of the users’ experience is actually “black-boxed” for the sake of reducing the complexity.
During the Validation phase this Black Box (may I say – the Pandora’s box?) is opened and the system is integrated with the gamut of entities that were “transparent” to the system’s developers – even if “stakeholders analysis” was really done by the contractor and not delegated to the customer, as I witnessed in many cases.
So I claim that the real Systems Integration happens after the system’s transfer to the users, during the Validation phase, and what we usually call Systems Integration is just the system’s assembly for the verification.
What’s ironic that there is almost no discussion about the systems’ fate after the transfer, at least not in the SE literature. The real Systems Integration is concealed from the inquiry.
May it be so that this SI is done not by professional SEs but by the users themselves?
Sanity Testing 