Verification Architecture
We shall examine key points of software verification process. Enterprise Architect is usually responsible for maintenance of the test architecture shown here.
1) Vertical red line delineates between projects and releases. There are three projects (A), (B) and ( C) that constitute the single release (Z). The high-level view shows each project involves a Unit Test (an individual activity when a developer verifies his/her piece of code) and a Feature Test (when a single feature is tested within a project team). Red line marks the point when agile iterations end. A group called System Test assumes responsibility to receive baton from project teams and to execute Cross Feature Test and then System Test. Golden Run is the very last execution pass that is expected to go smoothly. If defects show up during a Golden Run, the whole process is re-evaluated and repeated. The curved arrows at the bottom show the propagation of selected test cases from each previous test-type to the next one.
2) A thick blue line depicts a steep hip of accumulated defects within a phased approach. This very line is quite capable of decimating the whole software company. Management cost for thousands defects is quadrupled. Many organizations are unable to release any new features to customers and spent 100% of their effort on bug fixing. When a new bug arrives, how do you know whether this bug already been reported or it is a new bug? Duplicate resolution program could easily consume a large portion of engineering resources. Fixing defects close to the point of their origin has its apparent benefits. Red jigsaw line shows a sharp contrast with an agile approach, when defects are not accumulated but resolved quickly. In fact, defects that are fixed within an iteration have no corresponding stories. Another difficulty of a phased approach is balancing between capacities of testers and developers. Often testers find too many defects ,which developers are unable to fix. Or developers write too much code, which testers are unable to verify.
3) Within each iteration, test cases are written and executed. The specific regression for this iteration is executed also. A story is not accepted unless a corresponding test is successful. Iteration goal is satisfied when demo is done-done.
4) The goal of Iteration Hardening is to make sure all left-over issues, whatever fell from the wagon so-to-speak - are resolved. If a story is split of postponed, now is the time to test and accept it. Note that mature teams might not need a hardening iteration and skip the whole idea, which makes such teams are so much more effective.
5) An individual feature could run well by itself, however could show issues when run concurrently with other features. A Cross-Feature test aims to reveal compatibility and integration defects. The link below shows two features, first, “hand sanitizer,” and, second, “electric plug”. Each feature works well individually, however, their relative position does create a conflict. https://twitter.com/ughitsaaron/status/1255267863436431360?s=20
6) A multi-layered approach to software development is reflected in organization of the repository of test cases. There are several types of regression, which is opposite to a big-bang approach. Project-level regression differs from system regression. Feedback is quick, as defects are well-isolated and their resolution is well-focused. At any point of time, we know where we stand and we could reasonably estimate project parameters. At any point of time, a master code line has code that is shippable upon customer request.