The importance of full target environment simulation tests for architecture validation

Proposed architectural advancements focus on solutions for known deficiencies. Validation of the proposals sometimes is limited to simple scenarios where the deficiency exists. Due to simulation overhead, the validation testing may be limited to small test scripts, and may not include the entire target environment in which the proposed architectures would be eventually applied. A case study is presented regarding several proposals for Speculative Execution. In each proposal, simulation testing is limited to small test scripts executing as stand-alone binaries. No provision is made to execute these binaries within the context and control of a host operating system, including interaction with external hardware and software components. By avoiding the simulation of a complete target environment, key issues are not addressed that could repudiate the validity of these proposals. This paper hopes to impress upon the reader the importance of architecture validation through rigorous testing by simulation of the anticipated target environments. The lessons learned in this case study presented in this paper can be generalized and applied to other micro-architecture proposals.