Performance evaluation of an integrated instrumentation system

With increasing sophistication of parallel tool development technologies, runtime data collection and management activities are receiving greater attention from tool developers. We define an instrumentation system (IS) as a set of modules and functions that collect runtime data from concurrent processes and support an integrated tool environment by sharing this data along with specific control information. Presently, it is not standard practice to formally evaluate the performance and functionality of a tool early in its development. Usability and efficiency studies of prototypical tools are emerging to alleviate this situation. However; the underlying IS is removed from the end-user and is part of system infrastructure, thus necessitating more rigorous evaluation. We are developing a structured approach to plan, design, model, realize, and test an IS, called the Vista IS, to appropriately address these issues