Embedding online test and monitoring features in real time hardware systems

This paper presents several test and monitoring techniques that have been deployed in some sub-systems of the CMS electromagnetic calorimeter readout electronics. The embedded online testability features of the Selective Readout Processor boards and of the endcap Trigger Concentrator Cards greatly simplified functional validation of the real-time hardware, facilitated development of automated production test benches and played an important role during the commissioning phase of the ECAL readout system. The deployed test and monitoring mechanisms as well as the used remote firmware control schemes are generic enough and look worth to be applied to a wide range of real-time systems with the FPGA-based hardware.