Video codec framework (VCF): Novel firmware architecture for video hardware

The ever growing need for higher computation power for video coding (e.g. MPEG2, H.264, HEVC) lead to hardware solution for video codec, which are controlled by local RISC controller in a typical chip. The firmware running on local controller uses simple sequential model for video processing from hardware due to lack of availability of real-time OS (RTOS) as well as tight on-chip memory budgets. This works well for simple video playback scenario. In case of video surveillance and infrastructure, which requires large number of video channels of lower resolutions or video conferencing market with higher number of slice processing results in much lower utilization of video hardware as inefficiencies becomes significant resulting in lower performance. The paper proposes novel approach by defining software based video codec framework (aka VCF) to enable multi-threading in local RISC controller in firmware. The proposed framework (aka VCF) defines model for controller firmware (in terms of software acceleration, static model for data flow, state diagram etc). The proposed VCF also defines simple primitive (e.g. threads, queues, semaphores, messaging, scheduling scheme etc) to enable it without using any RTOS. The proposed video codec framework (VCF) is independent of video standard. The C simulation of proposed framework shows large improvement (by reducing overheads by more than 80%) in overall performance at system level in complex use-case of multichannel, high frame-rate and high slice rate.