Data format and view synthesis for free-viewpoint video streaming of super multiview video

Our goal is to propose and evaluate a new streaming data format that can be adapted to the limited bandwidth and capable of free-viewpoint video streaming using super multi-view video plus depth (MVD). Additionally, we proposed a view synthesis method for our data format. Given a requested free-viewpoint, we use the two closest views and corresponding depth maps to perform free-viewpoint video synthesis. The new data format consists of all views and corresponding depth maps in a lowered resolution, and the two closest views to the requested viewpoint in the high resolution. When the requested viewpoint changes, the two closest viewpoints will change, but one or both views are transmitted only in the low resolution during periods of large round-trip delay time. Therefore, the resolution compensation is required before view synthesis. Experimental results show that our proposed framework achieves view synthesis quality close to view synthesis using high resolution multi-view video plus depth.