The vast visual information are first broken down into several primitive visual features, or name... more The vast visual information are first broken down into several primitive visual features, or namely, feature maps.
In this paper, we first develop improved projective rectification-based view interpolation and ex... more In this paper, we first develop improved projective rectification-based view interpolation and extrapolation methods, and apply them to view synthesis prediction-based multiview video coding (MVC). A geometric model for these view synthesis methods is then developed. We also propose an improved model to study the rate-distortion (R-D) performances of various practical MVC schemes, including the current joint multiview video coding standard. Experimental results show that our schemes achieve superior view synthesis results, and can lead to better R-D performance in MVC. Simulation results with the theoretical models help explaining the experimental results.
Small screen sizes, limited bandwidth, and low computational power often prohibit streaming of hi... more Small screen sizes, limited bandwidth, and low computational power often prohibit streaming of high-resolution videos to mobile devices over a wireless network. Recent advances in interactive region-of-interest (IRoI) video streaming technology enable users to interactively control pan/tilt/ zoom, while provide bit-rate and complexity savings. One recent application of IRoI video streaming is ClassX developed at Stanford University. It offers an open-source experimental platform for interactive online lecture streaming. In this technical demonstration, we present ClassX Mobile, which extends the current ClassX system and delivers highquality interactive video to smartphones and tablets with multi-touch screens.
The ClassX open source project is a free experimental interactive video streaming platform design... more The ClassX open source project is a free experimental interactive video streaming platform designed for educators, researchers and software developers. With minimal infrastructure set-up, ClassX offers educational communities a cost-effective solution for online lecture delivery. Our goal is to encourage contributions from other researchers, developers and educators in building an open, cost-effective and state-of-the-art online education video viewing system for the general public.
Recent studies in signal detection theory suggest that the human responses to the stimuli on a vi... more Recent studies in signal detection theory suggest that the human responses to the stimuli on a visual display are non-deterministic. People may attend to different locations on the same visual input at the same time. To predict the likelihood of where humans typically focus on a video scene, we propose a new stochastic model of visual attention by introducing a dynamic Bayesian network. When describing the network of visual attention, the principle of the signal detection theory is introduced, namely, the position where a saliency response takes the maximim is the eye focusing position. Experimental results have demonstrated that our model performs significantly better in predicting human visual attention compared to the previous deterministic model.
This report proposes a new stochastic model of visual attention to predict the likelihood of wher... more This report proposes a new stochastic model of visual attention to predict the likelihood of where humans typically focus on a video scene. The proposed model is composed of a dynamic Bayesian network that simulates and combines a person's visual saliency response and eye movement patterns to estimate the most probable regions of attention. Dynamic Markov random field (MRF) models are newly introduced to include spatiotemporal relationships of visual saliency responses. Experimental results have revealed that the propose model outperforms the previous deterministic model and the stochastic model without dynamic MRF in predicting human visual attention.
Current view synthesis prediction (VSP) techniques for multiview video coding (MVC) rely on dispa... more Current view synthesis prediction (VSP) techniques for multiview video coding (MVC) rely on disparity-based view interpolation or depth-based 3D warping. The former cannot be applied to every camera view, whereas the latter may require coding of the depth information of a scene. To avoid these constraints, we propose an improved VSP-based MVC scheme based on the following three techniques: 1) view extrapolation, which allows VSP to be applicable to almost all camera views, 2) projective rectification, which improves the synthesis quality when neighboring camera planes are not parallel, and 3) synthesis bias correction, which uses the past synthesis biases to improve the synthesis quality of the current frame. Experimental results demonstrate that our scheme offers PSNR gains of up to 1.6 dB compared to the current MVC standard.
Low-complexity video encoding is important for mobile and power-sensitive applications. One way t... more Low-complexity video encoding is important for mobile and power-sensitive applications. One way to reduce encoding complexity is to drop frames at the encoder and perform motion-compensated frame interpolation at the decoder. We propose a method to estimate the quality of the interpolated frames at the decoder by transmitting a small amount of error control information in lieu of an omitted frame. Typically, this information is obtained from a projection of the frame on a suitable low-dimensional basis. The projection coefficients can be compressed by conventional techniques or, more efficiently, by Slepian-Wolf coding. Based on the quality estimate, the decoder can recognize and suppress occasional frames for which motioncompensated interpolation does not yield satisfactory picture quality. Experimental results demonstrate that our approach eliminates most interpolation artifacts and achieves much better visual quality at a negligible increase in bit-rate.
ClassX is an interactive online lecture viewing system developed at Stanford University. Unlike e... more ClassX is an interactive online lecture viewing system developed at Stanford University. Unlike existing solutions that restrict the user to watch only a pre-defined view, ClassX allows interactive pan/tilt/zoom while watching the video. The interactive video streaming paradigm avoids sending the entire field-of-view in the recorded high resolution, thus reducing the required data rate. To alleviate the navigation burden on the part of the online viewer, ClassX offers automatic tracking of the lecturer. ClassX also employs slide recognition technology, which allows automatic synchronization of digital presentation slides with those appearing in the lecture video. This paper presents a design overview of the ClassX system and the evaluation results of a 3-month pilot deployment at Stanford University. The results demonstrate that our system is a low-cost, efficient and pragmatic solution to interactive online lecture viewing.
Uploads
Papers by Derek Pang