3D Focus Image Stacks

We developed a novel multi-view image noise reduction algorithm for camera array using a data structure called 3D focus image stacks with which disparity map (from the target view) can be efficiently constructed, and preliminary denoising can be performed. To further refine image qualities at certain “un-reliable” pixels, we propose a novel detection method and apply an adaptation of non-local patch matching scheme to correct the values of these pixels. Experiment results indicate that this proposed multi-view denoising algorithm out-performs existing multi-view de-noising algorithms while consuming less computing time and potentially suitable for real-time applications.

Preliminary experimental results of the proposed algorithm are summarized here. We used four dataset from different database for our evaluation. For all datasets, white Gaussian noise with noise level s is added. In our experiments, we assume all datasets have maximum disparity of 15 between adjacent views. We evaluate the quality of denoising using peak signal-to-noise ratio (PSNR). We ran the algorithm on Intel(r) Core(tm) i7-4700MQ CPU (2.40GHz) and used MATLAB(r) to implement all the algorithms. For the efficiency of evaluation, “Knight”, “Tarot”, and “Truck” have been resized to 256×256, 256×256, and 400×300, respectively. “Ohta” remains unchanged and is of size 288×384. For subjective visual quality, we enlarge the images and select particular regions for comparison in the figure below. From the figure, we can observe that the proposed method shows great preservation of edges and textures, such as the whiteboard in “Ohta” and walls in “Knight”, while both NLM and BM3D tend to smooth or blur these regions. As a result, both quantitative measurements and visual inspections reveal that our proposed method achieves great denoising quality compared to single view denoising algorithms.


Fig 1. Comparison between proposed denoising scheme, the method in [1], NLM, and BM3D (s = 20)


A. Kanhere, K. Van Grinsven, C.-C. Huang, Y.-S. Lu, J. Greenberg, C. Heise, Y. H. Hu and H. Jiang, “Multi-camera laparoscopic imaging with tunable focusing capability,”IEEE/ASME Journal of Microelectromechanical Systems, 23 (6), pp. 1290 – 1299, 2014.


[1] M. Miyata, K. Kodama, and T. Hamamoto. “Fast multiple-view denoising based on image reconstruction by plane sweeping.” in Proc. IEEE Vis. Commun. Image Process., Dec. 2014, pp. 462-465.


Algorithms for Functional Emulations of Single Camera Operations Using a Multicamera Array

Here, we show further development of the use of 3D focus image stacks to allow for imaging at multiple focal planes. A 3D focus image stack is a 3D matrix consisting of all pixels from all cameras aligned against the image coordinates of a reference image for a given disparity value d. Each disparity value d corresponds to the focal plane at the depth associated with d, and if an object is on that plane, the corresponding pixel will appear clear in the multi-focus image, as the following figure shows.


Fig 2. Illustration of multi-focus images at different disparity values (depths), showing 2D visualization of 3D focus image stacks.

Fig. 3 shows an example of texture map of a multi-view dataset, where bright colors represent high textures and dark colors represent low textures.


Fig 3. (a) noisy image, (b) texture map.


Fig 4. Comparison of disparity estimation methods: (a) Taniai et al. [1]; (b) Lee et al. [2]; (c) Klaus et al. [3]; (d) Miyata et al. [4]; (e) Zhou et al. [5]; (f) proposed.

Since multi-view denoising is one of our primary goals, our algorithms are compared against others in Fig. 5.


Fig 5. Qualitative comparison of different denoising methods when sigma = 20.


[1] T. Taniai, Y. Matsushita, Y. Sato, and T. Naemura, “Continuous stereo matching using local expansion moves,” arXiv preprint arXiv:1603.08328, Mar. 2016.

[2] S. Lee, J. H. Lee, J. Lim, and H. I. Suh, “Robust stereo matching using adaptive random walk with restart algorithm,” Image and Vision Computing, vol. 37, 2015, pp.1-11.

[3] A. Klaus, M. Sormann, and K. Karner, “Segment-based stereo matching using belief propagation and a self-adapting dissimilarity measure,” in Proc. 18th IEEE Int. Conf. Pattern Recognit., vol. 3, 2006, pp. 15-18.

[4] M. Miyata, K. Kodama, and T. Hamamoto, “Fast multiple-view denoising based on image reconstruction by plane sweeping,” in IEEE Conf. Visual Commun. Image Process., Dec. 2014, pp. 462-465.

[5] S. Zhou, Y. H. Hu, and H. Jiang, “Patch-based multiple view image denoising with occlusion handling,” in Proc. IEEE Int. Conf. Accoustic, Speech, Signal Process. (ICASSP), 2017, pp. 1782-1786.

[6] L. Zhang, S. Vaddadi, H. Jin, and S. Nayar, “Multiple view image denoising,” in Proc. IEEE Conf. Comput. Vision Pattern Recognit., Jun. 2009, pp. 1542-1549.