2nd. Thermal Image Super-Resolution Challenge

Objective & Scope

The thermal image super-resolution (TISR) problem has become an attractive research topic in recent years, mainly due to the appealing results obtained with recent deep learning-based approaches. In order to define a common benchmark for evaluating the different contributions, the first challenge on TISR has been proposed at the PBVS-2020 workshop. Due to the success of that first challenge, and trying to keep improving the obtained results, this year the second challenge on TISR is proposed in the framework of the PBVS-2021 workshop.

The challenge stated for this year consists in obtaining super-resolution images at x2 and x4 scales from the given images.

Just the mid- and high-resolution images from last year dataset will be considered. Ground truth images for the x4 scale corresponds to the provided high-resolution images; in other words, each team should down-sample the given images by x4 and use these down-sampled images (by adding noise) as inputs to develop their solutions. Regarding the x2 super-resolution solution, it should be developed using as an input the given mid-resolution images acquired with the camera Axis Q2901-E and as an output, the corresponding high-resolution images, of the same scene, but acquired with the FLIR FC-632O camera. In other words, the x2 scale proposed solution should be able to tackle both problems, i.e., generating the super-resolution of the images acquired with the camera Axis Q2901-E camera; as well as mapping images from one domain (Axis Q2901-E camera) to another domain (FLIR FC-632O camera).

Winning Teams

Evaluation 1 (x4): SVNIT_NTNU team

Heena Patel*, Vishal Chudasama*, Kalpesh Prajapati*, Anjali Sarvaiya*, Kishor P. Upla*, Raghavendra Ramachandra+, Kiran Raja+ and Christoph Busch+

*:SVNIT, Surat, India, +:NTNU, Gjøvik, Norway

Evaluation 2 (MR2HR): ULB-LISA team

Feras Almasri, Thomas Vandamme, and Olivier Debeir

Universite Libre de Bruxelles, Belgium

Dataset

The dataset used for this second TISR challenge consists of the mid- and high-resolution images of last year's challenge. These images have been acquired at the same time with different cameras. Technical details are provided below in Table 1, some illustrations from each camera are depicted in Fig. 1.

Image Description   Brand Camera  FOV    Native Resolution  
Low (LR)+Axis Domo P12908mm160x120
Mid (MR)Axis Q2901-E9mm320x240
High (HR)FC-632O FLIR13mm640x512*
Table 1 - Thermal Camera Specification (* Cropped to 640x480) (+ images not used in this second challenge).


Figure 1 - Examples from each camera (+ images not used in this second challenge).

The original dataset contains a total of 1021 thermal images, which were simultaneously taken with each camera. All images are semi-paired. From all these images, 1001 are provided to the participants (951 for training and 50 for testing), while the remaining 20 are used for evaluating the results from the different teams (remember that from the given dataset just mid- and high-resolution images are needed, low-resolution images are also included since they are a part of the original dataset).

 

Evaluation

PSNR and structural similarity (SSIM) measures are going to be computed over a small set of images left for evaluating the performance of the proposed solution. Two kinds of evaluations are going to be performed. For the first evaluation, a set of 10 single down-sampled and noisy images will be shared for traditional evaluation as shown in Fig. 2. For the second evaluation a set of 10 real images, from the MR dataset, will be shared. In this case, the obtained SR images will be compared with the corresponding real High-Resolution semi-paired images, as shown in Fig. 3. For the second evaluation, HR images are going to be registered with the computed SR images in order to compute PSRN & SSIM metrics.


Figure 2 - First evaluation diagram (x4 for high).


Figure 3 - Second evaluation diagram* (x2 for MR2HR).

 

*Due to the semi-paired nature, for a fair comparison, the evaluation will be performed over a small region (50% of image size) centered in the image.

 

  • Evaluation 1: 10 noisy down-sampled images, from HR camera, will be provided to compute the corresponding SR. Bicubic function should be used for down-sampling at scale of 4 and Gaussian noise** with mean=0 and sigma=10 should be considered. Results will be evaluated as follows:
    where EVAL is PSNR & SSIM measures and N is the number of validation images.

    ** On python for Gaussian noise, use np.random.normal(mean, sigma, img.shape)

  • Evaluation 2: MR images as provided by the camera will be shared; this evaluation will be performed just for the x2 scale. The average evaluation value will be computed as follows:
    where EVAL is PSNR and SSIM measures, and N is the number of validation images (note that EVAL will be applied just on a part of the image to avoid regions without overlap due to the bias of the cameras).

 

Scale   Evaluation1 (SISR x4)  Evaluation2 (SISR MR2HR)
x2---PSRN/SSIM
x4PSRN/SSIM---
Table 2 - Evaluations measures.

 

Note: The super-resolution (SR) results must be submitted in a zip file together with a short description of the proposed approach to <pbvs21.tisr.challenge@gmail.com>. The approach with the highest performance in most evaluation metrics will be the winner of the challenge.

Benchmark

Feel free to check the 1st. Thermal Image Super-Resolution Challenge here for more information.

For benchmark, please check Rivadeneira et al. (2020). Thermal Image Super-Resolution Challenge – PBVS 2020. In The 16th IEEE Workshop on Perception Beyond the Visible Spectrum on the Conference on Computer Vision and Pattern Recongnition (CVPR 2020) (Vol. 2020-June, pp. 432–439) paper HERE.

Important Dates

  • Registration open & dataset released: January 18, 2021
  • Evaluation images distributed: March 2, 2021
  • Deadline for challenge & result submitted: March 12, 2021
  • Winner announcement: June 19, 2021

Register

 

Contact Us

Rafael Rivadeneira

Guayaquil, Ecuador

rrivaden@espol.edu.ec