CSBSR: Joint Learning of Blind Super-Resolution and Crack Segmentation for Realistic Degraded Images

Yuki Kondo, Norimichi Ukita
Toyota Technological Institute
IEEE Transactions on Instrumentation and Measurement (TIM) 2024
Paper Video Code arXiv Tech. Report
example

Crack segmentation challenges for synthetically-degraded images given by low resolution and anisotropic Gaussian blur. Our method (f) CSBSR succeeds in detecting cracks in the most detail compared to previous studies (d), (e). Furthermore, in several cases our method was able to detect cracks as successfully as when GT high-resolution images were used for segmentation (c), despite the fact that our method was inferring from degraded images.

Abstract

This paper proposes crack segmentation augmented by super resolution (SR) with deep neural networks. In the proposed method, a SR network is jointly trained with a binary segmentation network in an end-to-end manner. This joint learning allows the SR network to be optimized for improving segmentation results. For realistic scenarios, the SR network is extended from non-blind to blind for processing a low-resolution image degraded by unknown blurs. The joint network is improved by our proposed two extra paths that further encourage the mutual optimization between SR and segmentation. Comparative experiments with State of The Art (SoTA) segmentation methods demonstrate the superiority of our joint learning, and various ablation studies prove the effects of our contributions.

Video

CSBSR Overview

Technical Report of Additional Experiments

While CSBSR is proposed for crack segmentation, it is applicable to other similar problems. Here, CSBSR is applied to vessel detection in retinal images1.

1This experiment is a technical report showing the results of an additional experiment performed after our paper was accepted by IEEE TIM and was not published in the original paper.

BibTeX

@article{kondo2024csbsr,
        title={Joint Learning of Blind Super-Resolution and Crack Segmentation for Realistic Degraded Images},
        author={Kondo, Yuki and Ukita, Norimichi},
        journal={IEEE Transactions on Instrumentation and Measurement},
        year={2024},
        volume={73},
        number={},
        pages={1-16},
      }
Our previous work : CSSR [MVA'21]

Crack Segmentation for Low-Resolution Images using Joint Learning with Super-Resolution

Yuki Kondo, Norimichi Ukita
Toyota Technological Institute
17th international conference on machine vision and applications (MVA) 2021
(Oral Presentation, Best Practical Paper Award )
Paper Code

Abstract

This paper proposes a method for crack segmentation on low-resolution images. Detailed cracks on their high-resolution images are estimated by super resolution from the low-resolution images. Our proposed method optimizes super-resolution images for the crack segmentation. For this method, we propose the Boundary Combo loss to express the local details of the crack. Experimental results demonstrate that our method outperforms the combinations of other previous approaches.

BibTeX

@inproceedings{CSSR2021,
          title={Crack Segmentation for Low-Resolution Images using Joint Learning with Super-Resolution},
          author={Kondo, Yuki and Ukita, Norimichi},
          booktitle={International Conference on Machine Vision Applications (MVA)},
          year={2021}
        }