Adaptive Segmentation of Thermal Anomalies in Buildings Using Human-in-the-Loop Active Learning Capsule Networks

Authors

Yining Wen, University of AlabamaFollow
Kaiwen Chen, University of Alabama - TuscaloosaFollow

Abstract

Buildings account for 40% of energy consumption in the U.S., with significant energy losses stemming from poor insulation and leaks. To meet decarbonization goals by 2050, efficient methods for diagnosing and retrofitting thermal anomalies are essential. Infrared thermography (IRT), combined with emerging technologies like computer vision and deep learning, offers the potential for automating thermal anomaly detection and segmentation. However, challenges such as building diversity, scenario variations, and labor-intensive image annotation hinder model reliability and robustness. This study proposes a human-in-the-loop active learning approach to fine-tune the pretrained Capsule-based network (CapsLab) to enhance adaptability to new building scenarios efficiently. Using a Query-by-Committee (QBC) strategy, the method selects the most informative thermal images from a target dataset for expert verification and iterative model refinement. To streamline the expert annotation verification process, a weakly annotation strategy is introduced supporting human-in-the-loop training by Simple Linear Iterative Clustering (SLIC)-based superpixel segmentation and scribble prompts for efficient labeling. The pretrained CapsLab model is iteratively fine-tuned using these verified annotations to enhance segmentation performance. Three fine-tuning methods, including parameter freezing strategies, are evaluated for optimal results. This iterative workflow reduces annotation effort, improves model adaptability, and improves the precision of thermal anomaly segmentation, facilitating energy-efficient building retrofitting and maintenance.

The paper will be presented:

In-person

Primary U.N. Sustainable Development Goals (SDG)

Sustainable Cities and Communities - - Make cities and human settlements inclusive, safe, resilient and sustainable

Secondary U.N. Sustainable Development Goals (SDG)

Industry, Innovation and Infrastructure - - Build resilient infrastructure, promote inclusive and sustainable industrialization and foster innovation

Primary CIB Task Group OR Working commission

W086 – Building Pathology

Secondary CIB Task Group OR Working commission

W070 – Facilities Management and Maintenance

Recommended Citation

Wen, Yining and Chen, Kaiwen (2025) "Adaptive Segmentation of Thermal Anomalies in Buildings Using Human-in-the-Loop Active Learning Capsule Networks," CIB Conferences: Vol. 1 Article 241.
DOI: https://doi.org/10.7771/3067-4883.1830