Structure Guided Global and Local Attention Transformer for Image Inpainting of Obscured Ships in Maritime Surveillance

Abstract

To ensure the safety of maritime traffic, it is crucial for cameras to capture a complete appearance of moving vessels. However, in real marine environments, ships are often obscured by objects such as rocks or buoys. To overcome this challenge, this paper proposes a specialized Structure Guided Global and Local Attention Transformer (SGGLAT) for restoring the appearance of obscured ships. The SGGLAT contains two networks: an edge generation network and an image inpainting network. The Edge Global and Local Attention Transformer (EGLAT) module within the edge generation network fuses the information of global and local information across the image to generate a more accurate edge map. The Texture Global and Local Attention Transformer (TGLAT) module embedded in image inpainting network focuses on the contours of the edge map to synthesize textures, so that it can address the issue of smoothed textures presented in previous models that only used edge maps as additional input data. Additionally, we design an Active Masked Patch Removal Algorithm (AMPRA) that progressively erases the mask to restore the image, thereby enhancing the model’s performance in image restoration. Our experiments were conducted based on the custom ship image dataset developed by MGT, a corporation engaged in developing maritime control systems in South Korea. The qualitative and quantitative experiments on this data demonstrate that our model produces visually more plausible inpainting results, outperforming state-of-the-art models. Authors