Blurry Multi-View 3D Reconstruction Based on Neural Implicit Fields

Three-dimensional reconstruction from two-dimensional images has long been a fundamental challenge in computer vision and computer graphics, with applications spanning autonomous driving, robotics, and virtual reality. Traditional methods rely on multi-view stereo matching or depth estimation techniques, but recent advancements in deep learning have introduced neural implicit representations, which offer high-fidelity scene reconstruction by encoding geometry and appearance in continuous functions. However, most existing methods assume ideal input conditions, ignoring the practical challenges posed by motion-blurred images captured in real-world scenarios. This limitation significantly degrades reconstruction quality, leading to overly smoothed surfaces and missing geometric details. To address this gap, we introduce Deblur-NeuS, a novel approach for reconstructing sharp geometric surfaces from blurry multi-view images using neural implicit fields.

Introduction

Neural implicit surface-based reconstruction methods have gained prominence due to their ability to represent complex geometries with high fidelity. Techniques such as Neural Radiance Fields (NeRF) and its successors leverage volumetric rendering to synthesize novel views and extract detailed surfaces. However, these methods struggle when processing blurry inputs, which are common in real-world scenarios due to camera motion or object movement during exposure. Blur introduces ambiguities in geometry and appearance, making it difficult for neural networks to recover accurate surfaces. Existing solutions either focus on ideal input conditions or apply post-processing deblurring, which often fails to preserve geometric consistency.

Deblur-NeuS tackles this problem by explicitly modeling the blurring process during training, enabling the network to learn a sharp implicit representation despite blurry inputs. The method incorporates two key innovations: a blur kernel prediction module that simulates the motion blur effect and an implicit displacement field that refines the geometric surface representation. Additionally, sparse point cloud supervision is introduced to enhance geometric accuracy. During inference, the blur-related modules are removed, allowing direct extraction of high-quality surfaces.

Background and Related Work

Neural Implicit Surface-Based 3D Reconstruction

Neural implicit representations encode 3D shapes as continuous functions learned by deep neural networks. Early approaches like Occupancy Networks and DeepSDF model surfaces as occupancy or signed distance functions (SDF), enabling flexible shape representation. Subsequent methods, such as NeuS and VolSDF, integrate SDF-based rendering with volumetric techniques to improve reconstruction quality. These methods excel at capturing fine details but assume noise-free inputs, limiting their applicability to real-world blurry images.

Recent extensions, including HF-NeuS and PET-NeuS, enhance detail preservation through hierarchical optimization or hybrid architectures. However, none explicitly address motion blur, which distorts both geometry and texture. Deblur-NeuS bridges this gap by incorporating blur modeling directly into the training pipeline.

Neural Radiance Fields for Blurry Inputs

NeRF-based approaches have also explored handling blurry inputs. Deblur-NeRF introduces deformable sparse kernels to simulate motion blur, improving novel view synthesis. Follow-up works like PDRF and DP-NeRF refine this idea with progressive deblurring or physical priors. However, these methods primarily focus on improving rendered image quality rather than geometric accuracy. BAD-NeRF jointly optimizes camera trajectories and blur parameters but remains computationally intensive.

Deblur-NeuS distinguishes itself by prioritizing surface reconstruction quality. By combining blur-aware rendering with explicit geometric supervision, it achieves superior detail recovery compared to purely radiance-based approaches.

Methodology

Overview

Deblur-NeuS operates in two phases: training and inference. During training, the network learns to reconstruct blurry views while simultaneously recovering a sharp implicit surface. The process involves three main components:

Blur Kernel Prediction Module: Predicts per-pixel sparse blur kernels to simulate motion blur.
Implicit Surface and Displacement Fields: Models both the sharp base surface and the motion-induced deformations.
Sparse Point Cloud Supervision: Provides explicit geometric constraints to refine surface details.

At inference time, the blur-related modules are discarded, leaving only the sharp implicit surface representation for extraction.

Blur-Aware Neural Rendering

Motion blur arises when objects or cameras move during exposure, causing pixels to integrate light from multiple scene points. Deblur-NeuS models this effect by predicting a set of sparse blur kernels for each pixel. These kernels define how neighboring rays contribute to the final blurred color. The module predicts not only kernel weights but also ray origin and pixel offsets, enabling accurate simulation of 3D motion effects.

To prevent unrealistic distortions, an alignment loss ensures that one of the predicted rays remains close to the original input ray. This constraint maintains geometric consistency while allowing the network to learn plausible blur patterns.

Implicit Displacement Field for Surface Refinement

Blurred surfaces exhibit complex deformations that cannot be captured by a single SDF. Deblur-NeuS addresses this by decomposing the surface into a base SDF and a displacement field. The base SDF represents the sharp geometry, while the displacement field models how surface points move during exposure. By combining these fields, the network reconstructs a temporally averaged “blurred” surface that matches the input images.

The displacement field is regularized to prevent excessive deformations, ensuring that the learned base SDF remains geometrically plausible. This decomposition allows the network to disentangle motion effects from static geometry, leading to sharper reconstructions.

Geometric Supervision with Sparse Point Clouds

Neural implicit surfaces often suffer from geometric ambiguities when trained solely with photometric losses. To mitigate this, Deblur-NeuS incorporates sparse 3D point clouds obtained from structure-from-motion (SfM) as additional supervision. These points, filtered to remove outliers, provide direct constraints on the surface location.

A key innovation is the use of visibility-aware SDF loss, which accounts for occlusions and varying point densities across views. This loss ensures that the reconstructed surface aligns with the sparse geometry while preserving fine details not captured by the point cloud.

Experiments and Results

Datasets and Implementation

Deblur-NeuS is evaluated on the DTU and BlendedMVS datasets, which contain multi-view images with ground-truth 3D scans. Blur is synthetically introduced by convolving images with linear filters. The network is trained on blurry inputs but evaluated on both reconstruction quality and novel view synthesis.

Ablation Study

Ablation experiments validate the contributions of each component:

• The blur kernel module significantly improves both rendering (PSNR) and reconstruction (Chamfer distance) metrics.

• The displacement field further enhances geometric accuracy, particularly for thin structures.

• Sparse point cloud supervision reduces surface noise and improves detail recovery.

The full model outperforms baseline NeuS by a large margin, demonstrating the effectiveness of the proposed modules.

Comparative Analysis

Deblur-NeuS is compared against state-of-the-art methods including NeuS, VolSDF, and HF-NeuS. Quantitative results show consistent improvements in Chamfer distance, PSNR, and SSIM across all test scenes. Qualitatively, the method recovers fine details such as texture patterns and sharp edges that are lost in baseline reconstructions.

Notably, Deblur-NeuS handles complex scenes with rich backgrounds more robustly than prior work. For example, it accurately reconstructs the feathers of a bird model and the intricate cracks on a statue, whereas other methods produce overly smoothed surfaces.

Limitations

The main limitation is increased training time due to the additional modules. However, inference speed remains comparable to baseline methods. Future work could explore more efficient blur modeling or adaptive kernel strategies.

Conclusion

Deblur-NeuS advances the state of the art in neural implicit surface reconstruction by explicitly addressing motion blur. By integrating blur modeling, displacement fields, and geometric supervision, the method achieves high-quality surface recovery from blurry inputs. Experimental results demonstrate significant improvements over existing techniques, particularly in preserving fine details. This work opens new directions for handling real-world imperfections in 3D reconstruction, with potential applications in robotics, augmented reality, and digital heritage preservation.

doi.org/10.19734/j.issn.1001-3695.2024.03.0166

Was this helpful?

0 / 0