SIGGRAPH Asia 2026 Technical Papers

Sparse-view structured 3D reconstruction

CGGT: Curve-Grounded Geometry Transformer for Structured Curve Reconstruction from Unposed Sparse Views

From casually captured RGB images to compact, editable 3D parametric curves—without calibrated cameras, precomputed edge maps, or costly per-scene reconstruction.

Zhirui Gao Renjiao Yi Ruizhen Hu Yunfan Ye Chenyang Zhu Wei Chen Kai Xu
▣ Paper ↗ arXiv soon ⌘ Code ◇ Dataset
CGGT reconstructs structured 3D parametric curves from sparse unposed images

CGGT grounds persistent curve instances across sparse views, lifts them with predicted geometry, and produces clean CAD-style wireframes using lines, ellipses, and Bézier curves.

Overview

Pixels in. Editable curves out.

Recovering editable 3D parametric curves from 2D images is a fundamental challenge in computer graphics, bridging pixel-based perception and vector-based CAD modeling. Existing NeRF- and 3DGS-based methods often rely on dense calibrated views, precomputed 2D edge maps, and costly per-scene optimization, limiting their applicability to casually captured real-world inputs.

We propose CGGT, a Curve-Grounded Geometry Transformer that directly grounds 3D-consistent 2D curve instances in image space from sparse, unposed multi-view images. CGGT combines a geometry-aware transformer encoder for multi-view feature learning with a curve-aware masked-attention decoder for cross-view instance association. In one forward pass, it predicts camera parameters, dense depth maps, and instance-level 2D curve masks. These predictions are lifted into 3D and refined through fast parametric optimization to recover compact, editable curve primitives.

Trained only on the proposed Wireframe-100K synthetic dataset, CGGT generalizes to real-world images and separates persistent structural curves from view-dependent edges caused by silhouettes, textures, shadows, and appearance variations.

01

Curve grounding, not edge chasing

Instance-level queries identify the same structural curve across views instead of fitting every visible image edge.

02

Geometry-aware feed-forward inference

Camera, depth, appearance, and curve predictions share multi-view features, enabling sparse and unposed inputs.

03

Compact parametric output

Lifted observations initialize lines, ellipses, and Bézier curves that remain editable in downstream CAD workflows.

Method

A two-stage route from images to structure

CGGT first solves the better-constrained image-space association problem, then recovers continuous 3D geometry through lightweight optimization.

Overview of the two-stage CGGT pipeline
Phase I grounds multi-view curve instances and predicts geometry. Phase II lifts their pixels into 3D, initializes parametric primitives, and refines them with geometric consistency and regularization.
Phase I · Feed-forward

Curve-Grounded Geometry Transformer

  • Large unified encoder: VGGT geometry tokens query frozen DINOv2 appearance tokens.
  • Multi-scale pixel decoder: DPT-style fusion produces dense, depth-aware features.
  • Curve-aware mask decoder: 64 learnable queries predict cross-view masks and curve categories.
  • Joint outputs: camera parameters, depth maps, masks, and line/ellipse/Bézier labels.
Phase II · Fast refinement

Fast parametric optimization

  • Back-project grounded curve pixels with predicted depth and camera poses.
  • Initialize line, ellipse, and Bézier candidates with RANSAC fitting.
  • Optimize multi-view projection consistency over all curve instances.
  • Regularize endpoints and dominant angular relationships for clean, compact structure.

Wireframe-100K

A dataset for structured curve learning

Wireframe-100K couples realistic multi-view observations with parametric 3D wireframes, instance-level 2D curve masks, and persistent identities across views. It supports learning object-level curve priors instead of relying only on local image edges.

Access Wireframe-100K
Multi-view curve masks predicted by CGGT
Example multi-view annotations and predictions: CGGT maintains consistent curve identities while suppressing texture and silhouette edges.

Visual Results

Clean structural curves from synthetic and real images

The visual comparisons highlight cross-view consistency, suppression of appearance-induced edges, and compact parametric reconstructions.

Qualitative comparison on the ABC-NEF benchmark
Synthetic objects. CGGT suppresses texture, illumination, and view-dependent silhouette edges to recover coherent structural curves.
CGGT qualitative reconstruction results on real-world objects
Real objects. The model transfers from synthetic training data to casually captured images with shadows, reflections, and distracting appearance cues.

Citation

If you find CGGT useful

@inproceedings{gao2026cggt,
  title     = {CGGT: Curve-Grounded Geometry Transformer for Structured Curve Reconstruction from Unposed Sparse Views},
  author    = {Gao, Zhirui and Yi, Renjiao and Hu, Ruizhen and Ye, Yunfan and Zhu, Chenyang and Chen, Wei and Xu, Kai},
  booktitle = {SIGGRAPH Asia 2026 Technical Papers},
  year      = {2026}
}

Acknowledgment

This work is supported in part by the NSFC (62325211, 62132021, 62572477, 62522219, 62372457, 62322207), the Research Fund of Jiangsu Key Laboratory of AI for Industries (E6420016G8), the Young Furong Scholar Support Program of Hunan Province, and the National Natural Science Foundation of China (62402171).