I Built an Image to 3D Model App using LangGraph

Meta Description: How I built a full-stack image-to-3D pipeline using Gemini, Hunyuan3D-2, LangGraph, FastAPI, Trimesh, and WebGL.

I wanted to experiment with a problem that sounds simple:

Can a single 2D image be turned into a usable 3D model?

A single image gives us information about the visible surface, but says very little about the hidden geometry. The challenge isn't just generating a mesh; it is handling the entire pipeline around it.

So I built Core3D, a system that takes a single image, preprocesses it, adds geometric context, generates a 3D mesh, repairs and validates the geometry, and finally lets the user inspect the resulting .GLB model directly in the browser.

The end-to-end pipeline averages roughly 22 seconds in the project's benchmark.

Architecture

The application is split between a Next.js frontend and a FastAPI backend.

                ┌──────────────────────┐
                │    Next.js Frontend  │
                │      WebGL Studio    │
                └──────────┬───────────┘
                           │
                       HTTP / GLB
                           │
                ┌──────────▼───────────┐
                │     FastAPI API       │
                └──────────┬───────────┘
                           │
                    LangGraph Pipeline
                           │
        ┌──────────────────┼──────────────────┐
        │                  │                  │
        ▼                  ▼                  ▼
   Preprocessing      VLM Conditioning    3D Generation
     U2-Net               Gemini          Hunyuan3D-2
        │                  │                  │
        └──────────────────┼──────────────────┘
                           ▼
                    Mesh Processing
                        Trimesh
                           │
                           ▼
                      Final GLB

The frontend uses Next.js, React, TypeScript, Tailwind CSS, and Google's <model-viewer>. The backend uses FastAPI, LangGraph, LangChain, and Pydantic.

1. Preprocessing the Image

The first problem is background noise.

If the input image contains a table, wall, floor, or other objects, the 3D reconstruction model has to distinguish the actual subject from everything else.

I use rembg with U2-Net to isolate the foreground object.

Input Image
     ↓
U2-Net / rembg
     ↓
Foreground Mask
     ↓
Bounding Box
     ↓
Center + Pad
     ↓
1024 × 1024 RGBA

After segmentation, the subject is cropped from its bounding box, scaled with approximately 15% padding, centered, and resized to 1024 × 1024.

This gives the reconstruction model a much more consistent input regardless of how the original photograph was framed.

2. The Problem With a Single View

A single image only exposes one side of an object.

For example, a photograph of a chair might show the seat, front legs, and backrest while giving us no direct information about the rear structure.

The model therefore has to infer:

• Hidden surfaces
• Object thickness
• Rear geometry
• Structural connections
• Overall volume

Instead of relying only on the image, I added a multimodal conditioning step.

3. Using Gemini as a Geometric Conditioning Layer

The preprocessed image is sent to Gemini 3.1 Flash-Lite.

Gemini isn't responsible for generating the mesh.

Instead, it produces a compact description of the object's 3D structure, including its volume, materials, shape, and likely hidden geometry.

              Image
                │
                ▼
        Gemini Vision Model
                │
                ▼
     Geometric Description
                │
                ▼
       3D Reconstruction

This makes the vision-language model act as a conditioning layer for the 3D model.

For example, instead of giving the reconstruction model only:

"chair"

it can receive a more useful description such as:

"Minimalist wooden chair with a rectangular seat,
four tapered legs, solid rear support and smooth
wooden surfaces."

The implementation also has a fallback path so that if the multimodal request fails, the pipeline can continue using the user's prompt or a default description.

4. Generating the 3D Shape

Once the image has been prepared and conditioned, it is sent to Hunyuan3D-2 through a Hugging Face ZeroGPU Space.

The backend uses gradio_client to communicate with the remote inference service.

The current configuration includes:

Steps:              30
Guidance Scale:     5.5
Octree Resolution:  256
Num Chunks:         8000

The generated result is returned as a .GLB file.

The interesting part here is that the expensive 3D inference doesn't have to run directly inside the FastAPI process.

FastAPI
   │
   ▼
Hugging Face ZeroGPU
   │
   ▼
Hunyuan3D-2
   │
   ▼
Raw GLB

This keeps the application backend relatively lightweight while still exposing the model through a normal API.

5. Why the Raw Mesh Isn't the Final Mesh

A generated mesh isn't necessarily ready for rendering.

It can contain:

• Incorrect normals
• Inverted faces
• Small holes
• Unnecessary geometry
• Excessive polygon counts

So the generated GLB is passed through a separate mesh-processing stage using Trimesh.

Raw GLB
  ↓
Load Geometry
  ↓
Fix Inversions
  ↓
Fix Normals
  ↓
Fill Holes
  ↓
Decimate if Necessary
  ↓
Validate
  ↓
Final GLB

The pipeline uses Trimesh repair functions to fix inversions, normals, and holes.

It also applies polygon reduction when the mesh exceeds 45,000 faces.

This matters because a high-polygon model isn't necessarily a better model for an interactive browser application.

6. Validation and Retry Logic

I didn't want to blindly trust whatever geometry the model generated.

The pipeline tracks information such as:

face_count
is_valid
retry_count
max_retries

After mesh processing, LangGraph determines whether the result is acceptable.

                 ┌──────────────┐
                 │ Mesh Repair  │
                 └──────┬───────┘
                        │
                        ▼
                 Is Geometry Valid?
                    /         \
                  Yes          No
                   │            │
                   ▼            ▼
                  END          Retry
                               │
                               ▼
                        3D Generation

The current configuration allows one retry when validation fails.

This is one reason LangGraph fits the project well: the workflow isn't just a straight sequence of function calls.

It can make decisions and route execution accordingly.

7. LangGraph as the Orchestration Layer

The entire backend pipeline is represented as a state machine.

The shared state contains fields such as:

class MeshState(TypedDict):
    job_id: str
    input_image_path: str
    preprocessed_path: Optional[str]
    user_prompt: Optional[str]
    enhanced_prompt: Optional[str]
    raw_mesh_path: Optional[str]
    final_mesh_path: Optional[str]
    face_count: int
    is_valid: bool
    retry_count: int
    max_retries: int

The resulting workflow looks like:

START
  ↓
Preprocess
  ↓
Generate Context
  ↓
3D Inference
  ↓
Mesh Repair
  ↓
Validation
  ├── Valid → END
  └── Invalid → Retry

Each stage reads from and updates the same graph state.

That makes the system much easier to extend than a single large backend function.

8. FastAPI as the Model Gateway

The frontend communicates with the backend through a small API surface.

The main generation endpoint is:

POST /generate
Content-Type: multipart/form-data

The request contains:

image: binary file
prompt: optional string

The backend creates a job ID, stores the uploaded image, initializes the graph state, and executes the LangGraph pipeline.

When processing finishes, the final .GLB is returned.

Additional endpoints provide model retrieval and health checking:

GET /models/{filename}
GET /health

This keeps the frontend independent from the individual AI services involved in reconstruction.

9. Building a Browser-Based 3D Studio

Generating a 3D model isn't enough.

You also need a way to inspect it.

The /inspect page uses Google's <model-viewer> component to render the generated GLB directly in the browser.

The user can:

• Rotate the model
• Zoom
• Enable auto-rotation
• Change color
• Adjust roughness
• Adjust metallic properties
• Modify exposure
• Change transparency

Material properties are updated directly through the model-viewer's PBR material interface.

Generated GLB
      ↓
<model-viewer>
      ↓
WebGL Renderer
      ↓
Interactive 3D Model

This turns the output from a generated file into something the user can immediately inspect.

10. Browser-Side Model Persistence

Large 3D files don't need to be downloaded repeatedly.

The frontend stores generated GLB assets in IndexedDB.

Generated GLB
     ↓
 IndexedDB
     ↓
 Browser Reload
     ↓
Restore Local Blob
     ↓
Render Model

The application keeps the generated binary locally so that the inspection page can restore the asset after a reload without regenerating it.

11. Performance

The benchmark in the project shows approximately:

The numbers make one thing very clear:

3D generation is the bottleneck.

Almost all of the latency comes from the Hunyuan3D inference stage.

That means future optimization efforts would have a much larger impact if focused on the reconstruction model rather than the preprocessing pipeline.

Tech Stack

What I Found Most Interesting

The interesting part of an image-to-3D application isn't really the phrase "image to 3D."

There are several different problems hiding behind it:

2D Image
   ↓
Foreground Isolation
   ↓
Spatial Normalization
   ↓
Geometric Conditioning
   ↓
3D Reconstruction
   ↓
Mesh Repair
   ↓
Validation
   ↓
WebGL Rendering

Each stage solves a different problem.

The segmentation model removes irrelevant background information.

The vision-language model adds semantic information about geometry.

Hunyuan3D generates the actual mesh.

Trimesh makes that geometry more usable.

LangGraph coordinates the entire workflow.

And WebGL turns the result into something interactive.

What's Next?

The biggest improvement I'd like to explore is multi-view reconstruction.

A single image inherently limits what the model can know. Providing multiple views should reduce ambiguity around hidden geometry and improve structural accuracy.

Other improvements could include:

• Local GPU inference
• Texture generation
• Better mesh optimization
• Automatic quality scoring
• Multi-image input
• Persistent model history
• Additional export formats

Conclusion

Core3D started with a simple question:

Can a 2D image become a 3D model?

The interesting part was discovering how many systems are required to make that actually useful.

The final pipeline combines computer vision, multimodal reasoning, generative 3D reconstruction, geometry processing, stateful orchestration, API design, and browser-based WebGL.

          Single Image
               ↓
          U2-Net / rembg
               ↓
        1024 × 1024 Subject
               ↓
      Gemini Geometric Context
               ↓
          Hunyuan3D-2
               ↓
           Raw GLB
               ↓
        Trimesh Repair
               ↓
        Validation / Retry
               ↓
       Interactive WebGL
               ↓
         Final 3D Asset

The biggest takeaway for me was that generative AI becomes much more useful when it is treated as one component inside a larger engineering system.

The model generates the geometry.

The rest of the system makes that geometry usable.

Source Code

GitHub: https://github.com/iPrq/image-to-model