Source: Dev.to

Source: Dev.to

Architectural drawings already contain everything needed to build cabinet layouts:

• Dimensions
• Cabinet placements
• Appliance spacing

But most of this information lives in PDF drawings that are designed for humans, not machines.

When cabinet designs move into production, someone usually has to manually:

1. Interpret the drawing
2. Rebuild the layout in CAD
3. Generate a 3D model
4. Verify measurements

That process is slow and repetitive.

We recently built a system that automates this workflow by converting cabinet drawings directly into structured 3D data.
Read more about it here: Converting cabinet drawings directly into structured 3D data.

The interesting part wasn’t the AI model itself—it was building a reliable pipeline that makes the automation work.

System Architecture

The system converts cabinet drawings into 3D models through a multi‑stage pipeline. Each stage tackles a specific problem in the workflow, and modularizing the process makes debugging and accuracy improvements easier.

flowchart TD
    A[PDF Drawing] --> B[PDF → High‑Resolution Images]
    B --> C[View Region Detection]
    C --> D[Cabinet Detection (YOLO)]
    D --> E[Measurement Extraction (LLM + OCR)]
    E --> F[Coordinate Mapping]
    F --> G[3D Geometry Generation]
    G --> H[AutoCAD DWG Export]

Pipeline Overview

1. PDF Drawing – Source file containing the cabinet layout.
2. PDF → High‑Resolution Images – Convert each page to raster images for visual processing.
3. View Region Detection – Identify the portion of the image that contains the cabinet view.
4. Cabinet Detection (YOLO) – Use a YOLO model to locate individual cabinets within the view.
5. Measurement Extraction (LLM + OCR) – Combine a large language model with OCR to read dimensions and annotations.
6. Coordinate Mapping – Translate extracted measurements into a consistent coordinate system.
7. 3D Geometry Generation – Build the 3D model based on the mapped coordinates and measurements.
8. AutoCAD DWG Export – Output the final model in DWG format for downstream CAD workflows.

Step 1 – Converting PDF Drawings to Images

Architectural PDFs are not ideal inputs for computer‑vision models because they contain a mix of vector data, annotations, layers, and text.

To simplify processing, convert each page to a high‑resolution PNG image (300 DPI). Higher resolution improves:

• Text extraction accuracy
• Detection performance
• Line segmentation

Small‑dimension labels become unreadable at lower resolutions, so image quality matters more than expected.

Step 2 — View Region Detection

A single architectural sheet usually contains multiple views:

• Floor plans
• Elevations
• Section views
• Cabinet details

Processing the entire page creates too much noise. Instead, we segment the sheet into visual regions and classify them. The system prioritizes the base floor plan, which typically contains cabinet‑placement information. This step reduces false detections later in the pipeline.

Step 3 – Cabinet Detection Using YOLO

After the relevant region has been identified, we run an object‑detection model.
We trained a YOLO model to detect cabinet‑related objects such as:

• Base cabinets
• Upper cabinets
• Tall cabinets
• Appliances

Detection Output

Detections with confidence scores below a predefined threshold are filtered out before proceeding to the next stage. This step establishes the exact locations of cabinets within the layout.

Step 4 – Extracting Measurements

Detection tells us where cabinets are, but not how big they are. Cabinet drawings typically include dimension annotations such as:

30"
2'-6"
34 1/2"

These values may be rotated, overlap other text, or be linked to the cabinet via leader lines. Traditional OCR often fails in these cases, so we combine OCR with a vision‑enabled LLM.

Workflow for each detected cabinet

1. Crop the region surrounding the cabinet.
2. Send the cropped image to a vision model.
3. Request structured measurements from the model.

Example model output

{
  "cabinet_type": "base",
  "width": 36,
  "height": 34.5,
  "depth": 24
}

Validation

To reduce errors, we apply validation rules after extraction. If any measurement falls outside the expected range for a given cabinet type, the result is flagged for manual review.

Step 5 — Coordinate and Scale Detection

Architectural drawings often include scale references, for example:

1/8" = 1'-0"

If the scale isn’t interpreted, cabinet positions remain expressed in pixel units.
The system therefore:

1. Detects the scale marker in the drawing.
2. Converts pixel distances to real‑world measurements using the identified scale.
3. Assigns each cabinet an X/Y/Z position relative to the drawing’s origin.

This process enables the layout to be reconstructed accurately in 3‑dimensional space.

Step 6 — Generating the 3D Layout

To turn the detected cabinets, measurements, and real‑world coordinates into a usable model, we generate 3D geometry and display it with a Three.js viewer.

What we use

• Cabinet detections – bounding boxes and class labels from the detection model.
• Measurements – width, height, depth, and any custom dimensions extracted from the point cloud.
• Real‑world coordinates – transformed from image space to the building’s coordinate system.

How it works

1. Create parametric objects – each cabinet becomes a Three.js BoxGeometry (or a more detailed custom mesh) whose dimensions are set from the measured values.
2. Position the objects – using the real‑world coordinates, we place each cabinet in the scene so that the spatial relationships match the original floor plan.
3. Add visual cues – optional wireframes, color‑coding by class, or labels to help architects spot errors quickly.
4. Render the scene – a lightweight viewer lets users rotate, pan, and zoom to inspect the layout.

Why this step matters

• Validation – architects can instantly see whether the pipeline produced a plausible arrangement before committing to full‑scale modeling.
• Error correction – mis‑detections or inaccurate measurements become obvious, allowing quick manual adjustments.
• Workflow acceleration – even a rough 3D preview cuts down the time spent on repetitive manual modeling, moving the process toward semi‑automation rather than full automation.

Note: The goal isn’t a perfect, production‑ready model; it’s a fast, visual sanity check that reduces the amount of manual work required later.

Step 7 — Exporting to AutoCAD

The final stage converts the generated geometry into DWG files. Using the AutoCAD SDK, the system exports:

• Cabinet blocks
• Correct dimensions
• Real‑world coordinates

If the upstream data is correct, the export works reliably. Interestingly, this stage turned out to be one of the simplest parts of the system; most of the complexity lies in interpreting drawings correctly.

Challenges We Encountered

1. Drawings Are Inconsistent
   Architectural drawings vary widely depending on the designer. Annotation styles, measurement formats, and layout conventions are rarely standardized. The system needs to handle a wide range of variations.

2. Measurement Ambiguity
   Dimension labels are not always placed directly next to objects. They may refer to multiple cabinets or entire cabinet groups. Resolving these relationships requires contextual reasoning.

3. Legacy Drawings
   Older drawings often lack digital scale markers or use outdated notation, making automatic scale detection difficult.

(The article continues with further challenges, solutions, and future work.)

Scanned Drawings Introduce Additional Problems

• Blurred lines
• Noisy backgrounds
• Overlapping annotations

These reduce detection accuracy significantly.

Current System Status

The system is currently an MVP under active development.

Performance Highlights

• Strong for:

  • Clean digital drawings
  • Modern architectural layouts

• Edge cases:

  • Scanned plans
  • Dense dimension annotations
  • Complex sheet compositions

Summary

Even with these limitations, the system already provides meaningful time savings. Architects can start with a generated layout and correct it instead of creating it from scratch.

Final Thoughts

Projects like this highlight an important lesson about AI systems.

The hardest part usually isn’t the model.
It’s designing the workflow around it.

Solving this problem required combining:

• Document processing
• Computer vision
• LLM interpretation
• Geometry generation
• CAD integration

Individually, these technologies are powerful.
Together, they create a system capable of automating a real engineering workflow.