Skip to content
The ATC
Log in
What we do
Our capabilities
AI & DataAutomationCloudConsulting & EngineeringData CenterDigitalImplementation ServicesIT Spend OptimizationLab HostingMobilityNetworkingSecurityStrategic ResourcingSupply Chain & Integration
Industries
EnergyFinancial ServicesGlobal Service ProviderHealthcareLife SciencesManufacturingMedia & GamingPublic SectorRetailSports & EntertainmentUtilities
Learn from us
Hands on
AI Proving GroundCyber RangeLabs & Learning
Insights
ArticlesBlogCase StudiesPodcastsResearchWWT Presents
Come together
CommunitiesEvents
Who we are
Our organization
About UsOur LeadershipSponsorshipsLocationsSustainabilityNewsroom
Join the team
All CareersCareers in AmericaAsia Pacific CareersEMEA CareersInternship Program
Our partners
Strategic partners
CiscoDell TechnologiesHewlett Packard EnterpriseNetAppF5IntelNVIDIAMicrosoftPalo Alto NetworksAWSGoogle CloudVMware
What we do
Our capabilities
AI & DataAutomationCloudConsulting & EngineeringData CenterDigitalImplementation ServicesIT Spend OptimizationLab HostingMobilityNetworkingSecurityStrategic ResourcingSupply Chain & Integration
Industries
EnergyFinancial ServicesGlobal Service ProviderHealthcareLife SciencesManufacturingMedia & GamingPublic SectorRetailSports & EntertainmentUtilities
Learn from us
Hands on
AI Proving GroundCyber RangeLabs & Learning
Insights
ArticlesBlogCase StudiesPodcastsResearchWWT Presents
Come together
CommunitiesEvents
Who we are
Our organization
About UsOur LeadershipSponsorshipsLocationsSustainabilityNewsroom
Join the team
All CareersCareers in AmericaAsia Pacific CareersEMEA CareersInternship Program
Our partners
Strategic partners
CiscoDell TechnologiesHewlett Packard EnterpriseNetAppF5IntelNVIDIAMicrosoftPalo Alto NetworksAWSGoogle CloudVMware
The ATC
Physical AIAI Assistants and AgentsComputer VisionData Strategy and ArchitectureApplied AIGenAIAI & DataATCApplied ResearchUtilitiesData AnalyticsWhat we doSupply Chain and Integration Services
WWT Research • Applied Research Report
• November 17, 2026 • 26 minute read

Intelligent Extraction from Technical Drawings

Building a bill of materials (BOM) by hand from dense CAD drawings can take two to eight days per drawing set. WWT's R&D team built an AI pipeline that reads CAD-vector water and sewer drawings, drafts a traceable BOM in minutes and cuts prep time by an estimated 60% to 80% per sheet.

In this report

  1. Executive summary
  2. Motivation for technical drawings extraction
  3. Overview of existing solutions
  4. Dataset description
  5. Experiment methodology
    1. Component detection stage
    2. Text extraction stage
    3. Geometry extraction stage
    4. Label and dimension association stage
    5. Cross-view matching stage
    6. BOM generation
  6. Confidence scoring and explainability
    1. Confidence scoring
    2. Provenance tracing
    3. Human in-the-loop review
    4. Synthetic dataset
    5. Overall BOM metrics
  7. Results
    1. Overall BOM results
    2. Benchmarking against VLM baseline performance
  8. Business use cases and value 
  9. Recommendations for future work
  10. Appendices
    1. Appendix A. Dataset reference assets
    2. Appendix B. Dimension type classification
    3. Appendix C. Confidence-scoring weights
    4. Appendix D. Human-in-the-loop review workflow
    5. Appendix E. Synthetic dataset generation process
    6. Appendix F. BOM matching internals
    7. Appendix G. Per-drawing BOM results

Executive summary

Infrastructure development projects need to convert dense computer-aided design (CAD) drawings into a procurement-ready bill of materials (BOM) before quoting and purchasing can begin. While several existing automation tools support parts of the pre-quote workflow, producing reliable BOMs still depends primarily on manual review.

Automating BOM generation is challenging because CAD-vector PDFs store text as strokes, contain dense lookalike symbols, require localization to link labels to parts and are prone to double-counting errors. Standard optical character recognition (OCR) and off-the-shelf vision language models (VLMs) are therefore unreliable without domain-specific adaptations.

WWT's R&D team decided to tackle the challenge head-on. This Applied Research Paper shares our findings.

To convert CAD-vector drawings into structured BOM drafts, our custom pipeline workflow combines stroke-text recovery, AI-based component detection, vector geometry reconstruction, label-to-part association, view reconciliation, scope filtering and catalog normalization. 

Using a collection of water and sewer construction drawings, our custom pipeline workflow achieved 75.4% precision, 79.6% recall and an F1 score of 77.5% for component-level BOM extraction across the dataset. For correctly matched components, it recovered dimensions in 93.9% of applicable cases and quantity in 77.8%, with quantity recovery declining in repeated or clustered drawing contexts. 

Benchmarking against off-the-shelf frontier VLM models revealed a clear trade-off: Claude Opus 5 was more conservative and achieved higher precision and a higher F1 score, while our custom workflow recovered more real components and localized them on the drawing. That localization is essential for label-to-part association, connectivity reasoning, repeated-view reconciliation and downstream workflows, such as RFI responses, compliance checks, knowledge graphs and digital-twin handover. Our pipeline also included confidence, provenance, source evidence and human review, making uncertain outputs auditable and improvable.

Today, manually preparing a BOM takes roughly two to four hours per drawing sheet, and between two and eight person-days per drawing set, depending on complexity. Our solution compresses this manual, multi-day BOM preparation process into a traceable BOM draft generated in minutes, which a reviewer can correct rather than build one from scratch, cutting effort by roughly 60-80% to 0.4-1.6 hours per sheet. 

Our approach reduces pre-procurement effort, improves consistency and makes the BOM output more usable for quoting and estimating teams. Simultaneously, it creates a reusable foundation for broader drawing intelligence, enabling bid support, RFIs, revision comparison, compliance checks and asset handover, while enabling future knowledge-graph or digital-twin workflows with searchable drawings.

Motivation for technical drawings extraction

The challenge of converting raw infrastructure drawings into a procurement-ready BOM stems from multiple inconsistencies:

  1. Text may be stored as stroke geometry rather than selectable characters, while pipes, leaders, dimensions, symbols and incidental drawing marks can look similar at the vector level.
  2. Drawing sets are also dense and vary in symbols, components and conventions. This makes off-the-shelf VLMs unreliable without domain-specific tuning.
  3. BOM extraction can introduce other complications: Similar symbols can represent different components; the same part may appear in multiple views; and repeated appearances can lead to double-counting.

Existing solutions address parts of the workflow, but none currently solve end-to-end infrastructure drawing understanding and BOM generation.

Overview of existing solutions

Commercial estimating and workflow automation tools, such as Bluebeam Revu and PlanSwift, improve manual estimating workflows, but their automation stops short of truly understanding the drawings. 

Bluebeam's Visual Search primarily performs visual pattern matching and is sensitive to rotation, occlusion and overlapping text, while its OCR struggles to associate text with adjacent symbols through connecting geometry. 

PlanSwift supports estimator-driven measurement and assemblies, but it still requires the user to decide what each drawing object means and which cost item it belongs to. 

Neither tool performs stroke-text recovery from CAD-vector fonts, leader-line tracing, semantic dimension classification, PLAN/SECTION reconciliation, catalog normalization, confidence scoring or provenance-backed BOM generation. As a result, they help users measure and organize pre-quote work, but they do not automatically convert dense infrastructure drawings into a structured, auditable, procurement-ready BOM.

CAD/BIM platforms, such as Autodesk Civil 3D and Bentley OpenUtilities, assume assets already exist as structured CAD/BIM objects with attributes and connectivity. They don't address the earlier problem of extracting a part registry and BOM from flattened drawing sheets.

Document AI and OCR platforms, such as Azure Document Intelligence and Amazon Textract, recover text well from typical business documents, but they struggle with dense engineering drawings where overlapping text, symbols, leaders, dimensions and graphical clutter degrade OCR reliability. Even when text is recovered correctly, these systems don't establish relationships between symbols, geometry, dimensions and BOM entities.

General-purpose VLMs can read labels and notes but remain unreliable for precise localization, small-symbol detection, leader geometry and deterministic counting. They also lack the domain-specific reconciliation, traceability and confidence scoring this workflow needs.

A custom pipeline was therefore necessary. To succeed, the task would require stroke-text recovery, component localization, geometric reasoning, label association, cross-view reconciliation, confidence scoring and provenance all working together. The system we developed combines detector-based localization, geometry-driven rules, targeted text recovery and limited LLM reasoning. Each stage handles a specific aspect rather than relying on a single SaaS product or foundation model to solve the entire problem.

Dataset description

Figure 1: Example source drawing panel from the dataset, showing CAD-exported vector utility content used as pipeline input.

The evaluation dataset is sourced from Palm Beach County Water Utilities' Minimum Design Standards, a public set of standard details and specifications for water and sewer construction. The source files are vector PDFs in which text appears as roman type.shx line-stroke geometry rather than selectable characters. Each sheet is split into drawing panels, which are the working unit for text recovery, geometry extraction, detection, association and BOM generation.

The dataset is targeted at piping materials for water distribution and sanitary sewer. Civil, structural and electrical material estimation is outside the scope because these disciplines use different estimating conventions, item vocabularies and pricing units. Civil/sitework items such as earthwork, concrete pads, thrust blocks, fencing and pipe supports may still appear near piping, but they are typically handled by other trades or priced by volume/lump sum rather than counted as piping parts. To keep the scope consistent, civil/sitework entries were flagged and listed separately, thereby excluding them from the BOM.

Beyond the drawing panels, the project used four supplemental information assets: component labels from panel 2W for you only look once (YOLO) training, template crops from 2W for fallback detection, a manually created BOM as ground truth for panels 2W, 6W and 10W, and an item catalog to normalize generated BOM lines. The detailed reference mapping is provided in Appendix A.

Experiment methodology

Our pipeline workflow uses a shared schema to sequentially turn a single vector drawing sheet into a BOM. As Figure 2 shows, the pipeline can be divided into three functional stages: extraction (reading raw content off the drawing), structuring (turning that content into identified, matched parts) and BOM assembly with audit (counting, checking and normalizing those parts into a final list).

Figure 2: End-to-end pipeline flow across the three functional stages.

We explore the various stages below.

Component detection stage

The component detection task locates and classifies every symbol on the rasterized drawing sheet, including meter boxes, corp-stops and valves. It returns a bounding box, class label and confidence score per detection. It was built around two interchangeable backend approaches that share a single output schema: a fine-tuned YOLO model (default) and a template-matching backend, swappable without touching anything downstream. 

YOLO was fine-tuned on a water-utility installation drawing from a public Florida dataset, hand-labeled in CVAT and cropped into overlapping 640px tiles. 

Template Matching needs no training; it slides a hand-curated library of reference crops across the drawing at multiple scales, looking for strong pixel correlation. 

Pipes are handled separately by a rule-based geometric heuristic, since they're just two parallel lines and no classifier discriminates them reliably. Specifically, it looks for two roughly parallel line segments with a consistent short-distance separation, running alongside each other with no arrowhead at either end.

Figure 3: Component detection output, with class labels, boxes and confidence.

All three backend approaches were benchmarked against the hand-labeled ground truth before YOLO became the default (Table 1):

  1. DINO-X, a general-purpose visual-prompt detector never trained on engineering drawings, performed well on recall but was unusable on precision.
  2. Template Matching remained viable due to its high in-panel ceiling given good crops and thresholds.
  3. YOLO led because it was the only backend approach that actually learned this domain, with room to improve as more drawings get labeled.

Despite scoring lower on F1 than Template Matching, YOLO was chosen as the default because it generalizes, while Template Matching memorizes. As a probabilistic model, YOLO can still recognize a symbol even if its scale, angle or aspect ratio changes on a new drawing. Template Matching looks for a near-exact pixel correlation, so a slightly different rendering simply goes undetected, and it depends entirely on a large, carefully curated reference library even to work at all. YOLO's lower score reflects a harder, fairer test: detecting symbols on a drawing it has never seen. Template Matching's higher score only holds for the one drawing it was tuned to.

Limitations: The model was trained on a single sheet (2W), so what it learned is tied to that drawing's specific style: A symbol class, such as bends, may look different on another sheet (say, 10W), and detections reliable on 2W can degrade elsewhere. Template Matching, the alternate backend approach, collapses without a carefully curated reference library.

Table 1: Component-detection backend comparison.

Text extraction stage

The text extraction task in our workflow pulls every label, dimension and note off a drawing sheet into structured, searchable text. Since a vector drawing consists of line strokes that represent each letter (no actual image or text), the strokes are exploited directly rather than applying OCR to a rasterized drawing. 

   Figure 4: Stroke-to-text extraction flow.

Stroke fragments are reassembled into glyph clusters based on proximity and connectivity, then matched by shape overlap against a reference dictionary. Confident matches are assigned (others are left blank rather than guessed), a process followed by cleanup: splitting fused characters, retrying at a corrected baseline and detecting small punctuation marks missed by the main pass.

The output is a JSON file: one record per line with text, bounding box, rotation, per-word boxes and a confidence score, grouped into blocks. This output is read back during later stages of the workflow, such as Association, Dimension Parsing and the BOM stage.

Limitations

  • A single vertical stroke reads as "I", so two are needed to trust it; C/O and S/8 still confuse at small sizes.
  • Some exporters merge strokes so badly that the text is unrecoverable.

Geometry extraction stage

Geometry extraction reads the drawing's vector commands to recover line segments, arrowheads, dimension pairs and bent leaders. This rule-based stage builds the structural skeleton that will later be used to connect labels, dimensions and components. In practice, it identifies arrowheads from triangular geometry, merges split dimension lines when they form one continuous measurement, and traces bent leaders through connected line segments.

Figure 5a: Geometry extraction examples: Broken dimension-line merging.
Figure 5b: Geometry extraction examples: Bent leader tracing.

The performance of geometry extraction is assessed using a regression baseline rather than precision/recall. To this end, counts from a fixed reference drawing are pinned (60 arrowheads, 173 segments, 4 merged pairs, 93 pipe-pairs, 27 bent leaders), and any run drifting more than ±10% flags a regression to investigate. This metric is based on a stable snapshot, not an independently verified ground truth.

Limitations

  • Thresholds applied in these methods may not be directly transferable to new drawing styles without tuning.
  • A few CAD exporters merge geometry into unrecognizable blobs.

Label and dimension association stage

This stage traces arrowhead leaders back to their label text and pairs two-arrowhead leaders into dimension lines. The method is primarily rule-based and geometric, with a narrow fail-safe LLM step used only when geometry alone cannot resolve the association.

  • Grouping labels: Multi-line labels are first clustered by proximity, then split when leader geometry shows that different leaders point to different fragments. Only unresolved cases are sent to a single batched LLM call that judges the full neighborhood, avoiding pairwise merges that could combine separate labels. If the LLM step fails, ambiguous fragments remain separate to avoid false merges.
  • Resolving leaders: As a purely geometric method, it traces from an arrowhead's base to the far end of its line (checking for a bend first) to determine what it points at. Single-arrow leaders resolve first; two-arrow leaders confirm only when both ends point back at each other. Components are matched to the nearest shape.
Figure 6: Leaders traced from arrowhead to label text.

Once a label is linked, its text is parsed for dimensions and classified using simple drawing cues: one-arrowhead attachments indicate component size, two-arrowhead attachments indicate distances between parts, MIN/MAX wording indicates clearance limits, note references indicate specification references, and loose measurement callouts are treated as annotations. The full mapping is provided in Appendix B.

Limitations

  • A leader crossing text can break the stroke and attach to the wrong continuation, mislinking the label.
  • Vertical or rotated text sometimes recovers with characters out of order, corrupting the label before association runs.
  • Proximity-based dimension-to-part linking can rarely mismatch a nearby part.
  • Bent leader lines score lower confidence than direct ones.

Cross-view matching stage

Cross-view matching prevents double-counting by consolidating the same physical part when it appears across PLAN, SECTION or DETAIL views.

Figure 7: Cross-view matching flow.

The cross-view matching stage runs in three steps:

  1. View segmentation: The sheet is segmented into PLAN and SECTION views using view titles where available, or whitespace and drawing-density cues when titles are absent; notes-heavy regions are excluded, and each detected view is cropped and numbered for review.
  2. VLM perception: A model reviews each marked view, classifies every numbered mark as valid, corrected, rejected or unclear, and assigns a canonical part type. It also estimates assembly order and identifies visible unmarked parts by name, type and location.
  3. Deterministic matching: Parts are matched across views using their relative order, with sheet layout used to resolve ties. VLM output can corroborate a match but does not decide it, and accepted matches are merged so each physical part appears as a single BOM node.

This design keeps the perception model-driven while alignment and merge decisions remain deterministic. Failed VLM calls fall back to raw detections without fabricated verdicts, and each match carries its own trust score for targeted review.

Limitations

  • Scoped to PLAN and SECTION only for now. Matching assumes a simple, same-page layout, so complex or multi-page relationships are not yet handled.
  • A conflicting VLM claim can override the geometric, YOLO-based match on a single, unverified call, with no independent check on that decision

BOM generation

The BOM is generated from the sheet-level part registry resolved through cross-view matching, then enriched with dimension inventory for real sizes and constraints and supported by recovered drawing text.

Figure 8: Normalized BOM output against the project catalog.

BOM generation converts the reconciled part registry into a normalized bill of materials by grouping components by assembly, type and size, with group count determining quantity. Detector-confirmed components are counted directly, while estimate-only components require supporting drawing text. Same-type components observed in only a single view are flagged for review because they may indicate missed cross-view reconciliation and potential double-counting. 

Pipe runs are reported as a count of drawn segments rather than a purchasable per-unit quantity, while actual run footage is tracked separately. Sizes are initially derived from recovered labels and upgraded when a confidently associated dimension callout provides a more reliable value. Installation constraints are preserved as structured metadata alongside the generated BOM.

Scope guardrails

Guardrails apply an AI-assisted review pass to the drafted BOM so out-of-scope items are not treated as normal piping-material purchases. The pass primarily flags civil or site-construction work, such as concrete pads, fencing or landscaping, and annotates affected lines for reviewer attention without deleting, reordering or rewriting the BOM.

Catalog normalization

Catalog normalization maps each drafted BOM line to the project catalog using type, size and material, producing canonical supplier wording, units and item descriptions. The matching is conservative: Unclear or conflicting lines remain unmatched rather than guessed, and the same pass preserves measurement class, discipline scope and valid aggregations when multiple detected parts represent one catalog item.

Confidence scoring and explainability

Confidence scoring

Confidence scoring converts each BOM line into an audit score based on the strength of its supporting chain: cross-view reconciliation, component detection, recovered text, parsed dimensions, valid leader/arrow geometry, and label-to-component association. Reconciliation and detection carry the highest weight because they establish whether the system found the correct physical part; the remaining checks assess the reliability of its label, size, geometry and association. Detailed questions and weights are provided in Appendix C.

Presently, the composite score is a weighted average. Missing expected evidence counts as a gap and lowers the score; truly not-applicable checks are removed from the denominator. The worked examples in Table 2 and Figure 9 show why the same formula can produce opposite tiers: P06 keeps a High composite because five checks corroborate a weak Detect, while P07 loses its supporting checks entirely and falls to Low.

Table 2: Per-check confidence scores for the two worked examples.
Figure 9: Worked confidence examples: P06 (High) and P07 (Low).

Provenance tracing

The provenance trace walks backward from a BOM line to every artifact that produced it, including the guardrail verdict, size record, registry part, per-view occurrence, detection, AI verdict and any cross-view corroborating claim. Each step carries its own confidence, and the weakest link in that chain is automatically flagged for review. Clicking any component or line in the BOM traces it straight back to where it sits on the sheet, along with its attached label and any nearby labels for context.

Figure 10: Provenance crop for the estimated gate-valve.

In Figure 10, a gate valve lacks a detector box and is supported only by an AI-estimated position at confidence 0.50. The provenance trace therefore flags that occurrence as the weakest link for reviewer attention.

With –html option supplied, any traced step with a page position renders as a live crop from the source PDF. Here, that crop is the weakest-link step itself, so the reviewer sees the buried valve the AI inferred rather than only a JSON confidence value.

Human in-the-loop review

Confidence scoring and provenance make human review targeted rather than blanket. As shown in Figure 11, low-confidence BOM lines are surfaced with supporting drawing evidence and the reason for flagging, allowing reviewers to focus on genuinely uncertain outputs rather than inspecting every extracted item. Review is not limited to flagged lines; the full BOM stays open for audit, so reviewers can correct a high-confidence item, add a component the pipeline missed, or exclude an erroneous one, each backed by evidence and a stated reason. This is critical because BOM outputs influence procurement, estimating and project planning, where undetected errors can propagate. For example, a valve identified through weak cross-view reconciliation may be flagged for review, letting an expert inspect the evidence and approve or correct it before it reaches downstream workflows. 

Every review decision contributes to an auditable record of how the final BOM was produced. Looking ahead, these corrections can be recorded to a Knowledge Management System (KMS), retraining and refining the pipeline over time and building a reusable base of validated engineering knowledge for future projects.

Figure 11: Human-in-the-loop flag review screen showing a Low-tier BOM line 4.8. Evaluation Metrics

Synthetic dataset

The labor required to generate human-curated evaluation datasets makes their use impractical. In their place, we have devised an evaluation dataset based on synthetically generated engineering diagrams.  The synthetic benchmark automatically generates test sheets styled after panel 2W, with each sheet's contents set by a configuration and made repeatable via a seed value, so that every extraction stage can be scored against a known, correct answer rather than relying solely on drawings labeled by hand.

Each synthetic sheet plants component symbols, pipes, labels, leaders and dimension pairs, with every object recorded in a truth ledger as it is placed. Figure 12 shows a sample sheet paired with its ground-truth ledger, which is used for stage-by-stage scoring. The benchmark covers extraction primitives only, including text recovery, leader tracing, pipe extraction, dimension pairing and component detection; it excludes end-to-end BOM assembly, cross-view reconciliation and catalog normalization. Generation counts and layout variation are controlled by configuration and seed, with the full workflow in Appendix E.

Figure 12: Synthetic sheet generation and truth-ledger scoring.

Table 3 reports the synthetic benchmark at the extraction-stage level. The numbers show that clean-sheet reading is largely solved for this controlled corpus: text, leaders and pipes are at or near perfect. The two open gaps are diagnostic rather than fundamental: missed dimension pairs are pairing failures, not reading failures, and low strict component recall is mainly class confusion rather than a localization gap.

Table 3: Synthetic benchmark results by extraction stage.

Overall BOM metrics

Overall BOM metrics compare the generated bill of materials against the manually generated ground truth, then grade performance through three standard measures:

Table 4: BOM evaluation metrics.

Evaluation works like a funnel: first it checks whether the right component was found; then, only for true positives, it checks size and quantity. This separates recognition errors from attribute errors, so we can tell whether the system is missing parts or simply misreading details after the right part is found.

Matching logic: ground-truth rows are matched to generated-BOM candidates using rarity-weighted text, in four passes; unresolved rows and unmatched predicted lines are labeled separately. Appendix F gives the full pass sequence and label definitions.

Results

Overall BOM results

Table 5 reports component-level BOM extraction across the three evaluated drawings.

Table 5: Component-level BOM extraction across the three evaluated drawings.

Size and quantity are conditional checks: they are evaluated only for correctly matched components where that field is present in ground truth.

Table 6: Conditional size and quantity accuracy on matched components.

Table 5 shows the headline result: a component-level F1 score of 77.5%, with 75.4% precision and 79.6% recall. In practical terms, the system finds about four out of five real components, while roughly three-quarters of generated component lines are confirmed correct. The remaining precision loss is driven more by extra or unclaimed lines than by broad component-recognition failure.

Table 6 shows the conditional attribute results: once a component is found and the field is checkable, size performs strongly with an F1 score of 91.2% and 93.9% recall, while quantity is weaker with an F1 score of 70.9% and 77.8% recall. The quantity errors are concentrated rather than random: 8 of 36 checkable quantity cases, almost all involving repeated parts or busy clusters where counting is harder than reading a single labeled item and most pronounced on 10W. The full per-drawing breakdown is provided in Appendix G.

Limitations of the overall solution: These results hold where the input matches the pipeline workflow's design assumptions, and three limits define that boundary. Extraction depends on recoverable vector lines, arcs and stroke-text, so scanned or rasterized sheets are detected and safely skipped rather than processed and would need a separate OCR/image-reconstruction path. Validation covers three panels from a single Palm Beach County-style drawing family — enough to expose failure modes such as quantity errors on clustered items, but not to claim generalization across symbol libraries, CAD standards, or drawing templates. And the output is a piping-material BOM draft, not a multi-discipline or priced estimate. The next step is therefore broader-labeled coverage — more drawing families, more BOM ground truth, targeted raster-input tests and repeated-instance counting examples — rather than further tuning on one sheet family.

Benchmarking against VLM baseline performance

Feeding drawings directly to a frontier Vision Language Model (VLM) is a useful practical benchmark for the custom solution built in this work: it shows what a general-purpose model achieves on the same sheets with no domain training, geometric rules, or cross-view reconciliation. Claude Opus 5 was used on the drawings evaluated above (Figures 9-12).

Table 7: Pipeline versus Claude Opus 5.

The two approaches trade places on headline metrics. Claude Opus 5 is more precise (89.1% vs. 75.4%) with a higher F1 score (82.0% vs. 77.5%); our workflow recovers more real components (79.6% vs. 75.9% recall). The precision gap is not like-for-like: Claude declines ambiguous cases, while the pipeline attempts them and exposes each one with a confidence score and source crop. 

However, going beyond the numerical scores, there is a substantive difference between the two approaches: localization. Our pipeline ties every detection to the physical fitting; Claude's boxes are looser and often anchor to label text instead, like a corporation stop in Figure 13, and bends in Figure 14, where Claude reads the "90° BEND" callouts but boxes the text while our pipeline lands on the four fittings. 

Figure 13: Corporation-stop localization comparison. The pipeline box stays tight on the fitting; Claude's boxes are larger and can anchor to label text instead of the physical part.
Figure 14: 2W bend localization comparison. Green boxes show pipeline detections on the actual bend symbols; red dashed boxes show Claude guesses placed near label text.

Figure 15 shows the same failure in arrowheads and text, where Claude over-counts drawing marks as arrows and treats graphical strokes as confident text.

Figure 15: Arrowhead and text-confidence comparison. The pipeline requires leader-connected arrows and component-linked text.

This matters beyond the count. Localization is the basis for spatial reasoning: a component that isn't correctly placed cannot be related to its neighbors, so adjacency and connectivity are lost, and with them the knowledge graph that unlocks drawing interrogation, RFI response and compliance checking. Reading labels well is not the same as understanding the drawing.

Summary: The VLM is strongest as a conservative reader of labels and context; our pipeline is stronger where outputs must be localized, confidence-scored and traceable to a source crop, which is what makes its errors convertible into rules, thresholds and training data.

Business use cases and value 

This work builds the foundation layer for drawings intelligence: static engineering drawings turned into structured, searchable, confidence-scored data. This work cuts BOM preparation effort by roughly 60-80% against a manual baseline of 2-4 hours per sheet, based on internal estimates. Today's pipeline workflow covers piping on plan and section views; the same foundation extends to P&IDs, isometrics, electrical and civil sheets and to a connected knowledge graph. The use cases below follow that order of readiness.

  • Procurement and inventory: BOM generation yields component lists with sizes, counts and match confidence. Buyers order from it and reconcile against stock; low-confidence lines are verified before a PO goes out.
  • Estimating and bid preparation: For a typical set of about 10 sheets, that's 20-40 hours of manual work reduced by the same 60-80%, and competing bids become comparable line-for-line.
  • RFIs: Instead of re-reading a sheet to confirm a size or a revision change, teams query the extracted data, and every answer stays traceable to the drawing evidence.
  • Compliance checks: Structured, machine-readable drawings can be checked automatically against building codes, fire codes and design standards (like clearances, minimum sizes, mandated fittings, etc.) instead of by manual review.
  • Revision management: Two revisions of a sheet can be compared directly: what was added, removed, or resized, surfaced automatically rather than found by eye.
  • Asset handover and digital twin: Extracted components and their relationships populate asset registers at handover, replacing the manual re-keying that follows construction.

Once a drawing is machine-readable and every fact is traceable, it stops being a document to be read and becomes an asset to be queried.

Recommendations for future work

  • Civil and electrical disciplines: The piping-only scope reflects the source data, not a structural limit. Today's guardrail removes civil content from a piping BOM; producing one would need discipline-specific drawing sets, symbol vocabularies, catalogs and scope policies. Worth exploring, not yet scoped.
  • Rasterized drawings: Text and geometry recovery depend on native vector strokes, so scanned input is currently set aside. Component detection already works from page rasters and should extend naturally; text and geometry would need OCR or image-based reconstruction.
  • Knowledge graph: The part registry already emits a flat, graph-ready parts list per sheet. Building the graph should be planned alongside the dimensions module's schema work so part and dimension models do not diverge.
  • Knowledge management system: Each run produces an isolated, provenance-stamped output folder; nothing persists across projects. User interaction and feedback can also help the future improvement of the overall system. A management layer above the graph would make extracted knowledge searchable, reusable and comparable across projects.
  • Cross-sheet references and reconciliation: Callout resolution and part reconciliation stay within a single sheet, and dimensions have no cross-view reconciliation stage. Extending across views and sheets requires a project-level part and dimension registry.
  • Implicit and inferred dimensions: Spans are captured only when both endpoints sit on identified components and are reported standalone. They are not yet used to infer missing dimensions, total run lengths, or drawing scale — scale-aware reasoning would close that gap.
  • Catalog growth loop: Normalization is deliberately read-only: unmatched lines stay unmatched. A review queue surfacing frequent unmatched descriptions would allow a curator to add real entries without weakening that guarantee.

Appendices

Appendix A. Dataset reference assets

Table A.1: Reference datasets and item catalog.

Appendix B. Dimension type classification

Table B.1: Dimension types and the evidence used to classify each.

Appendix C. Confidence-scoring weights

Table C.1: Confidence-scoring stages and weights.

Appendix D. Human-in-the-loop review workflow

The review workflow is designed so low-confidence BOM lines receive expert attention without forcing reviewers to inspect every generated item. Only Low-tier or explicitly flagged lines are queued for review; High and Medium lines remain available with their provenance but do not require mandatory action unless a reviewer chooses to inspect them.

Table D.1: Two-step human-in-the-loop review flow.

After sign-off, the system writes a reviewed BOM JSON file with reviewer identity and timestamp per line. These reviewed outputs are the intended bridge to a future Knowledge Management System: reviewer corrections can become curated examples for rule refinement, threshold tuning, model improvement and project-to-project knowledge reuse. Any KMS integration should be treated as a roadmap step rather than a feature already wired into the current pipeline.

Appendix E. Synthetic dataset generation process

Table E.1: Synthetic sheet generation and scoring process.

Appendix F. BOM matching internals

Ground-truth rows are matched to generated-BOM candidates using rarity-weighted text. A match needs two shared words, or one shared word plus matching size and runs in four passes; size and quantity only break text-supported ties. Rows and predicted lines that remain unresolved are labeled per Table F.2.

 Table F.1: Four-pass matching sequence.
Table F.2: Labels for unresolved and unmatched lines.

Appendix G. Per-drawing BOM results

Table G.1: Per-drawing F1 breakdown for component, size and quantity performance.
WWT Research
Insights powered by the ATC

This report may not be copied, reproduced, distributed, republished, downloaded, displayed, posted or transmitted in any form or by any means, including, but not limited to, electronic, mechanical, photocopying, recording, or otherwise, without the prior express written permission of WWT Research.


This report is compiled from surveys WWT Research conducts with clients and internal experts; conversations and engagements with current and prospective clients, partners and original equipment manufacturers (OEMs); and knowledge acquired through lab work in the Advanced Technology Center and real-world client project experience. WWT provides this report "AS-IS" and disclaims all warranties as to the accuracy, completeness or adequacy of the information.

Contributors

Aditya Prabhakaron
Data Science Manager
Belle Ho
Analyst
Doug Rank
Lead Data Scientist

Contributors

Aditya Prabhakaron
Data Science Manager
Belle Ho
Analyst
Doug Rank
Lead Data Scientist

In this report

  1. Executive summary
  2. Motivation for technical drawings extraction
  3. Overview of existing solutions
  4. Dataset description
  5. Experiment methodology
    1. Component detection stage
    2. Text extraction stage
    3. Geometry extraction stage
    4. Label and dimension association stage
    5. Cross-view matching stage
    6. BOM generation
  6. Confidence scoring and explainability
    1. Confidence scoring
    2. Provenance tracing
    3. Human in-the-loop review
    4. Synthetic dataset
    5. Overall BOM metrics
  7. Results
    1. Overall BOM results
    2. Benchmarking against VLM baseline performance
  8. Business use cases and value 
  9. Recommendations for future work
  10. Appendices
    1. Appendix A. Dataset reference assets
    2. Appendix B. Dimension type classification
    3. Appendix C. Confidence-scoring weights
    4. Appendix D. Human-in-the-loop review workflow
    5. Appendix E. Synthetic dataset generation process
    6. Appendix F. BOM matching internals
    7. Appendix G. Per-drawing BOM results

What's Next

About ResearchAI Proving GroundAI Proving Ground PodcastAdvanced Technology Center
WWT
  • About
  • Careers
  • Locations
  • Help Center
  • Sustainability
  • Blog
  • News
  • Press Kit
  • Contact Us
© 2026 World Wide Technology. All Rights Reserved
  • Privacy Policy
  • Acceptable Use Policy
  • Information Security
  • Supplier Management
  • Quality
  • Accessibility
  • Cookies