The whole industry is reaching for LLMs to “read drawings.” That’s solving the wrong problem. A takeoff is a measurement of physical space, and you don’t measure space with a language model.
Ask an estimator what they do all day and the verb that comes up is “measure.” They count receptacles, they trace duct runs, they pull linear feet of conduit off a floor plan, they tally column footings against a structural grid. The output of a takeoff is a quantity: 847 fixtures, 1,240 linear feet, 36 footings. A number tied to a place on a page tied to a real thing that gets installed.
So when people say the way to automate estimating is to throw the drawings at a large language model and ask it to read them, I think they’ve quietly swapped the problem for a different one. Reading is not measuring. A model that’s very good at predicting the next token is not, by that fact, good at telling you how many feet of 3/4” EMT run along the north wall of level 2. Those are different machines for different jobs, and most of the construction-AI demos I see are running the wrong machine.
This isn’t an anti-LLM piece. We use language models, in the places where language is actually the substrate. But the core of estimating is geometry, and geometry has to be solved with geometry tools: vector extraction, page classification, measurement against a known scale. Here’s why the text framing breaks, and what the geometry framing looks like when you build it.
Not a document
The first mistake is treating a plan set like a PDF of text. It looks like one. It opens in the same viewer as your lease agreement. But a construction drawing is a compressed symbolic projection of a physical system onto paper, and almost none of the meaning lives in the words.
Consider what’s actually on an electrical sheet. There are symbols, which are not words, scattered across a 2D field. There are line segments that represent conduit, with a length that only means something once you know the drawing’s scale. There’s a panel schedule, which is a table, on one sheet, feeding devices drawn on a different sheet ten pages away. There’s a legend that defines what each symbol means, and the legend is local to that sheet or that set. The “text” on the page (a few abbreviations, some notes, the title block) is the least information-dense thing there.
A language model linearizes everything into a token stream. That’s its whole premise: turn the input into a sequence, predict over the sequence. Do that to a drawing and you throw away the two things that matter most. Geometry goes first, because (x, y) coordinates and lengths don’t survive being flattened into “an outlet symbol, then another outlet symbol, then more of the same.” Scale goes next, because the model has no idea an inch on the page is twenty feet in the building unless something measured it.
Once geometry and scale are gone, you cannot produce a quantity. You can produce a description. Nobody bids a job off a description.
Two page types
Here’s a concrete fork that the text framing never even sees. Construction PDFs come in two physical forms, and they demand opposite handling.
Some pages carry a real vector layer. The lines, symbols, and text are stored as geometry, the way the CAD tool exported them. You don’t OCR these; you read the embedded vector geometry directly and get exact coordinates and exact lengths. That’s the difference between guessing a length from pixels and reading the length the drafter actually drew.
Other pages are raster: a flattened image, a scan, a screenshot dropped into a sheet, with no machine-readable geometry. There’s nothing to extract directly, so you fall back to OCR and vision-based detection to recover symbols and text from pixels.
Before any “reading” happens, you classify every page: vector or raster, which trade (electrical, mechanical, plumbing, structural, civil), what scope, what scale. We store the trade and scope at classification time so downstream extraction knows which playbook to run. Get this wrong and everything after it is wrong, quietly. We’ve watched it happen: a sheet rendered in a way that broke the searchable-text assumption, so the text read came back empty, so detection found nothing, so the count was zero. No error, no crash, just a confident wrong number. That failure is invisible to a feed-it-to-the-model architecture, because the model was never told there were two kinds of page.
The real pipeline
Strip away the framing and the real pipeline is a sequence of geometry and vision steps, with language used only at the edges where there genuinely is language.
A plan set comes in. It’s split into individual pages, each rendered for pixel-level work. Two things run in parallel: detect the trades and scopes present, and read the text that does exist (title blocks, schedule labels, the legend). Classification routes each sheet to the right trade handler. From there it’s extraction against geometry: detect and count symbols, parse schedules and legends into structured tables, trace runs, measure lengths and areas against the page scale. The output is a quantity bound to a location bound to a sheet. That’s the takeoff.
Where do language models fit? In the seams. Normalizing a legend’s abbreviations is partly a text task. So is matching a detected symbol to a requested label, parsing a messy schedule, or reconciling the dozen ways a drafter abbreviates “ground fault circuit interrupter.” A model is the right tool there. But notice the shape. Language handles the labels and the notes. Geometry handles the measurement. The measurement is the product.
This is also why a bigger context window doesn’t rescue the text approach. A hospital electrical set runs 400 pages. Even when it fits, stuffing all 400 in hurts: attention spreads across hundreds of irrelevant sheets and accuracy drops. The fix isn’t more context, it’s selecting the right pages, which is a retrieval-and-classification problem, not a reading one. We’ve written separately about that page-selection layer. The point here is upstream of it. Even with perfect retrieval, the page you hand the model still has to be measured, not described.
Where multimodal breaks
“Multimodal LLMs can see images now” is the line that makes people think this is solved. It isn’t, and the failure is specific.
A modern multimodal model can look at one clean drawing and say sensible things. That demos beautifully. Then you hand it the real job: a 400-page set, mixed vector and raster sheets, repeating symbol notation, schedules that reference devices ten pages away, a scale that changes between details and overall plans. Four things break at once.
It can’t measure. Without an explicit scale and the real line geometry, a length read off pixels is a guess, and guesses don’t bid. It loses count on repetition. A bank of eighteen receptacles drawn as “symbol, symbol, symbol, ellipsis, symbol” is a convention a human reads instantly; a pixel pattern-matcher counts the first few and drops the rest, turning eighteen into a smaller number, or zero. It can’t reliably cross-reference, because a flat token stream has no stable notion that the circuit on sheet E-201 is the one listed on the schedule on E-101. And it’s expensive in the wrong way: pushing hundreds of high-resolution pages through a general model per takeoff, run after run, is a cost structure that doesn’t survive a customer doing dozens of bids a week.
None of these are prompt problems. They follow from flattening a spatial, multi-scale, multi-sheet system into a sequence of tokens. You can’t prompt your way back to geometry you already deleted.
The moat
The reason this matters beyond engineering taste is that the framing determines what you build, and what you build determines whether you have anything defensible.
If you believe estimating is a language problem, your roadmap is a thin wrapper over whatever the frontier lab ships next. Your moat evaporates the morning a better general model drops. If you believe it’s a geometry problem, you build a vision and measurement system specialized to drawings: a page classifier that knows the trades, a vector path and a raster path, symbol detection that understands drawing conventions, schedule and legend parsing, measurement against scale. A general-model release doesn’t replace any of that, because none of it is what general models are trying to be good at. Text-versus-geometry isn’t a preference. It’s the difference between building a feature and building a category.
I’ll put it plainly. We use language models for the parts of estimating that are about language. Everything that produces a number, we treat as geometry, because that’s what it is. A takeoff is a measurement of physical space. You measure space with vision and geometry, not with a model that was trained to predict the next word.
Victor Augusteo is the CTO of Boon AI.
