
Search CAD models with natural language
Find 3D models using natural language
Neat.
As a mechanical engineer, I feel the part of my job is safe from AI for the time being. I don't think quality training data for good mechanical design exists.
3D CAD is only part of good design. To a tinker-er that is 3D printing simple parts, an STL is fine. But most parts that matter require far more design consideration and detail than simply the geometry data that an STL (or other 3D file) provides.
The majority of parts are accompanied with a drawing, and that is where the real design actually is found: Tolerances, GD&T, materials, processing notes...
Even then, most of the calculations and considerations to build the model and drawing are not explicit in the design documents: Nothing about a drawing of a stainless steel part tells you WHY it must be a stainless steel part. I don't think there is a large set of well documented designs out there to act as training data for an AI system to design an assembly beyond basic 3D parts.
The authors identify this gap, but it's a fundamental problem with the wholesale move to AI in mechanical design.
Agreed. At the end of the day, manufactured parts are driven by constraints outside of the CAD environment so analyzing 3D data as the foundation of an AI system strikes me as attacking the problem from the wrong direction. i.e. Simple optimization of a part for injection molding can take it from requiring a bunch of side actions and collapsing cores to a simple 2 sided mold and save hundreds of thousands of dollars in tooling. None of that is obvious from 3D data alone.
That said, I am excited for AI assisted CAD tools. Things like creating and applying global variables to an existing part, complex assembly analysis for part reduction or just making a starting base part can be incredibly tedious and are low hanging fruit for improving CAD workflows with AI imo.
a lot of the why is encoded elsewhere in mechanical engineering at least - the tables, the formulas, textbook problems, engineering reports.
one of the challenges to making a good data set might be around bad designs and why they failed. if we get to a mechanical agent, its going to need to understand that brass was a mistake and redesign a part as steel and change the design for the new contraints
unlike code, that kind of train of experiment i think will be a lot more expensive to make, since you might actually want to create those parts along the way and not just pretend
Amusingly, I tried. Your job is safe. ;-) I'm a scientist. I tried using a couple of free 3d modeling tools that are programmable in Python, using Claude within VS Code to design a part according to my prompts. It was a simple plate with some holes for electrical connectors, switches, etc. Of course it let me make some noobie design mistakes, such as making the hole exactly the same diameter as the thing that went into it, plus or minus of course. And I would have had to really scrutinize the Python code to notice that I got one of the diameters wrong altogether.
Just judging from this experience, the effort would rise exponentially with the complexity of the part, not to mention assemblies. The designers earn their keep. I get really bad eyestrain with any task that requires staring at the screen while doing fine mouse work, so I just can't use CAD. On the other hand, I can code all day because I'm not closely focused on the screen when I'm typing.
hold my beer.
the apocolypse needs to happen sooner so we can pop this bubble.
We rendered the one million part ABC dataset from Deep Geometry, and open-sourced the data. We also built a fun demo with the following pipeline: CAD > render > caption > embed.
Open-sourced dataset: https://huggingface.co/datasets/daveferbear/3d-model-images-...
Blog writeup: https://www.finalrev.com/blog/embedding-one-million-3d-model...
The search function doesn’t seem to work at all, it provides nonsensical results.
For example if I search “supercolumns” I get regular household furniture.
Yeah I think the embedding are describing what can be seen from a picture of the model not what it is or what it is used for. search some things work like "Fan" but others don't so you can search for "plate with 5 holes" but not for specific engine part cover.
When I search for "chair" I get 48 results, about 3 of which are actually chair-like.
- Some are clearly miscategorized - ABC-00131096 is a coffee table but has a very detailed description of its chair attributes.
- Many others are weird nonsense geometry, like ABC-00991744, ABC-00807798, ABC-00349255 or ABC-00822766.
- Some have a partially accurate description (if you pretend it's a chair), like ABC-00685912 has a blocky geometric structure with a horizontal piece off a vertical piece, but then it starts talking about an armrest on one side that doesn't exist at all.
- ABC-00388826 is a silhouette of a cat, which the description misses completely, and I don't see how you would sit on this "unique chair design characterized by its fluid and sculptural form."
Overall the descriptions are pretty useless and ascribe a lot of chairness to things that are not chairs.
Is a dataset with this much junk in it good for something?
Neat, but also hilarious! Searching for "mug" gives results where the first item listed (ABC-00008297) is a mug model with a hole not only in the top to pour in your drink, but also in the side and bottom (just in case you wanted more access to your liquid).
Not the first one... but the "pair of interconnected mugs" that is described as "emphasizes connectivity and collaboration, suitable for serving beverages in a shared or communal setting" is pretty amazing too. I never knew I was missing out on communal mug holding.
I'll do you one better.
The first item returned by a search on "couch" is ABC-00927226, which "features a rectangular form resembling a couch, characterized by a solid, box-like structure." In other words: it's a box.
HOWEVER: We KNOW it's a COUCH, because: "One side prominently displays the word "COUCH" vertically, rendered in a bold, modern font."
It's BOLD! It's MODERN! It MUST be a "COUCH!"
Yeah, this looks like something you might hang from a wire or string with a slot for a dowel. A+ for effort though, lol!