Benchmark: CadQuery vs. OpenSCAD for agentic CAD work

(modelrift.com)

10 points | by jetter 1 hour ago

4 comments

  • dofm 30 minutes ago
    None of the "findings" here really needed an AI experiment to do it:

    - "CadQuery fails loudly and early... OpenSCAD fails silently and late"

    This should be obvious, from the documentation, from the way geometry construction works. OpenSCAD has no sense of failure where one shape means another won't work; you're just drawing the equivalent of 3D pixels in space. It will always potentially-meaninglessly succeed if the syntax is OK.

    - Renders caught nothing that mattered

    Objects with hollows are not going to show their major problems this way.

    - CadQuery can be interrogated, OpenSCAD cannot

    Isn't this explicit from the documentation of both? One works by iteratively building on top of a previous result that can be stored in a variable; the other doesn't.

    - OpenSCAD renders have no concept of a part edge

    Again — this should be explicit from the documentation, which describes no methods for operating on edges (and largely from the fact that it is declarative).

    - Speed favours OpenSCAD, and it barely matters

    Yes, being faster is no good if things are wrong.

    - "What decides it is verifiability rather than expressiveness, and CadQuery leads there by a wider margin than the syntax difference suggests."

    Yes, because the difference is semantic. Which you can get from the documentation. Building iteratively on the basis of existing geometry is inherently more verifiable, because stuff that can't work won't work.

    Honestly do people not try learning CAD before they try to build an AI tool to generate CAD models? This feels like yet another situation where people who have not done the foundational conceptual work seem to think that they can avoid it and just work around it with AI.

  • ariwilson 40 minutes ago
    not interesting as the writeup as very AI sloppish :(. i'm sad about that as I would really like to have an intuitive sense for how good the current frontier models are at coming up with interesting 3d printable parts.
    • ____tom____ 32 minutes ago
      I've been just getting started on this with Astra. Astra is one-shotting good 3D models from photographs for me. I was unable to get Sol to do this, even after multiple rounds.

      It's not a small change, it's from "not useful" to "pretty good"

      • elwebmaster 23 minutes ago
        With any tool?
      • dofm 29 minutes ago
        > Astra is one-shotting good 3D models from photographs for me.

        Right, but, so what? There is very little utility here. Parts that need to be useful need to be designed to spec.

        • ____tom____ 18 minutes ago
          Maybe not for you, but I'm building miniatures for custom board games. Works great.
          • dofm 13 minutes ago
            Right but there is no meaningful specification failure mode in creative CAD, is there? I mean, yes, image to CAD for a little miniature, I can buy that it is OK. Fun, even, especially when your use is sort of transitory.

            But this article is talking about functional parts. A functional part exists because of its spec, usually fails because of some combination of its material, manufacture, and spec. And the spec is chosen with that in mind.

            Even working from a text description of a part is error prone because language is imprecise and interpretable. I just don't understand why AI people want to race past actual proper sensible solutions to a problem just to automate generating from text.

            • elwebmaster 4 minutes ago
              People don't have time (or even mental capability / even more time) to learn CAD but they have a task at hand which they want done. If AI can generate a part which is "good enough" for the user at a cost that the user considers reasonable (I.e. zero) then AI gets used. If a part fails thats rarely a concern because these parts are just 3d printed and likely cost single digit cents in filament. User can always print a new one.
        • Mashimo 23 minutes ago
          Once you have the model, it's relativ easy to change the demensions / constrains later on, no?
          • dofm 20 minutes ago
            Not if the model has not been implemented from reasonable principles.
    • rjsw 35 minutes ago
      There was a blog post from a couple of days ago that had been initially flagged, it seemed to me to have a good overview of AI for CAD, will try to find the link.
    • dofm 7 minutes ago
      [dead]
  • tamimio 16 minutes ago
    Should try different models and harnesses, CC and opus5 aren’t the best combo. Also, I expect soon to have models that verify the mesh and even render it not just generating the code. So we got this for mechanical engineering, cupperhead and similar for electronics, usual agents for software, what is next?
  • brcmthrowaway 22 minutes ago
    The OP seems to be shilling this tool nonstop. Modelrift just seems to be an LLM wrapper.. the original models can handle OpenSCAD just fine.