How Can Large Language Models Help Humans in Design And Manufacturing? Part 2: Synthesizing an End-To-End LLM-enabled Design and Manufacturing Workflow

. The advancement of Large Language Models (LLMs), including GPT-4, provides exciting new opportunities for generative design. We investigate the application of this tool across the entire design and manufacturing workflow. Specifically, we scrutinize the utility of LLMs in tasks such as: converting a text-based prompt into a design specification, transforming a design into manufacturing instructions, producing a design space and design variations, computing the performance of a design, and searching for designs predicated on performance. Through a series of examples, we highlight both the benefits and the limitations of the current LLMs. By exposing these limitations, we aspire to catalyze the continued improvement and progression of these models.


Introduction To Part 2: Synthesizing an End-To-End LLM-enabled Design and Manufacturing Workflow
In Part 1, we described how LLMs can aid users in each step of a CDaM process; now, in Part 2, we describe how each LLM-augmented step can be composed into end-to-end realizations of physical, functioning devices.Following the approach of Part 1's Section ??, this Part focuses chiefly on the interplay between different aspects of the CDaM workflow, and demonstrates a strategy for uniting them.As before, we caution the reader not to expect flawless application of LLM technology without hiccups, but rather, a) to understand which aspects of the LLM-powered pipeline remain J u s t A c c e p t e d challenging for and b) to understand the realizability, adherence to specification, and the performance of real-world counterparts of virtual designs ideated and iterated on through natural language.As a capstone for this Part, we summarize capabilities, limitations, and dualisms of GPT-4 as a whole.
In this Part, we focus purely on our two running examples that served as a testbed for many of our investigations in Part 1: a static design problem (a cabinet) and a dynamic system design problem (a quadcopter).Vision.While not every part of a future LLM-augmented computational design workflow must incorporate the application of LLMs or other generative models, it is important to ensure that each LLM-powered piece of the pipeline can be integrated into typical workflows.Are the inputs and outputs of each step well-defined, and if so, through what data format should they be broadcast?Is an LLM such as GPT-4 able to gracefully reason about longer design and manufacturing processes?Does the synthesis of these steps introduce any new challenges that do not emerge when dealing with a subset of the processes; e.g. are optimized designs also designed for manufacturing?These represent a subset of the questions and potential challenges in building complex end-to-end systems.
While here we focus on showing end-to-end demonstrations for two examples of modest design difficulty, we envision a future where complex objects, devices, and cyberphysical machines could be ideated, specified, and created through AI-powered CDaM workflows, across scales.Not all steps must necessarily be powered by LLMs, nor by AI models at all; nor must a single model such as a generalist foundation model be used throughout the process.However, users must be able to reason persistently through the stages of the workflow and iterate as necessary.Knowledge and design automation provided by LLMs should minimize the amount of iteration needed, though some iteration -especially concerning formalizing objectives, requirements, and fixed specifications -is inevitable in any industrial workflow.Mistakes are made, humans change their minds, missions are altered, and LLM-enabled pipelines will eventually need to have the same flexibility as their traditional counterparts.Given their relative youth, any limitations seen in employing LLMs in these demonstrations should be seen as temporary roadblocks and technical opportunities for improvement, not as insurmountable barriers.
While this analysis is limited to GPT-4, multimodal models are increasingly growing in interest and popularity.In the future, we imagine that large models will accommodate inputs in the form of text, detailed images, hand-written sketches, video, audio, code, formal mathematics, and depending on the instrumentation of the input device being used and permissions enabled, emotional and contextual cues from interactive gestures, human expression, spoken tone, and more.Models will be able to more cleanly infer user intent and goals, especially as user data and feedback itself scales through deployment of these systems.Models will proactively work with users at every step of the pipeline and direct users' flow through that pipeline, including reasoning about how and when to decompose hard problems hierarchically into more tractable subproblems.Such models will aid each specific user in aspects they did not consider, with advice personalized to that user's level of expertise.

Part 2: Related Work
Integrating subcomponents of CDaM into a holistic end-to-end LLM-enabled design pipeline requires reasoning about each aspect of the pipeline and their interplay.Such LLM-enabled pipelines Figure 1.The Cabinet Design Process Using GPT-4.We showcase the comprehensive development process for a wood cabinet, highlighting GPT-4's roles in 3D design generation, part sourcing, manufacturing instructions, assembly guidance, and performance evaluations.
are a relatively new and rapidly growing area of research across domains; we note some notable recent examples of LLM and other here.
In concurrent work, LLMs have been employed in domain-specific pipelines.In robotics, (Stella et al., 2023) demonstrated how GPT-4 could also be used throughout the conceptual technical design and fabrication process.LLMs have also been applied to end-to-end ideation and synthesis in chemistry (Boiko et al., 2023).
Beyond the application of LLMs, other generative methods have been used for end-to-end design in other domains, such as materials synthesis (Hsu et al., 2022;Merchant et al., 2023) and soft robotics (Wang et al., 2023).

End-to-End Design Examples
In the preceding sections, we have explored how GPT-4 can benefit different stages of the design and manufacturing processes.Now, we consider the comprehensive end-to-end design processes for two examples that have been consistently referenced throughout this manuscript: the cabinet and the quadcopter.From initial design and evaluation to manufacturing and testing, we showcase how GPT-4 can be combined with traditional (modified) workflows at each stage to realize working physical objects.For the cabinet, GPT-4 automates 3D design, part sourcing, manufacturing instructions, assembly guidance, and performance evaluations.Similarly, for the quadcopter, GPT-4 facilitates design, part selection, manufacturing, assembly, and testing.These examples highlight the potential of GPT-4 in streamlining the product development process, while simultaneously exposing areas in which pain points persist.
3.1.Cabinet.In this subsection, we consolidate and elaborate on the comprehensive LLM-assisted development process for a wood cabinet with minimal human intervention.Throughout this process, GPT-4 played major roles in generating the 3D design, facilitating part sourcing based on the design specifications, generating machine-readable manufacturing instructions, providing human-readable assembly guidance, and conducting final performance evaluations (Figure 1).This holistic approach highlights the extensive capabilities of GPT-4 in automating and optimizing various stages of this product development process.
3.1.1.Design.We employed GPT-4 to generate Computer-Aided Design (CAD) models using such as OpenJSCAD (Part 1 Section 6.1.1).After a few iterations, GPT-4 successfully generated accurate CAD design files (see Part 1 Section 6 for more details).Additionally, we demonstrated utilizing GPT-4 to modify the cabinet design, including tasks such as adding a door and a handle (Part 1 Section 6.2.1) or adjusting the number of shelves (Part 1 Section 6.3).GPT-4 even integrated prefabricated elements like brackets into the cabinet design (Part 1 Section 6.2.2).We ultimately decided to fabricate the version of the cabinet presented in Part 1 Figure 2 with one shelf supported by four brackets.
3.1.2.Manufacturing.We then tested GPT-4's capabilities in tackling real-world manufacturing challenges encountered during the fabrication of the cabinet.These challenges encompass various aspects of the general manufacturing process, such as part sourcing, generating machine-readable instructions, addressing material and fabrication constraints, facilitating design modifications, providing assembly guidance, and offering human-readable instructions.
For this particular example, our manufacturing plan involved utilizing a universal laser system to cut commercially available wood boards.First, by leveraging the detailed design specifications in OpenJSCAD, GPT-4 provided accurate suggestions for sourcing specific items, including brackets from Home Depot and wood pieces from McMaster.Each suggestion was accompanied by a concise description of the item (Part 1 Figure 24 and Section ??).Furthermore, in order to realize the subtractive manufacturing approach, GPT-4 successfully generated precise 2D DXF files for laser cutting the wood pieces.These files were derived from the original 3D OpenJSCAD model, effectively guiding the laser cutter in executing the required cutting patterns (Part 1 Section Section 8.4.2).
We also showcased GPT-4's ability to adapt the design based on fabrication constraints.For example, the largest wood pieces we were able to obtain from McMaster that could fit into our laser cutter were 12 by 24 inches.Unfortunately, this size was too small to support the original cabinet design.As we presented this constraint to GPT-4, it successfully scaled down the entire design to ensure compatibility with the available wood pieces (Figure 3).Given that our laser cutter is capable of engraving patterns onto the cutting materials, we took advantage of this feature and requested GPT-4 to generate Python code for producing DXF files that could engrave words onto the side board of the cabinet.GPT-4 was able to fulfill this request, providing us with the necessary code to create the desired engraving.
Finally, once we had the laser-cut wood pieces, we proceeded to assemble them into a fully functional cabinet.During this process, we sought GPT-4's expertise in recommending the most suitable adhesive for wood.GPT-4 suggested Quick-Set Epoxy, 3M DP100, 1.64 FL. oz.Cartridge as the optimal glue for our purposes (Figure 4).Additionally, GPT-4 provided detailed treatment guidance and step-by-step procedures for the assembly, ensuring a smooth and successful construction process (Figure 4).The final fabricated cabinet is showcased in Figure 5, offering three different view angles.To provide a clear perspective of its size, a mug is placed on the middle shelf, serving as a reference for scale.
3.1.3.Performance evaluation.Using the 3D OpenJSCAD design as a basis, GPT-4 was prompted to develop functions to evaluate performance criteria: storage capacity, load capacity before sagging substantially, plywood material cost, and accessibility for individuals in wheelchairs.See Part 1 Section 9.1 for details on this type of design evaluation for numerical evaluation and Part 2 Section 9.2 for subjective metrics.When comparing these values for the original and scaled cabinets, the storage capacity, load capacity before sagging, material cost, and accessibility were computed to be 9,205 in 3 , $67, 14.336 lbs, and a score of 5/10 for the original cabinet, and 1,131 in 3 , $18, 8.601 lbs, and a score of 5/10 for the scaled-down cabinet.It failed to compute an appropriate strength and could not verify the equation, but other evaluations are correct.Based on these numbers, both J u s t A c c e p t e d Now let's fabricate this cabinet using wood.I have a universal laser system with a processing area of 32 by 18 inches.Can you recommend the materials I should buy?Ideally provide me links.
Based on your design, it appears you're working with a thickness of approximately 1/2 inch, which is a standard size for many types of wood.I'm going to suggest a couple of options for you: Baltic Birch Plywood -This is a good option for cabinetry because it is strong, has a nice grain pattern, and finishes well.It's also highly recommended for laser cutting because it cuts cleanly and has minimal smoke and resin.
Here's a link to purchase from McMaster-Carr: Baltic Birch Plywood Sheets.Medium-Density Fiberboard (MDF) -If you're looking for a more budget-friendly option, MDF might be the way to go.It doesn't have the strength or the aesthetic appeal of real wood, but it is very consistent in density and thickness, making it ideal for precision work like this.It's also known for its smooth finish, and it laser cuts very well.Here's a link to purchase from McMaster-Carr: Medium-Density Fiberboard (MDF) Sheets.
For brackets, you can find a variety of options from Home Depot or other similar stores.Depending on the size and weight of your cabinet, you can choose a bracket that suits your needs.Just make sure the brackets you select fit the dimensions in your OpenJSCAD code.Here's a general link to the bracket section of the Home Depot website: Shelf Brackets -Home Depot Remember that working with laser cutters requires adequate ventilation due to the smoke produced during the cutting process, and the edges of the wood will be darkened due to the heat from the laser.If you want to paint or finish the wood, it may need a bit of light sanding to clean up the edges.

J u s t A c c e p t e d
The largest wood piece I can find on McMaster that can fit into the 18 by 32 inches laser cutter is 12 by 24 inches.Therefore, we need to scale down the initial cabinet design such that every piece can be cut from the 12 by 24 inches plates.The thickness of the wood piece I selected is 1/2 inch.Can you regenerate the python code for dxf considering this fabrication constraint?
We need to make the width of the back board to be less than or equal to 12 inches.One possible solution is to divide the backboard into two equal halves, each with a width of half of the original.They could be joined together after cutting.Let me modify the code accordingly:  Here, the available wood board size is insufficient for the original cabinet design.GPT-4 successfully scaled down the entire design, ensuring manufacturability.cabinets were qualitatively classified as "strong" on a range of "strong," "moderate," or "weak," being far stronger than a typical cabinet used in an office.However, while the larger cabinet was categorized as "highly useful" on a scale of "useful," "moderate", or "not useful" due to its high strength and storage capacity, the smaller cabinet was only "moderately useful" due to its small size.Further aesthetic evaluation in text form is included in Figure 6, classifying without computation the original design as "Contemporary" and the scaled design as "Minimalist".
3.2.Quadcopter.In this subsection, our objective is to design a functional indoor quadcopter capable of flight, hovering, and landing, using GPT-4 with minimal human intervention.The Here's a simple procedure to assemble your cabinet: 1. Dry Fit: Start by fitting all the pieces together without any glue to ensure they all fit properly.2. Sanding: If necessary, sand the edges of your wood pieces for a better fit.Be careful not to alter the dimensions significantly.

Gluing Procedure:
• Back Board to Side Boards: Apply a bead of glue along the side edges of the back board.
Align it with the side boards such that the back board is sandwiched between the two side boards.Ensure the edges are flush.Clamp them together and wipe off any excess glue that squeezes out with a damp cloth.Allow the glue to dry according to the manufacturer's instructions.• Bottom Board: Apply glue along the bottom edges of the side and back boards.Position the bottom board so that its edges are flush with the bottom edges of the side and back boards.Clamp and allow to dry.• Top Board: Similarly, apply glue to the top edges of the side and back boards, align the top board, clamp, and allow to dry.• Shelf Board: Apply glue along the edges of the shelf board where it will contact the side and back boards.Slide it into position and clamp.Allow to dry.

Finishing:
After the glue is completely dry, you can finish your cabinet with a finish of your choice (paint, varnish, etc) to increase its durability and improve appearance.Remember, it's important to clamp your pieces together after you've applied the glue to ensure a tight bond.Also, ensure your workspace is level so your cabinet comes out square.
Please remember to follow all safety guidelines when working with tools and materials, including wearing appropriate personal protective equipment.Always work in a well-ventilated area when using glues or finishes.GPT-4 provides guidance on glue selection, pre-treatment, assembly sequence, and finishing options process involves sourcing parts, creating a 3D design incorporating the chosen parts, and ultimately, manufacturing, assembling and testing the quadcopter, as depicted in Figure 7.
3.2.1.Parts Sourcing.Many real-world systems, like drones, are built from pre-existing components with various kinds of specifications.Therefore, our first task is to utilize GPT-4 to select appropriate parts for our specific use case.This was successfully accomplished using GPT-4.The detailed process is elaborated in Part 1 Section 8.3, and the selected parts are shown in Figure 9 (left).
3.2.2.Generating Designs from Natural Language Specifications.Upon identifying the components, we employ GPT-4 once more to generate a viable quadcopter design incorporating those parts.
With minimal human intervention, we successfully crafted a geometric design for the quadcopter, as detailed in Part 1 Section 6.2.2.Now, we shift our focus to practical issues: 1) how parts Given its relatively small dimensions, your cabinet could fall into the "Minimalist" aesthetic category.
With its clean, uncomplicated design and single mid-height shelf, it would emphasize functionality and simplicity, which are key principles of minimalist design.Its smaller size would also contribute to an uncluttered appearance, further aligning with the minimalist aesthetic.are mounted onto the designed frame, and 2) whether the frame is manufacturable.Given that describing each part's geometric details is challenging and that GPT-4 doesn't fully comprehend how to design the frame for optimal physical balance, we provide GPT-4 with low-level instructions to guide adjustments to the current frame design rather than expecting it to independently modify the design.First, we adjust the frame bar's cross-sectional size using GPT-4 to ensure it's 3D-printable and adequately robust.Then, we combine all boxes and cylinders that form the frame in the geometric  design.To stabilize the battery placement, we semi-integrate it into the frame, and subtract it from the frame so the frame securely holds the battery.For the controller and signal receiver, which are much lighter and smaller than the battery, we simply glue them onto the battery, eliminating the need for additional accommodations.
Lastly, we mount the motors using screws for stability, requiring screw holes in the frame.To minimize human effort, we utilize GPT-4 to create holes in the frame.For each motor, we instruct GPT-4 to generate four cylinders representing the required holes, detailing the hole specifications via text according to the motor's specifications.Even though crafting mounting holes isn't trivial, we successfully produced the correct cylinders after a few prompts with GPT-4.The cylinders are shown in dark gray in Figure 8(Left).Once the hole cylinders for one motor are ready, we let GPT-4 group and duplicate them using our place function.GPT-4 managed to position the hole cylinders correctly but had issues with proper rotation, which we later manually corrected.
As seen, the hole cylinders overlap with the frame bar.Since we cannot change the frame bar's thickness due to the manufacturing concern, we manually adjusted the frame bar's tip thickness to prevent it from obstructing the holes.This was done manually, as it proved challenging to adequately convey the problem and solution to GPT-4.
Our experiments also revealed a limitation: adjusting designs we completed earlier proves difficult.After extensive interaction with GPT-4, referring back to previously discussed design elements becomes a challenge.Consequently, if any issues arise with earlier addressed parts, it becomes arduous to revisit them with GPT-4 and prompt modifications.Ideally, we should finalize each design without the need for future revisions, as adjustments later prove difficult.The final frame result is displayed in Figure 8(Right).
3.2.3.Design-to-Manufacturing.The only part that needs manufacturing is the frame.Once the frame is determined, we fabricate it using a 3D printer.Because the representation of the frame results from Boolean operations of boxes and cylinders, it is simple to directly convert them to .stlformat which is widely recognized by the 3D printers.We used OpenJSCAD to do it.Once we have the .stlfile, we manufacture the frame using Stratasys Fortus 400 since it has a sufficiently large build volume and it realizes precise, robust, and durable parts.We instruct the printer to use the least amount of infill possible in the print settings.By choosing a lower infill percentage, the printer will create a sparse or hollow internal structure rather than a solid one.This decision helps conserve material and reduce print time without significantly compromising the strength or weight of the copter's frame, given that it is not a load-bearing component.The resulting fabricated copter frame not only meets the required dimensions, but also balances the strength and weight, necessary for optimum flight performance.We visualize the printed frame in Figure 9 (middle).
3.2.4.Assembling and Real-World Verification.With the 3D printed frame ready, we proceed to the assembly stage, integrating the pre-prepared components.Given that assembly considerations were incorporated into our GPT-4-guided design process, the assembly of the quadcopter is straightforward.The battery is secured in the central frame slot using double-sided tape and wrappers.Similarly, the controller and receiver are placed atop the battery and secured with double-sided tape and wrappers.The four motors are attached using screws.All elements are affixed firmly and stably, resulting in a sturdy copter ready for flight, as shown in Figure 9 (right).
Once assembled, we conduct a series of tests.First, we administer an ascending test, directing the copter to lift off the ground and ascend to a specific altitude.This test gauges the combined thrust of the motors and the propellers' efficacy in converting the motors' rotary motion into lift.It also allows us to evaluate the copter's responsiveness to radio transmitter commands, the flight controller's interpretive capacity, and the copter's overall ascent stability.The motion is depicted in Figure 10 (left).
Following ascent, we undertake a hovering test.During this phase, the copter is directed to maintain its altitude and position for a set period.Hovering demands continuous, simultaneous operation of all four motors to counter gravity.This test significantly illuminates the copter's capacity to achieve and maintain stable flight, a vital characteristic of any functioning copter.The hovering motion is demonstrated in Figure 10 (mid).
Finally, we execute a descending test, instructing the copter to safely and gradually descend to the ground.This evaluates the copter's ability to control thrust reduction and the resulting downward motion, as well as the flight controller's capacity to interpret and carry out the descent command.It is also a crucial examination of the copter's landing abilities; a smooth, safe landing is essential to preserve the copter and its components.The descending motion is exhibited in Figure 10 (right).3.2.5.Computer Aided Engineering.We also investigate how GPT-4 can help with measuring the performance of a given quadcopter design.Given a current design iteration of the copter from Part 1 Section 9.1.2,GPT-4 is able to identify important trade-offs to optimize and subsequently implement optimization strategies to improve performance.One such trade-off GPT-4 identified is between weight and size, where smaller copters are generally able to stay afloat longer due to reduced weight and aerodynamic drag, while larger copters have more space to accommodate larger batteries which can provide more energy for longer flight times.Out of all the possible optimization methods to find the best combination of parameters that maximize flight time, speed, and distance while meeting constraints on weight and size, GPT-4 chose a very suitable numerical method of Particle Swarm Optimization (PSO) from the PySwarm library.Aside from being very efficient and simple to implement, PSO has a strong global search capability, which is beneficial when the optimal solution might be located in a large and complex space, and allows for real-time adjusting of the copter's weight and size based on performance data.GPT-4 has a strong grasp on the inherent trade-offs of such systems, and is capable of generating tailored ideas and feasible solutions to optimize performance.
We now turn to the details of using simulation to evaluate the quadcopter's performance.In the workflow of fabricating a functional robot, simulation is often used for both control and collecting performance metric statistics that can be used for optimization.Since our fabricated robot includes its own controller, we focus on using the robot's performance in simulation for design optimization.Our design space involves both a parameterized quadcopter whose frame bar lengths can vary but is otherwise constrained by the design created in Section 3.2.2 and the controller design.While it is possible to ask GPT-4 to provide suggestions on the type of controller to apply, we choose to have GPT-4 generate a LQR controller, which is widely used for UAVs.We break down multicopter optimization into three steps: 1) Given the OpenJSCAD design of the quadcopter, convert the design into a format specific to modeling multibody systems, such as URDF, and the means of computing relevant physical properties that inform the controller design, such as the robot's mass.
2) Given the robot's physical properties, generate a LQR controller for simulation.3) Given a robot design in URDF, functions for extracting the design's relevant physical properties, and a controller, synthesize an algorithm to optimize the robot's design.Converting OpenJSCAD to URDF.We start with the OpenJSCAD quadcopter design developed in Section 3.2.2.Because there is no straightforward equivalent of subtraction and union from OpenJSCAD in URDF, we omit the creation of the holes in the motor base and replace the union of motor base parts with placement of individual links while retaining the essence of the original design.We then take an object-oriented approach to having GPT-4 synthesize the equivalent URDF code.As seen in 11, we prompt GPT-4 to create a Component class whose instances store the geometry, mass, position, and orientation attributes of the corresponding OpenJSCAD primitives.Component instances also have distinct names to represent URDF links.We additionally prompt GPT-4 to generate helper functions for placing instances with different geometries in Fig. 11.This framework allows GPT-4 to generate a function that places components in terms of absolute coordinate positions and orientations and to replicate the Python equivalent of the OpenJSCAD, as shown in Fig. 12.
However, one difficulty is that unlike CAD-like designs, formats for representing robots also require relations between components to be represented as joints in order to accurately simulate the robot dynamics.We prepare for this when synthesizing the component placement script by prompting GPT to store the components as a dictionary, which allows easy access to the components.We tackle the challenge of generating the quadcopter's joints by relying on GPT-4's knowledge of the spatial relation between components in a quadcopter.After equipping the Component class with a function that sets the parent link, we use this interface to have GPT-4 synthesize a sequence of robot joints, as shown in Fig. 13.We find that although GPT-4 understands certain substructures, such as the fact that the motor is placed on top of the motor base and the propeller is connected to the top of the motor, its initial definition results in an invalid URDF format, as both of the frame bars are root links.We thus explicitly prompt GPT-4 to choose one of the frame bars as the root link.
Finally, GPT-4 is tasked with creating a full URDF file.Because the robot is represented with modular Component instances that contain all relevant information on individual links' mass and geometry as well as relations to parent links, it is relatively straightforward for GPT-4 to create helper functions that synthesize URDF links and joints.We note that creating a joint is a more involved task, since the link's absolute position and orientation must be converted to relative position and orientation to the parent link.It is necessary to explicitly prompt GPT-4 to use the appropriate rotation matrices in its calculation; otherwise, it does not appropriately account for how the parent link's rotation affects the child link's relative translation.Because we choose LQR as the robot's controller in simulation, we ask GPT-4 to compute the full assembly's mass and moment of inertia given the Python code it has generated thus far.It outputs reasonable Python code that computes the assembly's center of mass and uses the parallel axis theorem to combine the moment of inertia of the individual links.
In summary, we find that explicitly prompting GPT-4 to provide suitable object-oriented representations, such as the Component class, and modularizing the code generation as much as possible, e.g.asking GPT-4 to synthesize helper functions, placing components, and defining parent-child relations separately are key techniques in successfully converting from OpenJSCAD to URDF.Deriving a LQR Controller for the Quadcopter.Multicopter control is an extensively studied problem, for which various algorithms have been proposed such as PID controllers, LQR controllers, and more complex alternatives.We aim to synthesize a LQR controller, as it is not only a popular choice in the literature but also guarantees optimal control when the multicopter dynamics and stable fixed point are known.In particular, we focus on controlling a quadcopter, which is an underactuated system with 6 degrees of freedom but only 4 independent actuators, resulting in nonlinear dynamics even without aerodynamical effects.However, LQR provides optimal control by linearizing the system around a stable fixed point, which then has a closed form solution.
As LQR is widely used for control, the algorithm is easily accessible through the control Python library that performs LQR with a single function call.As such, the main challenge in using LQR  Figure 13.Constructing Relations between Quadcopter Links.After creating and placing links in the appropriate positions, GPT-4 must infer parent-child relations that are not available in the OpenJSCAD design to complete the URDF representation.In its first attempt, it produces an invalid file because only one of the frame bars can be the root joint.However, GPT-4 demonstrates an understanding of how the motor, motor base, and propeller should be connected and that the main body contains the battery, controller, and receiver.

J u s t A c c e p t e d
for quadcopter control is deriving its state space representation, the A and B matrices, used in the linear model of the system as shown in Equation 3.1.x is a 12-dof state vector, including coordinate position, linear velocity, roll, yaw, pitch, and angular position.u is the 4-dof control input, typically consisting of the upward thrust and the external torque applied in the xyz directions.Note that x in 3.1 is the difference between the current state and a user-specified target state.Similarly, u is with respect to the target control input, where the torques are 0 and the thrust balances out the gravitational force acting on the qaudcopter.GPT-4 is then asked to provide a suitable fixed point at which the quadcopter is stable, and it does provide a correct response. (3.1) In order to verify GPT-4's results, we ask GPT-4 to first synthesize the full kinematics model before using a a symbolic manipulation library, sympy, to compute the state space matrices.This forces GPT-4 to produce an interpretable dynamical model that a domain expert can verify, rather than directly outputting the A and B matrices.This proves to be essential as even though GPT-4 captures the high-level idea of assuming the quadcopter to be a rigid body and applying Newton-Euler equations to describe its acceleration and angular acceleration, it is unable to zero-shot provide the exact model of the system without user feedback.For instance, as shown in Fig. 14, GPT-4 formulates the correct rotation matrix but does not apply it correctly to convert from inertial frame (control inputs) to the body frame (linear acceleration).Although GPT-4 can correct its error when given feedback, this type of error is difficult to catch without rigorously checking the correctness of GPT-4's calculations and highlights the limitation that in some inverse design domains, a user cannot generate precise outputs from GPT-4 without the expert knowledge to perform verification.After this fix, the resulting code for deriving the A and B matrices is still incorrect as the simulated quadcopter sinks downward due to a sign error in the equations for acceleration.Similarly, generating the simulation loop in PyBullet requires several rounds of iteration on the initial code.Some of the mistakes are more obvious, such as incorrectly indexing into the control input vector when applying external force and torques, whereas others are more specific to PyBullet's API.For the former, we directly pointed out GPT-4's mistake.In the latter case, we find that simply giving GPT-4 the error message and asking it to fix its code suffices for our problem.Co-design of Quadcopter's Shape and Control.Now that the quadcopter design can be outputted to URDF and the LQR controller can be synthesized from a given design, we ask GPT-4 to optimize the quadcopter design.Much of the design space has been fixed by the fact that most components have unchangeable dimensions.However, as the framebars are not predefined components but rather constructed from carbon-fiber tubes, we focus on optimizing their lengths.The control then follows directly from a given design, as we only need to compute the design's total mass and moment of inertia to apply the LQR controller derived by GPT-4.With the objective of minimizing the number of simulation steps required to reach a goal height of 1m, we prompt GPT-4 with the full Python script that converts OpenJSCAD to URDF and performs simulation, and we find that it is able to provide an outline of the optimization, consisting of an objective function that involves creating a quadcopter with specified framebar lengths and performing the simulation loop, as shown in Fig. 15.GPT-4 also provides reasonable bounds on the frame bar lengths of (100mm, 500mm) when prompted further.GPT-4 then provides an outer optimization script that uses the helper objective function.We also explicitly prompt GPT-4 to complete its outline by updating component creation code to generate frame bars with the correct masses and to change To optimize a quadcopter's design, GPT-4 identifies proposes to wrap the code that produces the quadcopter's URDF file and that performs LQR control in an objective function, which enables it to perform optimization in the outer scope.However, its initial suggestion to perform SQSLP is unsuitable, as there are no gradients provided from simulation.
J u s t A c c e p t e d These echo familiar themes from Part 1 of GPT-4 struggling with domain translation and reference frame geometry.
Potential Solutions: Explicitly providing as many context cues about tasks, including changes to domain or specifications, can provide more information about which the LLM can reason.
L.3 Scalability.End-to-end workflows represent the longest processes, and as such, it is no surprise that GPT-4 struggles with memory.Adjusting completed designs at the manufacturing and testing stage remains difficult.
Potential Solutions: Careful organizing conversations into broad but shallow trees, and reprompting with context when necessary can help extend GPT-4's scalability.
Specific to long workflows is also the fact that at certain points, file conversion (for different processes) often becomes necessary.For the Quadcopter translation from OpenJSCAD to URDF, this ultimately becomes a geometry-intensive language translation problem.In these regimes, structure is required from the user to help guide the format translation, and we must remind GPT-4 to use appropriate rotation matrices for its calculation.

Discussion
In this chapter, we summarize capabilities, limitations, and dualisms in applying LLMs across all aspects of the CDaM pipeline.If nothing else, we hope you, the reader, carefully consider and examine these points.
5.1.Observed Capabilities and Limitations.Our examination of GPT-4's performance in the fields of design and manufacturing has underscored several crucial capabilities instrumental in supporting the design processes, in addition to pinpointing significant limitations.Next, we summarize these observations and discuss various methods to mitigate the identified limitations.Capabilities: C.1 Extensive Knowledge Base in Design and Manufacturing: GPT-4 boasts a comprehensive knowledge base in the realms of design and manufacturing.Its capabilities extend to solving a wide array of problems and automatically completing specifications, making it a versatile tool across various domains.C.2 Iteration Support: GPT-4 incorporates an iterative approach to problem-solving.When feedback is provided on errors, it attempts to rectify them.This ability to adapt, although not always successful, is a valuable facet of an GPT-4's performance.C.3 Modularity Support: GPT-4 supports modular design, demonstrating the ability to reuse or adapt previous designs or solutions when explicitly instructed.While it does not inherently retain memory of past interactions, explicit instructions can help leverage its modular capabilities effectively.Limitations: L.1 Reasoning Challenges: GPT-4 encounters difficulties with certain types of reasoning, particularly those involving analytical reasoning and computations.These limitations can manifest as notable challenges in the design and manufacturing domain, for instance, a general lack of spatial reasoning capabilities.Potential Solutions: Implementing well-crafted domain-specific languages (DSLs) can help address these challenges.DSLs, widely used in computer science, encapsulate recurring knowledge, rules, and valuable abstractions, thereby filling knowledge gaps.Alternatively, APIs that can perform the complex computations can be integrated.GPT-4's proficiency in creating high-level abstractions can be utilized to generate inputs that be processed through APIs by computational solvers.L.2 Correctness and Verification: GPT-4 often produces inaccurate results or justifications for its solutions and lacks the ability for self-verification.Text output by GPT-4 should be verified before being trusted, rather than the converse.Potential Solutions: Apart from relying on human verification, automated verification can be accomplished by utilizing APIs that conduct checks and validations.By leveraging GPT-4's iterative capabilities (C.2), we can create a feedback loop that continues until a satisfactory solution is obtained.L.3 Scalability: As tasks become larger or more complex, GPT-4's performance can deteriorate, often struggling to manage multiple tasks concurrently.Potential Solutions: One strategy is to partition larger tasks into multiple sub-tasks.For instance, rather than requesting it to evaluate multiple performance metrics simultaneously, it may be more effective to request them individually.When constructing more complex models, employing an incremental design process can prove beneficial.Components can be designed and verified separately J u s t A c c e p t e d before being assembled into the final model.GPT-4's modularity support (C.3) can be used to facilitate the creation of complex models from a series of instructions.L.4 Iterative Editing: When a design needs modifications, specifying those changes as a prompt will often lead to unsatisfactory behaviors.This is because the GPT-4, upon receiving a change prompt, will regenerate the design, often overlooking elements specified in previous prompts.This situation poses challenges to design editing.Potential Solutions: Our solutions involved either feeding the prompts back in to create a full specification in a single prompt, though this also caused challenges due to the scalability limitations discussed above.The alternative was explicit modularization to enable parts to be designed and reused, though the latter was not possible in some cases.Dualism: D.1 Context Information: GPT-4's performance improves significantly with the provision of context information.The more detailed the domain description, the better it performs.Furthermore, GPT-4 is adept at providing context for its actions, making it an asset in sequential workflows.This characteristic proves particularly beneficial when using GPT-4's generated content in subsequent tasks, as these tasks can utilize the context included in the output from the initial task.However, it may also be harmful if the user does not provide enough context, or if the user would like to create an unusual design, but GPT-4 is unable to overcome the biases it associates with a particular domain.D.2 Unprompted Responses: GPT-4 often infers aspects that are not specified in the prompt, either auto-completing specifications or finding ways to make decisions without enough information.While this is interesting in the design context in terms of allowing for partial specifications that can be auto-completed, it can sometimes be overly proactive, guiding the design in some aspects, which may limit creativity.

Ethical Considerations
While powerful LLM-augmented CDaM technology has the potential to improve design, manufacturing, and livelihoods, it also raises risks.Here, we briefly outline some of these.We emphasize that this list is not exhaustive.
Job Displacement: At its best, the integration of generative AI technology into CDaM workflows has the potential to make design at the industrial and hobbyist levels accessible to everyone.This could expand the possibility of what designers and engineers can create, and accelerate their workflows, in turn enabling individuals and companies to take bigger creative risks in products and open new markets.Even in this best-case scenario, without efforts to train a workforce in how to use and/or develop AI technology, this could further inequality between skilled and unskilled workers.If, however, generative AI does not enable enhanced worker creativity (or if it is not applied for that purpose), and instead it allows companies to churn out products with less labor or less skilled labor, then generative AI not only runs a serious risk of damping industries, but also will displace workers, especially at the skilled level.
Design Data Privacy And Security And Intellectual Property (IP): Interrogative LLMs generate output in response to human query.This language and these responses encode "foundational knowledge" across domains; however, where does this knowledge come from?Current LLMs are trained from public data, privately procured exclusive data, and in some cases, private user data.Who owns that data?If it is the user, and that data is then used to train LLMs, then how can the user be sure that their data is not being leaked through the models to other users?This is especially important when dealing with sensitive user data, or sensitive company data, in the case of IP.If LLM-augmented CAD programs are used to invent new technologies, it must be determined if that IP belongs to the user, and if so, how LLM service providers will ensure that data (or the knowledge it encodes) is not improperly ever transmitted to unprivileged parties.
Dual-use Nature and Potential Military Applications: Already we have seen demonstrations that LLMs can be used for creating dangerous malware, which could have immediate serious ramifications and application.It is not hard to imagine LLM-augmented CDaM technology being used for other weaponized purposes, such as creating new military vehicles, payloads, or other equipment.While some of this technology can be used for strategic defensive and life-saving operations, it can also be used for offensive capabilities, and often-times, the classification of defensive and offensive capabilities are contextual and/or subjective.To what level such technology should (and even can) be regulated and deployed must be a subject of policy debate at both governmental and industrial consortia levels.We require international collaboration and active and open debate on ethical use in military applications.
Design Reliability and Accountability: As we have seen, LLMs get things wrong, and in certain domains, not infrequently.LLMs can rarely be trusted blindly right now, but in the future, they may become robust enough to operate autonomously on difficult queries.In both these advanced situations and even today as AI proliferates through society and users learn more about it, we must ensure that downstream consumers are broadcast accurate measurements of task uncertainty and appropriately vet those answers.Further, when generative AI models do make mistakes (perhaps without any human-in-the-loop), we must apply existing or develop new models of accountability and, when harm is committed, appropriate compensation for damages.
Limited Access to Tools: AI-powered CDaM technology can empower everyone, but only if everyone has access to them.If running such technology requires access to advanced hardware or embargoed chipsets, then only privileged classes will have access to this technology.Rather than empower and reduce inequities, AI-empowered LLM technology can then become a vehicle for inequality, especially by amplifying existing gaps.

Conclusions
In conclusion, we find that GPT-4 possesses numerous capabilities that can be leveraged within the domain of design and manufacturing.This area, where creativity converges with practicality, presents exciting opportunities for advancements that can potentially bring about a significant shift in the way we ideate, prototype, and manufacture a broad range of products.
However, it is essential to recognize that substantial work remains to be done to fully support the integration of these tools within this field.A fundamental issue is that design for manufacturing involves a delicate balance between creativity and formal verification.Engineering design presents a paradox; it requires precision and exactness, yet thrives on an iterative and exploratory spirit.While our experiments have managed to circumvent limitations for formal reasoning through user guidance, DSL crafting, and APIs that call computations, there is still much to understand about how best to implement these strategies.
For instance, we hope that our analysis can stimulate new insights about DSL design.Historically, DSLs have been developed with human users in mind.However, when we shift our perspective to creating DSLs for an AI coder, new questions and possibilities emerge.What should these DSLs

Figure 2 .
Figure 2. Part Sourcing Guided by GPT-4 Based on the Specific Cabinet Design and Manufacturing Method.In this example, we employ laser cutter and premanufactured hardware.

Figure 3 .
Figure 3. Handling Manufacturing Constraints with the Assistance of GPT-4.Here, the available wood board size is insufficient for the original cabinet design.GPT-4 successfully scaled down the entire design, ensuring manufacturability.
Great.I purchased Play Wood from McMaster with 1/2 inches.I want to use glue to assemble them together.Please provide a link for the glue I should buy and give me the details treatment procedure.For plywood assembly, a good choice would be Titebond III Ultimate Wood Glue.It offers superior strength, a long open-assembly time, and is safe to use.You can find it on the McMaster site here.

Figure 4 .
Figure 4. Comprehensive Assembly Instructions Provided by GPT-4.GPT-4 provides guidance on glue selection, pre-treatment, assembly sequence, and finishing options

Figure 5 .
Figure 5.The Manufactured Cabinet.Real-world cabinet manufactured from different view angles, showcasing its scale with a mug as a reference.

Figure 6 .
Figure 6.Aesthetic Description.GPT-4 comments on the aesthetic category of the designed and fabricated cabinets based on its Geometry.

Figure 7 .
Figure7.The Quadcopter Design Process using GPT-4.The process involves sourcing parts, creating a 3D design, manufacturing, assembling and testing the quadcopter.We successfully manufactured a working quadcopter.

Figure 8 .
Figure 8.The Quadcopter Frame.Here we show that GPT-4 can create cylinders for the motor mounting holes.Using boolean operations, we successfully created a valid frame.

Figure 9 .
Figure 9.The Parts and the Printed Frame of the Copter.Left: Selected parts.Middle: Printed frame.Right: Assembled copter.

Figure 10 .
Figure 10.The Flight Test.Left: the ascending test.Mid: the hovering test.Right: the descending test.
Figure 11.Creating Object-Oriented Representations of Robot Components.Using the Component class for representing a robot link facilitates the placement of links with the place() function.Later in the example, the Component class is modified to additionally keep track of an instance's parent link.
Figure12.Replicating Quadcopter Component Placement.Disregarding the union and subtract operators in OpenJSCAD, the above Python code accurately reproduces the locations of the components as the ones in the OpenJSCAD design.The only error is that certain components do not have unique names, which GPT-4 later corrects on additional prompting.

I
Figure15.Code Skeleton for Optimizing Quadcopter Design.To optimize a quadcopter's design, GPT-4 identifies proposes to wrap the code that produces the quadcopter's URDF file and that performs LQR control in an objective function, which enables it to perform optimization in the outer scope.However, its initial suggestion to perform SQSLP is unsuitable, as there are no gradients provided from simulation.
Initial Position (b) Final Position Figure 16.Simulating Quadcopter Performance.Left: Following the OpenJSCAD to URDF conversion script provided by GPT-4, we show the quadcopter rendered in PyBullet's simulator.The quadcopter has optimized frame bar lengths discovered with the script provided by GPT-4 and is resting at its initial position.Right: Using the LQR controller designed by GPT-4, the quadcopter is able to lift off and hover at the goal position 1m off the ground.