Tim Fu explores the transformative potential of AI in the architectural field. From enhancing design workflows to envisioning a future where AI-generated 3D models and robotics become integral, he unravels the gradual yet promising impact of AI technology on the architectural world.
Today we are standing at a unique turning point in history. With the emergence of generative AI models such as diffusion models, for the first time, we are seeing machine intelligence being readily adaptable to the creative discourse of architectural design and construction. I have had the opportunity to experience the early adoption of diffusion AI in the industry working at Zaha Hadid Architects, as well as through my independent continued research in this field, which led to me to found my own design firm in order to orient more towards AI adoption. We are in a unique circumstance where we are experiencing the naissance of technology. Amongst a plethora of use cases for AI, some are already widely introduced in the industry, while others are more speculative and will advance as the technology processes. I will first discuss the more readily available AI use cases that can enhance the current architecture design workflow, but will also reveal some methods of advanced AI adaptation in my research, which to highlight, is subject to continued development as AI technology processes.
My experiments with diffusion AI began in July of 2022. During the time I was working as a designer at Zaha Hadid Architects, I began noticing a great deal of AI-generated imagery of architecture that started flooding social media. I was fascinated by how purely “generative” the ideas are, considering they are derived from concepts described by text directly into imagery. At this phase, I started exploring the early versions of text-to-image diffusion AI platforms like Midjourney and Dalle2. As various AI images of buildings “designed by Zaha Hadid” circulated on the internet, I took note. From my experience designing at ZHA, I knew the AI fell short in many regards, so I wondered how I could bring more refinement into a design that prompted “Zaha Hadid”. My early experiments focused on refining the form of the design, realism, and constructibility. I produced each building as a standalone concept of architectural expression and began sharing them on social media. As I gradually gained traction, Parametric Architecture Academy reached out to me to teach my unique methods of using AI to design. It is where I started to methodically document observations and design workflows that aided the control of diffusion AI and the quality of image output.
With Midjourney, there’s already plenty of ways to adopt it into the design workflow. As of writing this, Midjourney features the most advanced text-to-image diffusion model. It currently produces the most photorealistic output and resolved geometry, all while creatively negotiating your prompt concepts, be it form, typology, style, materiality, or others. This is great for the concept phase in architecture.
1) A standard approach to a concept design would start from gathering precedent images for reference of style and material system, or to create a mood board. Now AI AI-generated results can take the role of the precedent study, but it will create much more relevant results than what is queried.
2) Beyond serving as a mood board, the Midjourney platform can also be used for creative concept generation. When no specific idea of form or materiality comes to mind, very abstract prompts can be written to generate designs that are more spontaneous and less expected, this “creative idea searching” process can often result in ‘happy accidents’, which can be taken into development.
3) Once satisfactory designs are found, further iterative work can be done to bring the design towards the goal of the project. For example, you can use “remix” to further nudge the design towards intentions that can be described by words, or you can use “vary region” to adjust individual components of the architecture towards the goal of the project.
The prospective future of using diffusion AI in architecture lies in its inevitable capabilities to generate spatial concepts in 3D. As of writing this, Midjourney V6 is already in development (https://www.pcguide.com/apps/midjourney-v6/). In addition, there are speculations that the team is working on text-to-3D models. This will surely change the way we adopt this model. There are two methods of 3D generation. One is through Image-based depth estimation using a deep neural network, creating a depth map for the image to be represented in 3D. The second method is the training of diffusion models from a direct 3D database. Once the 3D models are associated with their text prompts, we would be able to generate 3D models in similar ways we generate 2D images. The former method already has early versions available but it lacks geometric refinement and accuracy. The latter would eventually develop to the same fidelity as the current 2D diffusion models, thus there is high potential to be used directly in modeling software in architectural practice. The rough diffusion 3D output will likely be in the form of an approximated mesh, where manual modeling can take over and fine-tune the result. Likely in the coming days, a myriad of AI tools will come in the form of plugins that will directly create designs and assets in a 3D modeling ecosystem.
When early concepts can generate immediate visuals, these assets become useful to client discussions. Although the architecture of the spaces is not defined or configured, the use of AI media can convey the visual essence of the design and mood, which becomes evident to the respective client acquiring the design service. To further this use, another AI tool that can provide a better idea of the design concept is the use of image-to-video AI. For example, programs like Runway ML can be used to animate an AI-generated image into a video walkthrough. Neural networks predict the video output based on one image, so inputting an AI-designed architecture interior can result in a live walkthrough of the spaces, providing better early visuals during the design process with the client.
The next step of using AI in architectural design is to allow diffusion models to help rapidly render realistic imagery from accurate 3D models. This is a big step forward in the practical application of AI as it allows for accurate control of architectural output based on programmatic requirements. LookX is the first program that comes to mind on the frontlines of this technological development as it offers a diffusion model service that is trained off of an architectural image dataset. This greatly improves its use case in the context of architectural design practice. With lookX, you can input images of any kind and use the trained model to reproduce the image into an architectural render. For example, inputting an image of a concept building massing into LookX is already enough for it to generate a seemingly finished building with industry-style rendering (albeit quality is still an ongoing improvement). This means that program massings can be primitively modeled using Sketchup or Rhino, and LookX will render a completed architecture, with building envelope details, and full materiality. This form of rendering is a novel concept, as generative rendering is not only a visualization process but now also becomes a design process. Materiality, construction, and tectonics can all be explored rapidly by offering a plethora of results using this AI.
Beyond rendering with AI, it is a tool that can introduce new design workflows. With programs like LookX, we redefine the concept of rendering from the realm of visualization to the realm of design. Since images can be turned directly into architectural renders, we can introduce a myriad of alternative design workflows. One can provide a conceptual hand sketch of the building and convert that into an AI render with resolved construction details. One can put together a massing model made of basic geometry, like children playing with blocks, and convert them into a finished architecture visualization. One can even crumple a piece of paper, and ask the AI to reinterpret it as a finished building. The possibilities are endless. From my end, I’ve also converted images of forms found in nature, such as a pinecone or a flower, into an office or residential building. The idea is that we can now take inspiration from various forms found around us and in nature and readily deploy AI to explore architectural expressions that mimic these forms.
In the realm of traditional architectural workflow, we can readily adapt AI rendering as an early form of visualization alongside accurate data. For example, we can develop the building into detailed CAD drawings of floor plans, and ask the AI to interpret these drawings as 3D floor-plan renders. This is made possible with the configurable AI models that LookX features. With LookX, we can readily attach LORA (Low-Rank Adaptation) models to their trained model. LookX also provides a feature that allows various users to train their own LORA models from their own selection of images. These models can be of different contexts, such as 3d-floor-plan renders, detail renders, interior renders, aerial renders, etc… Attaching these LORA models allows the generation of a specific style of content that is needed. LookX also provides a forum where users can share their own LORA models they trained, thus creating a wealth of crowd-sourced resources. This process of model sharing can help democratize the use of advanced AI tools for the future generation of architects and designers.
On the theoretical side of architecture, using AI to design can be seen as a heavily formalist approach to design. It is important to understand AI tools as a design aid, supplementing the other crucial architectural processes of resolving contextual considerations, from programs and circulation to the tectonicity of detailing. With that said, AI is a powerful tool that can provide a wide variety of complex geometric expressions. My work prior to AI pertained heavily to parametric architecture and complex façade systems. The AI very quickly became a design aid when I worked on projects at the concept level. As AI can blend patterns and forms with architectural imagery, it can quickly generate parametric structures and seemingly algorithmic forms. The line of parametric design becomes a great benefactor of the use of AI. I’ve generated various parametric forms, from parametric timber structures, to parametric office façade systems. The latter half of the stage requires manual interpretation/rationalization of the AI image through 3D modeling. The deployment of parametric modeling tools like Rhino Grasshopper becomes a powerful tool to use in this context. Alternatively, we can also start by generating massings using these algorithmic tools and then allowing AI to imagine the details. Thus, AI and parametric design possess overlapping features and can be further explored.
AI diffusion models can also be adapted with other types of AI computational models. The umbrella term “AI” covers a myriad of approaches to using machine intelligence that can also work with diffusion AI. Within architecture, there exist various fields of research that can categorically be considered to be within the domain of “AI” and machine intelligence. One model I’ve worked with is the agent-based algorithm. In such instances, individual agents/particles are programmed to behave a certain way, responding to the proximity of the surrounding agents as well as hard-wired behavior. Collectively they exhibit emergent behavior as a swarm. This computational process mimics the swarm behaviors we see in nature, such as how birds flock in swarms, a school of fish swims as a collective, or how physarum(slime mold) can optimize the most efficient network of paths. This form of AI can be used to find form. For example, a series of particles can flow and generate geometry that simulates the growth of plants while optimizing for circulation. Diffusion AI can be used here to convert the diagram of paths in the plan into circulations for landscapes and urban environments, again with the use of LORA models that are trained on urban data.
Another combination of AI I’ve explored is the use of genetic algorithms with diffusion AI. Genetic algorithms (GAs) are computational models that explore design options while optimizing for quantifiable requirements by a selection process that mimics natural selection found in the evolution process in nature. For example, an urban housing project requires various parameters to be optimized, such as solar radiation, unit typology, GFA, and unit views. The high number of units and their individual considerations can be more efficiently solved with GAs. The parameters can be quantified into values, in which the GA either tries to maximize, or minimize. The computational geometric model can be set up with many input variables that can control the layout and geometry of the output building, and these outputs can then be analyzed for those required programmatic or sustainable considerations. The performance of each solution to these requirements can then be output as values. GA would take control of these inputs and generate a high population of solutions for each generation. And the output performance values will decide which solutions are kept in a generation. With subsequent generations improving from the last, we result in a selection of optimized solutions from a wide search space. Hooking this process onto diffusion AI allows for immediate visual feedback in this solution-searching process that would inform the designer how the set-up conditions of the algorithm can affect the design. With the immediacy of renders readily available, we can make informed qualitative design choices while AI makes quantitative optimizations. This is an ideal combination of human and machine intelligence at play.
The field of architecture is gradually being increasingly disrupted by the emergence of AI. We have various methodologies to utilize the current wave of tools released. However, due to the acceleration of development in AI technologies, we should mark our current time as the mere naissance of this technology. These particular diffusion models we see today are only the beginning of their development, and our current methods of deploying these tools will undergo various changes. With diffusion models, we are already looking towards better versions, such as Midjourney V6 or Dalle3 (not yet out as of writing this article). These models are in a constant path of improvement to perform more accurately to the user’s prompt and make better design decisions. All of these programs use 2D pixel-based algorithms, but 3D is next on the horizon. When these models are trained directly on 3D data, we will see AI with an understanding of form and space. These 3D AIs will surely develop more functional architecture and spaces. The introduction of large language models (LLMs) will also improve this process. Dalle3 utilizes OpenAI’s GPT4 to prompt, this feature combines the intelligence of LLMs with diffusion models. This means we will be able to describe the details of the design and reproduce them from various angles and potentially in plan drawings. We may also eventually link AI intelligence (perhaps LLMs) with BIM modeling. There could be a future where after describing the requirements, we will be given solutions that automatically calculate detailed construction drawings and budget calculations. AI having an accurate representation of architectural data will surely change the game in the near future.
Beyond taking over software used in architectural design, we may see AIs being incorporated into the fabrication process. Robotic fabrication and construction are already becoming more widely available in the construction process, but incorporating AI into the automation of robotics can allow efficient deployment of the robot construction team to fulfill most of the physical labor needed in construction. Humans right now are far more adaptable and intelligent than AIs when it comes to locomoting in the real world, but that may no longer be the case. When robots are deployed and mass organized by a central artificial intelligence system, they can self-organize and be deployed to perform tasks much more efficiently and accurately than the human task force. We already see many such examples, like robotic arms used to lay bricks in parametric formations, a feat that humans cannot do accurately. A machine task force will also eliminate the associated risks and dangers of working on-site. Labor costs at some point will exceed those of machine-driven fabrication processes, and there will eventually also be a financial incentive for machines to take over.
Overall, we are at the naissance of AI technology and there are many ways to take advantage of diffusion models for ideation. These methods will continuously change as the technology improves. We are only entering this era of machine intelligence and we already see the vast variety of methodologies in incorporating AI in design. The immediacy of using diffusion AI to render ideas into details is already shifting the process of design, and the use of intelligent computational algorithms will keep pushing architecture toward efficiency and speed. The entire architectural design-to-production process might soon be dominated by AI and robotics as we incorporate various AIs to interact in various modalities. We cannot know definitively how the development of AI technologies will define the field of architecture and design, but we know it will dominate all our processes in the coming era.
Author: Tim Fu
Designer and Founder, Studio Tim Fu Ltd. Tim Fu is a renowned architectural designer specialized in advanced computation and artificial intelligence (AI). Emerging from Zaha Hadid Architects, he has founded Studio Tim Fu, a high-tech design practice pioneering the integration of AI into visionary design. As an active educator, he has run workshops at Harvard GSD, PA Academy, and lectured in various universities and conferences globally. Leveraging digital platforms, Tim has also built a notable online presence, sharing insights into the overlap of technology and design. His AI explorations have been featured worldwide, including Sky News, Bloomberg, ARTE, AD, GenAI Summit, NXT BLD and has been exhibited during the Venice Biennale.
If you would like to ready more case studies like the one above please check our annual publication
Architecture Competitions Yearbook
The post Using AI for architecture in practice appeared first on Competitions.archi.