Currently underway, I am currently exploring the use of conditional adversarial networks (CAN), a machine learning strategy, on images to generate 3D models of building facades.
To begin, I utilized a pre-trained models from Affine Layer’s ‘pix2pix - Tensorflow’, which is based on an academic paper from UC Berkeley researchers. I used datasets of over 600 images from Center for Machine Perception’s facade database. The images are from a variety of urban European facades. Each image is labeled & divided into 12 classes, which describe the elements of the facades.
These classes include: facade, molding, cornice, pillar, window, door, sill, blind, balcony, shop, deco, background.
I added my own class of labels to account for the size of each element so that the pixels were scaled uniformly. This would help Revit apply the appropriate dimensions to families which were eventually placed.
When modifying the pre-trained model, the images were split into testing & training sets. The testing set is used to validate the model trained on the training set. A few of the images from the testing set were also eventually used to generate the Revit model.
The results from the research paper yielded a promising outlook for using CANs on images of the built environment for use in computational design. The inputs were PNG files with shapes drawn to correspond to the labels from the dataset. The outputs are new images of facades.
From there, there was the task to convert these outputs to Revit elements. All walls, windows & Revit families related to the labels are placed according to the layout in the image.
The script then resizes the Revit elements according to their predicted dimensions.