UI design is at a stage of disruption where Machine Learning models understand basic UI design patterns and use generative models to generate variations of UI patterns. This experiment demonstrates one of the generative design process components that can augment human designers in the real world.
I wanted to build a basic image recognition system that identifies the hand-drawn pattern and matches it with the UI element. It is the first step towards building a system that can generate fully developed UI design using image recognition. For this experiment, I used Google's Teachable Machine to build a model (based on image classification transfer learning).
Launch Demo
Google's Teachable Machine is just one of the many ways to use pre-trained models for building new use cases. There is another option from Microsoft called Lobe that offers similar functionality. Quality of prediction directly depends upon the variety of data collected for labeling purposes. In short, there are ways to get started in the model building process without relying on data scientists for primary use cases.