Tommaso

Bridging Web and Figma: Automating Large-Scale UI Dataset Generation for AI-Enhanced Design

ACM SIGCHI Symposium on Engineering Interactive Computing Systems (Companion) (EICS Companion), 2025
Best Late Breaking Result Award

Abstract

Large-scale User Interface (UI) data is essential for developing Artificial Intelligence (AI)-driven tools that can support designers in creating interfaces. However, many publicly available datasets are either manually annotated, a time-consuming and costly process that limits their scale or lack crucial structural information, such as semantic labels and hierarchical relationships, necessary for effective design assistance. Moreover, no existing dataset offers a standard format designed for seamless integration of AI models into real-world design tools. In this work, we introduce a pipeline that automatically converts any HTML content into structured, Figma-compatible representations. To validate our pipeline, we apply it to WebUI, a large-scale HTML-based dataset, and conduct a comparative evaluation by training five state-of-the-art layout generation models on our data and the manually annotated Rico dataset. Experimental results demonstrate that the models achieve comparable performance across both datasets and suggest that our pipeline can effectively produce high-quality data suitable for training AI models integrable into design workflows.

BibTeX

			
@inproceedings{russo2025web2figma,
author    = {Francesca Russo and Tommaso Calò and Luigi De Russis},
title     = {Bridging Web and Figma: Automating Large-Scale UI Dataset Generation for AI-Enhanced Design},
booktitle = {Companion Proceedings of the 17th ACM SIGCHI Symposium on Engineering Interactive Computing Systems (EICS Companion '25)},
year      = {2025},
pages     = {13--20},
doi       = {10.1145/3731406.3734974},
url       = {https://doi.org/10.1145/3731406.3734974}
}