Detection and recognition of hand drawn website UIs dataset
The increasing importance of User Interfaces (UIs) for companies highlights the need for novel ways of creating them. Currently, this activity can be slow and error prone due to the constant communication between the specialists involved in this field, e.g., designers and developers. The use of machine learning and automation could speed up this process and ease access to the digital space for companies who could not afford it with today’s tools. A first step to build a bridge between developers and designers is to infer the intent from a hand drawn UI (wireframe) or from a web screenshot. This is done by detecting atomic UI elements, such as images, paragraphs, containers or buttons. The provided data set consists of 4,291 hand drawn images inspired from mobile application screenshots and actual web pages. Each image comes with the manual labeling of the positions of the bounding boxes corresponding to each UI element and its type. To avoid any ambiguity, a predefined shape dictionary with 21 classes is used, e.g., paragraph, label, header. The development set contains 3,218 images while the test set contains 1,073 images.
The dataset was validated during the 2020 and 2021 ImageCLEF - Multimedia Retrieval at the Conference and Labs of the Evaluation Forum (CLEF).
To download the data, please follow this link.
For more details see:
- Fichou, D., Berari, R., Brie, P., Dogariu, M., Stefan, L. D., Constantin, M. G., & Ionescu, B. “Overview of ImageCLEFdrawnUI 2020: the detection and recognition of hand drawn website UIs task”. In CLEF2020 Working Notes. CEUR Workshop Proceedings, CEUR-WS, Thessaloniki, Greece (September 22-25 2020).
- Berari, R., Tauteanu, A., Fichou, D., Brie, P., Dogariu, M., Ştefan, L. D., Constantin, M. G., & Ionescu, B. “Overview of ImageCLEFdrawnUI 2021: the detection and recognition of hand drawn and digital website UIs task”. In CLEF2021 =Working Notes. CEUR Workshop Proceedings, CEUR-WS, Bucharest, Romania, (September 22-25 2020)
Acknowledgements:
If you plan to make use of the Drawnui dataset, or refer to its results, please acknowledge the work of the authors by citing the paper listed above.
This dataset was conceived using the data gathered during the ImageCLEFdrawnUI tasks, and was funded by teleportHQ. Mihai Dogariu, Liviu-Daniel Ștefan, Mihai Gabriel Constantin and Bogdan Ionescu's contribution to this work was supported under project AI4Media, A European Excellence Centre for Media, Society and Democracy, H2020 ICT-48-2020, grant #951911.