Building UI/UX Dataset for Dark Pattern Detection and YOLOv12x-based Real-Time Object Recognition Detection System

With the accelerating pace of digital transformation and the widespread adoption of online platforms, both social and technical concerns regarding dark patterns-user interface designs that undermine users’ ability to make informed and rational choices-have become increasingly prominent. As corporate online platforms grow more sophisticated in their design strategies, there is a pressing need for proactive and real-time detection technologies that go beyond the predominantly reactive approaches employed by regulatory authorities. In this paper, we propose a visual dark pattern detection framework that improves both detection accuracy and real-time performance. To this end, we constructed a proprietary visual object detection dataset by manually collecting 4,066 UI/UX screenshots containing dark patterns from 194 websites across six major industrial sectors in South Korea and abroad. The collected images were annotated with five representative UI components commonly associated with dark patterns: Button, Checkbox, Input Field, Pop-up, and QR Code. This dataset has been publicly released to support further research and development in the field. To enable real-time detection, this study adopted the YOLOv12x object detection model and applied transfer learning to optimize its performance for visual dark pattern recognition. Experimental results demonstrate that the proposed approach achieves a high detection accuracy of 92.8% in terms of mAP@50, while maintaining a real-time inference speed of 40.5 frames per second (FPS), confirming its effectiveness for practical deployment in online environments. Furthermore, to facilitate future research and contribute to technological advancements, the dataset constructed in this study has been made publicly available at https://github.com/B4E2/B4E2-DarkPattern-YOLO-DataSet.

💡 Research Summary

**
The paper addresses the growing societal and technical concern of “dark patterns” – UI/UX designs that manipulate users into making irrational decisions – by proposing a visual detection framework that combines a newly constructed dataset with a state‑of‑the‑art object detection model. The authors first built a proprietary dataset consisting of 4,066 screenshots collected from 194 websites and mobile applications across six major industry sectors (e‑commerce, travel & hospitality, finance, media & content, public administration, and press) in South Korea and abroad. To capture the visual manifestations of dark patterns, they manually explored high‑risk user flows (registration, login, search, cart addition, payment, subscription cancellation) and captured full‑screen images that preserve context such as button size, color contrast, and popup layout.

The dataset is annotated with five UI component classes that frequently serve as building blocks for dark patterns: Button (1,691 instances), Checkbox (1,045), Input Field (1,045), Popup (1,537), and QR Code (210). Annotation was performed using the RoboFlow platform, with each image reviewed by at least two labelers to ensure consistency. By focusing on these elementary elements rather than attempting to label whole “patterns,” the authors enable a conventional object detection model to learn the visual cues that underlie deceptive designs.

For the detection engine, the paper selects YOLOv12x, a one‑stage detector that treats object detection as a single regression problem, thereby offering superior inference speed compared with two‑stage methods such as Faster R‑CNN. The authors fine‑tuned a pre‑trained YOLOv12x model on their dataset using transfer learning, applying data augmentation (random scaling, color jitter, horizontal flips) to mitigate the modest dataset size. Evaluation shows a mean average precision at IoU = 0.5 (mAP@50) of 92.8 % and an average inference speed of 40.5 frames per second on a single GPU, confirming that the system meets real‑time requirements for deployment in web browsers or edge devices. Comparative experiments with YOLOv5 and Faster R‑CNN demonstrate that YOLOv12x achieves the best trade‑off between accuracy and latency for this task.

The authors discuss several limitations. First, the annotation scheme captures only five generic UI components, which may not fully represent complex dark patterns such as “forced continuity” or “hidden costs” that involve combinations of elements and subtle visual cues. Second, the dataset size (≈4 k images) is relatively small for deep learning, potentially restricting model generalization to unseen domains or newer design trends. Third, the approach ignores textual information that often accompanies UI elements; a multimodal model that fuses OCR‑derived text with visual features could reduce false negatives.

Future work is outlined along three main directions: (1) expanding the taxonomy to include additional UI primitives (sliders, dropdowns, progress bars) and explicit pattern‑level labels; (2) incorporating temporal UI sequences to capture dynamic dark‑pattern behaviors such as delayed pop‑ups or progressive disclosure; (3) developing multimodal architectures that jointly process visual and textual cues, and evaluating lightweight variants suitable for on‑device deployment. The authors also propose establishing an international standard for dark‑pattern annotation to facilitate cross‑cultural studies and benchmarking.

In summary, the paper makes two substantive contributions: (i) the release of a curated, publicly available UI/UX dark‑pattern dataset that fills a notable gap in the research community, and (ii) a demonstration that a modern YOLOv12x detector, fine‑tuned on this dataset, can achieve high detection accuracy while maintaining real‑time performance, thereby offering a viable technical countermeasure to deceptive UI designs.