HapticGen🤖

Generative Text-to-Vibration Model
for Streamlining Haptic Design (CHI '25)

(1)KAIST     (2)ASU

*Equal Contribution     
†Corresponding Authors

🔎 HapticGen is a generative model that streamlines the complex process of vibrotactile design by allowing designers to create diverse haptic feedback simply by typing text prompts.

Abstract

Designing haptic effects is a complex, time-consuming process requiring specialized skills and tools. To support haptic design, we introduce HapticGen, a generative model designed to create vibrotactile signals from text inputs. We conducted a formative workshop to identify requirements for an AI-driven haptic model. Given the limited size of existing haptic datasets, we trained HapticGen on a large, labeled dataset of 335k audio samples using an automated audio-to-haptic conversion method. Expert haptic designers then used HapticGen's integrated interface to prompt and rate signals, creating a haptic-specific preference dataset for fine-tuning. We evaluated the fine-tuned HapticGen with 32 users, qualitatively and quantitatively, in an A/B comparison against a baseline text-to-audio model with audio-to-haptic conversion. Results show significant improvements in five haptic experience (e.g., realism) and system usability factors (e.g., future use). Qualitative feedback indicates HapticGen streamlines the ideation process for designers and helps generate diverse, nuanced vibrations.

HapticGen Model Diagram

HapticGen Model Diagram

HapticGen features a comprehensive architecture that transforms audio and text inputs into haptic feedback patterns.

The pipeline begins with a captioned audio dataset that undergoes preprocessing including speech filtering and haptic label augmentation. The core model utilizes a Transformer LM (~1.5B parameters) with parallel signal and text encoders (operating at 8kHz), followed by a signal decoder and normalization stage. The system is further refined through expert-voted datasets and DPO fine-tuning to align with human preferences.

HapticGen UI

Our UI allows users to generate haptic signals directly from text prompts or select from existing samples.

XR scenarios with HapticGen

Game, Simulation, Sports, Interaction, ...and Emotion!
HapticGen is a versatile tool for creating haptic feedback in a variety of XR scenarios.


Video Teaser

Resources

Explore more on HapticGen

ACM Reference Format:

Youjin Sung, Kevin John, Sang Ho Yoon, and Hasti Seifi. 2025. HapticGen: Generative Text-to-Vibration Model for Streamlining Haptic Design. In CHI Conference on Human Factors in Computing Systems (CHI '25), April 26--May 01, 2025, Yokohama, Japan. ACM, New York, NY, USA 24 Pages. https://doi.org/10.1145/3706598.3713609