On December 11, 2024, at the COSMAX Beauty AI Conference, our team unveiled the HEVA Magic Mirror, a voice-to-voice beauty agent designed to transform how users engage with makeup recommendations. Imagine consulting with a personal beauty advisor and instantly seeing their suggestions applied to your face. The HEVA Magic Mirror blends voice interaction, AI, and real-time visualization to create an immersive, intuitive beauty experience.
The Goal: Crafting a Smarter Beauty Companion
Our mission was simple but ambitious: build a system where users could speak directly to an AI agent, receive personalized recommendations in natural-sounding speech, and instantly see those suggestions virtually applied to their faces. The HEVA Magic Mirror merges conversation with visual feedback, offering users an experience that feels personal, seamless, and interactive.
Turning Vision into Reality
To build the HEVA Magic Mirror, we combined advanced AI models with technologies that could process voice, interpret user preferences, and visually simulate makeup applications. This involved integrating multiple components that worked together to deliver smooth, natural interactions and accurate virtual makeup results.
A critical part of development involved reducing delays, enhancing speech quality, and enabling users to interrupt or refine their consultations dynamically — ensuring that conversations felt as fluid as speaking with a human advisor.
Overcoming Key Challenges
Throughout development, we faced several hurdles:
- Latency: Real-time performance was crucial to avoid delays between user input and AI response.
- Speech Quality: Early versions lacked the natural intonation necessary for engaging interactions, particularly in Korean.
- Dynamic Conversations: We needed to ensure users could interrupt, adjust, or clarify requests without waiting for full responses.
- Prompt Efficiency: Managing long or complex queries required optimizing how information was processed and presented.
By refining the system’s architecture and integrating new solutions, we successfully improved response times, enhanced voice naturalness, and introduced a more interactive, free-flowing experience.
Enhancing the Experience with Real-Time Interaction
To elevate the HEVA Magic Mirror beyond standard voice assistants, we integrated real-time video capabilities. This enabled the AI to not only respond but also apply makeup virtually, matching user preferences live through a connected camera feed.
Technologies that support low-latency communication, video, and voice were essential to this process, ensuring a consistent and engaging user experience.
Live Demo at the Conference
During the COSMAX Beauty AI Conference, our PM showcased the HEVA Magic Mirror with a real-life example:
- The question: “Could you recommend makeup for the EOY party I’m throwing this weekend?”
- The response: HEVA initially suggested a bold, glittery eyeshadow paired with a soft pink lip. After feedback from the PM, the model refined the look to a more mature, reddish-pink aesthetic — showcasing its ability to adjust recommendations based on user input in real time.
The final twist? The recommended makeup appeared instantly on the user’s face through the virtual camera, demonstrating the seamless fusion of voice interaction and real-time visualization.
The Future of BeautyTech
The HEVA Magic Mirror marks a significant step forward in merging AI with beauty tech. By addressing key challenges and enhancing the interactive experience, it opens new possibilities for personalized, real-time beauty consultations.
During the Q&A session at the COSMAX Beauty AI Conference, the HEVA Magic Mirror stole the spotlight, sparking the most curiosity and engagement from the audience. Questions ranged from its underlying technology to potential applications, highlighting the excitement around its innovation and practicality.
With its ability to provide personalized recommendations and visually demonstrate them instantly, the HEVA Magic Mirror redefines how users experience beauty consultations. As we refine and expand this technology, it promises to shape the future of smarter, more interactive beauty solutions.