At B*Factory, our mission to push boundaries in beauty technology is embodied in Ameli, our thoughtfully innovative cosmetics brand. Through our technical expertise and dedication, we’ve developed a virtual try-on (VTO) system for makeup that not only rivals but often surpasses offerings from major global beauty brands.
Today, we’d like to pull back the curtain and share the journey behind our groundbreaking VTO technology and LViton system.
Building a Real-Time Virtual Try-On System
When tasked with developing a VTO platform from scratch for Ameli, the challenge was clear: create a system that meets stringent requirements while processing images in real time. And we did it — all in just a month.
Features That Transform the Experience
From foundation and blush to complex elements like eyeliner, aegyosal, and lipstick, our VTO system supports:
- Color customization: Pick any color for a specific makeup area.
- Thickness adjustments: Control opacity for a perfect application.
- Softness settings: Achieve natural looks by seamlessly blending with the skin.
- Effects:
- Gloss: for realistic reflections on lipstick
- Glitter: for striking looks on eyeshadows
- Shimmer: for metallic reflections
This system was initially implemented as a Gradio demo, enabling Ameli’s team to test and iterate seamlessly.
Scaling Performance with C++
While the initial version written in Python validated the concept, the processing time per image was a roadblock. To address this, we rewrote the image processing core in C++, delivering:
- Massive speedup: Processing time dropped to a few milliseconds.
- Custom optimizations: Optimized color space conversions, a custom Gaussian blur implementation, and direct memory manipulation reduced overhead significantly.
- Community contributions: Our enhancements to open-source libraries like dlib benefited the broader development community.
The Evolution: LViton for Browser-Based Virtual Try-On
With our C++-based system achieving near-instantaneous processing, the next step was accessibility. Enter LViton — our browser-native virtual try-on technology.
But what’s behind the name LViton? It’s a little inside joke with a touch of flair. Initially, we called our system “Viton,” short for “Virtual Try-On.” Then, as a playful nod to a certain luxury fashion house, we added an “L” to the front. And voilà — LViton was born! It’s a name that blends high-performance technology with a touch of luxury, perfectly mirroring the aspirations of Ameli’s brand.
Why Browser-Based?
Deploying GPU-reliant systems at scale is resource-intensive. By leveraging MediaPipe, we simplified our tech stack and compiled our C++ code to WebAssembly, ensuring:
- Privacy-preserving operation: No images leave the user’s device.
- Infinite scalability: Computations happen entirely in the browser.
From C++ to Zig: A Bold Leap Forward
While the WebAssembly port in C++ was functional, we sought to push boundaries further. Zig, a modern programming language designed as a C replacement, enabled us to:
- Eliminate dependencies like Emscripten.
- Reduce compilation times drastically for faster iteration.
- Write a streamlined codebase with minimal overhead.
Our zig implementation is now the backbone of LViton, featuring:
- Optimized image processing: Blurring, transformations, and color conversions.
- Glitter effects: Spatial and temporal consistency for dazzling results.
- Foundation without segmentation: Real-time rendering using face landmarks and convex hull calculations.
What sets LViton apart?
LViton isn’t just a technology; it’s a paradigm shift in how users interact with makeup. Its unique strengths include:
- High performance on low-end devices: Real-time processing without the need for powerful GPUs.
- Seamless user experience: No loading delays, thanks to a lighter codebase and optimized features.
- Continuous innovation: From glitter effects to depth-free foundation rendering, we constantly expand its capabilities.
Driving Beauty Innovation Forward
By harnessing the power of modern programming languages, machine learning, and systems-level optimizations, Ameli’s virtual try-on system and LViton embody our commitment to innovation. These technologies reflect our vision of making beauty experiences universally accessible, privacy-focused, and effortlessly enjoyable.
As we reflect on our journey with LViton and our exploration of Zig, we’re proud to share Zignal, an image processing library inspired by the amazing dlib. Building LViton wouldn’t have been possible without the open-source tools and communities that supported us, and Zignal represents our way of giving back to the community.
Zignal offers a collection of generally useful image processing algorithms and, to the best of our knowledge, is the first library of its kind in Zig.
By open-sourcing Zignal, we aim to empower developers, contribute to the growth of the Zig ecosystem, and foster further innovation in image processing and systems programming. With Zignal, we hope to inspire collaboration and creativity, just as open-source tools have inspired us.
As we continue to push boundaries in beauty tech — and beyond — our journey is only just beginning.