Enabling Real-Time Experience in a Digital Fitting Room

Description

A blockchain company turned to Altoros to develop two mobile apps that would enable fashion retailers to adopt the anticounterfeit capabilities of the decentralized economy.

Brief results of the collaboration:

• The customer developed two mobile apps—a digital wardrobe on Android and a universal profile on iOS—that facilitate the adoption of the proprietary anticounterfeit blockchain by retailers.
• Now, fashion brands can secure and simplify the purchase of digital designer items across multiple stores on the blockchain network, enable provenance, and satisfy the demand for virtual fitting rooms.
• After successfully validating the idea of bringing the blockchain capabilities to mobile, the provider is now able to move on with the development of cross-platform solutions, saving costs on maintaining 4 individual apps.

The customer

Counterfeits are estimated to cost the fashion industry $600 billion per year. Meanwhile, brands strive to satisfy consumer interest in digital experience, such as owning virtual replicas of designer products or trying on clothes online.

Based in Germany, the customer develops a blockchain network that brings the perks of decentralized economy to retail. In 2020, the company raised $18 million to evolve its blockchain into an anticounterfeit platform that secures transactions and enables provenance. On top of that, the provider saw an opportunity to deliver a digital fitting room driven by AR and a universal profile for multiple shops.

The need

Being a small company with limited engineering resources, the customer relied on the mobile expertise at Altoros to accelerate development—validating integrability of augmented reality tools, blockchain, single sign-on, and e-commerce technologies for iOS and Android users.

The challenges

Under the project, the team at Altoros had to address the following issues:

• As the universal profile was to store and process sensitive data (names, addresses, etc.), it was crucial to ensure security.
• In contrast to the abundance of tooling that enables blockchain integration with web solutions, web3swift was one of the few libraries on the market for mobile apps. However, it also had certain limitations around smart contract management.
• It was important to prevent performance issues caused by the infusion of augmented reality into the digital wardrobe.
• The existing AR libraries did not meet the customer's requirements around the precise mapping of digital items on different body shapes and sizes.

The solution

Stage 1. Mobile developers at Altoros delivered two mobile apps: a digital wardrobe on Android and a universal profile on iOS. Powered by AR the first app allows to virtually try on digital items. Using WalletConnect, the second app performs transactions across a multitude of digital stores on the blockchain network via a single account.

Stage 2. The measures taken to secure data in the universal profile included transaction encryption and using keychain to store sensitive data (password, certificates and identities, etc.).

Stage 3. To overcome limitations, our engineers contributed to the development of the web3swift library: enabled the authentication of blockchain transactions via a mobile app and implemented metadata decoding for smart contract parsing.

Stage 4. Utilizing the MediaPipe library, the developers delivered immersive experience of a fitting room. In particular, the engineers employed Pose Estimation, Selfie Segmentation, Face Detection, and Face Mesh modules. Each of the tools features lightweight model architectures combined with GPU acceleration throughout the pipeline. This helped to achieve near real-time performance.

Stage 5. To improve the precision of fitting digital items, our developers optimized inverse kinematic calculations. This is a process of rotating skeleton joints to get them as close as possible to the points generated by MediaPipe. As a result, the clothes adjustment rate reduced by 3x (from 0.09 to 0.03 seconds per frame).

Stage 6. Firebase was integrated for incident monitoring. Finally, QA engineers conducted unit, performance, and UI testing across the two apps.

The outcome

Partnering with Altoros, the customer built two mobile apps, contributing to the adoption of a proprietary anticounterfeit platform by fashion retailers and improving digital experience. Now, brands can secure digital purchases across numerous stores from a single profile, as well as enable virtual fitting rooms.

As the provider validated its idea, the company is planning to deliver cross-platform systems for both a digital wardrobe and a universal profile rather than building and maintaining 4 individual apps.