AR Shopping Apps: How to Use Artificial Intelligence in Retail

Why do retailers and E-commerce owners like AR? Users prefer to buy online, but some goods are difficult to choose remotely. Before making a purchase, they need to try different models, watch them in a 360-degree view, etc. This is where augmented reality comes in handy: users buy online or via mobile apps, visit high-tech online stores, use virtual fitting rooms, and make orders without leaving home. Let’s take a closer look at the augmented reality retail industry and find out how to use it for business.

Definition of AR

To better understand the details of working with augmented reality, you need to know the meaning of augmented or artificial reality. It is an environment that complements the physical world in real-time. It works on several devices, such as tablets or smartphones. There is also a complex software component behind them. Examples of augmented reality implementation are smart glasses, apps for trying on watches, glasses and sneakers, or even the Iron Man helmet.

AR statistics for 2021

Virtual reality, augmented reality and similar technologies help people in education, work, medicine, shopping, and military operations. As a result, the demand for quality products is growing, and statistics show that the number of users has been increasing too.

Source: Statista

Why retailers implement AR

There are many examples of augmented reality use in retail. Here are some reasons why augmented reality is attracting sellers and service providers. It helps:

• personalize the shopping process,
• attract a young audience,
• get more opportunities for online sales,
• show the customers the whole product line,
• sell new products even before they are marketed,
• save on extra physical stores.

Additionally, the use of augmented reality in shopping gives sellers the opportunity to know their audience better because they can get information about what people are looking for and eventually buying. Consequently, they can create more targeted advertising, understand customers, and offer them everything they need.

In-Store Experience

In-Store Experience

One of the most important areas of AR use in retail is its application in retail stores. Let’s discuss it in more detail.

When coming to a new store, especially a large one, people have various questions, such as, where can I find the product I’m looking for? How do I get to the fitting room? Where are the cash registers? AR-driven navigation systems can solve such issues. They may use marks on the floor. The user would then point a smartphone camera at these marks, and the app determines the location of a person. When clients move the camera to other areas of the store, it shows the name of the store department. Having a smart shopping mall like this gives people a more positive shopping experience. It would allow them to quickly find the products they want, thus increasing the likelihood of a purchase.

Another way to simplify the augmented reality shopping experience is by eliminating the need to walk around the sales area and choose a product. Thus, for example, an augmented reality mirror shows clothes based on the “reflection” of a person. This could radically change retail stores and shopping experience. One day, they may only consist of fitting rooms with AR mirrors, warehouses, and even cash registers. People will just try on clothes in augmented reality. Using this, the flow of customers will increase, and the rent expenses will decrease.

Home shopping

Apps can be used in retail not only in outlets but also for online sales. AR shopping apps make clients feel like they’re in an augmented reality retail store. Thus, the communication between AR retailers and buyers becomes even more productive. Here are some of the possibilities that you can implement with such apps:

• fitting in the augmented reality,
• placement of goods (for example, furniture) in the room,
• customization of goods (clothes, food, or any others),
• instructions for working with the goods,
• recommendations and advice.

For example, users of the Heinz app can point a smartphone camera at a product label. They will see more detailed information about it but also video recipes with it. It makes the shopping experience more exciting.

One of the hot trends of online shopping is trying on clothes in AR. Customers can see what clothes would look like on them from the comfort of their home, using photos or videos. They won’t have to put on dozens of different things, or waste time walking around, and look for a suitable store. Moreover, more online shopping opportunities will lower the transportation load and, as a result, harmful emissions.

Furniture Shopping

Apps for buying furniture must also be truly of high quality and close to reality. A classic example of an augmented reality furniture shopping app is IKEA Place. To try a table or rack and see how it fits the room, the user points the camera to the desired part of the room and selects an object from the catalog. Another function of the app is the recognition of IKEA products. Thus, if a user likes an object in a cafe, they can point a camera on it, find it in the catalog, and order it immediately.

Another furniture AR app made by IKEA is AssembleAR. This app is for those who have already bought furniture, but find it difficult to put its parts together. To use the app, users need to scan the barcode on the box. After that, an animated version of the manual appears on the screen, along with an online version of the box on the floor. The app also contains a list of tools needed for assembly.

Our experience in developing an AR Watch App for Retail

Augmented reality retail projects are gaining popularity. For example, fitting apps are relevant for accessories, including watches. One of our US clients, Leef Company, decided to step ahead of competitors in the digital world and create an app for the retail business. They asked us to develop an app for trying on watches. Here is an overview of how we developed the app and the results.

What is the Leef app?

The Leef app allows users to open up the catalog and choose the watches they like. Then, simply by pointing a smartphone camera at their hand, they can try the watches on as if they were real.

Our approach

To develop the product, we decided to use markerless augmented retail stores. In comparison with marker-based AR requiring visual markers, markerless augmented retail uses real-world objects as markers. In our case hands were the markers.

The task was to train neural networks so they can show watches on the hand and let it move together with the hand like it’s real. We began with an app prototype on Python to check that how the neural networks are working. Then, we built iOS and Android apps.

How We Trained Neural Networks

To make a training dataset we drew a cross on the wrists of our colleagues to detect the watches’ position on their hands. We took photos of hands from different angles on a green background.

To increase the accuracy, we chose several backgrounds. So, we extended the dataset up to 60 thousand photos. Then, we developed an algorithm that indicates the cross on the hand.

How We Built AR for iOS and Android

During the next stage, we imported a 3D model of watches into the iOS application using the SceneKit framework, which works with older iPhone versions. A question came up: how do we place virtual watches on the hand realistically? There is always a watch band  that is hidden under the wrist. We found an innovative solution: we drew a virtual transparent hand that erases the invisible band part.

When we built the Android augmented retail app, we had to train a new neural network model. The first algorithm didn’t work on Android. The problem was that when the hand position changed, the model reacted very slowly and the AR turned into a slide show.

We quickly trained a new model to detect the wrist size and hand position. This also increased the app’s speed. To draw the watch on Android we used the Rajawali library.

The Result

Our client, Leef Company, offers the app prototype to different brands, and some retailers are already interested in the product. The outcome was so impressive for our client that they returned with a new task: to develop an augmented reality app for trying on shoes.

Conclusion

Even in 2021, not all retailers understand the value of technology for business growth. This is great news because you can create an amazing product that will conquer the market. If you need help creating a retail project, don’t hesitate to contact us. We will be glad to work with you.