Introduction

The Unlock Box is one of the always-on signature services of Live Concept stores. Mystery box stocked with distinctive gifts & benefits, surprise and delight local members and build anticipation in every visit. A vending machine filled with Mystery Boxes is placed near the front gate of the store, and when consumers input a code from their receipt, a mystery box from a random rail will drop. The purpose of this study is to help teams to have a better understanding of how the Unlock Box influence our business.

The Unlock Box Vending Machine

This study focuses on four main aspects:

  1. Unlock Box Performance Metrics
  2. Performance Before VS After Gift Refreshment
  3. Did consumer notice the Unlock Box?
  4. Evaluation of the Unlock Box based on Cost Analysis

Data

1. Unlock Box Redemption Data 1

Redemption data includes the date, gift, digital_order_id and store_name. The time period is from 02/01/2021 to 11/30/2021.

2. Member Data 2

Member data includes information about the members who made inline purchases in Live Concept stores, including gender, UPMID, and so on.

3. Beijing and Shanghai Live inline Transaction Data 3

All transactions occurred in Beijing and Shanghai Live since opening.

4. Scalper Data 4

Scalpers were selected based on Yunhai’s Scalper Detection Machine. Scalper data includes scalpers’ UPMID.

5. Full Path Analysis Raw Data 5

FPA data is generated by computer vision technology, the data includes the area, dwell time, number of consumer of SH TKH Live.

Unlock Box Performance Metrics


General Metrics

Using the data from 02/01/2021 to 11/30/2021, we calculated some basic metrics (excluding scalpers and transactions from @WO):

Service # of Redemptions Redemption Rate Demand Demand / Redemption # of Members Member Demand ULB Member AVG Demand
Unlock Box BJ 31.7K 84% RMB32.7M RMB1,033 23.1K RMB31.2M RMB1,348
Unlock Box SH 6.4K 77% RMB6.9M RMB1,081 4.8K RMB6.8M RMB1,407

Store # of Transactions Demand ADPT # of Members Member Demand Live Member AVG Demand
BJ SLT Live 69.4K RMB53.0M RMB764 41.0K RMB48.1M RMB1,174
SH TKH Live 14.7K RMB12.2M RMB830 9.8K RMB11.3M RMB1,152


  • According to the proportion of stores’ metrics, and redemption rate, the Unlock Box achieved high consumer/member engagement.
  • The Unlock Box is closely related to high-value transactions and members.
  • Unlock Box in BJ Live generally has a better performance, but SH has higher Demand/Redemption and ULB member average demand.


Monthly ULB Redemption Rate

  • From the above line chart, we can see that over time, the attractiveness of gifts to consumers will gradually decline, resulting in a decline in the redemption rate. There was a gift refresh on July 9th, which attracted consumers again. Based on the past trend, we suggest to refresh gifts every 4-5 months.


Monthly DPM of ULB Members

  • DPM of SH Live has greater volatility, which may be due to the relatively smaller consumer base.


Order-Split Rate

  • Order-Split Rate = # of consumers with multiple above ￥800 transactions in one day / # of consumers spent more than ￥1600 in one day
  • Order-Split Rate can indirectly measure the attractiveness of ULB to consumers. Excluding scalpers, the only reason for order splitting is to gain extra ULB chances.
  • Most of the consumers with orders above ￥1600 chose to split their orders to gain extra Unlock Box chances, indicating their interest in this service. They are willing to spend more time during check-out for this experience.
  • The trends of order-split rate are similar to the redemption rate’s, proving the 4-5 months gift refresh cadence we suggested.
  • Also, this kind of action will cause lower ADPT. The gap between ADPT and ADPMPD (Avg Demand Per Member Per Day) is mainly caused by order-split. ADPT is lower due to Unlock Box


Performance Before VS After Gift Refreshment

There was a gift refreshment on July 9th, using the data from 03/01/2021 to 10/31/2021, we got the following comparison:


  • All the metrics are indicating better performance after the gift refreshment. Consumers did react to the more premium gift offering.
  • The gift refreshment increased the redemption rate back, and remained steady for 3 months.
  • The consumers who redeemed ULB gifts spent more.
  • More consumers split orders for extra ULB chances, showing interest in gifts.


Did consumers notice the Unlock Box?

Even the Unlock box has shown a strong performance. We still want to know if consumer notice this unique service and if it can do better. Powered by computer vision technology, Full Path Analysis 6 project can help us answer this question. It uses cameras to monitor consumers’ shopping journey, recording their dwell time in each zones.

Computer Vision Zone Layout


Month Store Traffic # of Consumers with 5s+ in ULB Area %
Jun 7942 471 5.9%
Jul 7481 279 3.7%
Aug 6988 450 6.4%


Only around 5% of consumer have Unlock Box machine in mind before beginning to shop in store. This is very important insight we can leverage in considering incremental revenue by Unlock Box, which will be discussed in the following section. This also means that we have opportunities to build awareness of ULB service, drive member engagement online & offline and elevate its business impact:

  • Enhance in-store communication on gift assortment & drive excitement
  • A more conspicuous in-store location and appearance

Evaluation of the Unlock Box based on Cost Analysis

Unlock Box Gift Cost Pre-refreshment Unlock Box Gift Cost Post-refreshment



  • Considering the Unlock Box is one of the unique services of Live Concept and has achieved high consumer engagement, the ￥23.7 average cost per ULB redemption is quite cost-efficient.
  • As the software development fee is an one-time initial investment, the next year’s average cost can be controlled more flexibly.


ROI Evaluation Method for Current Unlock Box Strategy

91% of the consumer were one time buyer, and based on our computer vision data, 95% of the consumer never engaged with The Unlock Box vending machine right after entering the store. With all these insights, along with store athletes’ feedback, we can assume that most consumers do not have the information about Unlock Box during their shopping stage, and most of them are informed about the Unlock Box by the athletes during the checkout stage. This assumption means:

    1. If consumer already have items that cost more than 800RMB at checkout before knowing the existence of Unlock Box and pays, then no incremental revenue from Unlock Box.
    1. If consumer have items that cost less than 800RMB at checkout before knowing the existence of Unlock Box, and athletes suggest to add item(s) to meet the 800RMB threshold, then the incremental revenue from Unlock Box is the revenue from the additional items added to meet the threshold.

The price of the cheapest item (59RMB) in Live is higher than Average Cost per Redemption (23.7RMB), which means the ROI must be positive

Price of cheapest item is higher than Cost per Redepmtion

ROI Evaluation Method for Next Steps

Free Gift in SH Live

  1. Effective Traffic attracted by Unlock Box free gift can be tracked using existing Computer Vision system: Only those traffic who redeem a gift and go further into the store and browse are effective traffic (redeem gift and engage store).

  2. Cost per redemption will be calculated for this phase. Incremental revenue is the revenue from all redeemed consumer, a similar comparison of Cost per Redemption and incremental revenue will be made. But keep in mind that ineffective traffic will hurt the ROI.

Sports and Fitness Purchase Threshold in BJ Live

  1. In this case, I suggest let consumer know about Unlock Box and qualification in advance. So we can assume revenue from qualified items is the incremental revenue.

  2. Cost per redemption in BJ will be different, and it will be calculated exclusively for BJ Live.

Summary



  • A huge proportion of the sales in Live Concept Stores is related to Unlock Box, the Unlock Box is highly engaged by consumers and members.
  • Gift refreshment in July was successful, all metrics improved since the refreshment.
  • ADPT is high in both Live stores, but because of the splits of transaction due to Unlock Box Gift’s attractiveness, the ADPT is lower than what it suppose to be.
  • Currently with the ￥800 threshold, for consumers with orders close to ￥800, they would buy extra cheap items like socks for the ULB chance. Taking socks, one of the cheapest item, as an example, the minimum price is around ￥59, higher than the average cost per ULB redemption. In this case, the incremental revenue is already higher than the cost. That is to say, the ULB is cost-efficient.
  • As the Unlock Box service is not totally member-gated currently, we will evaluate the ULB’s member acquisition ability in the next phase.
  • We will also compare the Unlock Box userprofile like age and gender to Live trade area using LBS data.

This is a on-going report, new items will be added

Acknowledgment

Thank Yunhai Zhang for the supervision and guidance.

  1. Source: recorded by ULB vendor and sent by Nippon Steel ()↩︎

  2. Source: Nike Membership Data↩︎

  3. Source: Nike Inline Order Line↩︎

  4. Based on the scalper detection machine built by Yunhai Zhang ()↩︎

  5. Source: Owned by STORE OPS & EXCELLENCE team, Marco Gao ()↩︎

  6. Source: Owned by STORE OPS & EXCELLENCE team, Marco Gao ()↩︎