Today, we’re excited to announce intelligent user segmentation powered by machine learning (ML) in Amazon Personalize, a new way to deliver personalized experiences to your users and run more effective campaigns through your marketing channels.
Traditionally, user segmentation depends on demographic or psychographic information to sort users into predefined audiences. More advanced techniques look to identify common behavioral patterns in the customer journey (such as frequent site visits, recent purchases, or cart abandonment) using business rules to derive users’ intent. These techniques rely on assumptions about the users’ preferences and intentions that limit their scalability, don’t automatically learn from changing user behaviors, and don’t offer user experiences personalized for each user. User segmentation in Amazon Personalize uses ML techniques, developed and perfected at Amazon, to learn what is relevant to users. Amazon Personalize automatically identifies high propensity users without the need to develop and maintain an extensive and brittle catalog of rules. This means you can create more effective user segments that scale with your catalog and learn from your users’ changing behavior to deliver what matters to them.
Amazon Personalize enables developers to build personalized user experiences with the same ML technology used by Amazon with no ML expertise required. We make it easy for developers to build applications capable of delivering a wide array of personalization experiences. You can start creating user segments quickly with the Amazon Personalize API or AWS Management Console and only pay for what you use, with no minimum fees or upfront commitments. All data is encrypted to be private and secure, and is only used to create your user segments.
This post walks you through how to use Amazon Personalize to segment your users based on preferences for grocery products using an Amazon Prime Pantry dataset.
Overview of solution
We’re introducing two new recipes that segment your users based on their interest in different product categories, brands, and more. Our item affinity recipe (aws-item-affinity) identifies users based on their interest in the individual items in your catalog, such as a movie, song, or product. The item attribute affinity recipe (aws-item-attribute) identifies users based on the attributes of items in your catalog, such as genre or brand. This allows you to better engage users with your marketing campaigns and improve retention through targeted messaging.
The notebook that accompanies this post demonstrates how to use the aws-item-affinity and aws-item-attribute recipe to create user segments based on their preferences for grocery products in an Amazon Prime Pantry dataset. We use one dataset group that contains user-item interaction data and item metadata. We use these datasets to train solutions using the two recipes and create user segments in batch.
To test the performance of the solution, we split the interactions data into a training set and test set. The Amazon Prime Pantry dataset has approximately 18 years of interaction data from August 9, 2000, to October 5, 2018, with approximately 1.7 million interactions. We hold out 5% of the most recent interactions and train on the remaining 95%. This results in a split where we use interactions from August 9, 2000, through February 1, 2018, to train the solution and use the remaining 8 months of interactions to simulate future activity as ground truth.
Results
When reproducing these tests in the notebook, your results may vary slightly. This is because when training, the solution the parameters of the underlying models are randomly initialized.
Let’s first review the results by looking at a few examples. We ran queries on three items, and identified 10 users that have a high propensity to engage with the items. We then look at the users’ shopping histories to assess if they would likely be interested in the queried product.
The following table shows the results of a segmentation query on gingerbread coffee, an item we might want to promote for the holiday season. Each row in the table shows the last three purchases of the 10 users returned from the query. Most of the users we identified are clearly coffee drinkers, having recently purchased coffee and coffee creamers. Interestingly, the item we queried on is a whole bean coffee, not a ground coffee. We see in the item histories that, where the information is available, the users have recently purchased whole bean coffee.
Gingerbread Coffee, 1 lb Whole Bean FlavorSeal Vacuum Bag: Bite into a freshly baked Gingerbread Coffee
USER_ID
Last Three Purchases
A1H3ATRIQ098I7
Brew La La Red Velvet Cupcake Coffee
Ola’s Exotic Super Premium Coffee Organic Uganda B
Coffee Masters Gourmet Coffee
ANEDXRFDZDL18
Pepperidge Farm Goldfish Crackers
Boston Baked Beans (1) 5.3 Oz Theater Box Sizecont
Boost Simply Complete Nutritional Drink
APHFL4MDJRGWB
Dunkin’ Donuts Original Blend Ground Coffee
Coffee-Mate Coffee Mix
Folgers Gourmet Selections Coconut Cream Pie Flavo
ANX42D33MNOVP
The Coffee Fool Fool’s House American
Don Francisco’s Hawaiian Hazelnut
Don Francisco’s French Roast Coffee
A2NLJJVA0IEK2S
Coffee Masters Flavored Coffee
Lays 15pk Hickory Sticks Original (47g / 1.6oz per
Albanese Confectionery Sugar Free Gummy Bears
A1GDEQIGFPRBNO
Christopher Bean Coffee Flavored Ground Coffee
Cameron’s French Vanilla Almond Whole Bean Coffee
Cameron’s Coffee Roasted Whole Bean Coffee
A1MDO8RZCZ40B0
Master Chef Ground Coffee
New England Ground Coffee
Maxwell House Wake Up Roast Medium Coffee
A2LK2DENORQI8S
The Bean Coffee Company Organic Holiday Bean (Vani
Lola Savannah Angel Dust Ground
New England Coffee Blueberry Cobbler
AGW1F5N8HV3AS
New England Coffee Colombian
Kirkland Signature chicken breast
Lola Savannah Banana Nut Whole Bean
A13YHYM6FA6VJO
Lola Savannah Triple Vanilla Whole Bean
Lola Savannah Vanilla Cinnamon Pecan Whole Bean
Pecan Maple Nut
The next table shows a segmentation query on hickory liquid smoke, a seasoning used for barbecuing and curing bacon. We see a number of different grocery products that might accompany barbecue in the users’ recent purchases: barbecue sauces, seasonings, and hot sauce. Two of the users recently purchased Prague Powder No. 1 Pink Curing Salt, a product also used for curing bacon. We may have missed these two users if we had relied on rules to identify people interested in grilling.
Wright’s Natural Hickory Seasoning Liquid Smoke, 128 Ounce This seasoning is produced by burning fresh cut hickory chips, then condensing the smoke into a liquid form.
USER_ID
Last Three Purchases
A1MHK19QSCV8SY
Hoosier Hill Farm Prague Powder No.1 Pink Curing S
APPLE CIDER VINEGAR
Fleischmann’s Instant Dry Yeast 1lb bagDry Yeast.M
A3G5P0SU1AW2DO
Wright’s Natural Hickory Seasoning Liquid Smoke
Eight O’Clock Whole Bean Coffee
Kitchen Bouquet Browning and Seasoning Sauce
A2WW9T8EEI8NU4
Hidden Valley Dips Mix Creamy Dill .9 oz Packets (
Frontier Garlic Powder
Wolf Chili Without Beans
A2TEJ1S0SK7ZT
Black Tai Salt Co’s – (Food Grade) Himalayan Cryst
Marukan Genuine Brewed Rice Vinegar Unseasoned
Cheddar Cheese Powder
A3MPY3AGRMPCZL
Wright’s Natural Hickory Seasoning Liquid Smoke
San Francisco Bay OneCup Fog Chaser (120 Count) Si
Kikkoman Soy Sauce
A2U77Z3Z7DC9T9
Food to Live Yellow Mustard Seeds (Kosher) 5 Pound
100 Sheets (6.7oz) Dried Kelp Seaweed Nori Raw Uns
SB Oriental Hot Mustard Powder
A2IPDJISO5T6AX
Angel Brand Oyster Sauce
Bullhead Barbecue Sauce
ONE ORGANIC Sushi Nori Premium Roasted Organic Sea
A3NDGGX7CWV8RT
Frontier Mustard Seed
Da Bomb Ghost Pepper HOT SaucesWe infused our hot
Starwest Botanicals Organic Rosemary Leaf Whole
A3F7NO1Q3RQ9Y0
Yankee Traders Brand Whole Allspice
Aji No Moto Ajinomoto Monosodium Glutamate Umami S
Hoosier Hill Farm Prague Powder No.1 Pink Curing S
A3JKI7AWYSTILO
Lalah’s Heated Indian Curry Powder 3 Lb LargeCurry
Ducal Beans Black Beans with Cheese
Emerald Nuts Whole Cashews
Our third example shows a segmentation query on a decoration used to top cakes. We see that the users identified are not only bakers, but are also clearly interested in decorating their baked goods. We see recent purchases like other cake toppers, edible decorations, and fondant (an icing used to sculpt cakes).
Letter C – Swarovski Crystal Monogram Wedding Cake Topper Letter, Jazz up your cakes with a sparkling monogram from our Sparkling collection! These single letter monograms are silver plated covered in crystal rhinestones and come in several sizes for your convenience.
USER_ID
Last Three Purchases
A3RLEN577P4E3M
The Republic Of Tea
Alyssa’s Gluten Free Oatmeal Cookies – Pack of 4.
Double Honey Filled Candies
AOZ0D3AGVROT5
Sea Green Disco Glitter Dust
Christmas Green Disco Glitter Dust
Baby Green Disco Glitter Dust
AC7O52PQ4HPYR
Rhinestone Cake Topper Number 7 by otherThis delic
Rhinestone Cake Topper Number 5This delicate and h
Rhinestone Cake Topper Number 8 by otherThis delic
ALXKY9T83C4Z6
Heart Language of Love Bride and Groom White Weddi
Bliss Cake Topper by Lenox (836473)It’s a gift tha
First Dance Bride and Groom Wedding Cake TopperRom
A2XERDJ6I2K38U
Egyptian Gold Luster Dust
Kellogg’s Rice Krispies Treats
Wilton Decorator Preferred Green Fondant
A1474SH2RB49MP
Assorted Snowflake Sugar Decorations Disney Movie
Darice VL3L Mirror Acrylic Initial Letter Cake Top
Edible Snowflakes Sugar Decorations (15 pc).
A24E9YGY3V94N8
TOOGOO(R) Double-Heart Cake Topper Decoration for
Custom Personalized Mr Mrs Wedding Cake Topper Wit
Jacobs Twiglets 6 Pack Jacobs Twiglets are one of
A385P0YAW6U5J3
Tinksky Wedding Cake Topper God Gave Me You Sparkl
Sweet Sixteen Cake Topper 16th Birthday Cake Toppe
Catching the Big One DecoSet Cake DecorationReel i
A3QW120I2BY1MU
Golda’s Kitchen Acetate Cake Collars – 4.
Twinings of London English Breakfast Tea K-Cups fo
Chefmaster by US Cake Supply 9-Ounce Airbrush Clea
A3DCP979LU7CTE
DecoPac Heading for The Green DecoSet Cake TopperL
Rhinestne Cake Topper Number 90This delicate and h
Rhinestone Cake Topper Letter KThis delicate and h
These three examples make sense based on our editorial judgement, but to truly assess the performance of the recipe, we need to analyze more of the results. To do this broader assessment, we run the aws-item-affinity solution on 500 randomly selected items that appear in the test set to query a list of 2,262 users (1% of the users in the dataset). We then use the test set to assess how frequently the 2,262 users purchased the items during the test period. For comparison, we also assess how frequently 2,262 of the most active users purchased the items during the test period. The following table shows that the aws-item-affinity solution is four times better at identifying users that would purchase a given item.
Test Metrics
Hits
Recall
Personalize – Item Affinity
0.2880
0.1297
Active User Baseline
0.0720
0.0320
Although these results are informative, they’re not a perfect reflection of the performance of the recipe because the user segmentation wasn’t used to promote the items which users later interacted with. The best way to measure performance is an online A/B test—running a marketing campaign on a list of users derived from the aws-item-affinity solution alongside a set of the most active users to measure the difference in engagement.
Conclusion
Amazon Personalize now makes it easy to run more intelligent user segmentation at scale, without having to maintain complex sets of rules or relying on broad assumptions about the preferences of your users. This allows you to better engage users with your marketing campaigns and improve retention through targeted messaging.
To learn more about Amazon Personalize, visit the product page.
About the Authors
Daniel Foley is a Senior Product Manager for Amazon Personalize. He is focused on building applications that leverage artificial intelligence to solve our customers’ largest challenges. Outside of work, Dan is an avid skier and hiker.
Debarshi Raha is a Senior Software Engineer for Amazon Personalize. He is passionate about building AI-based personalization systems at scale. In his spare time, he enjoys traveling and photography.
Ge Liu is an Applied Scientist at AWS AI Labs working on developing next generation recommender system for Amazon Personalize. Her research interests include Recommender System, Deep Learning, and Reinforcement Learning.
Haizhou Fu is a senior software engineer on the Amazon Personalize team working on designing and building recommendation systems and solutions for different industries. Outside of his work, he loves playing soccer, basketball and watching movies, reading and learning about physics, especially theories related to time and space.
Read MoreAWS Machine Learning Blog
