How Synthetic Personas Can Help Test Personalized Web and Mobile App Experiences

Software users expect web and mobile apps to deliver individually tailored content. While personalized experiences drive greater user satisfaction and engagement, they pose a challenge for software development teams that need to be able to test dynamically changing apps. 

However, recent advances in artificial intelligence (AI) have made it possible to build large language models, which can be used to create synthetic personas for testing. Synthetic personas, which are essentially AI-generated user personas, can help with testing web and mobile applications. But how to create a synthetic persona? And how can synthetic personas aid your test automation strategy?

By nature, traditional test automation scripts give the impression of a simple, uni-dimensional persona, which makes it difficult to test hyper-personalization features.

For example, consider search terms: because the search keywords within test automation scripts never change, the app under test assumes the user’s behavior is the same every time they interact with the app. While the test can be customized for certain search terms, it cannot exercise the full breadth of hyper-personalization features. The same holds true for browsing history.

The solution to this challenge? Synthetic personas.

A synthetic persona is an AI-generated user persona that leverages learned characteristics to predict the behavior of actual humans. In the web and mobile app development world, synthetic personas use known information such as search and browser history to predict the content a user wants to see. One approach for creating synthetic personas is to draw upon knowledge from a neural network, which is a subset of machine learning where algorithms learn to perform certain tasks by analyzing training examples.

The challenge is how to easily create internally consistent synthetic personas that can exercise a broad array of app hyper-personalization features. For testing purposes, a synthetic persona mostly needs to provide a sequence of related words that can be used either as search terms or to select content items for display. Essentially, creating a synthetic persona boils down to creating a sequence of internally consistent words. Enter large language models. 

A large language model is a neural network trained to understand the co-occurrence frequencies of words. For example, if you see the word “banana,”  then the following word is likely to be “peel” or “split” and  unlikely to be “lathe.” The concept of “attention” generalizes this idea to look for co-occurrence frequencies not just in word pairs, but by looking 2, 10, 50, or more words ahead.

A neural network can be trained on large collections of documents, such as “The Pile,” which contains 825 GiB of documents. With such a large dataset, it’s possible to learn how a certain word might co-occur with other words that appear in a large neighborhood. Once trained, the model can be used to generate text. For example, a prompt phrase is provided, such as “Software testing is an important part of software quality,” and then the large language model is asked to complete the phrase. For this specific prompt, InferKit’s large language model completes the sentence by writing, “assurance and quality control, since the success or failure of software applications cannot be gauged by outside factors.” The result is an internally consistent sentence that is correct and meaningful in context.

Large language models can be used to create synthetic personas for testing hyper-personalization by generating internally consistent search terms from clever prompt strings. For example, consider the following prompt text:

Ishaan is going shopping at an Indian food store and is looking forward to cooking vegetarian Indian food. Ishaan has a shopping list with the following items:

1. dal

2. asafetida

If this prompt is given to the large language model at [3], one sample output is the following:

3. red chilies

4. green chilies

5. garam masala

6. salt

7. pepper

8. curry powder

In other words, the output is a series of grocery store items that are consistent with the persona of Ishaan as a shopper interested in Indian cooking. As a software tester, this model was provided with nothing more than the prompt text. The knowledge about Indian food is stored within the model in the form of co-occurrence frequencies learned over gigabytes of documents.

The outputs from the large language model change with each model run (inference). A “temperature” parameter lets you control how different the generated results are in each run. For example, with a high temperature, the same prompt above generates the following outputs in a second run:

3. kasuri methi

4. chana

5. fresh onions

6. garlic

7. red chilly

8. fresh coriander

9. green coriander

10. green chili

11. fenugreek seeds

Parsing the outputs is important because large language models sometimes produce sentences instead of more list items. For example, one output generated a shopping list followed by, “However, as he is on his way, he runs into Jyoti, and she needs to go to the hospital.”

Testing the personalization features of apps is important. Now, using large language models we can easily create synthetic test personas by writing effective prompts and then parsing the outputs for search terms.