What We Learned from the Beta Version of the Gamer Profile Tool

Quantic Foundry's June 2015 post describes how the company designed and beta-tested the individual reports delivered by its Gamer Motivation Profile Tool. While factor analysis for clustering motivations is standard in psychology, presenting personalized results back to participants is understudied, since researchers typically publish aggregate findings. The team therefore had open questions about perceived accuracy, desired level of detail, how much math to show, and whether to use percentiles or text labels.

The report used a three-tier hierarchy. At the top, MBTI-style textual markers flagged high (above the 65th percentile) and low (below the 35th) scorers on each of five motivation groups, such as "Driven" versus "Relaxed" for Achievement, with no marker for average scores. A radar graph plotted percentile scores across the five groups, and bar graphs detailed each underlying motivation with descriptions and game examples. The resulting report ran roughly 1,200 words.

The beta test drew on an initial sample of 1,127 gamers, of whom 625 were on the mailing list and received customized reports. A follow-up evaluation survey was completed by 125 participants. Four lessons emerged. First, 82% rated the profile "Very" or "Extremely Accurate," with only one respondent rating it slightly or not at all accurate. Second, 86% judged coverage of their primary motivations as good or great. Third, and most surprising to the team, 73% felt the level of detail was about right while 25% wanted more, contrary to expectations that users would want less; more game examples were added in response. Fourth, respondents expressed considerable dislike of Facebook when asked about a hypothetical app version.

That last finding led the company to host the profile on its own website and remain agnostic about how results are shared, rather than building around Facebook. The sample is self-selected and modest in size, drawn from engaged mailing-list subscribers, but the results supported the long, hierarchical report format.