Over the past few years we’ve seen one aspect of our products become more and more important. This is the ability to reliably answer questions about how players interact with our games. In order to facilitate this, we often bundle our games with an online dashboard.
In this article we’ll take an in depth look at the ways these dashboards supplement our games. We’ll also walk through the design process we employ to make sure we present and visualize data in the most efficient format.
A different dashboard for everyone
When starting development on a new dashboard we start with defining who the target audience will be and what they want to get out of the dashboard. This is usually a combination of the following five groups:
- The customer: to get insight in how the game is helping them to reach their goals, and to find out how the game is performing in terms of reach and player retention. The dashboard also typically allows them to manage the licenses they supply to users.
- The development team: to get insight in how players interact with the game in order to be able to improve the engagement factor of the game.
- The domain experts: to get insight in the results of the game and the impact it has on the players, in order to be able to validate and/or improve the effectiveness of the game.
- Coaches or trainers that use the game for their students: to get insight into the performance of individual students and give personalized guidance.
- The players: to gain extensive insights into their own behaviour and learnings in the game.
Each of these groups wants different questions answered by the dashboard. But finding out what these questions are, by itself, means asking them the right ones.
Asking the right questions
When talking to members of these different user groups, even when they are part of the development team, they will often have a clear idea of what they would like the dashboard to show them. People quite often show us colourful pie charts, or surprise us with an extensive list of the data they would like to see represented.
We, however, prefer to start our design from another point of view. Instead of focusing on the data, and how to best show it, we like to ask users about the goals the dashboard should help them accomplish. If a user tells us he’d like to see the length of a session, we first ask them ‘Why?’ in response.
This will often trigger them to reveal their reasons for wanting to see certain data. In the case of the session length example, it might lead them to elaborate that they really want to be able to find out if players are actually enjoying the game. This, in turn, will lead us to identify additional or sometimes entirely different metrics which would help to answer that question.
To document these goals and sub goals we like using the following “User story” format:
<Role> should be able to <Feature> so that <Result>.
By documenting the features in this manner, our development team has the creative freedom to come up with the best way to achieve the Result, taking it into account when implementing all the minutiae that are involved with actually realizing a Feature. By specifying the Role as well, we make it explicit from the start which types of users should be able to get access to certain features: a privacy-must.
Defining the data
The next step in designing our dashboards is defining which data we are going to record. When looking at what data we should store, we look at measurements we can do in two major categories: behavior and performance.
The difference between those two categories is that the behavior category describes how the user interacts with the game. In contrast, the performance category describes the result of that interaction. For example: the duration of a single play session takes is behavioral data. What the highest achieved score was in that session is performance data.
This distinction will become important when trying to answer questions with this data, because the data tells us very different things. A player’s high score will not really tell us anything about how engaged they are by the game. Likewise, the duration of a play session tells us very little about the difficulty level of the game.
The combination of these two types of data allows us to answer interesting questions. For example, we need both types of data to answer the question “Do players who lose interest in the game quickly also have low scores?”. If so, the game might be too difficult or poorly explained. This conclusion would allow us to make more informed design decisions during continued development of the game.
The main takeaway here is that ideally you want to make sure you have data in both categories since you’ll need it to reach conclusions that are actionable.
Collecting metrics
From the perspective of a user, the dashboard is the thing you see in your browser when you press that login button. In reality, the full system consists of three distinct parts:
- The game integration: Collecting metrics starts with measuring them. This may seem like the obvious and straightforward part, but it often turns out to be rather involved.When working on this part of the dashboard, we have to make sure that data integrity gets safeguarded when internet connections are flaky or even non-existing for longer periods of time. To handle this, we store statistics locally on the user’s device until we get an opportunity to send it to the server. To further complicate things, our users may also want to play the game on different devices. We typically solve this problem by allowing only one active session, automatically logging the user out of their oldest one.
- The back end: When the data traffic reaches the server, it needs to be validated, throttled and stored in a database. These problems have been solved before, so we started looking for an established framework to help us deal with this. We examined several options for storing and presenting game metrics first, such as Unity Analytics. However, these frameworks typically either didn’t allow for enough customization, raised some privacy-concerns due to the location of their servers or simply weren’t user-friendly enough for our customers. In the end we settled on using ‘Laravel’, an established light-weight framework for general web development, which does help us with all of the mundane tasks of creating dashboards without limiting our creative freedom. We have been loving our choice so far and didn’t look back once.
- The front end: Then, finally, there’s the front end: the actual thing people see when they interact with our dashboard. An interesting thing of note here: in addition to presenting the game-usage metrics here, we also often collect metrics about the usage of the dashboard itself. Don’t forget that some of the user groups we’ve listed above (see: “A different dashboard for everyone”) will mainly interact with our game through this dashboard. Giving them a high-quality user experience is therefore just as paramount as offering it to our players. In the future this is something we’d like to expand upon: our dashboards are really rather functional right now, but they don’t feel as engaging as the games we make for our end-users. We think we could accomplish more if it’s actually fun for our domain experts to use these dashboards.
Privacy by design
We have all had those e-mails asking about your explicit permission to keep sending you newsletters: the GDPR is here and it’s here to stay. We have always been very much aware of the need to protect the privacy of our users, especially from a technical point of view, but meeting the full demands of the GDPR has put additional emphasis on this.
Earlier in this article, we’ve talked about the benefit of asking your customers why they want to see certain data. Asking this question is not only a good design practice, it also helps address the privacy of our users.
Sure, we can show our customers exactly where and when individual user have been playing our localization-based games, but do they really need that information? Often, what are our customers are really interested in accumulated data: they want to be able to reach broad-strokes conclusions quickly for the whole player base, or a specific subset of players. Asking the why-question not only helps to reveal this, having the answer to this question is also needed as a basis for the dashboards privacy policies.
In closing
Metrics are at the basis of solid decision making. When you focus on what your users want to accomplish with these metrics it opens up the design space, allowing for creative problem solving and a goal-oriented development process.
Our reuse of a light-weight web framework prevented us from reinventing the wheel, while offering out-of-the-box advantages like secure authentication and fast back end development.
By offering all types of users a dashboard, we allow them to interact with, and reflect upon the entirety of our game-based solution. For some of these users, the dashboard will even be their primary means of interaction.
We expect dashboards to be of clear added value to most of our future productions and are continually looking to expand its base functionality to make them just as fun and effective to use as our games.
Would you like to chat with us about the possibilities of serious gaming and metrics collection, or do you happen to have examples of fun ways to efficiently present metrics to users, then we would love to talk to you!
[sc name=”newsletter-signup-EN” ]
