In this article, we will take a close look at how serious games are effectively validated, specifically in the environmental sustainability and healthcare industries.
There are still those who question the effectiveness and efficiency of serious games as a tool for training, education, and awareness creation. These doubts are usually borne out of the inaccessibility and in other cases the unavailability of evidence that proves serious games’ effectiveness.
By the end of this submission, we want to establish as a matter of fact that serious gaming can be validated scientifically, accessing cases from the health and sustainability sectors.
There are two ideas we need to define before we can proceed to the meat of the matter. These definitions will help put us on the same page.
- Firstly, serious gaming as a term naturally elicits nothing but entertainment. Well, there is nothing wrong with that perception. The only concern is that that perception does not capture the whole truth. In our context, serious gaming is an adopted methodology that explores the entertainment elements of games for skills training or enhancement, to educate and create awareness. In essence, serious games have a predetermined purpose besides entertainment.
- The second term is science. This might seem pretty straightforward and obvious but it is crucial to make sure you understand what we mean by scientific validation in a serious game’s context. Science has been defined as the pursuit and application of knowledge and understanding of the natural and social world following a systematic methodology based on evidence ( Science Council ). There are definitely different definitions but this definition captures all the essentials.
So the question is, how does serious gaming fit into the world of science? Here is how:
- Serious games seek to solve or address existing problems, just like science
- If serious gaming is to be taken seriously, then it’s approach should be systemic and taken through rigorous tests, like in scientific research.
We will demonstrate why serious games can be scientifically validated with two examples from our projects.
CASE 1 (Water Industry)
Problem to be solved
Water, as globally recognized, is key to man’s existence. The world in recent times has been reminded of this reality through global warming and climate change. Different countries are faced with different water challenges, including water supply deficits.
In the Netherlands for instance, water supply is generally not problematic. Nonetheless, water companies such as Vitens, identified serious concerns with water usage during peak hours. To avoid future operational tragedies, Vitens partnered with Grendel Games to explore serious gaming solutions.
Question / Hypothesis
The question to answer then was, “Can serious gaming be a solution to water conservation?” The collaboration aimed to reduce water usage during peak hours.
The hypothesis was that, if serious gaming can be used to teach then it could also be used to enlighten households on water conservation and eventually teach people to save water.
Method / Solution
Vitens and Grendel Games decided to create an application, Water Battle, with three elements. The product is a game for children from ages 6 to 11, it also comes with educational materials on water conservation, and finally, households can also track their water usage.
Vitens and Grendel Games figured that the best way to change water consumption behaviour was through the collaborative efforts of household members, that is, children and parents/guardians. So, this is how the product worked:
- Parents and children predetermine water consumption targets that are entered into the game/application (Water Battle). The children then play the game and learn water-saving tips.
- Most importantly, the children can only finish the game if the predetermined targets are not exceeded.
- This situation automatically sparks conversations in the household, which leads to controlled water usage and water saving.
Validation / Results
In scientific terms, there are two options; a between-group comparison or a within-group comparison.
These options mean we could have households that used the product compared with those that do not for comparison or just a single group of households and a comparison of their performance before and after product use.
Vitens and Grendel Games adopted the within-group approach. They recruited over 300 households and monitored their water usage during peak hours for 3 months as they used the product, Water Battle.
The ratio between the peaks and valleys in water usage significantly improved. Before the start of the campaign, this ratio was 1.87. In other words, during peak hours, water usage was 1.87 times higher than the rest of the day.
As participating households used Water Battle, the ratio lowered to 1.77 (with a significance of P=0.01). At the end of the 3-month test period, Vitens reported that participating households recorded a 7 percent reduction in water usage.
Future works
Since the UK is also unfortunately faced with a dire water situation, Water Battle will be fully deployed there in the coming years. The ultimate goal would be to change water consumption behavior through serious gaming.
CASE 2 (Health Sector)
Problem to be solved
In surgeries such as laparoscopy, trainees traditionally use simulation trainers for skills acquisition. This methodology comes with a drawback. Trainees are less motivated to continue with systematic follow-up sessions once training goals are reached. The reason is that simulators are considered uninspiring and boring.
Question / Hypothesis
Can serious gaming as a tool help motivate the trainees to continue systematic follow-up training?
The hypothesis is, if games are generally entertaining and engaging, then serious gaming is an effective motivation tool for trainees.
Method / Solution
The University of Groningen Medical Center (UMCG: the Netherlands) and Grendel Games develop a product called Underground. The game is played on a Nintendo Wii-U. In the game’s cave scenery and setup, players use tailor-made controllers to build roads and create paths. The game has no medical or anatomic element to it but is meant to train the pycho-motor skills of laparoscopy trainees.
At the Radbound University Medical Center, a study was conducted by Ijgosse and colleagues (2018), with 107 participants, divided into a control group and an Underground training group. The control group had four training sessions on a simulator while the Underground group played Underground for three sessions and transferred to the simulator on the fourth session. This transfer was to test the engaging effect of Underground.
The study compared the control group’s first-session performance to the fourth session of the Underground group. The fourth session of the Underground group to understand the effect of playing Underground in the three previous sessions on the last session (simulator).
Validation / Results
- The results reported indicate a significant difference in performance between the two groups on 3 levels (Cameral navigation, Instrument navigation. and Coordination, p<0.01). The Underground group performed significantly better than the control group.
- The authors also acknowledged the effect of Underground as a medium for skills transfer.
Future works
Underground admittedly has a huge potential. The conclusions from the studies conducted so far suggest there a few additions that could heighten engagement elements amongst trainee surgeons. For instance, there have been suggestions to include a scoring and ranking system.
The inclusion is expected to feed the competitive nature of the target audience.
These suggestions can only make the product better and help sharpen the skills of trainee surgeons for societal benefit. This is exactly what serious gaming strives to achieve.
Concluding remarks
We have illustrated the scientific approach to serious games validation, with 2 briefly described case studies. So, you are completely right if you ever thought serious gaming is a viable solution for societal problems. And YES, the solution can be scientifically validated.
Ongoing discussions should be in the lines of, how validation techniques in serious gaming can be improved and creative ways the solution can be adapted to solve societal problems.
If you already have some ideas or questions, let’s continue the conversation, contact Tim Laning via email / tel: +31681565635 / Linkedin
Information sources / Further reading
Serious Games and Entertainment
https://link.springer.com/book/10.1007%2F978-3-319-51645-5
Science Council
https://sciencecouncil.org/about-science/our-definition-of-science/
The effects of serious games on laparoscopic simulator skills
https://www.rug.nl/research/portal/files/14930477/Chapter_2.pdf
Saving robots improves laparoscopic performance: transfer of skills from a serious game to a virtual reality simulator
