Gamification
Annotated Bibliography
Compiled from Multiple Sources
August 2019
Aguilar, S., Holman, C., & Fishman, B. (2013). Leveling-Up: Evolving Game-Inspired University Course Design (pp. 46–52). Presented at the GLS’13: Proceedings of the 9th international conference on Games + Learning + Society Conference.
The study involves a high-enrollment political science course at the University of Michigan and a software project called GradeCraft. The metaphor of “leveling-up” is used in the experiment to showcase the nature of course design and research design as an iterative process.
Barata, G., Gama, S., Jorge, J., & Gonçalves, D. (2013). Engaging Engineering Students with Gamification: An Empirical Study. 2013 5th International Conference on Games and Virtual Worlds for Serious Applications (VS-GAMES), 1–8.
The authors of this conference presentation propose a study of engagement in advanced computer courses. They intend on building multiple version of a gamification system to measure the effectiveness of various game mechanics. This article contains a fair Introduction and Related Work sections with a good review of general concepts in gamification. Because this article is not a presentation of results of the study, it stresses the need to study student engagement.
Black, A. E., & Deci, E. L. (2000). The effects of instructors’ autonomy support and students’ autonomous motivation on learning organic chemistry: A self-determination theory perspective. Science Education, 84, 740-756.
The study included 380 students in an organic chemistry class. Initially 289 completed questionnaires, later 41 dropped and 137 provided feedback to comprise the primary sample of the study. 42 student instructors applied principles of self-determination theory in group work. The authors concluded that students who entered the course with more autonomous motivation had more positive experiences in the course and were more likely to adjust to the course and complete it. However, the correlation could not be made between initial autonomy and achievement in the course. The relative autonomy developed during the course did predict better academic performance in the course. The study provides an important distinction in initial autonomy and relative autonomy. The relative autonomy being granted to students during the course helps in promoting better grades.
Brown, S., & Vaughan, C. (2009). Play how it shapes the brain, opens the imagination, and invigorates the soul. New York: Avery.
Brown considers play in animals and humans as a basic communication method and learning technique. He finds behavioral and medical connections with play. He states: “The opposite of play is not work, it’s depression”. The book makes a good connection between enjoyment, motivation, and learning. However, Brown provides a vague definition of play, therefore making the identification and reproducibility of play more difficult. He states that play may involve finding novelty, embracing risk, using the imagination, or “play is a state of mind, rather than an activity”. A narrower definition of play will be more useful in designing classroom activities to balance rigor with learning objectives.
Burke, B. (2014, April 4). Gartner Redefines Gamification. Retrieved from http://blogs.gartner.com/brian_burke/2014/04/04/gartner-redefines-gamification/
Gartner continues to be an important source of research on various industries. In 2012 they predicted that majority of gamification projects will fail due to poor design. In this blog Burke provides a new definition for gamification based on Gartner’s publication. Therefore, gamification is defined as the use of game mechanics and experience design to digitally engage and motivate people to achieve their goals. This definition includes the goals of the players in the definition, which builds on previous definitions, which tend to focus on the use of mechanics or the design of activity. When the goals of players are considered first, gamification is likely to be more successful. This blog and Gartner’s research touches on the major flaw in gamification projects, which is surface approach and application of simple mechanics that generates novelty and excitement, but does not accomplish strategic goals.
Cheong, C., Filippou, J., & Cheong, F. (2014). Towards the gamification of learning: Investigating student perceptions of game elements. Journal of Information Systems Education, 25(3).
As a precursor to the development of a game-like learning system, we survey 51 undergraduate IT students to obtain their perceptions on game elements, which are the building blocks of what makes a game identifiable as such. All game elements that were presented to the respondents were highly rated. It was found that undergraduate students have a positive perception of systems that use game elements and are interested in its use for learning. Overall, students favored social interaction, engagement, feedback, and increased learning, which suggests that gamification is particularly suited to learning approaches such as social constructivism.
Connolly, T. M., Boyle, E. A., Macarthur, E., Hainey, T., & Boyle, J. M. (2012). Computers & Education A systematic literature review of empirical evidence on computer games and serious games. Computers & Education, 59(2), 661–686. http://doi.org/10.1016/j.compedu.2012.03.004
This paper examines the literature on computer games and serious games in regard to the potential positive impacts of gaming on users aged 14 years or above, especially with respect to learning, skill enhancement and engagement. Search terms identified 129 papers reporting empirical evidence about the impacts and outcomes of computer games and serious games with respect to learning and engagement and a multidimensional approach to categorizing games was developed. The findings revealed that playing computer games is linked to a range of perceptual, cognitive, behavioural, affective and motivational impacts and outcomes. The most frequently occurring outcomes and impacts were knowledge acquisition/content understanding and affective and motivational outcomes. The range of indicators and measures used in the included papers are discussed, together with methodological limitations and recommendations for further work in this area.
de-Marcos, L., Domínguez, A., Saenz-de-Navarrete, J., & Pagés, C. (2014). An empirical study comparing gamification and social networking on e-learning. Computer’s and Education, 75, 82–91
The study compares the effect on academic achievement of three teaching approaches to an introductory college course in computer technologies including office applications. The three approaches are gamification (114 students), social networking (184), and hybrid e-learning (74). All three groups were in hybrid format and the third group was set as a control group.
de Schutter, B., & Vanden Abeele, V. (2014). Gradequest — Evaluating the impact of using game design techniques in an undergraduate course. Foundations of Digital Games.
The study applies to an undergraduate computer science course, which can be considered an experimental course. The innovative features of the course include student avatars with their own stories, course quests, a course story, and a game environment with leaderboards and trophies. There was also a control course present without the experimental features. The findings of the study were that there were no significant improvement in intrinsic motivation of students in the experimental course. The authors admit that the level of engagement of students may depend on the skill and experience with which the game mechanics are applied to the course. The relationship between the presence of game mechanics in the course and engagement results must include the consideration of the gameful design to promote engagement.
DomíNguez, A., Saenz-De-Navarrete, J., De-Marcos, L., FernáNdez-Sanz, L., PagéS, C., & MartíNez-HerráIz, J. J. (2013). Gamifying learning experiences: Practical implications and outcomes. Computers & Education, 63, 380-392.
This paper attempts to verify theories that lay the groundwork for the success of gamification in learning settings. The authors designed and built a gamification plugin for a well-known e-learning platform. They made an experiment using this plugin in a university course, collecting quantitative and qualitative data in the process. Their findings suggest that some common beliefs about the benefits obtained when using games in education can be challenged. Students who completed the gamified experience got better scores in practical assignments and in overall score, but their findings also suggest that these students performed poorly on written assignments and participated less on class activities, although their initial motivation was higher.
Early, D. M., Rogge, R. D., Deci, E. L., & Early, D. M. (2015). Engagement, Alignment, and Rigor as Vital Signs of High-Quality Instruction: A Classroom Visit Protocol for Instructional Improvement and Research, 97(4), 219–239. http://doi.org/10.1353/hsj.2014.0008
The study investigates the EAR Classroom Visit Protocol to measure the quality of instruction in the classroom. The data to confirm the effectiveness of the tool was collected by implementing the tool in 4 high schools with about 6000 enrollment total. 2,171 samples were collected during the study.
Faiella, F. & Ricciardi, M. (2015). Gamification and learning: a review of issues and research. Journal of e-Learning and Knowledge Society, 11(3),. Italian e-Learning Association. Retrieved August 28, 2019 from https://www.learntechlib.org/p/151920/.
This paper provides an overview of some literature on gamification and gamified learning. If investigates and provides data on what happens when gamification is introduced in class, especially on student’s motivation, engagement, and performances
Freeman, S., Eddy, S. L., McDonough, M., Smith, M. K., Okorafor, N., Jordt, H., and Wenderoth, M. P., (2014).Active learning increases student performance in science, engineering, and mathematics. Proceedings of the National Academy of Sciences (PNAS), 111(23),8410-8415.
This foundational study to peer instruction and active learning covers 225 studies. It finds that active learning lecturing increases student performance by 0.47 SDs based on 158 studies, while traditional lecturing increased the odds of failing by 1.95. Further findings were that examination scores improved by 6% in active learning courses and students in traditional lectures were 1.5 times more likely to fail. This is important data, which follow my definition of a short game approach. Using peer instruction or other active learning techniques often means implementing gameful thinking in short periods of lecturing. However, the overall approach to the semester long teaching requires also the long game approach. This opens up autonomous student approach and availability of diverse content to invite students into exploration of the subject matter.
Gasiewski, J., Eagan, M., Garcia, G., Hurtado, S., & Chang, M. (2012). From Gatekeeping to Engagement: A Multicontextual, Mixed Method Study of Student Academic Engagement in Introductory STEM Courses. Research In Higher Education, 53(2), 229-261. doi:10.1007/s11162-011-9247-y
The article recognizes the problem of engagement in introductory college courses and identifies faculty attitude toward science by calling them gatekeepers. In this academic view, scientists are born, they are not made. Therefore, introductory courses are supposed to be difficult to filter out students who are not the right material. As a result, introductory courses are based on memorization and shallow application of knowledge, which discourages students and leads to their changing of majors. The article concludes that the instructor can adjust into an “engaging professor” by using student focused teaching.
Günüç, S., & Kuzu, A. (2014). Factors Influencing Student Engagement and the Role of Technology in Student Engagement in Higher Education: Campus-Class-Technology Theory. Turkish Online Journal Of Qualitative Inquiry, 5(4), 86-113.
The study attempts to create a relationship between student engagement and the use of classroom technology. 45 student teachers were inquired, 25 through face-to-face interviews and 20 through written communication. Technology, which was well integrated into the teaching process, was found to benefit student engagement. However, when technology was poorly integrated, it was distracting. This study seems to follow a theme in measuring engagement in the classroom, while experimenting with tools. When instructors use the tools skillfully and with strategic design, the tools can be of benefit. Whether this is technology or gamification, the simple use of an experimental tool does not automatically create engagement.
Hakulinen, L., Auvinen, T., & Korhonen, A. (2013). Empirical Study on the Effect of Achievement Badges in TRAKLA2 Online Learning Environment. Learning and Teaching in Computing and Engineering, 47–54.
The gamification of this undergraduate computer science course involved digital badges. The goal of the study was to modify student behavior in curbing procrastination, improving attention to detail in assignments, and increasing retention of the material. The badges were granted based on academic achievement in the course. A control group was established with the same assignments, but without the badges. The findings demonstrated that the digital badges did have a positive effect in achieving higher grades in the gamified class. This was especially true for advanced badges within a declared major of the student. My reflection on this approach is in the desire of students to have an external evidence of their time investment and a mini-credential. Digital badges help to provide such credential and in that way are very loosely related to deeper gamification principles in intrinsic motivation.
Hanus, M. D., & Fox, J. (2015). Assessing the effects of gamification in the classroom: A longitudinal study on intrinsic motivation, social comparison, satisfaction, effort, and academic performance. Computers and Education, 80, 152–161.
Two undergraduate courses were selected for the study. One was a control group without gamification elements and the other with specific game mechanics applied. The gamification elements included digital achievement badges and leaderboards. The study found that the gamified courses did not improve general academic behavior of students and did not improve motivation of students in the course. The final exam scores were worse in the gamified group than in the control group. The authors concluded that the competition encouraged in the gamified courses left students unsatisfied and unmotivated. The gamification was also a required element, not an optional feature. This may be another study, which does not apply gamification based on gameful principals like the need for autonomy in play to be enjoyable.
Hamari, J., Shernoff, D. J., Rowe, E., Coller, B., Asbell-Clarke, J., & Edwards, T. (2016). Full length article: Challenging games help students learn: An empirical study on engagement, flow and immersion in game-based learning. Computers In Human Behavior, 54170-179. doi:10.1016/j.chb.2015.07.045
The article asks good research questions about student engagement and game-based learning. The questions are: Do challenge and skills predict engagement and immersion in game-based learning? _Do engagement and immersion predict perceived learning in game-based learning? _How engagement and immersion mediate the effect of challenge and skills on perceived learning in game-based learning? The connection of schoolwork to professional work and how the flow can help to create happy students and happy employees is very important.
Huang, B., Hew, K. F., & Lo, C. K. (2018). Investigating the effects of gamification-enhanced flipped learning on undergraduate students’ behavioral and cognitive engagement. Interactive Learning Environments, 1-21.
This study examined whether gamification can enhance student engagement in a flipped course. A comparison study was conducted, involving two classes of undergraduate students in an Information Management course. The results indicated that students in the gamification-enhanced flipped learning group (n = 48) were more likely to complete the pre-class and post-class activities on time than those in the non-gamified flipped learning group (n = 48). Students in the gamification-enhanced flipped learning group also produced higher quality artifacts than the non-gamified flipped learning group in the pre-class thinking activities. Moreover, students in the gamification-enhanced flipped learning group scored significantly higher in the post-course test than did their non-gamified counterparts.
Juul, J. (2013). The Art of Failure: An Essay on the Pain of Playing Video Games. MIT Press. Retrieved from https://books.google.com/books?id=5sC1knAj_isC
This book is a careful and detailed study of failure in computer games. While most people may think that we play computer games for fun, Juul submits that we play to seek out failure. Humans desire the feeling of competence and success, but only when it feels deserved. Games create the environment, where grunt work or increase in skill allows the player to have a satisfaction and fun as a result of handling failure well. In gamification of education this is a key strategy that in traditional courses makes for a poorly designed game. Failure in courses is typically discouraged and avoided. However, failure is how we learn
Kim, B. (2015). Designing Gamification in the Right Way. Library Technology Reports, 51(2), 29-35.
In chapter 5 of the book Understanding Gamification Kim suggests that designing research around gamification is very difficult. There are too many variables to account for. While many teachers and business people like the idea of involvement through using lightweight gaming principles, the data proving that it is effective is not evident. Kim discussed classroom and library gamification projects, which lack a clear purpose or goal of why gamification is implemented. Some projects use gamification to advertise the library or class, some use gamification as an experiment. Kim advocates recognizing the research of gamification in designing for various gamer personality types and for cultural backgrounds of participants. Kim points out that classroom gamification must go beyond engagement in order to support academic achievement. She mentions appropriate game dynamics and supporting game mechanics to promote repetition and association, which promotes acquiring demonstrative knowledge. This is exactly the main problem with many of gamification studies and experiments. While games are fun in general, creating games to promote specific learning goals require deep reflection and skillful application.
Kim, S., Song, K., Lockee, B., & Burton, J. (2018). Theories for Gamification in Learning and Education. In Gamification in learning and education (pp. 39-47). Springer, Cham.
This book provides a comprehensive background on the theories behind the gamification of learning
Krause, M., Mogalle, M., Pohl, H., & Williams, J. J. (2015, March). A playful game changer: Fostering student retention in online education with social gamification. In Proceedings of the Second (2015) ACM Conference on Learning@ Scale (pp. 95-102). ACM.
Many MOOCs report high drop off rates for their students. Among the factors reportedly contributing to this picture are lack of motivation, feelings of isolation, and lack of interactivity in MOOCs. This paper investigates the potential of gamification with social game elements for increasing retention and learning success. Students in our experiment showed a significant increase of 25% in retention period (videos watched) and 23% higher average scores when the course interface was gamified. Social game elements amplify this effect significantly – students in this condition showed an increase of 50% in retention period and 40% higher average test scores.
Kuo, M., & Chuang, T. (2016). Full length article: How gamification motivates visits and engagement for online academic dissemination – An empirical study. Computers In Human Behavior, 55(Part A), 16-27. doi:10.1016/j.chb.2015.08.025
Online education is a big business and applying gamification poses a number of challenges. Play, however, is a range of voluntary, intrinsically motivated activities, which can affect learners. As play is molded into games, by applying rules and rewards, humans can be directed into certain behaviors. Gamification is a process of applying playful components with a non-game target. This allows for non-game applications of the mechanics in order to make activates more enjoyable and engaging. Gamification can be applied in a commercial environment to enhance organizational performance, or in an academic environment to help students focus on the course content. Games can be considered as tools to provide joyful, intrinsic motivation. Gamification attempts to parse those characteristics in order to apply it to non-game environments.
Nelson, C. (2010). Dysfunctional illusions of rigor: Lessons from the scholarship of teaching and learning. In L. B. Nilson & J. E. Miller (Eds.),To improve the academy: Resources for faculty, instructional, and organizational development, 28(177-192). San Francisco: Josse-Bass. Major sections of this article available at: http://cgi.stanford.edu/~dept-ctl/tomprof/posting.php?ID=1058
Indiana University Professor Emeritus of Biology Dr. Craig Nelson is the author of “Dysfunctional Illusions of Rigor”. These illusions are reflected in difficulties of adoption of gamification teaching practices the same as any other alternative approaches to undergraduate teaching. The following misconceptions are the illusions: “1. Hard courses weed out weak students.” When students fail it is primarily due to inability, weak preparation, or lack of effort.
Ong Chao, X., Teh Tuan, A., Chan Ying, H., & Lee Tse, Y. (2014). Effectiveness of Gamification in Vocational Technical Education. Proceedings Of The European Conference On Games Based Learning, 2636-644.
The study is based on 450 students enrolled in a technical university in Europe. Two courses were selected for the gamification project. One at the undergraduate level, Computer Organization, and a master’s level course on Cloud Computing. The results of the study showed 75% of students passing on the first try. The study concludes that the increase in the percentage of passing students and participation in voluntary activities and challenging assignments is caused by the gamification efforts. The teacher is identified as coach, motivator, and facilitator. Gamification made the revision of content more interesting. The study has a fair amount of students, but the comparison of students who were passing could have been attributed to other factors like the excitement of the instructor, who was awarded the Teacher of the Year recognition.
Pavlus, J. (2016, March 7). Why We Love the Games That Enrage Us Most. Scientific American. Retrieved March 10, 2016, from http://www.scientificamerican.com/article/why-we-love-the-games-that-enrage-us-most
The article in Scientific American connects attitudes toward games with academic activities. The scaffolding present in games, the opportunity and encouragement to fail, the practice that produces progress, they all can be applied in school. The author discusses a balance needed between tasks that are too easy and tasks that are too difficult. Easy tasks can create boredom. Too difficult tasks can create frustration. To instill competence instructors, need to find a balance by “easing players up the learning curve with early levels that act as self-guided tutorials for mastering basic moves and controls”. Introductory courses run the risk of applying curriculum of assignments created with a single student in mind to a large and diverse group of students. Tasks that are easy and tasks that are too difficult, when disconnected from each other, can turn an introductory course into an ongoing quest for boredom and frustration. Application of gamification to an introductory course must include an automated way of matching the task difficulty to the student ability.
Pierce, K. and Palsole, S.V. (2015). Gamification as an Engagement Strategy in Online Courses. 8th Annual Emerging Technologies for Online Learning, International Symposium. Dallas
This paper presents results by the implementation of gamification strategies in a stratified business course that had students with multiple levels of expertise and a large format introductory science course. The paper presents the theoretical framework on gamification and comparative results from the gamified courses vs non-gamified courses.
Prensky, M. (2010). Teaching Digital Natives: Partnering for Real Learning. London: Sage Publishers.
The book is a pedagogical exploration of a teacher dilemma associated with teaching with technology: What elements of traditional teaching should be retained and what elements of technology should be adopted to be most effective in education? The author agrees that an overwhelming majority of teachers love and want to help students. With these good intentions, there is a body of knowledge, that may help to better identify the methods adapted to a new generation, Prensky called Digital Natives. A major difference being the focus on a new reward system, which focuses on building passion in students instead of applying demotivating punishments. The learners who are Digital Natives are different from former generations. School is boring to kids, who grew up with games and know what engagement feels like. Teachers need new ways to motivate this new generation of learners.
Sailer, M. & Homner, L. Educ Psychol Rev (2019). Gamification of Learning, A Metaanalysis. https://doi.org/10.1007/s10648-019-09498-w
This meta-analysis was conducted to systematically synthesize research findings on effects of gamification on cognitive, motivational, and behavioral learning outcomes. Results from random effects models showed significant small effects of gamification on cognitive (g = .49, 95% CI [0.30, 0.69], k = 19, N = 1686), motivational (g = .36, 95% CI [0.18, 0.54], k = 16, N = 2246), and behavioral learning outcomes (g = .25, 95% CI [0.04, 0.46], k = 9, N = 951). The key shortcoming of the study is the lack of data around implementation of gamification that could have long ranging implications for success.
Seaborn, K., & Fels, D. I. (2015). Gamification in theory and action: A survey. International Journal Of Human – Computer Studies, 7414-31. doi:10.1016/j.ijhcs.2014.09.006
The article attempts to validate gamification as meaningful for engagement and provide evidence of effectiveness as a tool for motivation. Games, like the elephant test, are easy to recognize, but difficult to define. Gamification is the use of game elements and mechanics in non-game contexts. Gamification is criticized as focusing on pointisification, badges, and leaderboards even though they are the least essential game concepts. It is also called exploitationware, because it uses game approach to take financial advantage of consumers. While gamification is a popular subject of blogs and conference keynotes, the theoretical definitions are poorly defined and require clarification. Applying theoretical definitions to gamification will help to stabilize the research of gamification and fuel the development of the practice. The survey was designed to explore the concept of gamification, examine available research on gamification, provide links between theory and practice of gamification. Survey of literature was performed with meta-synthesis approach. The rigorous search included EBSSOhost, JSTOR, Ovid, ProQuest, PubMed, Scopus, and Web of Knowledge. Findings: Average age of gamers is 30, 45% are female, 62% of population play games socially. The terms have been clarified and supported by peer-reviewed references. A number of important follow up research questions have been also asked.
Sheldon, L. (2012). The multiplayer classroom: Designing coursework as a game. Boston, MA: Course Technology/Cengage Learning.
Lee Sheldon writes about his course being changed into a multiplayer classroom experience. This is a good case study for practical application of gameful principles. The design makes great use of characters and stories. Students are engaged in a playful learning experience. The gamification takes place throughout the course and modifies the standard approaches. Presentations are turned into quests, fighting monsters is taking quizzes, and crafting means writing game-analysis papers. The class is made out of 40 students divided into six zones for group activities. To this end, this practice is not applicable to all academic environments, such as coordinated courses, where curriculum and assessments are inflexible. Many faculty do not have permission to modify course milestones. This book and practices work well for faculty who are in full control of the course and course outcomes.
Swap, R. J., & Walter, J. A. (2015). An Approach to Engaging Students in a Large-Enrollment, Introductory STEM College Course. Journal Of The Scholarship Of Teaching And Learning, 15(5), 1-21.
Educators are sometimes pessimistic about the possibility of engaging students in large enrollment introductory courses. This is due to difficulties in engaging students and especially doing so in large enrollment sections. The term “engaged” may be understood in many ways. Starting from the concept of interest, through applicability of the course content, to a broader conversation outside the classroom. Another view on engagement is a positive influence on educational outcomes. The challenge of large enrollment courses is that, while they are favorable in terms of organizational profit, they can also carry a risk of limited engagement. Large enrollment means wide variety in knowledge and experience of students, challenges in assessment and classroom activities. Such courses seem necessary as the goal is the exposure of students to the subject matter. Large enrollment courses (160 – 250 students) suffer from lack of engagement of majority of students. Purpose: Provide a hybrid approach of teaching in a large enrollment introductory course to generate engagement. Findings: The study documents how engagement can be generated in large enrollment courses by integrating traditional lectures, with student-centered social constructivist pedagogies. These include use of small group activities, diversity of instructional media, leveling of the classroom hierarchy to invite broader participation, creation of cognitive dissonance, high instructor availability.
Zichermann, G., & Cunningham. C. (2011). Gamification by Design. Sebastopol, CA: O’Reilly Media.
This book is a refined theory of gamification by an early pioneer of gamification, Gabe Zichermann. The concepts of fun quotient, evolution of loyalty, human motivation are all foundation of both enterprise gamification and gambling establishments. Included are case studies of Nike Plus, Health Month and others. While aiming at enterprise application the book provides good theory of gameful thinking. The thorough discussion of game mechanics and application into business processes helps to adopt gamification into academic application.