Date of Award

Spring 4-2022

Document Type

Dissertation

Degree Name

Doctor of Philosophy in Information Systems (PhDIS)

Department

Business and Information Systems

First Advisor

Cherie Noteboom

Second Advisor

Omar El-Gayar

Third Advisor

Kevin Smith

Abstract

Along with the popularity of gamification, there has been increased interest in using leaderboards to promote engagement with online learning systems. The existing literature suggests that when leaderboards are designed well they have the potential to improve learning, but qualitative investigations are required in order to reveal design principles that will improve engagement. In order to address this gap, this qualitative study aims to explore students' overall perceptions of popular leaderboard designs in a gamified, online discussion. Using two leaderboards reflecting performance in an online discussion, this study evaluated multiple leaderboard designs from student interviews and other data sources regarding the potential of each leaderboard to improve user engagement.

Analysis of the leaderboard designs was conducted using a single case study. The data was collected from semi-structured interviews, transcripts from the discussion data, and surveys. Interview data was analyzed using Corbin & Strauss’ (2014) open coding method. The result of the data collection was 221 minutes of recorded conversation which converted to 135 pages of transcribed text. The transcribed data was tagged with open codes, sorted, and grouped into related conceptual clusters resulting in 68 individual codes. These codes were then grouped into 16 concepts as part of the axial coding phase. The next phase of coding was theoretical or selective coding. In this phase, concepts were abstracted to eight broader categories or, in this project’s case, design principles that formed the essence of the emergent theory.

The eight categories that emerged from the data formed the essence of a theoretical model for system engagement using global, relative, and team leaderboards. The model communicates that factors which lead to positive system engagement include clear instructions, challenge/skill balance, and timely feedback. Within each of these areas are elements of the eight design principles that act as positive or negative system engagement factors.

Three significant findings were identified in the study in relation to factors influencing engagement in settings where leaderboards are used as the primary game element. First, clear instructions must include both clear goals but also a clear understanding of the system in which the leaderboard game element is employed. Second, team leaderboards must foster team accountability through the design of the leaderboard and through social influences. Data in this study demonstrated that team leaderboards which employ rankings within teams creates power social comparison on two fronts: intra competition (evaluating scores within the team) and extra (evaluating the scores among teams). Team accountability is increased as each individual’s contribution to the overall team performance is clearly seen and is reinforced via social influences of other team members and game moderators.

Finally, and most significantly, this project demonstrated that global, relative, and team leaderboards each have specific design features that create differing levels of challenge-skill balance. Global leaderboards should be redesigned to use “sliced” leaderboards to avoid negative engagement from lower ranking users. Level leaderboards should employ levels that are perceived as realistic and achievable. Team leaderboards should develop accountability with ranking of team members both between and within teams. The design decisions associated with each leaderboard are, thus, critical to ensuring optimal positive system engagement and avoiding significant negative system engagement outcomes.

Download

Share

COinS