Date of Award

Spring 3-2025

Document Type

Dissertation

Degree Name

Doctor of Philosophy in Information Systems (PhDIS)

First Advisor

Cherie Noteboom

Second Advisor

Deb Tech

Third Advisor

Jun Liu

Abstract

Bias in large language models (LLMs) poses substantial risks to fair and accurate information processing, particularly in high-impact contexts such as news dissemination and content moderation. These models often learn and unintentionally amplify systematic biases, ranging from confirmation and negativity to anchoring and partisanship, present in the data on which they are trained, thereby distorting public discourse and potentially fueling misinformation. Building upon theories in cognitive psychology and AI-decision-making, the Cognitive Bias in Artificial Intelligence Theory (CoBAIT) becomes the interdisciplinary theoretical foundation of this work.

Drawing on both cognitive bias theory and a design science approach, this dissertation develops a novel CoBAIT-Informed Fine-Tuning (CIFT) bias-detection artifact that integrates short, text-based “context snippets” during LLM fine-tuning. The framework is empirically evaluated using the BABE dataset, which contains over 3,700 sentences from politically oriented news articles annotated by expert raters. By prepending domain specific cognitive bias language known to evoke those heuristics in humans, such as confirmation or negativity bias, to each sample, the LLM gains a clearer “mental model” of what constitutes biased language. When compared to a baseline DistilBERT model, results show an approximate 1–2% improvement in F1 score and a notable reduction in false positives, indicating that the enhanced models are more effective at discerning biased and neutral content. Although the number of false negatives increases, this trade-off proves beneficial in domains where the cost of overclassifying bias far outweighs the risk of occasionally missing biased text, such as where impartiality is particularly important.

Beyond demonstrating technical feasibility, this dissertation contributes to the literature by bridging psychological theory, design science methodology, and advanced Natural Language Processing (NLP) practices, showcasing how explicit cognitive bias frameworks can yield measurable improvements. The findings have implications for large-scale platforms such as social media networks, where small percentage gains can translate into thousands or even millions of accurately identified biased posts, and for smaller, specialized communities, where higher precision and transparency are critical to sustaining trust. Future directions include extending these techniques to multilingual contexts, more complex bias typologies, or otherLLM architectures (e.g., GPT or Llama), thereby refining how cognitive principles are harnessed to build more equitable and reliable AI-driven content analysis systems.

Download

Share

COinS