Analyzing Brand Perception In LLMs

Abstract

This thesis investigates brand perception in different Large Language Models (LLMs), focusing on three brands: Apple, Samsung, and Huawei. We first established an understanding of brand perception and the construction of psychometrically sound tests. Leveraging this foundation, we defined four metrics across two dimensions, sentiment and preference, to facilitate a comprehensive analysis. In the sentiment dimension, we observed that the Gemma LLM exhibited consistent bias across all brands, whereas ChatGPT3.5 and ChatGPT4 displayed similar behavior for Apple and Samsung, with notable differences for Huawei. In the preference dimension, all studied LLMs demonstrated transitivity consistency, consistently preferring Apple over Samsung and Samsung over Huawei. Our findings highlight the potential for extensive analysis using the defined metrics, limited here by time constraints. We suggest several avenues for future research, including expanding the range of brands and LLMs analyzed, improving the question bank through collaboration with psychologists, and incorporating varied question connotations and mask questions to enrich the study’s depth. This study provides a methodological framework for assessing brand perception in LLMs, with implications for broader applications beyond the specific brands and models examined.