LLM for Comparative Narrative Analysis

In this paper, we conducted a Multi-Perspective Comparative Narrative Analysis (CNA) on three prominent LLMs: GPT-3.5, PaLM2, and Llama2. We applied identical prompts and evaluated their outputs on specific tasks, ensuring an equitable and unbiased comparison between various LLMs. Our study revealed that the three LLMs generated divergent responses to the same prompt, indicating notable discrepancies in their ability to comprehend and analyze the given task. Human evaluation was used as the gold standard, evaluating four perspectives to analyze differences in LLM performance.

@article{kampen2025_2504.08211,
  title={ LLM for Comparative Narrative Analysis },
  author={ Leo Kampen and Carlos Rabat Villarreal and Louis Yu and Santu Karmaker and Dongji Feng },
  journal={arXiv preprint arXiv:2504.08211},
  year={ 2025 }
}