Benchmarking Vision-Language Transformer Models Across Diverse Datasets

Abstract

This project develops a common benchmark harness for comparing vision-language models across visual question answering, captioning, detection, classification, preference, rating, and prompt-reconstruction tasks. The final evaluation focuses on hosted API models using 20 examples per benchmark and identical execution settings. GPT-4o and Claude Sonnet 4.5 are compared on 44 jointly completed benchmarks; GPT-4o wins 30, Claude wins 10, and 4 are tied. A second comparison covers the 22 Type A/B/C benchmarks completed by GPT-4o, GPT-4.1, GPT-5, Gemini 3.5 Flash, and Claude Sonnet 4.5. Gemini leads this restricted Type A/B/C set, while GPT-4o is the stronger model in the direct GPT-4o–Claude comparison. The results show that model rankings depend on task family and that matched benchmark coverage is necessary for fair aggregation. Because each benchmark uses a small sample and hosted models may change, the conclusions should be interpreted as a reproducible experimental snapshot rather than a permanent leaderboard.