A Report on Performance Differences of Mainstream Foundation Models

Abstract

AI technology has expanded rapidly in recent years, and a great number of pre-trained foundation models have been released to the public. However, there is no unified standard to measure how well these models perform. Many companies and individual developers struggle to pick a model that fits their actual needs, which often leads to issues like high operating costs or insufficient model capacity. This paper picks three mainstream closed-source commercial models, namely GPT-4o, Claude 3.5 and Gemini, alongside three widely-used open-source models Llama 3, Qwen and DeepSeek for research. I draw comparisons and share real usage experience to sort out the strengths and drawbacks of each model. Three core findings can be concluded from this research. First, closed-source models deliver better overall performance, yet they come with data leakage risks and recurring subscription fees. Second, open-source models support offline local deployment, making them suitable for confidential work and personal use, though their overall capacity is relatively weaker. Third, there is no one-size-fits-all foundation model. People should pick models based on their working scenarios, budget and data security requirements, as well as personal needs.