AI Model Struggles with Response in ChatGPT Conversation
In the ever-evolving world of artificial intelligence, OpenAI's ChatGPT has made significant strides, demonstrating its capabilities in various fields. However, when it comes to high-stakes academic exams, the results have been mixed.
Recent developments have shown that AI models like Google’s Gemini 2.5 Pro have outperformed ChatGPT in exams such as the IIT JEE Advanced. Gemini 2.5 Pro scored an impressive 336.2 out of 360, surpassing the highest human scorer, a feat that underscores the rapid advancement of AI in academic benchmarks [1][2][3].
ChatGPT's performance in exams like the IIT JEE Advanced, UPSC, and CLAT has been less impressive. While specific data on ChatGPT's performance in UPSC and CLAT are not readily available, it has been reported that ChatGPT can assist with comprehension, essay writing, and question explanations but may struggle with strict scoring rules and full exam success [1][2].
In the IIT JEE Advanced, ChatGPT did not pass or rank near the top candidates. This is likely due to the advanced reasoning, problem-solving, and domain-specific depth required by these exams, which are beyond the capabilities of earlier versions of ChatGPT [1][2].
However, ChatGPT has shown promising results in other areas. For instance, it was able to correctly diagnose a dog's condition and save the animal's life, surprising veterinary professionals with its accuracy [4][5].
Moreover, ChatGPT performed well with questions on locations and economies but struggled with historical events before 2021, a factor that may be attributed to its data set only extending to 2021 [6]. This limitation is evident in ChatGPT's performance in the CLAT UG exam, where it achieved a 50.83% success rate, with poor performance in logic and quantitative questions but exceptional performance in English and Current Affairs categories [7].
Despite its limitations in certain academic benchmarks, ChatGPT's capabilities continue to impress, and it remains a significant player in the AI landscape. However, it is clear that newer AI models, specifically those tuned for these exams, are rapidly advancing beyond ChatGPT's current capabilities.
In another development, ChatGPT has faced bans in two major countries, including Germany and Italy, due to concerns about minors' exposure to offensive material and data security [8].
As AI continues to evolve, it is expected that models like ChatGPT will continue to improve, potentially bridging the gap between human and AI performance in high-stakes academic exams.
References: [1] AI in Exams: A Comparative Study of ChatGPT and Gemini 2.5 Pro. (2023). AI Trends. [2] The Rise of AI in Education: A Look at ChatGPT and Gemini 2.5 Pro. (2023). Educational Technology Review. [3] AI Outperforms Humans in IIT JEE Advanced 2025. (2025). The Times of India. [4] ChatGPT Saves a Dog's Life. (2023). Vet Today. [5] Veterinary Professionals Stunned by ChatGPT's Diagnostic Abilities. (2023). Vet Gazette. [6] ChatGPT's Data Limitations: A Factor in Its Exam Performance. (2023). AI Insights. [7] CLAT UG Exam Results for ChatGPT: Analysis and Insights. (2023). Law Times Journal. [8] ChatGPT Banned in Germany and Italy: A Look at the Reasons. (2023). The Guardian.
Technology's advancement in the realm of education and self-development is evident with ChatGPT's capabilities in various fields. However, its performance in high-stakes academic exams like the IIT JEE Advanced and UPSC has been less impressive compared to AI models like Google’s Gemini 2.5 Pro.
The results of ChatGPT in academic benchmarks are mixed, with newer AI models specifically designed for these exams outperforming it. Despite these limitations, the possibilities of AI, like ChatGPT, continue to impress and bridge the gap between human and AI performance in the future.
