With a performance that blurs the line between silicon and carbon intelligence, ChatGPT (o4-mini-high) has stormed the legendary JEE Advanced battlefield, leaping past 1.5 million brilliant minds to claim a spot in the coveted top 20.
To our Indian audience, “Sharma ji ka beta” has been replaced by “GuPTa ji ki santaan.”
TL;DR
Let’s summarize the findings for those who are lazy.
| ChatGPT o4-mini-high | ||||
| JEE Advanced 2025 | Physics (60 marks) | Chemistry (60 marks) | Math (60 marks) | Total(180 marks) |
| Paper 1 | 46 | 48 | 60 | 154 |
| Paper 2 | 49 | 50 | 60 | 159 |
| Grand Total (360 marks) | 95 | 98 | 120 | 313 |
| All India Rank (AIR) | 18 | |||
The good news is that JEE topper (two humans) beat ChatGPT o4-mini-high by a healthy margin by scoring an astounding 332. The bad news (or is that good news?) is that if s is silicon intelligence and c is carbon intelligence,
In the next section, we will provide some background information for JEE Advanced. People familiar with the JEE can skip this section and proceed directly to the methodology section.
JEE Advanced: The Toughest Engineering Entrance Exam in the World
JEE Advanced is one of the toughest undergraduate entrance exams in the world. To even qualify for this exam, you need to compete with approximately 1.5 million students in JEE Mains. Only 187,223 students qualified for JEE Advanced, out of which 17,740 students will be admitted into the prestigious Indian Institutes of Technology (IIT).
This exam tests the conceptual understanding, analytical ability, and problem-solving skills of candidates in Physics, Chemistry, and Mathematics at the high school (10+2) level.
JEE Advanced is a computer-based test consisting of two compulsory question papers, Paper 1 and Paper 2, each three hours long. You can score a maximum of 360 marks (points) (180 marks in each paper). Both papers cover Physics, Chemistry, and Mathematics. The questions are usually of three types.
- Multiple-choice questions with one or more correct options
- Numerical-answer type (NAT) questions where you key in a real number (no guessable choices)
- Matrix-match or paragraph-based sets that bundle several micro-questions around a common stem or data set.
Marks per question vary by section. Multiple-choice questions carry negative marks for incorrect answers. Partial credit is possible in some multiple-correct sections, while NAT items typically carry no penalty for an incorrect entry.
Both papers are taken back-to-back in a single day with a lunch interval, and you must attempt both for your responses to be evaluated. Raw scores are “normalized” across the two papers and then aggregated to produce subject-wise and overall totals.
A difference of just 10 marks can result in a difference of a thousand ranks!
How do humans perform in JEE Advanced?
Short Answer: Poorly.
If the exam were taken by all humans aged 16-60 years worldwide, the mode of the distribution would be either zero or negative!
In JEE Mains, a much easier exam compared to JEE Advanced, the median score is 30 out of 300. The top performers in JEE Mains take JEE Advanced, and yet the median score in JEE Advanced is only 120 out of 360.
Evaluation Methodology
We used a ChatGPT Pro account, which gives us access to the latest ChatGPT models.
Model Used: o4-mini-high
We evaluated ChatGPT o4-mini-high by giving screenshots of every problem. We are a computer vision and AI consulting company, and we are very interested in ChatGPT’s ability to interpret images.
We used o4-mini-high, which, according to OpenAI, is “great at coding and visual reasoning.” The o3 model is also great, but it is much slower than the o4-mini-high model.
Prompt
The prompt was simply
“Solve and explain.”
No Deep Research
ChatGPT has a Deep Research mode in which it searches for the solution on the internet in addition to trying it on its own. To prevent it from finding the solution on the internet, we did not use Deep Research mode.
ChatGPT Response to JEE Advanced 2025
We are sharing the raw responses in the links below.
- Paper 1. There were some human errors by us on Paper 1 link. You can find the corrections Chemistry 1, Question 9 and Chemistry, Question 10
- Paper 2: Physics, Chemistry, Mathematics.
Note: While evaluating Paper 1, we didn’t separate the three subjects, but we split Paper 2 to speed up the process. Additionally, it’s unfortunate that ChatGPT does not display screenshots on shared links.
We have collated the screenshots for every problem, ChatGPT’s response, the correct answer, the total marks, and the time taken by ChatGPT in this spreadsheet.
Please link to this article if you use the spreadsheet.
ChatGPT Response Analysis for JEE Advanced
Let’s analyze ChatGPT’s response to JEE Advanced to see if we discover some patterns.
How much time did ChatGPT take to solve Paper 1 and Paper 2?
ChatGPT o4-mini-high was much faster than its human counterparts. It completed Paper 1 in 27 minutes and 58 seconds and Paper 2 in 40 minutes and 8 seconds.
The fasted response was to Paper 1, Chemistry Q1. The problem was solved in solved in just 4 seconds!
The slowest response was to Paper 2, Chemistry Q3. The problem took 338 seconds of compute time.
Subject-wise Analysis: Physics, Chemistry, and Mathematics.
Before we started this experiment, based on our previous experience, we were confident that ChatGPT would excel at math questions, and it did not disappoint. It scored 100% in mathematics. We thought it would struggle a bit with geometry problems, as they require spatial and visual reasoning, but it nailed all of them.
We also thought ChatGPT would excel in Chemistry. ChatGPT is good at memorizing rules and exceptions, making it well-suited for this task. However, it scored the worst in Chemistry, with a score of 98 out of 120.
We were expecting ChatGPT to have the hardest time with Physics because it requires sophisticated reasoning based on just a few rules, and you could create a complex problem combining those simple rules. In addition, Physics problems, like geometry problems, often have a visual diagram associated with them. Interpreting the diagram is non-trivial. ChatGPT got 95 out of 120 in Physics, but not for the reasons I suspected.
What questions did ChatGPT answer incorrectly?
The table below displays all incorrect responses from ChatGPT.
| Subject | Question | ChatGPT Answer | Correct Answer |
|---|---|---|---|
| Physics 1 | Q7 | B, D | A, D |
| Physics 1 | Q8 | 1 | 2 |
| Physics 1 | Q12 | 38 | 75-79 |
| Chemistry 1 | Q3 | A | B |
| Chemistry 1 | Q4 | C | B |
| Chemistry 1 | Q13 | 236 | 175 |
| Physics 2 | Q6 | A | A, B |
| Physics 2 | Q14 | 2.4 | 1.2 |
| Physics 2 | Q16 | 16 | 26-33 |
| Chemistry 2 | Q1 | C | A |
| Chemistry 2 | Q8 | A, B, C | B, C |
Why did ChatGPT get some answers wrong?
First, ChatGPT made no mistakes in Mathematics. Perfect!
Second, let’s discuss the three controversial questions: Physics 1, Question 8, and Physics 2, Question 16, which are problematic for similar reasons, and Physics 1, Question 12, which has a different problem.
In Physics 1, Question 8, and Physics 2, Question 16, they are asking for maximum variation, which is an ambiguous term that can be interpreted in two ways: either as the variation from the mean (as by ChatGPT and many other students) or as the variation between extremes (as by the JEE committee).
This video (in Hindi) explains the problem in good detail. The same video also explains why Physics 1, Question 12 (another question that ChatGPT did not get right) is problematic.
Tragic!
In Physics 1, Question 7 omitted a negative sign, resulting in an incorrect final answer, despite well-reasoned steps.
Physics 2, Question 6 o4-mini-high considered only one of two options. It failed to take into account a second case where two reflections are possible. The o3 model on the other hand gave the right answer.
Physics 2, Question 14 was straightforward, but ChatGPT made an error in interpreting the diagram and incorrectly calculated the thicknesses tA and tB.
In Chemistry 1, Question 3, ChatGPT o4-mini-high gets the facts wrong, but when we asked the same question to ChatGPT o3, it gave the correct answer. Permanganate (even “only” to MnO₂) is still a strong enough oxidant in neutral water to convert I⁻ to IO₃⁻, which is why iodate – not iodine – appears among the products.
In Chemistry 1, Question 4, the structures were read correctly, so the image interpretation was correct, but the reasoning failed. ChatGPT overlooked how powerful the aromatic stabilization of an indenyl anion is compared with a “plain” enolate plus a benzylic ring.
Chemistry 1, Question 13 was a massive failure as it failed to estimate the structure of P and it was all downhill from there.
Chemistry 2, Question 1 was surprisingly missed by o4-mini-high but answered correctly by o3 in 23 seconds.
Chemistry 2, Question 8 it got almost right but made a mistake about the optical properties of the molecule.
Final Thoughts
Overall it is impressive to see a $200/mo AI tool solve JEE Advanced questions at the skill level of an All India Rank (AIR) 18!
Coming from a computer vision background, we thought the image interpretation part would fail a lot but it didn’t. Almost all failures were reasoning failures and not because of ChatGPTs inability to interpret the image.
We see the stunning success of ChatGPT on JEE questions as a boon to Physics, Chemistry and Mathematics education. Currently kids born to parents with means have a massive advantage in JEE preparation because JEE coaching is expensive, and in recent years almost all AIR 1 to 100 have been professionally coached.
This system leaves our bright young minds living in rural india without the means to afford the fees of coaching institutes.
Imagine the impact on STEM education when an AI based tutor mobile app that costs only Rs 999 ($12/mo) reaches millions of these students. This tutor will patiently break down a complex JEE problem into simpler fundamental problem for a student to understand. It will be able to pinpoint where the student is lacking understanding, and create a few mini-problems for them to solve around the concept they are struggling with. It will finally put together the little concepts to solve the complex JEE problem.
Imagine an AI tutor who embraces human values more than some teachers who lack the basic empathy necessary to deal with kids who are slow at grasping concepts.
Imagine an AI tutor that encourages kids, and makes them see the beauty in science and the joys of mathematics beyond competitive entrance exams. Imagine an AI that corrects gently, never humiliating them, and builds their confidence in tackling difficult problems.
We are on the cusp of a massive revolution in education.
Acknowledgement:
We would like to thank Akansha Mallick for helping with analysis of the Chemistry questions.
