Testing times for JEE

The presence of several mistakes in the IIT-JEE exam this year kicked up a storm that refuses to abate. It stands to affect the future careers of scores of serious applicants.
Given the enormity of the issue, there is a need for objective analysis by an interested outsider. As an alumnus, I empathise with the student angst. As faculty member, I understand the difficulties involved in ensuring a fair outcome when the exam had mistakes. I do not belong within the IIT system, I have no axe to grind. Let us start with one of the key mistakes in the IIT-JEE paper, which has led to a verbal match between Prof. Rajiv Kumar of IIT Kharagpur and Prof. T.S. Natarajan of IIT Madras. The questions in the IIT-JEE paper are multiple choice. Specifically, in section II of the paper, each question could have one or more correct answers. Therefore, this section allowed for partial marks. The instructions in the IIT-JEE paper for section II read as follows: “For each question in section II, you will be awarded three marks if you have darkened only the bubble corresponding to the correct answer and zero mark if no bubbles are darkened. Partial marks will be awarded for partially correct answers. No negative marks will be awarded in this section.” Since negative marks will not be awarded in this section, Prof. Rajiv Kumar accuses that students can score 93 points without having studied for the exam by blackening all four bubbles in this section. In my opinion, IIT Madras — the organisers of this year’s IIT-JEE — need not concern themselves with this issue.

To understand why, let me draw a parallel. Imagine a cricketer playing in a Ranji Trophy match hoping to get selected into the Indian Test cricket team. He finds a wide gap between third slip and gully. However, he will realise that the opposing team has left that gap intentionally to get him out caught at slips or gully. He will not attempt to qualify into the team by repeatedly gliding the ball between third slip and gully and hoping to notch up a high score in the process. Instead, the serious cricketer would attempt to play as straight as possible and score his runs in front of the wicket, particularly when the wicket is behaving viciously. In a similar vein, serious students at the IIT-JEE are unlikely to adopt Prof. Rajiv Kumar’s method. However, to clarify Prof. Kumar’s accusation, Prof. Natarajan makes the egregious mistake of laying new rules after the exam is over. Consider the following parallel to understand such ex-post rule setting. Imagine umpire Steve Bucknor telling Sachin Tendulkar after he has scored a fabulous hundred on a dicey wicket that the one four that Sachin edged between slip and gully will not count. After Sachin’s innings is over, Umpire Bucknor comes with the rule that only runs scored in front of the wicket will count. On a dicey wicket, there may be some unintentional edges even when Sachin is making his best effort to score by playing as straight as possible. Therefore, it is a mistake to deny Sachin a well-deserved century by changing rules after his innings is finished. Prof. Natarajan retorts to Prof. Kumar’s accusation by saying: “If all four options are shaded for a question for which there are less than four correct answers then the candidate gets zero. To avail partial mark the number of choices shaded should not exceed the number of correct choices and must include at least one of the correct choices.” To illustrate the flaw in this rule, consider question 41 in Section II in the Mathematics section in Paper 1 (code 5). Here, the correct answers are B and C options. 

Going by the rules specified in the exam, even a student who is not sure whether A is correct may answer A, B and C since the student would have verified that B and C are certainly correct and A appears (!) to be correct. Such a student should get partial credit for answering B and C correctly. However, using the rule laid down by Prof. Natarajan, this student will get 0 since his number of choices (3) exceed the number of correct choices (2). In contrast, a student who answered only B (which is more obvious than C) and did not take the effort to verify whether C was correct would get 1.5 points. Worse, a student who marks B and D will get 1.5 points while the diligent student who marked A, B and C will get 0. Thus, the rule will be unfair to several deserving students. The other key mistake related to the interchanging of Physics and Mathematics in the response sheet, which led to considerable confusion and loss of time for students. Furthermore, there was wide disparity in the communication about the mistake at the different examination centres. A question was missing and there were printing mistakes in the instruction sheet of the Hindi version. Finally, the question paper had logarithmic tables appended but there were no instructions mentioned about the same. Given the chaos due to the other mistakes and the time lost from having to re-mark the response sheets, it is very likely that many students did not notice it. There was one question each in the Chemistry & Physics sections that needed the use of log tables. For many borderline students, these two questions could make all the difference. Given all these mistakes, an expert committee must be constituted with a tight deadline to comprehensively examine the effect of these mistakes on candidate scores. Stratified sampling and statistical techniques can be used to search for systematic patterns in performance emerging from these mistakes. If no such patterns exist, then ex-post allocating marks to compensate for the mistakes would be an arduous task. In this case, a re-examination is a necessity. If re-examination is not necessary, then the ex-post fix to be applied must be transparently communicated together with an opportunity for re-evaluation requests from borderline students. While these steps will be time-consuming, justice delayed will be better than justice denied.

The writer is assistant professor of finance, Goizueta Business School, Emory University.