Reflecting Back on the Online Evaluation : Examinations 2020-21
Online Exams, some of us were perplexed when we first heard about it, some were happy, and some were sad, but what we all felt together was being a rookie. How will the exams be conducted? How will they be graded? Will attendance play a significant role? What happens if I catch Covid? How will the lab exams be conducted? Will they be conducted at all? These were some of the most prevalent questions in the minds of the students. In this article, Team Monday Morning walks you through what happened and how the institute prepares for the future.
The students recently received their cumulative grades for the session 2020-2021. Team MM conducted a survey across all courses and departments to boil down the major tell of online examinations. The responses accounted for 52.3% for the first year, 28.5% for the second year,15.2% for the third year, and the remaining 4.1% for the fourth and fifth-year students.
The Entropic Turn of Events
The coronavirus pandemic brought unprecedented challenges in the face of education throughout institutes, and NIT Rourkela was no exception. As the uncertainty of the future started taking shape, the evaluation mode was put in a quandary. The semester initially began with firm determination from the administration to conduct the examination (end semester and mid-semester) offline. However, as the pandemic peaked, this decision lay hold only on the end semester examination. The situation showed no signs of normalization as the semester came to an end.
In contrast to other institutes, NIT Rourkela decided to postpone the odd end semester exams and continued with the even semester curriculum. Similarly, the mid-semester exams were scrapped for the even semester, and the students were to be evaluated on the end semester examination. This was the shift from offline to online mode in conduction the end semester examinations simultaneously for the first and second semesters.
The situation spanned out with a tad bit more difficult for the pre-final year and final-year students. Final year students were called back to the campus initially, but the pandemic breakout on the campus grounds led to a demand for answers and protests. To know more about the issue, click here. Along with the consecutive end semester exams, the pre-final year ones also had their ongoing internships.
The question of utmost salience is whether the continuous evaluation or the pro-rata will be more beneficial for the students in the upcoming semester in consideration of the ill-favoured turn of past events regarding the same.
Team MM reached out to a few authorities, and here is what they had to say.
MM: In these uncertain times, it might have been difficult for the professors to arrange all the documents concerning exams. Would it have been better to employ a continuous evaluation system rather than pro-rata?
Prof. Susmita Mishra (Ex-PIC Exam): We have discussed this matter in the last senate, and many senators have agreed to this. Initially, I had also agreed to continuous evaluation, but many professors didn’t agree. It is impossible to conduct exams every 15 days and evaluate their mark sheet because of the huge classrooms. So we agreed to have two exams that are mid sem and end sem. Some faculties were in favor of that, some weren’t, but as per the last senate meeting, it was decided to have two exams in a semester. But it is still being discussed, and I don't know about the students' opinions.
Because of this internet issue, many feel that two exams are better because if somehow one of the exams goes wrong because of the internet issue or so, then at least they can recover their marks in the following exams. But some professors are saying that since we are going to have a different set of modalities this time, a more standard set of modalities, conducting two sets of exams will be tiresome, and students can rather have one exam in a very detailed manner. But the Senate is in the opinion of having two exams.
MM: As a professor, what do you think will suffice with the students, continuous evaluation, or a pro-rata system?
We are always discussing this issue in the Senate meetings, the Deans, and Head of Department groups. Continuous Evaluation is always good, but the problem arises when it comes to a large class. So it is not possible to take 60-70 students, but in my case, where I’m taking 30-40 students, it is possible.Overall, it is a good option and has been adopted by IITs as well.
-Prof. Santanu K Behera (HOD ECE)
We are also in favor of continuous evaluation by conducting tests every month. But there are certain problems. For instance, if I schedule a test on a specific day, the students might complain about having another exam of a different subject. Thus to conduct examinations for all the courses of a particular semester, we will have to allocate an entire week. The semester lasts for approximately four months and the exams will take up to four weeks. Therefore, the Senate has not approved it, but the faculties can conduct tests and continuously evaluate. The Senate is of the opinion of conducting two exams per semester.
-Prof. Dipti Patra (HOD EE)
The Certain Uncertainty of Modalities
Amidst these seemingly unending problems, NIT Rourkela could successfully manage and execute the online conduction of two end sem exams. Professors employed different ways of evaluation for the theory and practical subjects determined by the need of the particular course. Subjective and Objective patterns were used commensurably.
Quizzes were conducted through Test Portal, MS Teams, Google Forms, Moodle, etc. Rough sheets were asked in some courses. Subjective questions were evaluated through pdf submission at Microsoft Teams or its equivalents. Viva Voice was also conducted in certain subjects through MS Teams calls, Google Meet, or phone calls. If any student somehow missed the test’s deadline due to some issue, they could mail their respective course teachers for consideration and further proceedings.
Though the modalities were well thought out, one can never trust what might happen regarding technical gadgets and exam forums. Usage of different modalities did have an impact in contributing to the examination difficulties. Here is what the Dean Student Welfare had to say regarding it:
MM: A lot of issues arose in the examination due to the usage of different modalities. Would it have been better if a standard set of modalities were followed? What are your views on this?
Prof Snehasish Chakraverty (Ex-Dean Student Welfare): We cannot generalize the modalities for every subject. Even in offline exams, the modalities are different. I cannot speak for other professors, but you can check how I have conducted my exam. The trick for me was to have questions of both subjective and objective types. Thirty questions were MCQ type, 20 were subjective type, and marks were distributed evenly from P to Excellent(Ex) grades. Until a versatile software like CAT is used, we cannot expect to have a generalized exam, but even then, there is no guarantee that it would be hundred percent foolproof.
There are difficulties, there are problems in this online exam, there are limitations, and everyone knows that. But, no student came up with a solution; complaining is very easy, but what is the solution?
MM: Students weren't provided with a totally corrected answer sheet to compare their answers with; this led to a lot of confusion in the grade they received. Do you think this decision was apt?
Only end sem examinations were conducted, and almost all the faculty members showed the marks to the students. For the ID department, they had done complete justification to the students regarding the marks. It should be shown to the students.
-Prof. Deepak B.B.V.L ( Department of Industrial Design )
I have recently checked the mark change portal, and there, the faculty members commented that they have verified correctly and reverified after receiving a request for mark change. Very seriously, they have checked so that no student gets anything lesser than what they are supposed to get. The faculty have rechecked thoroughly to avoid any confusion any errors in calculation. In some cases, they even allotted some extra marks where they seemed necessary. With enormous strengths of courses like analogue electronics or digital electronics, it is very difficult, at least online, to show all these things.
-Prof. Santanu K Behera (HOD ECE)
The Dynamics of Grade Assessment
Grade assessment was a significant space of critical change. Doling out grades to students is an acknowledged and set up training among teachers as an essential aspect of teaching. The numerous uncertainties regarding the modalities of conducting online exams loomed large over the students and professors alike. While the fairness of exams is an utmost priority for all the stakeholders (administration, professors, and students), the genuine constraints of unstable and uneven availability of network connection cannot be disregarded.
TA Marks: The Dark Matter of Online Exams
Teacher’s Assessment carries 20% of the total weightage in grade evaluation. The online assessment had its fair deal of difficulties in giving out grades for TA. Different professors used different methods to calculate the final TA marks that catered to the need of their respective courses. The survey concluded that the majority of the professors graded TA based on Class Tests(80.3%) followed by Assignments(75.5%), Class Performance(23.3%), and Attendance(24.9%). In comparison, a significant number (18.8%)of students stated that no specific evaluation was done.
The online classrooms did have a major setback in student-teacher interaction and thus played a vital role in the showcased discontent between teachers and students.
The following are some opinions of students in concern with the TA evaluation-
Marks weren't transparent, and also results of class tests weren't declared.
In a few subjects, the level of questions was quite high compared to regular semester examinations.Tons of similar kinds of tests, assignments, and presentations were taken in the name of the TA, but in the end, the marking was quite sporadic and arbitrary.
Further, when the problems regarding TA marks were taken to the authorities, here is what they had to say respectively:
MM: Why are the TA marks always undisclosed to students? Don't you think this generates unnecessary confusion?
Prof. Saroj Kumar Patel(Ex-Dean Academics): The modality of TA evaluation may vary from teacher to teacher, but it should be disclosed at the beginning of the semester. Any lapse in this regard should be reported to the concerned teacher. If not done, report to HOD; if still not done, report to Dean.
Students should cooperate in giving feedback on such lapses so that necessary corrective action can be taken. So students should be proactive in reporting such lapses if any.
Prof. Santanu K Behera(HOD ECE): The TA marks should be disclosed, but due to the untimely examinations one after the other, there is not sufficient time to work on this. Due to the lack of gap between the autumn and spring semester, it is a bit difficult. Otherwise, we try our best to show the marks to the students even when it concerns the mid-semester and end-semester examinations, even before the publication of the final grade card.
Lab Evaluation: An Experimental Collapse?
The laboratory examinations account for 18 - 20% of the total grade percentage. Due to the absence of physical on-ground student presence, labs' conduction and grade allotment was a rather tricky process. The major drawback that arose is the lack of lab classes and last-minute declaration for the evaluation process. As per the survey, grade evaluation for the labs was done via Quiz and Assignments (21.6%), Viva(7.8%), and in the majority of the classes, through both of the above (60.8%). Some courses like Engineering Drawing and Basic Programming resorted to continuous evaluation in the first-year curriculum through weekly assignments spanning 7 to 8 weeks and a final test.
In most of the subjects, the online lab classes were neglected entirely, even in terms of theory. The students were provided end-moment study material and videos based on which the examination and vivas were conducted. This stirred noticeable mayhem among the students.
Here are few problems as stated by the students-
For the lab exams, we were given very little time to answer questions.
No specific time was given for preparation. Just told us three days ago, and our end semester was about to happen
In some lab courses, not a single class was held, but still, they conducted viva, which was difficult. In some lab courses, not a single class was held, but still, they conducted viva, which was difficult.
On questioning the Deans regarding this, here is what the Ex-Dean Academics and Ex-PIC Exams had to say-
MM: A few laboratory courses didn’t have any classes throughout the semester. A few days before the examination, the entire material was shared with students, and they were asked to prepare their own and give the exams. What are your views on this?
Prof. S K Patel: There was never a circular last year for not taking laboratory classes. Every class, theory or practical, was taken in online mode. No classes were suspended. Many teachers have done video displays of how to conduct experiments. Whatever is possible in online mode for laboratory classes has been done by many teachers. A few lapses may have been there. But no students have earlier reported such lapses, although I had shared my mobile number with all of the students. Otherwise, necessary corrective actions could have been taken much earlier.
For practicals like Chemistry, workshop practice, etc., students will be doing experiments in their own hands when they are back on campus. There will be one more round of evaluation to finalize the grade by giving proper weightages to previous and new grades. Otherwise, L-T-P 0-0-3 will be written as 0-2-1 and 0-0-2 as 0-1-1, keeping credit unchanged to avoid any problem of derecognition of degree in future due to online mode of doing practical.
Prof. Susmita Mishra: I remember that it was planned to allow students to return to the institute to get hands-on experience in practical courses. We had decided to call the final year followed by the pre-final year, and in the summers, the 1st and 2nd-year students also. But everything went wrong. We could not do any of these. Only the final-year students could get some experience in the lab. As a result, those faculties with an experimental laboratory had no other option and had to prepare at the last moment and share it with the students. But I know I had taken a lab which was also an experimental lab. I demonstrated the experiments online. Maybe few departments and few particularly experimental laboratories could not be conducted because of the last moment decision.
I don't think that will be repeated this time because we have already instructed everybody to go for online laboratories.
End Semester Exams: A Potential Catastrophe?
The real challenge was to conduct the end-semester examination of over 200 courses for such a vast student strength, each with different access to resources like network connection, electronic gadgets, and suitable study space. 96.7% of students who participated in our survey had their online exams through MS Teams, Quizzes, and Assignments. 63.9% of students had their exam through the Test Portal, and 62.2% had their exam on Google Forms.
From the survey results, it is pretty evident that many students attempted exams on mixed platforms. People have faced problems while giving the exams with insufficiency of time being at the pinnacle. The next most reasoned is the most obvious one, unstable internet connection. A whopping 73.6% of people who participated in the survey tackled the first issue and 59.9% the second. Following these are other issues like difficulty in scanning and uploading documents (55.1%), accessibility of the question paper (17.6%), proctoring methods (15.7%), etc.
With a never-ending pile of requests and colossal amounts of difficulties the students faced, the professors adopted many methods to try and resolve those. 56.8% of students who attended the survey had increased time intervals to finish answering their exam. For 27.8% of the students, their professors accepted alternate methods of submission. 24.2% of the students were allowed a retest, and a tragic 23% of students’ requests were unattended by their respective professors.
The following is a response of a student,
Actually, if the Endsem mark was not much good, then it had a direct effect on the midsem mark, as it's given on the basis of end sem only. If this time also, an online session will be conducted, then there should be separate end sem and midsem. So that they don't affect each other.
Problems were not a paucity in these online exams, albeit students never refrained from hope. Professors were expected to respond and rectify the issues but the survey results, as shown below, indicate otherwise.
One of the issues for the pre-final year students is the overlapping of their internship with their exams. When asked about the same, the Ex-Dean Academics and Ex-PIC Exams said,
MM: Pre-final years were called back to campus because there wasn't supposed to be any hindrance in their internship period. Still, online exams were conducted for them with 1st, 2nd yrs rather than final year students, which could have benefited them. What led to this decision?
Prof. S K Patel: Covid was highly unpredictable. We had no previous experience with this. There was no prediction of the 2nd wave. When colleges started opening everywhere after the 1st wave, we carefully and systematically planned to call students back to campus batch-wise. But all plans got disturbed. We were pretty flexible in taking decisions depending on the situation throughout the year.
Pre-final year students' examinations were required to be completed to avoid their problems in facing campus placements starting from July/Aug.
Prof. Susmita Mishra: When the final year students’ exams were conducted, it was so sudden that we were not prepared to conduct the exams for so many students. Therefore, the Senate decided to prioritize the final year students because they were supposed to graduate timely to join the different companies they had committed to. So exams were held for them. Since two exams were yet to be completed, we had to take the step.
We knew that the third years are having their internships, so we held the exams in such a way that their internship period is least hampered. We also conducted the exams on Saturdays in the first hour, 7 am to 9 am.
MM: The examinations were conducted during the peak of the second wave of Covid. Were any relaxations given to the students who had difficulty in attending the examinations?
It is not known to others, but especially in our department and even in other departments, we were lenient because the conduction of online exams has its own set of problems due to the network and other things. It becomes a little difficult for the students to follow instructions; therefore, we tried to be very lenient while evaluating.
-Prof Santanu K Behera (HOD ECE)
Not many students faced problems during the examination. Only 1-2 students from the department faced such issues. We're recommending them to give supplementary exams. Especially from our department, our students are completely happy.
-Prof. Deepak B.B.V.L(Dept. of Industrial Design)
A Coherent Superposition
We have so far seen how the exams were conducted, what modalities the lecturers employed, and what means they used to hold the examinations. Furthermore, we delved into how the laboratory exams were held, how the TA marks were allotted (which has always been an age-old mystery), and how all the exams were graded. Let’s burst the bubble and look at what our confreres from other institutions experienced in the name of online examinations. The following is information gathered from student sources of the respective institute.
NIT Tiruchirappalli (Trichy)
- The zenith of the NITs (according to NIRF rankings) had a distinctive way to conduct the exams. Professors of NIT Trichy adopted a continuous evaluation that was not inclusive of the semester exams. The semester exams were conducted individually.
- The class representatives(CRs) had the authority to change the time and date of lab exams. They could change the mode of the theory exams and decide the platform in which the exams would be conducted.
- Since many students faced network issues, they weren't asked to turn on their cameras, and the lab exam was an open-book assessment.
- The institute chose a relative grading system to grade its students.
- The theory exams for IIT Kanpur students were conducted on Mookit, an examination platform developed in-house, and Google Forms with virtual proctoring happening on Zoom.
- For the laboratory exams, the professors used Codetantra and Prutor, another in-house platform. Few courses were graded on a subjective exam, a few more on an objective exam, and some on both.
- Every professor had the choice of either grading relatively or absolutely for their course.
- The exams were primarily held on moodle, but they were conducted on Microsoft Teams for lesser accredited courses.
- Most of the courses were graded relatively. The evaluation method was continuous for all courses.
- For grading laboratory exams, the professors required the students to solve weekly assignments that had to be submitted within the class hours. For some theory subjects, some frequent quizzes and assignments were used for evaluation.
- In the odd semester of the year, the institute shifted from continuous evaluation to pro-rata, making the students’ day frenetic. This frenzy among students caused the administration to change its decision. Successively the assessment was done both based on all of the tests and quizzes conducted and also pro-rata.
- Up there in Delhi, exams mainly were conducted on moodle. Professors used Google Meet or Microsoft Teams for proctoring.
- Courses evaluated over objective exams were held on moodle, and Gradescope was used to conduct subjective exams.
- To further prevent the usage of unfair means, the institute has implemented plagiarism checkers.
Kurzgesagt - In a Nutshell…
Ultimately, the subsequent pie chart shows how satisfied or unsatisfied the students who attended the survey were with their CGPA.
Profuse meetings were held, and numerous arguments and suggestions were heard, and only then did the administration take decisions. Every moon has its spots, and NIT Rourkela isn’t an exception. We are all novices to the online exams, and so are the professors, but co-operation and communication between both made the exams relatively seamless.
The era of the first online examinations unfolded as a bittersweet affair for the students and the administration. We learn from our own mistakes and hope that the upcoming semester is held and evaluated seamlessly and none of the mistakes is repeated. However, all we can do is sincerely hope this pandemic seizes and the world goes back to normal.