Cinneadh
Mr Y and Department of Education
Ó Oifig an Choimisinéara Faisnéise
Cásuimhir: OIC-107664-C7T5D8
Foilsithe
Teanga: Níl leagan Gaeilge den mhír seo ar fáil.
Ó Oifig an Choimisinéara Faisnéise
Cásuimhir: OIC-107664-C7T5D8
Foilsithe
Teanga: Níl leagan Gaeilge den mhír seo ar fáil.
Whether the Department was justified, under section 15(1)(a) of the FOI Act, in refusing the applicant’s request for the exact Junior Certificate results used to calculate his Leaving Certificate 2020 calculated grades on the ground that no such record exists
20 December 2021
The applicant in this case was a Leaving Certificate candidate for the examination year 2020. On 12 March 2021, he sought his “personal data used in the calculation of all [his] calculated grades for Leaving Certificate 2020”. Specifically, he sought details of the Junior Certificate subject and grade “that was actually used” to determine his calculated grades for Leaving Certificate 2020. On 15 April 2021, the Department issued its decision in which it said it had decided to grant the request. It provided details of the applicant’s Junior Certificate results for six subjects.
On 16 April 2021, the applicant sought an internal review of that decision. He said he suspected the data provided to him was attained by looking at his Junior Cert results directly, rather than through the examination of the data used during the calculated grades process. He explained that he was seeking the data that was actually used as opposed to the data that should have been used.
On 11 May 2021, the Department issued its internal review decision in which it refused the request under section 15(1)(a), i.e. that the record sought does not exist. It said it was not possible to provide the details regarding the Junior Cycle data which were used as part of the Calculated Grades process as the records contained within the Calculated Grades model did not include a static record of each individual student’s results. Rather, it said, all Junior Cycle results for all students were fed into the model and the relevant results were used in the course of running the statistical model. On 17 May 2021, the applicant sought a review by the Commissioner of the Department’s decision.
During the course of the review, the Investigating Officer sought submissions from the Department on searches undertaken to locate the relevant records and a summary of these submissions were provided to the applicant with an invitation for him to provide a submission of his own. The applicant provided a submission. He indicated that the two-week period inviting him to make submissions was insufficient. Consequently, the Investigating Officer granted the applicant an extension but no further substantive comments were received.
I have now completed my review in accordance with section 22(2) of the FOI Act. In carrying out my review, I have had regard to the submissions made by the Department and the applicant’s comments in his application for review and in further communications with this Office. I have decided to conclude this review by way of a formal, binding decision.
This review is concerned with whether the Department was justified, under section 15(1)(a) of the Act, in refusing access to records of the applicant’s personal data used in calculating his Leaving Certificate 2020 calculated grades on the grounds that no records exist.
Section 15(1)(a) of the Act for the refusal of a request where the records sought do not exist or cannot be found after all reasonable steps to ascertain their whereabouts have been taken. The Commissioner's role in a case such as this is to review the decision of the FOI body and to decide whether that decision was justified. This means that I must have regard to the evidence available to the decision maker and the reasoning used by the decision maker in arriving at his/her decision and also must assess the adequacy of the searches conducted by the FOI body in looking for relevant records. The evidence in “search” cases generally consists of the steps actually taken to search for the records along with miscellaneous and other information about the record management practices of the FOI body, insofar as those practices relate to the records in question.
As I have outlined above, the Department provided this Office with details of searches it undertook in an effort to locate relevant records and of its reasons for concluding that no relevant records exist. While I do not propose to repeat the details in full here, I confirm that I have had regard to them for the purpose of this review. The calculated grades process is somewhat complex and I think it useful to first provide general background to the process before addressing the specifics of the applicant’s request.
The Department explained that as part of the calculation process, schools were asked to provide an estimated percentage mark for students in each subject. These estimated percentage marks were then returned to the Department’s Calculated Grades Executive Office (CGEO), which was set up for the purpose of administering this process, where they underwent a process of national standardisation.
It said that the process of national standardisation, which forms a key part of the Calculated Grades process, was applied to the information provided by schools in order to ensure comparability between the standards applied by individual schools and the national standard. In order to be fair to the class of 2020, the teacher judgements made at the level of the school had to be adjusted so that a common national standard was applied. It was inherent to the system of calculated grades that school estimates would be subject to adjustment through this standardisations process. These adjustments resulted in the school estimates staying the same or being revised upwards or downwards. The standardisation process operated on the premise that the school estimates should only be adjusted through the standardisation process where there was credible statistical evidence to justify changing them.
The use of Junior Cycle data in the standardisation process in the model is set out in detail in the Report of the National Standardisation Group. Other than in certain cases of subjects studied outside of a school setting, individual Junior Cycle results were not used to directly determine that individual’s Calculated Grades. Rather, the Calculated Grades process took account of the overall Junior Cycle performance of the Leaving Certificate class of 2020 in each school and used this data to help in predicting the likely distribution of Leaving Certificate performance of that group using related information about the relationship between performance at Junior Cycle and Leaving Certificate based on national data over time for that subject at that level.
In the system of calculated grades, the standardisation process applied uniformly across all subject and levels and school types. While schools carried out internal in-school alignment processes in producing estimated percentage marks, there was no guarantee of inter-school equity and common standards being applied between schools. This was the purpose of national standardisation. The degree to which mark changes occurred related to the degree of over or underestimation in the school estimates for each subject and each level. This means that some students experienced mark changes from the school estimates but no changes to the grades based on the school estimates; while others will have experienced marks changes from the teacher estimates leading to grade changes in one or more of their subjects. Following standardisation, the estimated percentage mark was converted to a calculated mark and subsequently, a calculated grade which was provided to students on 7 September 2020.
The Department outlined that on 30 September, the Minister for Education made a statement advising that two errors had been identified in the Calculated Grades process. These meant that incorrect grades had been issued to some students when they received their results on 7 September. The first error was identified by an external contractor as having taken place when processing students’ data. The error, in one out of 50,000 lines of code, was in relation to the processing of Junior Cycle data. The system was meant to draw on the core Junior Cycle subjects of English, Irish and Maths, and combine them with the students’ 2 best non-core subjects for all of the students in each class in each school. The coding error instead combined the core subjects with the students’ 2 weakest non-core subjects. The error meant that, in some subjects, some students received Calculated Grades that were lower than they should have been, while some students received grades that were higher than they should have been. However, because students’ Junior Cycle performance was taken into account at an aggregate level under the system of Calculated Grades, the error did not result in individual students’ grades being affected as a result of the use of their incorrect individual non-core subject data.
In the course of a review which the Department then undertook, Departmental staff found a further error in the section of the code dealing with Junior Cycle results. The Junior Cycle subject Civil, Social and Political Education (CPSE) was meant to be disregarded as part of the model but was not. This second error, however, had a negligible impact on results. The Minister provided a further update on this matter on 03 October 2020, announcing that improved Calculated Grades would issue to impacted students that day. This announcement was made following a review of essential aspects of the coding by Educational Testing Services (ETS), a non-profit organisation based in the United States. ETS completed their review and provided it to the Department on 3 October. The review identified one further error, relating to how the code handled cases where students did not sit all three core subjects (Irish, English and Maths) at Junior Cycle level. As a result of rectification of these errors, a total of 6,100 students received higher grades.
In his application for review, the applicant argued that his personal Junior Certificate data was inputted into the calculated grades model and used directly to modify his teachers estimated grades. He suggested that the use of these grades resulted in errors for some student and led to grade amendments. He argued that the Department couldn’t reasonably say no records exist as it was the data he requested which was amended for 6,100 students when errors were discovered. The applicant argued that “…it was not just ‘aggregate Junior Cycle results that was used in the model. An individual’s Junior Cycle result was also used to directly calculate my grades. As is publicly known, it is this individual’s result that was correct for 6,100 students, once errors were discovered”.
In response the Department submitted that the applicant’s assertions are incorrect and represent a misunderstanding of how the Calculated Grades model operated. The Department said that the framing of the applicant’s request and subsequent correspondence imply a direct link between an individual’s Junior Cycle results and their Calculated Grades results. It said that Junior Cycle data was used at an aggregate level for class groups. It said that the applicant’s assertions appear to be based on the assumption that the errors identified were a result of incorrect data used for those 6,100 students only. It said the errors in how Junior Cycle data was treated in the Calculated Grades model applied to all students. For the majority of students, however, there was no material impact. For example, a student who received a mark of 72% after standardisation but prior to rectification of the errors, and 74% after the rectification of the errors would see no material impact, as the grade remained the same. The 6,100 students were those where the change in percentage mark resulted in a change in grade being awarded.
It said that the applicant appears to assume that Junior Cycle data was added to the model in a manual, or semi-automated, manner and the basis of the request rests on whether correct data was entered to the model. The Department re-emphasized that an individual’s Junior Cycle results were not used to determine an individual’s Calculated grades.
It said that Junior Cycle results are held on a secure databased by the State Examinations Commission (SEC). Following the decision to postpone the Leaving Certificate examinations, this database was shared with the Calculated Grades Executive Office in the Department under a data sharing agreement. It said that students are identifiable by their examinations ID, date of birth, surname and forename. The roll number for the school may also be used to verify the school in which the candidate is registered. All Junior Cycle results for the entire Leaving Certificate class of 2020 were accessed by the Calculated Grades model. The model then extracted the relevant results from the database to determine the aggregate Junior Cycle data for class groups. This allowed the model to compile relevant results from students who may have changed schools between their Junior Cycle and Leaving Certificate.
The manner in which the model extracted the Junior Cycle data is where the errors occurred. The process of extracting the relevant Junior Cycle results is done on an automated basis and the model produced no records of what results were used at student level in its calculations.
Therefore, personalised, individualised records and calculations at individual student level, such as those that the applicant is seeking, do not exist and were not generated in the Calculated Grades process. The Department said it previously explained this to the applicant, and as is stated in published material, that the model drew on candidates’ Junior Cycle results for Irish, English, Mathematics, and their best two other subjects, excluding Civic, Social and Political Education (CSPE).
The Department said its Curriculum and Assessment Policy Unit liaised with the Calculated Grades Executive Office. The CGEO provided information regarding the model and whether records would have been generated at student level. It said that the CGEO carried out searches for records but considering the relatively narrow scope of the request it determined that no further searches were required.
It seems to me that the applicant’s true interest is in seeking assurance that his correct Junior Certificate grades were taken into account in the course of the determination of his calculated grades. It is important to note that this Office has no role in examining the administrative actions of public bodies in the performance of their functions. Our role is limited to determining whether the public body’s decision on the applicant’s FOI request was justified.
The applicant’s request was for details of the Junior Certificate subject and grade that was actually used to determine his calculated grades for Leaving Certificate 2020. It is important to note that while the purpose of the Act is to enable members of the public to obtain access to information held by public bodies, the mechanism for doing so is by accessing records held by those bodies. In other words, a person wishing to obtain information from a public body must make a request for records that contain the information sought. Requests for information, as opposed to requests for records, are not valid requests under the Act, except to the extent that a request for information can reasonably be inferred to be a request for a record containing the information sought.
In this case, the question arising is whether the Department holds a record that contains details of the applicant’s Junior Certificate grades that were actually used to determine his calculated grades for Leaving Certificate 2020. The Department’s position is that the process of calculating Leaving Certificate grades did not use personalised, individualised records and calculations at individual student level such as those that the applicant is seeking. Rather, Junior Cycle data was used at an aggregate level for class groups. No evidence has been presented to this Office to suggest that this was not the case. As such, it seems to me that the Department was justified in deciding that the record sought does not exist. Moreover, in so far as the applicant’s Junior Certificate grades may have been included in the aggregate class group information, I note that the Department provided details of those grades to the applicant in its original decision on the request.
Having regard to the nature of the request and extent of the searches conducted and the Department’s explanation as to why no records exist, I find that it has undertaken all reasonable steps to locate relevant records. As such, I find that the Department was justified in refusing access to any records under section 15(1)(a) of the FOI Act on the ground that no records exist or can be found.
Having carried out a review under section 22(2) of the FOI Act, I hereby affirm the decision of the Department in this case. I find that the Department was justified, under section 15(1)(a) of the Act, in refusing access to records on the ground that no records exist or can be found.
Section 24 of the FOI Act sets out detailed provisions for an appeal to the High Court by a party to a review, or any other person affected by the decision. In summary, such an appeal, normally on a point of law, must be initiated not later than four weeks after notice of the decision was given to the person bringing the appeal.
Stephen Rafferty
Senior Investigator