Methodology

Comprehensive Methodology

This Comprehensive Methodology
provides a structured tool for
assessing the quality of implementation
of a Right to Information Law.

Central Measures

This section of the methodology looks at how the RTI law is implemented at the central level. Central institutions serve two main functions: processing appeals and undertaking promotional measures.

Learn More

Institutional Measures

This section looks at what individual public authorities have done to implement the RTI law. In any particular jurisdiction, there will always be a great number, normally hundreds and sometimes thousands, of different public authorities of different types and levels.

Learn More

Proactive Disclosure

This section looks at proactive disclosure or the release of information by public authorities without an RTI request. This is often done through websites and annual reports.

Learn More

Reactive Disclosure

Whereas proactive disclosure looks at whether public authorities make information available even in the absence of a request for it, reactive disclosure is about how public authorities respond to RTI requests.

Learn More

Methodology Roadmap

041-pros-and-cons-300x300.png

1. Assess Feasibility of Doing Assessment

036-teamwork-300x300.png

2. Put in Place the Governing Structure

001-idea-300x300.png

3. Develop an Implementation Plan

029-review-300x300.png

4. Apply the Assessment Tools to Collect Information

010-research-300x300.png

5. Analyse Information and Score Results

011-solution-300x300.png

6. Draft Report and Recommendations

023-ui-design-300x300.png

7. Publish Report

031-startup-300x300.png

8. Conduct Follow-up Advocacy and Activities

Applying the Methodology
The Comprehensive Methodology evaluates the implementation and performance of a jurisdiction’s RTI laws by its central authority and public authorities. In any given country, there will be 100s or even 1000s of authorities, and it will not be possible to assess all of them. The Methodology, therefore, calls for a minimum of ten public authorities to be selected for assessment. The central and public authorities are evaluated at the institutional level. The public authorities are further assessed based on their performance of proactive and reactive disclosure requirements. According to a three-part scale, both the jurisdiction and the individual institutions are assigned grades: poor, mediocre, excellent.
The following constitutes an overview of the methodology. The complete methodology is available for download here.
Central Measures

This section of the methodology looks at how the RTI law is implemented at the central level. Central institutions serve two main functions: processing appeals and undertaking promotional measures. In almost every country where an independent oversight body (often called an information commission) exists, it bears primary responsibility for the first function. In many countries, the information commissions both process appeals and are the main official body that is responsible for promotional measures. However, in others, the oversight body is supported by a nodal body—either internal or external to the government—that undertakes the promotional functions, including assisting individual public authorities. For the purposes of this Methodology (available for download in full here), the information commission and nodal body (if any) should be evaluated as one.

Assessment

In this section, assessors evaluate whether the central authority has successfully implemented the RTI law at an institutional level, for example, by establishing an independent oversight body and whether the institution fulfills its RTI obligations in practice.  In most cases, responsibility for the former lies with the government or parliament. The latter focuses on the effectiveness of the oversight body in discharging its responsibilities.

 
Main Substantive Issues
Assessment Tools
Grading

Structural

Performance
  • Have independent, effective governing members of the oversight body (commissioners) been appointed?
  • Has the body been provided with reasonably sufficient funding to enable it to carry out its tasks?
  • Does the body recruit its own expert staff (as opposed to staff being allocated from the civil service whose primary loyalties tend to lie with the civil service)?
  • Does it have a full complement of staff (i.e. compared to its organigram or official documents)?
  • Are they on long-term employment contracts? Are they on long-term employment contracts? Is the staff diverse, including in terms of gender?
  • Does the oversight body make an effort to be geographically accessible to complainants?
  • Does the oversight body process appeals appropriately and in accordance with the law?
  • Does the oversight body take active (suo moto) steps to ensure that public authorities are implementing the law properly
  • Does the oversight body discharge its regulatory functions properly? This will depend on what the law requires/authorises it to do in terms of regulation.
  • Does the oversight body take steps to raise public awareness about the RTI Law?
  • Has the oversight body supported the provision of training to IOs?
  • Does the oversight body publish an annual report on overall implementation?
  • Does the oversight body actively use whatever powers it has under the law to impose appropriate sanctions on individuals/entities who obstruct access?
  • Has the oversight body commented on draft laws that affect RTI?
  • Does the oversight body provide advice to public authorities which ask for it? What about members of the public?
  • Has the oversight body taken any other steps to improve implementation?

The following assessment tools should be deployed:

  • Key informant interviews
  • A self-assessment by the public authority
  • A desk-based literature review
  • Another desk-based review of actual decisions on complaints/appeals

The central authority’s institutional measures are graded through five binary (0, 1) questions and eight ternary (0, 0.5, 1) questions that are then averaged. A data entry spreadsheet (available for download here) can be used to facilitate the grading process.

The five binary (yes-no) questions are:

  1. Has funding been allocated (i.e. to the body)?
  2. Does the body recruit its own staff (as opposed to this being done by the government, for example)?
  3. Are the body’s appeals decisions available online?
  4. Has the body produced and published an annual report for both of the last two years?
  5. Has the body published a guide for requesters?

The eight more qualitative questions are:

  1. Have the members been appointed?
  2. Are the members of the body independent and effective?
  3. Is the funding provided to the body reasonably sufficient for it to discharge its functions?
  4. Does the body decide appeals in a timely fashion?
  5. Are the due process rights of parties respected during appeals?
  6. Has the body made reasonable efforts to raise public awareness?
  7. Have effective measures been taken to provide training to officials?
  8. Has the body made a reasonable effort to comment on draft laws which affect the right to information?

Here, assessors are asked to determine whether the system performs Strongly (1 point); Partially (0.5 points) or Weakly (0 points).

The overall point score is calculated by taking the 13 individual scores and averaging them. This score should then be converted to a final colour grade using the following table:

Red Yellow Green
0-0.33 0.34-0.66 0.67-1.0

Public Authorities

This section looks at what individual public authorities have done to implement the RTI law.  In any particular jurisdiction, there will always be a great number, normally hundreds and sometimes thousands, of different public authorities of different types and levels. Acknowledging that it is unrealistic for assessors to review every single authority, the Methodology requires the Project Lead to select a subset of authorities representing the whole. In this subset, the Project Lead should only include public authorities that receive a volume of RTI requests that makes proactive disclosure—which this methodology measures—reasonable. This subset of authorities will be reviewed across all three assessment areas: Institutional Measures, Proactive Disclosure and Reactive Disclosure.

Institutional Measures

In this section, assessors evaluate whether the public authority has successfully implemented the RTI law at an institutional level, for example, by hiring a public information officer, and whether the institution fulfills its RTI obligations in practice.

 
Main Substantive Issues
Assessment Tools
Grading

Structural

Performance
  • Has the authority appointed an IO?
  • Has the IO been provided with RTI training? If so, how long was the training and who provided it? Was it just a one-off or has upgrade training been provided?
  • Have other staff been formally instructed to cooperate with the IO in discharging his or her functions, in particular in relation to the processing of RTI requests?
  • Does the authority have an overall implementation plan or standard operating procedure (SOP) for RTI? If so, is the plan or SOP reasonably detailed and does it seem realistic?
  • Has the authority developed/issued internal guidelines for receiving and responding to RTI requests?
  • Has the authority appointed someone to receive and process internal complaints?

 
  • Are there political pressures that make it difficult for the IO to do his or her job? Is he or she treated as a sort of “spy in the office”? Are there institutional incentives or just disincentives (most of this is quite subjective but it can be assessed in various ways, such as by asking the IO and civil society representatives)?
  • Is it easy to lodge RTI requests with the authority?
  • Does the authority prepare and publish annual reports, including statistics on RTI requests? If so, when was the last report published? How detailed is it (for example, how many types of information does it provide about RTI requests)?
  • Has the authority undertaken any public awareness-raising efforts? If so, what?
  • Has the authority put in place any systems or standards to improve its records management? Are these monitored, applied or enforced in any way?

The following assessment tools should be deployed:

Each public authority’s institutional measures are graded on ten binary (0, 1) questions and six ternary (0, 0.5, 1) questions that are then averaged. A data entry spreadsheet (available for download here) can assist with this calculation.

The ten objective evaluations are:

  1. Has an IO been appointed?
  2. Has the IO formally been given terms of reference or a job description?
  3. Has the IO been provided with training?
  4. Has an overall implementation plan or set of standard operating procedures (SOPs) been adopted?
  5. Has a set of guidelines for how to process RTI requests been adopted?
  6. Is it possible to lodge requests electronically? Is it easy to obtain an RTI request form? Is it easy to find the contact details of the IO? (YES is given for two or more positive answers, NO for one or less)
  7. Has a person who is different from the IO been appointed to deal with internal complaints?
  8. Did the public authority publish an annual report for the last two years?
  9. Has the public authority conducted any public awareness-raising activities over the last year?
  10. Has the public authority put in place any system or taken any action to improve its records management?

For each of these evaluations, one point is given for a YES and zero points are given for a NO. In some cases, these questions do not generate simply YES-NO answers but the assessor should nevertheless allocate binary scores here.

The six qualitative evaluations are:

  1. Does the IO have appropriate qualifications for the job and has he or she been allocated time to do the job?
  2. There is no political pressure on the IO that makes it difficult for him or her to do the job properly.
  3. How strong is the overall implementation plan or SOP?
  4. How strong is the annual report?
  5. How extensive are the awareness-raising activities?
  6. How effective are the measures taken to improve records management

For each of these evaluations, an evaluation of STRONGLY, PARTIALLY or WEAKLY is awarded, depending on how well the assessor believes the public authority has done.One point is awarded for STRONGLY, one-half point for PARTIALLY, and zero points for WEAKLY.

The final point score for the jurisdiction as a whole should be calculated by averaging the results of all public authorities.

The 16 point scores for each public authority are then averaged (added up and divided by 16) to get a final point score for each authority. These averages by public authority are then averaged again to get the final point score for the jurisdiction. The final point scores for each public authority and for the jurisdiction should be converted to a final colour grade using the following table:

Red

 Yellow Green
0-0.33 0.34-0.66

0.67-1.0

Proactive Disclosure

This section looks at proactive disclosure or the release of information by public authorities without an RTI request. This is often done through websites and annual reports.

Formally, public authorities’ proactive publication obligations are limited to what the RTI law requires them to do. In other words, performance should be assessed against the list of proactive publication obligations set out in the law. The types of documents that are required to be disclosed can be listed, and the assessment can be conducted against that list. Where the law is vague, the assessors should make and report on any required interpretations.

RTI laws are, however, slow to change. Therefore, it is recommended that the assessors evaluate the public authorities against a set of better practice standards. The final report should delineate between the requirements prescribed by law and those by generally accepted standards.

 
Main Substantive Issues
Assessment Tools
Grading

Structural

Performance
  • Is the public authority’s website WCAG 2.1 compliant?
  • Is it reasonably easy to find specific information from among all of the information that is being published online?
  • Is information, where relevant, both collected and then disseminated in a gender-disaggregated format?
  • Does the public authority use social media or other ways to draw the attention of the public to its proactive publications (and to provide key information directly to the public)?
  • Does the public authority take advantage of its public service points (i.e. offices to which the public has direct access) both to engage directly in proactive publication and to highlight its online proactive publication efforts?
  • Does the public authority make an effort to create understandable versions of at least the most important documents (such as its budget)?
  • What efforts does the public authority make to disseminate information other than simply via its website?
  • Has the public authority taken steps to reach women, rural communities, persons with disabilities or other groups who may face barriers to accessing information, such as by making information available in different languages and in ways that are accessible in practice to women and rural residents?

By definition, information made available on a proactive basis should be relatively easy to access either in person via the public authority’s office or website. The following assessment tools should be deployed:

Proactive disclosure is graded on two scoring criteria. The first—a compliance score—involves measuring the public authority’s compliance with each legally prescribed proactive disclosure requirement against a quinary scale (0, 0.25, 0.50, 0.75, 1.0). In other words, the assessment of proactive disclosure involves making a list of what should be disclosed proactively (according to the law) and then seeing if it is in fact available. For each item on the list, the public authority should be given one of the following evaluations: Full, Full to Partial, Partial, Partial to None or None.

It is not possible to provide precise instructions as to how to allocate evaluations here, given the wide range of different types of information covered. Care should be taken to be sufficiently stringent in allocating evaluations. Thus, a FULL evaluation should only be given if all of the factors are met.

Factors to consider here include:

  • Is the information which is available complete, in the sense of including everything in the relevant category
  • Is the information easy to find
  • Is the information up-to-date, taking into account how frequently that type of information changes

Points should then be assigned according to the below scale:

Full

 Full to Partial Partial Partial to None None
1.0 0.75 0.50 0.25

0

 

 

 

 

The results of each legal requirement are then averaged.

The second criteria—a convenience score—is comprised of five ternary (0, 0.5, 1) questions that are then averaged. The five issues that should be assessed here are:

  1. The extent to which the website is WCAG 2.1 compliant.
  2. The extent of the efforts the public authority takes to disseminate information other than simply via its website.
  3. The extent to which the public authority makes use of social media and other means to draw the attention of the public to its proactive publications and to disseminate information proactively.
  4. The extent to which the public authority makes an effort to create understandable versions of at least the most important documents (such as its budget).
  5. The extent to which it is reasonably easy to find specific information from among all of the information that is being published online.

For each of these issues, an assessment of STRONGLY, PARTIALLY or WEAKLY is awarded, depending on how well the assessor believes the public authority has done. One point is awarded for STRONGLY, one-half point for PARTIALLY, and zero points for WEAKLY. The point score by public authority for this (second) part of the proactive area is the average of these point scores (add them up and divide by three).

Each public authority’s final score is comprised of 3/4 compliance score and a 1/4 convenience score.

The final point scores for each authority are then averaged to get a final point score for the jurisdiction as a whole.

The final point scores for each public authority and for the jurisdiction should be converted to a final grade using the following table:

Red

 Yellow Green
0-0.33 0.34-0.66

0.67-1.0

 

 

 

 

A data entry spreadsheet (available for download here) can be used to assist with recording and averaging point scores.

Reactive Disclosure

Whereas proactive disclosure looks at whether public authorities make information available even in the absence of a request for it, reactive disclosure is about how public authorities respond to RTI requests.

 
Main Substantive Issues
Assessment Tools
Grading

  • Is only the minimum information required by the law demanded when making an RTI request or is other (additional) information demanded?
  • Can RTI requests be made in commonly used local languages or only official languages?
  • Is it easy to submit RTI requests (electronically, in other ways)? Do you need to use the form? Is the form easily available? Do you need to prove citizenship? If so, is this easy to do?
  • Is assistance provided when needed?
  • Is a receipt provided when an RTI request is lodged?
  • If the public authority does not hold the information, do they transfer the RTI request to the public authority which does hold it (or at least refer the requester to that authority)? In a timely manner? Are transfers made which the law does not authorise (i.e. where the request should not be transferred because the original public authority holds the information)?
  • How long does it take to process RTI requests? Are responses provided as soon as possible? Within the maximum time limits? Are any extensions legitimate in terms of the rules in the law for this? Where extensions are claimed, are responses provided within the extended period?
  • Is information provided in the format stipulated by the requester? If not, are reasons for this given? Are these reasons in line with the law (i.e. in line with the conditions regarding not respecting the requester’s preferred format set out in the law)?
  • Are only reasonable fees charged for RTI requests (i.e. in line with what the law and any rules on this allow, including no fee for lodging the request)?
  • If an RTI request is refused, is appropriate notice in line with the legal requirements provided?
  • Are claims for exceptions reasonable or overbroad (this is a subjective issue but can be assessed both directly by reviewing these claims and also by looking at the percentage of the appeals which are based on refusals that the authority loses, if that information is available)?
  • Are any guidelines adopted by the public authority followed when RTI requests are processed?
  • To the extent that this can be determined, do responses to RTI requests appear to vary as a function of the gender of the requestor? Where this is the case, this should be reflected and analysed in the final report, including any statistically relevant information on this.

The primary assessment tool here is to make several actual RTI requests to test the public authority’s response. The other assessment tools are:

Each RTI request is graded on two scoring criteria. The first type–a processing score—is made up of three binary (0, 1) sub-scores that are then averaged. The three sub-scores are as follows:

  1. A receipt score (based on whether a receipt was provided)
  2. A timeliness score, based on whether the request was answered within the statutory time limits, with any extensions being assessed for both compliance with the legal rules and reasonableness
  3. A fee score, based on whether any fee charged was in line with the legal requirements

The second type of scoring criteria for each RTI request—a results score—is comprised of two binary (0, 1) sub-scores and one ternary (0, 0.5, 1) sub-score that are then averaged. The sub-scores are as follows:

  1. For full disclosure of the information, which is always valid, one point is given.
  2. For oral refusals or mute refusals (failures to respond at all within the time limits), as well as for cases in which it was not even possible to lodge the RTI request in the first place, all of which are never valid, zero points are given.
  3. Three other responses – a written refusal (in whole or in part, with partial information having been provided), transfer of the RTI request or referral of the requester to another authority and indicating that the information is not held – may be more or less valid, depending on the circumstances. The assessor should decide whether these responses are LIKELY, MAY BE or is UNLIKELY to be valid, based on all of the circumstances. One point is given for LIKELY, one-half point for MAY BE and zero points for UNLIKELY. Providing only part of the information and not indicating that the rest of the information is either not held or is exempt is at least partially invalid and so should never receive full points. Where such a response is deemed to be more in error than a deliberate strategy to deny access to the rest of the information, one-half point may be awarded.

The final score of each RTI Request is comprised of 1/3 processing score and 2/3 results score.

The final point score for each public authority is calculated by averaging the results of the RTI Requests made to it.

The final point score for the jurisdiction as a whole should be calculated by averaging the results of all RTI Requests.

The final point scores for each public authority and for the jurisdiction should be converted to a final grade using the following table:

Red

 Yellow Green
0-0.33 0.34-0.66

0.67-1.0

 

 

 

Final Grading
Scoring Instructions

Final grades may be assigned in a number of different areas including to the jurisdiction as a whole, to each individual public authority overall, overall in each of the four assessment areas and to each public authority in each assessment area.

Each public authority should have been assigned three final point scores, one for Institutional Measures, one for Proactive Disclosure and one for Reactive Disclosure. To calculate the overall grade for each authority, simply average the three final point scores from each assessment area and then convert to a final grade using the following table:

Red

 Yellow Green
0-0.33 0.34-0.66

0.67-1.0

 

 

 

 

For the jurisdiction overall, there should have been four final point scores in each assessment area (i.e. one each for Central Measures, Institutional Measures, Proactive Disclosure and Reactive Disclosure). The overall final point score should be calculated by averaging these four final point scores and then converting this to a final grade using the following table:

Red

 Yellow Green
0-0.33 0.34-0.66

0.67-1.0