Comprehensive Methodology

This Comprehensive Methodology
provides a structured tool for
assessing the quality of implementation
of a Right to Information Law.

Central Measures

This section of the methodology looks at how the RTI law is implemented at the central level. Central institutions serve two main functions: processing appeals and undertaking promotional measures.

Institutional Measures

This section looks at what individual public authorities have done to implement the RTI law. In any particular jurisdiction, there will always be a great number, normally hundreds and sometimes thousands, of different public authorities of different types and levels.

Proactive Disclosure

This section looks at proactive disclosure or the release of information by public authorities without an RTI request. This is often done through websites and annual reports.

Reactive Disclosure

Whereas proactive disclosure looks at whether public authorities make information available even in the absence of a request for it, reactive disclosure is about how public authorities respond to RTI requests.

Methodology Roadmap

041-pros-and-cons-300x300.png

1. Assess Feasibility of Doing Assessment

036-teamwork-300x300.png

2. Put in Place the Governing Structure

001-idea-300x300.png

3. Develop an Implementation Plan

029-review-300x300.png

4. Apply the Assessment Tools to Collect Data

010-research-300x300.png

5. Analyse Data and Score Results

011-solution-300x300.png

6. Draft Report and Recommendations

023-ui-design-300x300.png

7. Publish Report

031-startup-300x300.png

8. Conduct Follow-up Advocacy and Activities

Applying the Methodology
The Comprehensive Methodology evaluates the implementation and performance of a jurisdiction’s RTI laws by its central authority and public authorities. In any given country, there will be 100s or even 1000s of authorities, and it will not be possible to assess all of them. The Methodology, therefore, calls for a minimum of ten public authorities to be selected for assessment. The central and public authorities are evaluated at the institutional level. The public authorities are further assessed based on their performance of proactive and reactive disclosure requirements. According to a three-part scale, both the jurisdiction and the individual institutions are assigned grades: poor, mediocre, excellent.
The following constitutes an overview of the methodology. The complete methodology is available for download here.
Central Measures

This section of the methodology looks at how the RTI law is implemented at the central level. Central institutions serve two main functions: processing appeals and undertaking promotional measures. In almost every country where an independent oversight body (often called an information commission) exists, it bears primary responsibility for the first function. In many countries, the information commissions both process appeals and are the main official body that is responsible for promotional measures. However, in others, the oversight body is supported by a nodal body—either internal or external to the government—that undertakes the promotional functions, including assisting individual public authorities. For the purposes of this Methodology, the information commission and nodal body (if any) should be evaluated as one.

Assessment

In this section, assessors evaluate whether the central authority has successfully implemented the RTI law at an institutional level, for example, by establishing an independent oversight body and whether the institution fulfills its RTI obligations in practice.  In most cases, responsibility for the former lies with the government or parliament. The latter focuses on the effectiveness of the oversight body in discharging its responsibilities.

 

Implementation

Performance

  • Have independent, effective governing members of the oversight body (commissioners) been appointed?
  • Has the body been provided with reasonably sufficient funding to enable it to carry out its tasks?
  • Does the body recruit its own expert staff (as opposed to staff being allocated from the civil service whose primary loyalties tend to lie with the civil service)?
  • Does it have a full complement of staff (i.e. compared to its organigram or official documents)? Are they on long-term employment contracts?
  • Does the oversight body make an effort to be geographically accessible to complainants?
  • Does the oversight body process appeals appropriately and in accordance with the law?
  • Does the oversight body take active (suo moto) steps to ensure that public authorities are implementing the law properly
  • Does the oversight body discharge its regulatory functions properly? This will depend on what the law requires/authorises it to do in terms of regulation.
  • Does the oversight body take steps to raise public awareness about the RTI Law?
  • Has the oversight body supported the provision of training to PIOs?
  • Does the oversight body publish an annual report on overall implementation?
  • Does the oversight body actively use whatever powers it has under the law to impose appropriate sanctions on individuals/entities who obstruct access?
  • Has the oversight body commented on draft laws that affect RTI?
  • Does the oversight body provide advice to public authorities which ask for it? What about members of the public?
  • Has the oversight body taken any other steps to improve implementation?

The following assessment tools should be deployed:

  • Key informant interviews
  • A self-assessment by the public authority
  • A desk-based literature review
  • Another desk-based review of actual decisions on appeals

The central authority’s institutional measures are graded through five binary (0, 1) questions and eight ternary (0, 0.5, 1) questions that are then averaged. A data entry spreadsheet (available for download here) can be used to facilitate the grading process.

The five binary (yes-no) questions are:

  1. Has funding been allocated (i.e. to the body)?
  2. Does the body recruit its own staff (as opposed to this being done by the government, for example)?
  3. Are the body’s appeals decisions available online?
  4. Has the body produced and published an annual report for the last two years?
  5. Has the body published a guide for requesters?

The eight more qualitative questions are:

  1. Have the members been appointed?
  2. Are the members of the body independent and effective?
  3. Is the funding provided to the body reasonably sufficient for it to discharge its functions?
  4. Does the body decide appeals in a timely fashion?
  5. Are the due process rights of parties respected during appeals?
  6. Has the body made reasonable efforts to raise public awareness?
  7. How effective are the measures taken to provide training to officials?
  8. Has the body made a reasonable effort to comment on draft laws that affect the right to information?

Here, assessors are asked to determine whether the system performs Strongly (1 point); Partially (0.5 points) or Weakly (0 points).

The overall point score is calculated by taking the 13 individual scores and averaging them. This score should then be converted to a final colour grade using the following table:

Red Yellow Green
0-0.33 0.34-0.66 0.67-1.0
Public Authorities

This section looks at what individual public authorities have done to implement the RTI law.  In any particular jurisdiction, there will always be a great number, normally hundreds and sometimes thousands, of different public authorities of different types and levels. Acknowledging that it is unrealistic for assessors to review every single authority, the Methodology requires the Project Lead to select a subset of authorities representing the whole. In this subset, the Project Lead should only include public authorities that receive a volume of RTI requests that makes proactive disclosure—which this methodology measures—reasonable. This subset of authorities will be reviewed across all three assessment areas: Institutional Measures, Proactive Disclosure and Reactive Disclosure.

Institutional Measures

In this section, assessors evaluate whether the public authority has successfully implemented the RTI law at an institutional level, for example, by hiring a public information officer, and whether the institution fulfills its RTI obligations in practice.

 

Implementation

Performance

  • Has the authority appointed a PIO?
  • Has the PIO been provided with RTI training? If so, how long was the training and who provided it? Was it just a one-off or has upgrade training been provided?
  • Have other staff been formally instructed to cooperate with the PIO in discharging his or her functions, in particular in relation to the processing of RTI requests?
  • Does the authority have an overall implementation plan or standard operating procedure (SOP) for RTI? If so, is the plan or SOP reasonably detailed and does it seem realistic?
  • Has the authority developed/issued internal guidelines for receiving and responding to RTI requests?
  • Has the authority appointed someone to receive and process internal complaints?
 
  • Are there political pressures that make it difficult for the PIO to do his or her job? Is he or she treated as a sort of “spy in the office”; are there institutional incentives or just disincentives (most of this is quite subjective but it can be assessed in various ways, such as by asking the PIO and civil society representatives)?
  • Is it easy to lodge RTI requests with the authority?
  • Does the authority prepare and publish annual reports, including statistics on RTI requests? If so, when was the last report published? How detailed is it (for example, how many types of information does it provide about RTI requests)?
  • Has the authority undertaken any public awareness-raising efforts? If so, what?
  • Has the authority put in place any systems or standards to improve its records management? Are these monitored, applied or enforced in any way?

The following assessment tools should be deployed:

Each public authority’s institutional measures are graded on ten binary (0, 1) questions and six ternary (0, 0.5, 1) questions that are then averaged. A data entry spreadsheet (available for download here) can assist with this calculation.

The ten objective evaluations are:

  1. Has a PIO been appointed?
  2. Has the PIOI formally been given terms of reference or a job description?
  3. Has the PIO been provided with training?
  4. Has an overall implementation plan or set of standard operating procedures (SOPs) been adopted?
  5. Has a set of guidelines for how to process RTI requests been adopted?
  6. Is it possible to lodge requests electronically? Is it easy to obtain an RTI request form? Is it easy to find the contact details of the PIO? (YES is given for two or more positive answers, NO for one or less)
  7. Has a person who is different from the PIO been appointed to deal with internal complaints?
  8. Did the public authority publish an annual report for the last two years?
  9. Has the public authority conducted awareness-raising activities over the last year?
  10. Has the public authority put in place any system or taken any action to improve its records management?

For each of these evaluations, one point is given for a YES and zero points are given for a NO.

The six qualitative evaluations are:

  1. Does the PIO have appropriate qualifications for the job and has he or she been allocated time to do the job?
  2. There is no political pressure on the PIO that makes it difficult for him or her to do the job properly.
  3. How strong is the overall implementation plan or SOP?
  4. How strong is the annual report?
  5. How extensive are the awareness-raising activities?
  6. How effective are the measures taken to improve records management
  7. For each of these evaluations, an evaluation of STRONGLY, PARTIALLY or WEAKLY is awarded, depending on how well the assessor believes the public authority has done.
  8. One point is awarded for STRONGLY, one-half point for PARTIALLY, and zero points for WEAKLY.

The final point score for the jurisdiction as a whole should be calculated by averaging the results of all public authorities.

The 16 point scores for each public authority are then averaged (added up and divided by 16) to get a final point score for each authority. These averages by public authority are then averaged again to get the final point score for the jurisdiction. The final point scores for each public authority and for the jurisdiction should be converted to a final colour grade using the following table:

Red

Yellow Green
0-0.33 0.34-0.66

0.67-1.0

Proactive Disclosure

This section looks at proactive disclosure or the release of information by public authorities without an RTI request. This is often done through websites and annual reports.

Formally, public authorities’ proactive publication obligations are limited to what the RTI law requires them to do. In other words, performance should be assessed against the list of proactive publication obligations set out in the law. The types of documents that are required to be disclosed can be listed, and the assessment can be conducted against that list. Where the law is vague, the assessors should make and report on any required interpretations.

RTI laws are, however, slow to change. Therefore, it is recommended that the assessors evaluate the public authorities against a set of better practice standards. The final report should delineate between the requirements prescribed by law and those by generally accepted standards.

Assessors should investigate both the implementation and performance of the RTI law with respect to proactive disclosure.

 

Implementation

Performance

  • Is the public authority’s website WCAG 2.0 compliant?
  • Is it reasonably easy to find specific information from among all of the information that is being published online?
  • Does the public authority use social media or smartphone apps to draw the attention of the public to its proactive publications (and to provide key information directly to the public)?
  • Does the public authority take advantage of its public service points (i.e. offices to which the public has direct access) to both engage directly in proactive publication and highlight its online proactive publication efforts?
  • Does the public authority make an effort to create understandable versions of at least the most important documents (such as its budget)?
  • What efforts does the public authority make to disseminate information other than simply via its website?

By definition, information made available on a proactive basis should be relatively easy to access either in person via the public authority’s office or website. The following assessment tools should be deployed:

Proactive disclosure is graded on two scoring criteria. The first—a compliance score—involves measuring the public authority’s compliance with each legally prescribed proactive disclosure requirement against a quinary scale (0, 0.25, 0.50, 0.75, 1.0). In other words, the assessment of proactive disclosure involves making a list of what should be disclosed proactively (according to the law) and then seeing if it is in fact available. For each item on the list, the public authority should be given one of the following evaluations: Full, Full to Partial, Partial, Partial to None or None, according to the below scale:

Full

Full to Partial Partial Partial to None None
1.0 0.75 0.50 0.25

0

       

These results of each legal requirement are then averaged.

The second criteria—a convenience score—is comprised of five ternary (0, 0.5, 1) questions that are then averaged. The five issues that should be assessed here are:

  1. The extent to which the website is WCAG 2.0 compliant.
  2. The extent of the efforts the public authority takes to disseminate information other than simply via its website.
  3. The extent to which the public authority makes use of social media and smartphone apps to draw the attention of the public to its proactive publications and to disseminate information proactively.
  4. The extent to which the public authority makes an effort to create understandable versions of at least the most important documents (such as its budget).
  5. The extent to which it is reasonably easy to find specific information from among all of the information that is being published online.

For each of these issues, STRONGLY, PARTIALLY or WEAKLY is awarded, depending on how well the assessor believes the public authority has done. One point is awarded for STRONGLY, one-half point for PARTIALLY, and zero points for WEAKLY. The point score by public authority for this (second) part of the proactive area is the average of these point scores (add them up and divide by three).

Each public authority’s final score is comprised of 3/4 compliance score and a 1/4 convenience score.

The final point scores for each authority are then averaged to get a final point score for the jurisdiction as a whole.

The final point scores for each public authority and for the jurisdiction should be converted to a final grade using the following table:

Red

Yellow Green
0-0.33 0.34-0.66

0.67-1.0

        A data entry spreadsheet (available for download here) can be used to assist with recording and averaging point scores.
Reactive Disclosure

Whereas proactive disclosure looks at whether public authorities make information available even in the absence of a request for it, reactive disclosure is about how public authorities respond to RTI requests. Assessors should investigate both the implementation and performance of the RTI law with respect to reactive disclosure.

 

Implementation

Performance

  • Is only the minimum information required by the law demanded when making an RTI request or is other (additional) information demanded?
  • Can RTI requests be made in commonly used local languages or only official languages?
  • Is it easy to submit RTI requests (electronically, in other ways)? Do you need to use the form? Is the form easily available? Do you need to prove citizenship? If so, is this easy to do?
  • Is assistance provided when needed?
  • Is a receipt provided when an RTI request is lodged?
  • If the public authority does not hold the information, do they transfer the RTI request to the authority which does hold it (or at least refer the requester to another public authority)? In a timely manner? Are transfers made which the law does not authorise (i.e. where the request should not be transferred because the original public authority holds the information)?
  • How long does it take to process RTI requests: are responses provided as soon as possible? within the maximum time limits? are any extensions legitimate in terms of the rules in the law for this? are responses provided within the period set for any extensions?
  • Is information provided in the format stipulated by the requester? If not, are reasons for this given? Are these reasons in line with the law (i.e. in line with the conditions regarding not respecting the requester’s preferred format set out in the law)?
  • Are only reasonable fees charged for RTI requests (i.e. in line with what the law and any rules on this allow, including no fee for lodging the request)?
  • If an RTI request is refused, is appropriate notice in line with the legal requirements provided?
  • Are claims for exceptions reasonable or overbroad (this is a subjective issue but can be assessed both directly by reviewing these claims and also by looking at the percentage of the appeals which are based on refusals that the authority loses, if that information is available)?
  • Are any guidelines adopted by the public authority followed when RTI requests are processed?

The primary assessment tool here is to make several actual RTI requests to test the public authority’s response. The other assessment tools are:

Each RTI request is graded on two scoring criteria. The first type–a processing score—is made up of three binary (0, 1) sub-scores that are then averaged. The three sub-scores are as follows:

  1. A receipt score (based on whether a receipt was provided)
  2. A timeliness score, based on whether the request was answered within the statutory time limits, with any extensions being assessed for reasonableness
  3. A fee score, based on whether any fee charged was in line with the legal requirements

The second type of scoring criteria for each RTI request—a results score—is comprised of two binary (0, 1) sub-scores and one ternary (0, 0.5, 1) sub-score that are then averaged. The sub-scores are as follows:

  1. For full disclosure of the information, which is always valid, one point is given.
  2. For oral refusals or mute refusals (failures to respond at all within the time limits), as well as for cases in which it was not even possible to lodge the RTI request in the first place, all of which are never valid, zero points are given.
  3. Four other responses – a written refusal (in whole or in part, with partial information having been provided), transfer of the RTI request or referral of the requester to another authority, providing only part of the information, and indicating that the information is not held – may be more or less valid, depending on the circumstances. The assessor should decide whether the response is LIKELY, MAY BE or is UNLIKELY to be valid, based on all of the circumstances. One point is given for LIKELY, one-half point for MAY BE and zero points for UNLIKELY.

The final score of each RTI Request is comprised of 1/3 processing score and 2/3 results score.

The final point score for each public authority is calculated by averaging the results of the RTI Requests made to it.

The final point score for the jurisdiction as a whole should be calculated by averaging the results of all RTI Requests.

The final point scores for each public authority and for the jurisdiction should be converted to a final grade using the following table:

Red

Yellow Green
0-0.33 0.34-0.66

0.67-1.0

        A data entry spreadsheet (available for download here) can be used to assist with the grading process.
Final Grading

Final grades may be assigned both to each individual public authority for purposes of feedback and to the jurisdiction as a whole.

Each public authority is assigned three final point scores: one for Institutional Measures, one for Proactive Disclosure, and Reactive Disclosure. Each authority’s overall final point score is calculated by averaging the three final point scores from each assessment area. This should then be converted to a final grade using the following table:

Red

Yellow Green
0-0.33 0.34-0.66

0.67-1.0

 

 

 

Overall, the jurisdiction is assigned four final point scores, one for Central Institutional Measures, one for Public Authority Institutional Measures, one for Proactive Disclosure and one for Reactive Disclosure. The overall final point score is calculated by averaging the four final point scores from each assessment area. This should then be converted to a final grade using the following table:

Red

Yellow Green
0-0.33 0.34-0.66

0.67-1.0