Too Much of a Good Thing? Balancing Transparency and Government Effectiveness in FOI Public Interest Decision Making

AIAL Forum No. 82 (949 Kb)

TOO MUCH OF A GOOD THING?

BALANCING TRANSPARENCY AND GOVERNMENT EFFECTIVENESS IN FOI PUBLIC INTEREST DECISION MAKING

Danielle Moon and Carolyn Adams*

‘A democracy requires accountability, and accountability requires transparency.’[1]

The ‘new age’ of transparency heralded a raft of reforms to the Freedom of Information Act 1982 (Cth) (FOI Act 1982) that shifted control
of information away from government in a bid to increase transparency and accountability. Conclusive certificates, for example, were abolished in 2009
[2]

and a number of exemptions, including the exemption for deliberative documents, were made conditional on a single public interest test in 2010.
[3]
Transparency is not, however, an absolute and cannot be an end in itself; it has value only insofar as it enhances accountability. Even then,
the proper balance must be struck between transparency, efficiency and effectiveness.

This paper considers the role and importance of transparency and its relationship to the public interest test in the FOI Act 1982. It examines
the basis and impact of recent reforms and asks whether they do, in fact, strike the right balance in respect of the deliberative processes of government.
Have the reforms resulted in more accountability or less? Is there a danger that we now have ‘too much of a good thing’, that is, transparency, but
that efficiency, effectiveness and even accountability have been inappropriately compromised?

A culture of secrecy

While our public institutions exist to serve the community and should, therefore, be open to public scrutiny, they have their own internal drivers that
militate against transparency and public accountability. Early to mid-20th century scholars studying government and bureaucracy, such as Max Weber
and Carl J Friedrich, came to the conclusion that one of the defining characteristics of such organisations is a tendency to protect, rather than share,
information. Friedrich based his analysis on an empirical examination of the central administrative bodies in a number of countries including England
and the United States.[4] He noted that there was a time when arcana imperii, or State secrets, was the prevailing characterization of information in the hands of government,
which was not routinely shared with those outside government.

Friedrich’s empirical studies highlighted the fact that organisations consistently put rules and regulations in place to enforce secrecy, particularly
in relation to controversial or competitive matters. This is certainly true at the federal level in Australia. In a 2009 report, the Australian Law
Reform Commission identified over 500 provisions in 176 pieces of legislation that imposed some obligation of secrecy.^[5] In addition, legal obligations of confidence, both at common law and in equity, will apply to government bureaucrats in some circumstances.

More recent research into organisational theory considers the informal elements that permeate an organisation such as group dynamics and culture. If one
of the cultural values of an organisation is secrecy it is likely that decision makers within the organisation will also place a value on secrecy because
compliance with cultural norms and values is rewarded. Florence Heffron notes the difficulty of changing organisational culture because the values
of the organisation are often internalized and unconscious.[6] There is evidence that government bureaucrats, like members of any other organized group, tend to identify with their group and because of this:

In making decisions their organizational loyalty leads them to evaluate alternative courses of action in terms of the consequences of their action for
the group.[7]

The need for and limits of transparency

Transparency, it seems, does not come naturally to governments and even where disclosure of documents is allowed, or even required, by law it may be that
conflicting cultural or organisational factors are at work. This apparent tendency to secrecy has been widely criticised in relation to the approach
of the current Australian Government on issues such as border protection;[8] foreign aid;[9] the dismantling of the Office of the Australian Information Commissioner[10] and also more generally.[11] The tendency towards secrecy must always be borne in mind when discussing accountability measures, including transparency.

On the other hand, it is worth considering whether too much transparency might have a downside and whether the 2010 reforms have found the most appropriate
balance between too little disclosure and too much. Transparency is often brandished as a value in its own right. Attorney-General Ramsey Clark, in
introducing FOI legislation for the first time in the United States in 1967, referred to disclosure under the Public Information Act as a
‘transcendent goal’.[12] Transparency in liberal democratic theory is seen as one of the pillars supporting integrity in government and public policy and as an antidote to
corruption.

Albert Meijer, however, discusses some of the problems with transparency such as the costs, including opportunity costs, of realisation; the avoidance
strategies that may evolve in response; and the possible erosion of trust and confidence as a flood of unsorted information is disclosed leading to
confusion and uncertainty. He notes the work of Mark Bovens, ‘who warns against the dark side of transparency and its potential to drag government
through the mud time and time again.’[13] He concludes that transparency has an upside and a downside and that it is necessary to consider both if the debate is to be helpful.

There is a need for a more nuanced and instrumental approach to transparency: transparency is only valuable when it is actually contributing to effective
decision-making and accountability. David Heald lists a range of other values that may be traded off with increasing levels of transparency including
effectiveness; trust; autonomy and control; confidentiality, privacy and anonymity; fairness; legitimacy; and even accountability itself.^[14] He illustrates his point with the apt metaphor that while some sunlight is a good thing, overexposure can be damaging.^[15]

This paper focuses on two areas in which the drive for transparency needs to be carefully balanced: effectiveness and accountability. Heald suggests, for
example, that too much transparency, or the wrong kind of transparency, can disrupt organisational functioning. He suggests that overexposure of the
process of policy making is likely to have the result that ‘real policy-making shifts backwards into secret confines, with proposals less subject to
challenge … and poorly documented’.^[16] This will result in less effective decision-making and less accountability. Statements by senior federal bureaucrats indicate that this is, indeed,
what is happening^[17] and in response it is important to consider carefully whether the 2010 amendments to the FOI Act 1982 found the correct balance between requiring
too little disclosure and too much.

Freedom of Information legislation as a balancing act

FOI legislation is often viewed as a tool for promoting transparency. Moira Paterson, for example, writes of freedom of information laws as belonging to
the category of ‘laws which contribute to the objective of transparency’ contrasted with ‘laws which operate to detract from transparency’.^[18]

Considering, however, that a large part of the FOI Act 1982 is concerned with establishing exceptions and exemptions from the general right of
access to information, a better approach is to consider FOI legislation as a tool for achieving the balance between the disclosure of too much and
too little information. On the one hand, there are the overall goals of the legislation, which can broadly be described as enhancing accountability
of policy and decision making and encouraging public participation in the democratic process.^[19] On the other hand is the recognition, expressed primarily in the form of exemptions to the general right of access to information, that this right
is not absolute and is limited by other public interest concerns.

These competing interests have to be balanced when responding to requests for information. In Harris v Australian Broadcasting Corporation, Beaumont
J said ‘in evaluating where the public interest ultimately lies … it is necessary to weigh the public interest in citizens being informed of the
processes of their government and its agencies on the one hand against the public interest in the proper working of government and its agencies on
the other.’^[20]

Problems arise, however, in relation to how a decision maker ought to decide, in individual cases, whether the interest in confidentiality outweighs the
interest in disclosure. In particular, there is a question about the legitimate role of executive government in balancing the public interest in individual
cases.^[21] Richard Mulgan has suggested that in freedom of information cases ‘it is appropriate that the government should not act as judge in its own cause but
should refer the decision to an independent body’.^[22] The implication is that it is unwise to trust to government the task of balancing public interest arguments, and that self-interest, rather than public
interest, might motivate government decisions to withhold information.^[23] This perception has problematic consequences; unless there is public confidence in government FOI decisions, it is unlikely that the regime will deliver
the promised benefits of enhanced accountability and public participation.

The 2010 amendments to the FOI Act 1982 attempted to tackle the problem of public interest decision making. This paper examines those changes
in the context of internal working documents and concludes that they compromise the ability of the decision maker to balance the various public interest
considerations in play and that they are likely to lead to disclosure avoidance behaviour which will reduce, rather than increase, accountability.

FOI Act 1982 prior to amendment

The focus of this paper is the public interest in the disclosure of government’s ‘internal working documents’: documents that relate to the ‘deliberative
processes’ or ‘thinking processes’ of government.^[24] In order to understand the recent amendments, it is first necessary to understand the approach taken to balancing the competing public interests of
transparency and effectiveness in the original legislation.

Balancing tools

If the role of FOI legislation is to strike the balance between too much and too little information, then two of the key tools that the FOI Act 1982 used to strike that balance in relation to internal working documents were the principle of maximum disclosure and a public interest test.

Principle of maximum disclosure

Prior to the amendments, the principle of maximum disclosure was established in the objects clause (s 3) and in s 11 as follows:

3. (1) The object of this Act is to extend as far as possible the right of the Australian community to access to information in the possession
of the Government of the Commonwealth by—

…

(b)creating a general right of access to information in documentary form in the possession of Ministers, departments and public authorities,
limited only by exceptions and exemptions necessary for the protection of essential public interests and the private and business
affairs of persons in respect of whom information is collected and held by departments and public authorities.

11. Subject to this Act, every person has a legally enforceable right to obtain access in accordance with this Act to—

(a)a document of an agency, other than an exempt document; or

(b)an official document of a Minister, other than an exempt document.

[Emphasis added]

The effect of these provisions was to establish disclosure as the default position unless the government could demonstrate that this would be contrary
to the public interest (or private and business interests). Thus, the starting point was that there was an over-arching public interest in disclosure.

Public interest test

At the same time, however, the legislation recognised, through the inclusion of exemptions, the importance of government being able to withhold information
where necessary. The onus was on government to show why, in particular cases, an exemption applied such that disclosure would be contrary to the public
interest. In relation to internal working documents, the relevant exemption was set out in section 36:

Internal working documents

36 (1) Subject to this section, a document is an exempt document if it is a document the disclosure of which under this Act—

(a)would disclose matter in the nature of, or relating to, opinion advice or recommendation obtained, prepared or recorded, or consultation or deliberation
that has taken place, in the course of, or for the purposes of, the deliberative processes involved in the functions of an agency or Minister or of the Government of the Commonwealth; and

(b)would be contrary to the public interest.^[25]

Alongside this was the power to issue conclusive certificates^[26],
the effect of which was that the responsible Minister could conclusively certify that internal working documents were exempt from release under the
Act because disclosure would be contrary to the public interest.

Simon Murray has noted that section 36 ‘is concerned with instances where public disclosure of a document would prejudice the integrity and viability of
the decision making process’.^[27] The concept of public interest was not defined in the legislation. This lack of definition can be seen either as the legislation’s biggest problem,
or its greatest advantage. It was a strength because it meant that all relevant factors could be taken into account and given appropriate weight, making
it highly adaptable to circumstances and changes over time. But it was a weakness because it left open two questions:

• what public interest concerns might outweigh the interest in disclosure, and

• what weight ought to be given to those public interests?

There was no universal agreement in relation to these issues, and early Administrative Appeal Tribunal (AAT) case law took a relatively cautious approach,
finding that a number of factors—including the seniority of those involved, the possible inhibition of frankness and candour in future, and the
likelihood of confusion or unnecessary debate—would lead towards a finding that the information in question ought not be disclosed.^[28] Academics, including Paterson,^[29] Peter Bayne and Kim Rubenstein,^[30] and Rick Snell^[31] criticised this approach, suggesting that reliance on these factors as a matter of course was inappropriate, and contrary to the objects of the Act.
Over time, the approach of the AAT shifted, with the Re Fewster^[32] line of cases, for example, taking a more restrictive view of the application of these factors, such that by 1995 Snell noted that the AAT had begun
to favour a presumption of disclosure.^[33] The case law seemed to be moving gradually in the direction of greater transparency, without the need for any legislative action.

In 2006, however, the High Court in McKinnon v Secretary, Department of Treasury^[34] found that it had limited power to review the Government’s assessment of the public interest in cases where a ‘conclusive certificate’ had been issued.
Judith Bannister concluded that the decision in McKinnon effectively precluded any real review of government decision-making in this area,
^[35]
a concern that was echoed by mainstream journalists.^[36]In
the following year, the Australian Labor Party placed FOI reform at the heart of its election platform,^[37] a pledge which led, in turn, to a series of legislative amendments.

2009/2010 amendments

In 2010, the Freedom of Information (Reform) Act 2010 (Cth) inserted into the FOI Act 1982 a new objects clause and public interest test.
Whilst several other changes—including the abolition of conclusive certificates^[38] and the establishment of the position of Information Commissioner with full powers of merits review^[39] amplified the impact of these amendments on internal working documents.

In the Review of the Freedom of Information Act 1982 and Australian Information Commissioner Act 2010 (Hawke Review) Allan Hawke
AC stated:

The purpose of exemptions is to balance the objective of providing access to government information against legitimate claims for the protection of sensitive
material. The exemptions provide the confidentiality necessary for the proper workings of government.^[40]

This suggests that the need for balance is still at the heart of the legislation. A closer look, however, suggests that the legislation has been re-focussed
away from balancing competing interests in favour of promoting transparency.

Still a balancing act?

As noted above, properly conceived, the function of FOI legislation is to strike the balance between too much and too little disclosure. The original objects
clause expressly acknowledged that the right of access to information was limited ‘by exceptions and exemptions necessary for the protection of essential
public interests’. The new objects clause substituted by the FOI (Reform) Act 2010, however, removes the express reference to this limitation
and provides in part:

3 Objects—general

(1) The objects of this Act are to give the Australian community access to information held by the Government…

(2) The Parliament intends, by these objects, to promote Australia’s representative democracy by contributing towards the following:

(a)increasing public participation in Government processes, with a view to promoting better-informed decision-making;

(b)increasing scrutiny, discussion, comment and review of the Government’s activities … [Emphasis added].

The exemptions themselves still exist, although in slightly different form; but the change in the objects clause indicates that the purpose of the legislation
is no longer to strike the balance between transparency and the other interests, but to promote transparency.

Blunting the balancing tools?

In addition to shifting the focus of the legislation away from balance, the amendments have ‘blunted’ the tools available to decision makers assessing
the public interest in order to promote transparency outcomes. As noted above, the tools used to strike the disclosure balance in relation to internal
working documents in the original legislation were the principle of maximum disclosure and the public interest test. The amendments strengthened the
principle of maximum disclosure through the changes to the objects clause set out above. The real change, however, has been in relation to the public
interest test.

The Freedom of Information (Reform) Act 2010 (Cth) separated exemptions into two categories. In the first category are ‘absolute’ exemptions,
meaning that if the document falls under the definition of the exemption, it is exempt, with no further consideration of public interest.^[41]

The second category covers ‘conditional’ exemptions. Even a document that falls within the exemption will be released unless the government can demonstrate
that to do so would be contrary to the public interest. The ‘internal working documents’ exemption previously found in section 36 was re-cast as a
conditional exemption in section 47C, and a single public interest test was applied to all conditional exemptions by section 11A(5):

The agency or Minister must give the person access to the document if it is conditionally exempt at a particular time unless (in the circumstances) access
to the document at that time would, on balance, be contrary to the public interest.

The application of a public interest test in relation to deliberative documents is a longstanding practice; what is striking about the amendments is the
attempt, if not quite to define the public interest, then at least to determine which factors must and must not be taken into account when considering
the public interest by the insertion of section 11B into the FOI Act 1982.

Relevant factors cannot be taken into account

New section, 11B, states that it is to be used for the purpose of ‘working out whether access to a conditionally exempt document would, on balance, be
contrary to the public interest’. It does so by listing factors that may, and may not, be taken into account when conducting the public interest balancing
test, dividing them into ‘factors favouring access’ and ‘irrelevant factors’.

As a result of the insertion of section 11B the following factors cannot be taken into account:

(a)access to the document could result in embarrassment to the Commonwealth Government, or cause a loss of confidence in the Commonwealth Government;

(aa) access to the document could result in embarrassment to the Government of Norfolk Island or cause a loss of confidence in the Government of Norfolk
Island;

(b) access to the document could result in any person misinterpreting or misunderstanding the document;

(c) the author of the document was (or is) of high seniority in the agency to which the request for access to the document was made;

(d) access to the document could result in confusion or unnecessary debate.

These factors, or variations of them, have in the past been used to support an argument that to disclose internal working documents would compromise effective
government decision making. Most notably, the AAT in the Howard case, discussed above, found that the likelihood of disclosure causing confusion
or unnecessary debate, or the high seniority of the author of the document, were factors which might suggest that disclosure was not in the public
interest. Reliance on these factors is now prohibited following the amendments to the legislation. As noted above, the approach of the AAT had begun
to shift by the time of the amendments, and it was less routinely accepted that the presence of these factors could lead to compromised government
effectiveness if the information was disclosed.

This shift in understanding of the public interest does not necessarily mean, however, that these factors will never, in any circumstance, or at any time
in the future, be relevant to a decision on disclosure. Mulgan has noted that judgements about the public interest are essentially political in nature,
^[42]
meaning that what constitutes the public interest shifts with time, circumstances and changing political views. Indeed the 1995 ALRC report on
the FOI Act recommended against legislative guidelines on the public interest because:

Just as what constitutes the public interest will change over time, so too may the relevant factors. For this reason, the Review considers that administrative
guidelines issued pursuant to the Act are generally preferable to legislative guidelines.^[43]

It is unclear why the government went against this advice. Whatever the reason, the result is that we can no longer be confident that the FOI Act is capable of ensuring that decision makers can take all relevant factors into account in all cases. If, as a result of the inclusion of this list
of factors in section 11B, the government is unable to make out a legitimate case that effectiveness will be compromised by disclosure, then an