Update on Audit Quality Indicators

Introduction

I appreciate this opportunity to update you on the Board's audit quality indicator project, or AQI as we call it. As a reminder, my comments reflect my personal views, and not necessarily those of the Board or other staff.

At its root, the Board's project seeks to answer two questions:

• Can we develop a portfolio of quantitative measures that provide new insight into audit quality?
• If so, how can we deploy those measures in a manner that best promotes quality?

We expect to issue a Concept Release in the first quarter seeking public input on these questions.

We premise the project on the notion that audits are a bit like icebergs: the buyers of audit services, audit committees and to a lesser extent, investors, can see only the tip, while important ingredients of audit quality lay opaque, under the waterline.

I have found the AQI project intriguing because it runs to the heart of the incentive system for auditors:

• When much audit effort is opaque to the buyer of audit services, it creates an incentive for the auditor to emphasize what the buyer can see, and potentially to shortcut what is opaque.
• Meanwhile, the buyer, having limited information, tends to focus on the tip of the audit while less able to question and challenge what is below. This undermines the market's ability to demand and reward the underlying ingredients needed for the highest quality work.

In an ideal world, AQIs assist the dialog between auditors and the market. That, in turn, could promote competition among auditors based on audit quality; a world where auditors win work, retain work and are compensated based on the quality of what they do.

By their nature, regulatory endeavors tend to be long on sticks and short on carrots. We issue rules, inspect for compliance, publicly note deficiencies, and when needed, discipline folks.

Ideally, the AQI project could add a welcome incentive to the regulatory tool kit. By providing audit buyers with insight on quality, we can help align auditor interests with market interests.

There is reason for optimism that measuring quality can affect behavior. Accountants and auditors have long appreciated that people manage what they measure, which we note in many aspects of human endeavor. A few examples include reporting the contents of food in a standardized label on food packaging; reporting formerly off-balance sheet OPEB obligations as liabilities; and reporting health department inspection scores prominently on the windows of NYC restaurants. In each case, users focused on the measures and in doing so affected both user and company decisions.

The notion of measuring audit quality and using the measures to promote quality enjoys widespread support, at least at the conceptual level:

• The US Treasury's Advisory Committee on the Auditing Profession recommended a few years ago that the Board undertake the project.
• The project enjoys the strong support of the Board's advisory groups.
• Of course, auditing firms have long measured their audit practices, in part for audit quality, and some are enhancing the insight and uses of their measures. We have tried to be students of their evolving practice.
• The CAQ continues with its project on AQIs, which has been helpful in the Board's project.
• The SEC staff has thought a lot about audit quality and has provided the Board with good insight.
• Outside the US, professional bodies, standard setters, and audit regulators have been considering audit quality indicators in separate projects.
• Perhaps there is a window of opportunity for AQIs.

Yet, despite widespread support, as always, the devil is in the details. Ideas for specific AQIs and ideas for possible uses of AQIs are controversial. Let me hum a few bars.

The nature of audit quality indicators

We speculate that a portfolio of about 15 indicators may be sufficient to present a "balanced scorecard." No single indicator is likely to be determinative. AQIs are more likely to cause the user to ask the right questions; they are unlikely to definitively answer those questions.

Thus, AQIs are not algorithms, nor safe harbors or even benchmarks.

AQI data likely requires context, that is, a qualitative analysis of what the data imply. The meaning of the numbers depends on the situation. Thus, AQIs + context = insight.

AQIs will offer the most value if they are comparable, both over time and across engagements or firms. We accountants instinctively understand the importance of comparability.

The AQI project is closely aligned with our Inspection Division's effort to encourage root cause analysis of audit problems and successes by audit firms. Root cause analysis is part of its broader effort on the audit quality improvement cycle. I view root causes and AQIs to be two sides of the same coin: in both cases we need to identify, measure and improve at the root cause level of audit quality.

To date, the staff has identified about 30 indicators that are most promising

To do so, we first sought to identify the broadest range of possible indicators, a process that has surfaced over 80 candidates. We considered indicators cited in academic research, previous efforts related to audit quality, and discussions with the Board's advisory groups, firms, other regulators, audit committees and academics.

To ensure a robust array, we created an audit quality framework, a COSO-type model for an audit practice. For framework elements without a candidate measure, the staff brainstormed about indicators that could inform that element, which surfaced more candidates.

The staff is now narrowing the list to the most promising indicators for discussion with the Board. We previously asked members of our advisory groups to rank the usefulness of the indicators. In addition, a cross-divisional team within the PCAOB is evaluating all candidates based on a set of criteria. Examples of criteria include usefulness to audit committees and investors, potential for unintended consequences, availability of data, scalability across firms and correlation to audit quality. We expect the Concept Release will result in comments that will help narrow candidates to a viable and useful portfolio.

I speculate that some of the candidate indicators are likely to be less controversial. Examples include:

• Partner to staff leverage.
• Staff utilization.
• Industry expertise.
• Percentage and timing of audit effort by the partner, manager and engagement quality reviewer.
• Findings of the PCAOB's inspection process.

Yet, I suspect other candidates will be more controversial. For example:

• Percentage and nature of work outsourced to service centers.
• Anonymous independent survey of firm personnel about "tone at the top" in their firm.
• Absence of a going concern paragraph before a bankruptcy or similar financial distress.
• Survey of audit committees about the quality of communications from their auditor.

Possible uses of AQIs

Deciding on the best use of the indicators requires answers to basic questions: Who can best use the indicators? How can they be used most effectively? How should the necessary data be obtained and distributed? The answers to these questions will determine the scope, timing and potential effect of the ultimate project.

There are alternative ways to provide benefits of AQIs to possible customers. The range of those alternatives depends upon several factors, including:

• The nature of customers that AQIs will intend to serve, whether audit committees, investors, regulators, audit firms, management, or others.
• Another factor is the unit of account (i.e. the entity that the AQI data relates to); options include audit engagements; national audit firms; global audit networks; or perhaps we need separate reporting for audits in specific industries.
• A third factor is the identity of entities that collect and disseminate AQIs, whether engagement teams, audit firms, third parties or regulators.
• Fourth is whether the AQI program is voluntary or mandatory.
• A fifth factor considers the scope of audits and audit firms that may be subject to AQI reporting; for example should we exempt smaller firms; or audits of smaller issuers; or audits in specialized industries that might require unique indicators?
• A sixth factor is the possibility of phasing-in, over time, steps toward a complete AQI program.

We view these factors as a menu of items, from which one could devise an AQI program. For example:

• A simple AQI program might begin with engagement teams voluntarily discussing AQIs with their audit committees.
• A more robust program could mandate those discussions based on specified engagement and firm level indicators.
• Still further could be firms publicly publishing and discussing AQIs in their reports on audit quality.
• Perhaps regulators could prepare an annual report on the state of audit quality citing relevant indicators?

We expect the Concept Release will take no position on these questions. Rather, it will identify alternatives and seek comment about them.

Closing

Some of you may be fans of AQIs, but question what value a regulator can provide. Why not let the market alone drive the nature and extent of AQIs? Perhaps not surprisingly, I believe the Board can add unique value to the AQI issue, which the market alone is unlikely to provide:

• Perhaps we can help flag best practice, across firms, study groups, and geography.
• Perhaps we can standardize vocabulary and definitions thereby improving comparability of the data.
• Perhaps we can help ensure a comprehensive discussion, one that considers both favorable and unfavorable data.

In closing, as the next step, we envision a Concept Release in 3 parts:

• The first part discussing the background and purpose of the project.
• The second discussing the nature of AQIs and the list of promising AQIs.
• The third part discussing possible uses of AQIs to promote quality.

Of course, a Concept Release is not a proposal and is not rulemaking. Subsequent steps in the project, which may involve rulemaking, will depend on the nature of public comments and, of course, the Board's deliberations.

Speaking of comments, we encourage as many of you as possible to provide us with your views. We have benefitted greatly from outside input to date, and we're sure to learn a great deal more in the public comment process and possible hearings that follow.

Thanks for your kind attention and we look forward to your comments and suggestions.