Can Artificial Intelligence Transform Auditing and Our Fear of That Transformation?

Remarks as prepared for delivery

Thank you for the introduction. I am honored to have been invited to this event. Before I get started, I want to make clear that the views I express today are my own and do not necessarily reflect the views of the PCAOB Board, other PCAOB Board Members, or PCAOB staff.

Artificial Intelligence is Here to Stay

This week, President Biden signed an Executive Order on Safe, Secure, and Trustworthy Artificial Intelligence (AI).¹ I read the over 60-page document² this week and using the word “sweeping” to describe it is an understatement. Out of the 24 CFO Agencies ³, the Executive Order directs 20 of them to take more than 120 actions. The 24 CFO Agencies represent over 95% of the federal budget. At a high level, it directs federal agencies to 1) set standards on AI safety and security; 2) provide guidance on the responsible use of AI in the private sector on privacy, health, criminal justice, housing, education, and more; expands capacity in AI expertise in government and industry; and prescribes the federal agencies’ use of AI. As a pioneer who has led a major government reform initiative, I know from firsthand experience the significance of this Executive Order. Not only does this Executive Order serve as an important signal to the world that the United States is serious about managing AI and leveraging it for good, the Executive Order also mobilizes the entire apparatus of the federal government to implement and institutionalize the policies outlined. The only thing that the Executive Order does not provide is funding, other than the reference to the Technology Modernization Fund. However, I expect that federal agencies will find ways to implement at least some components of this Executive Order without additional funding.

If there has been any doubt or debate about how significant AI technology is and will be to our economy, welfare and liberty, this Executive Order puts that debate to rest.  

AI Transforming Auditing

Against this backdrop, I want to talk to you today about my views on whether AI can transform auditing. While I don’t have a crystal ball, my personal view is that AI can transform auditing, and I hope that it will. My view was supported by a research paper released prior to the explosion of AI activities last year, and I expect more research to have been done since then.⁴ AI was conceptualized in the 1950s, but it did not mature until computer storage capacity was no longer a constraint in the 2000s.⁵ Fueled by the age of “Big Data”, it continues to thrive, and its recent speed of development is stunning. For example, ChatGPT is already on its 4th version, released four months after its initial release in November 2022. It has over 180 million active monthly users worldwide as of September 2023, achieved in less than one year after the initial release.        

I have said publicly before that data and technology can improve audit quality and enhance investor protection, but embracing it effectively and responsibly requires expertise, collaboration, and ongoing iterations in an agile and transparent environment. I want to share a few ways that I believe AI technology can transform auditing. Before I do that, I would like to first define AI, as this term is often used broadly and at times inconsistently. A legal definition for AI can be found in 15 U.S.C 9401(3), which defines AI as “a machine-based system that can, for a given set of human-defined objectives, make predictions, recommendations, or decisions influencing real or virtual environments. Artificial intelligence systems use machine- and human-based inputs to- (A) perceive real and virtual environments; (B) abstract such perceptions into models through analysis in an automated manner; and (C) use model inference to formulate options for information or action.” If you are like me, you will need to read this many times to make sense of it. To put this into practical terms, AI involves rapidly ingesting an enormous volume of data in all formats from disparate sources, then use techniques such as machine learning, natural language processing, computer vision, neural networks, etc. in order to perform activities or predict outcomes. Here are three specific, high-level areas where AI can transform auditing:

• Routine and Repetitive Audit Tasks. As a former auditor, I know that a great deal of audit tasks are mundane, repetitive, and tedious. These tasks include creating workpaper templates, documenting interview notes, cross-referencing numbers, counting inventories, extracting key terms from contractual agreements, and performing routine recalculation – just to name a few. These tasks require consistent execution. AI can automate all of these tasks and can execute them consistently and better than humans. The AI models for these tasks should be simpler because they are deterministic and require minimal judgment. This type of automation shifts the mundane and routine tasks from a human to technology, allowing the human to focus on more critical audit work that requires judgment. Further, if auditors can rely on machines to perform these routine tasks efficiently, they could then expand the testing population to 100% for some significant accounts. Under the current paradigm, audit sampling methodology is used to achieve reasonable assurance within a short period of time and at a reasonable cost. Another value for automating these audit tasks is that the data created through this process can be further analyzed to extract additional insights. I want to share a real-life example to illustrate the power of AI in automation. I had asked ChatGPT to summarize the 63-page Executive Order and provide a complete list of all actions required. It took an hour to get that information. Now, it took me about two and a half hours to read the Executive Order myself. It would probably take me at least another five hours to summarize it in writing and compile a list of the actions stipulated in the Executive Order. The accuracy of ChatGPT was also surprisingly good. I still believe my non-algorithmic brain has done a far superior job in understanding the context given my broad knowledge of the federal government. I therefore would still want to read it, but if I could save myself time from summarizing and listing, I could use that time saved on thinking and generating ideas. 
• Fraud Detection.  When I was at the U.S. Department of the Treasury, I spent a significant amount of time contemplating ways the Federal Government could prevent and detect fraud and improper payments, which were reported to total $247 billion in 2022.⁶ Prior to my appointment to the PCAOB Board, I led a data science consulting practice helping several federal agencies build AI models to detect fraud and anomalies, among other things. I equate looking for fraud to looking for the needle in a haystack. That’s why we need to leverage AI technology to help us. Specifically, AI techniques, such as machine learning can help organizations identify potentially fraudulent activities using all available and relevant data. AI combined with other tools, such as graph analytics can also help organizations identify and link relationships that may not be obvious or intentionally concealed. AI is now commonly used in flagging suspicious financial transactions in real-time within the financial services sector. It is also used to predict potential insider trading, financial crimes, and other illegal activities. Similar techniques can bolster auditors’ capability to detect fraud related to company financial statements. Even the data created from the automation, such as AI voice recorded interview notes can be further analyzed using sentiment analysis to detect potential inconsistencies.  
• Risk Assessment.  AI can especially assist with ongoing risk assessments of relevant and significant current events. Based on my observation, every time a public company fails, the public and investors would question the auditors’ work. We often hear people say, “hindsight is 20/20”. We do live in an uncertain world and that’s why absolute audit assurance is not possible. However, many current events can have an impact on a company’s financial performance. Auditors’ timely identification and assessment of their impact is crucial. It is not possible for human brains to process the massive amount of data available effectively and rapidly, but machines do have this ability. I believe that AI can help identify the impact of financial performance based on current events on a timely basis, so that auditors can reassess risks promptly and frequently as needed.    

If done right, any of these three areas can be a game changer in improving audit quality and investor protection. Experimentation with emerging technologies such as AI and machine learning can best be fostered iteratively in a collaborative environment. Based on my experience in leading large scale government reform in data and technology, innovation requires expertise, as well as willingness to experiment across the entire ecosystem, which the President’s Executive Order clearly recognizes. Section 5.2 emphasizes the need for public-private partnerships to advance, among other things, AI innovation, and the President directed the National Science Foundation, the Department of Energy, and the Department of Health and Human Services to develop pilot programs or partnerships to foster such collaboration with industry, academia, and others. 

Further, just like the President is directing key federal agencies to set standards on AI safety and security, while promoting and leveraging AI for public good, PCAOB auditing standards should also evolve to promote the responsible use of AI to improve audit quality and provide guardrails to ensure objectivity and transparency, in the interest of investor protection. For examples, if auditors are relying on AI tools to perform tasks or make predictions, what controls should be in place to ensure that they are free of bias, are auditable, and meet quality standards? What considerations are needed for tools that are developed by third parties? How will the audit evidence be documented if all or part of the audit processes become digitally native? How would data validation be performed, especially when the raw data input may be in an unstructured format, or information may be in the form of images and videos? How can regulators confirm that the AI tools can be trusted and free of bias?

Challenges of the AI Transformation

We clearly have more questions than answers, and there are many challenges ahead. However, I want to highlight two significant challenges of this transformation: data and talent. I tried to make a diagram to illustrate this point.

 

• Data: As I tried to define AI in practical terms earlier, data is the key ingredient to AI. Without quality data, AI will be of no use. Data quality and accessibility is a key challenge in realizing the benefits of AI. As you can see in this diagram, data goes through multiple stages to reach its final uses. The effort required to access/collect and validate/wrangle data is the most tedious, labor-intensive, and best suited for machines to handle at scale. It is well known that data professionals spend 80% of their time cleaning and wrangling data. Once data has reached a stage that requires more judgment, that is when algorithms cannot compete with human brains. That takes me to the second challenge.
• Talent: President Biden recognizes the crucial need for AI expertise in both the private sector and the federal government to implement his Executive Order. For example, he directed each agency to create a Chief AI Officer role; he directed the State Department and the U.S. Department of Homeland Security (DHS) to streamline the immigration process to attract this talent from abroad, and he also directed the Office of Personnel Management (OPM) to accelerate the hiring process for individuals with AI expertise into the federal government. I have been speaking repeatedly about the accountant pipeline crisis since last year. Accountants and auditors are context experts, who exercise the judgment needed to analyze and interpret the data, as well as form conclusions. Given the dual talent shortages here, it is vital for us to make the accounting profession relevant and attractive to young students.

Accountants have been creating, validating, analyzing, and interpreting data since writing was invented (coincidentally by the need to account for and check commercial transactions).⁷ Accountants have therefore been innovators and problem solvers since the beginning of time. My daughter who loves Harry Potter told me that “accountants” were mentioned once in the book series. However, it was categorized as a “muggle” profession. In my opinion, this is a compliment because this profession is about building value with diligence and ingenuity, not magic. I am proud to be both an accountant and a “muggle”. When I was a little girl growing up, I never imagined that one day my accounting degree would lead me to becoming a Deputy Assistant Secretary within the U.S. Department of the Treasury, nor playing a historic role as the first Asian American PCAOB Board member. The opportunities of this profession are endless.  

From Fear of Transformation to Transformation of Fear  

In closing, I want to acknowledge that it can be intimidating to be in the midst of major transformations because we don’t want to let go of what we already know in exchange for the unknown. I am reminded of a short inspiring piece by Danaan Parry titled “Turning the Fear of Transformation into the Transformation of Fear.”  It eloquently speaks to the challenges of life transitions through the illustration of being on a trapeze bar. I won’t read the entire piece here, but I will share the last sentence to whet your appetite, “We cannot discover new oceans unless we have the courage to lose sight of the shore."⁸  

Thank you for sharing your time with me today.