Risk Assessment in a Dynamic Environment
by Cathy Lockman
University of Illinois accountancy students got an up close and personal look at the work being done by the Public Company Accounting Oversight Board when Joe St. Denis, the director of the Office of Research and Analysis at the PCAOB, visited campus for the recent Department of Accountancy Lyceum. A CPA who began his career as an auditor at Coopers and Lybrand, and later served as assistant chief accountant in the Division of Enforcement at the Securities and Exchange Commission during the challenges of the dot com crash and the Enron scandal before joining the PCAOB, St. Denis shared his expertise on the role and functions of the ORA and his insights into how the PCAOB works to advance the preparation of informative, fair, and independent audit reports.
Created by the Sarbanes-Oxley Act of 2002, the PCAOB is a private sector, nonprofit corporation that oversees auditors as a way to protect the interests of investors. St. Denis explained that his office, the ORA, supports the oversight function by providing risk-assessment, surveillance, business intelligence, and policy support to the various divisions of the Board.
Central to that mission is assisting the Registration and Inspection Division in identifying which audits to inspect each year. The process, he says, involves some sophisticated data-mining techniques and analysis.
"The PCAOB can only look at about 10 percent of the audits completed each year, so we do a risk-based review to provide information to the Inspection Division as they make their selections for both the annual and triennial reviews," says St. Denis. The Business Intelligence group in the ORA pulls together all the data and engages in "hand-to-hand combat every day in getting the data into a format where it can be useful," he says. Then the Surveillance group gets to work, screening the data, analyzing the company, and writing a "referral memo," that summarizes all the risks that inspectors should be concerned about when undertaking their work.
"The referrals provide a kind of actionable intelligence for the inspectors," explains St. Denis. "Our surveillance analysts are people who have been in the audit industry for a decade or more. They have a natural skepticism that serves them well in their role, plus they have the benefit of having information about the accounting firm that is unique to the PCAOB. Each issuer audit is run through screens and when certain thresholds are hit, flags are raised, which trigger further analysis by our surveillance team. This is our internal risk application model."
The ORA employs a quant model as well. "This model takes 115 dimensions and performs a data-mining routine to look for patterns along these dimensions," says St. Denis. "It's a kind of profiling software that allows us to find patterns of companies that have restated earnings in the past, for instance. It provides a list of issuers that the quant model determines has the highest probability of restatement. It's a quantitative analysis not an intuitive one, and we have to evolve our quant models after a couple of quarters."
How effective are the PCAOB's efforts to uncover audit problems? St. Denis says one measure that indicates success is an assessment of "negative events," such as restatements, litigation, and SEC investigations. "The negative event rating has been going down over the last three years," he says. "We're happy to take some credit for the lower incidence, but we know there are lots of things we can't control."
For those who question why the PCAOB doesn't conduct a random sample of inspections, St. Denis explains that they would need three times the number of inspections, and thus inspectors, to complete a true representative sample. He also explains that there are some "automatic" inspections, with the top 120 companies in market cap inspected annually.
Another role of the ORA, according to St. Denis, is a peer-reviewed research function. "We have a great deal of proprietary data that we can't share, but we try to bring something to the discussion by engaging where we think we can add something useful," he says. "For example, we have examined the impact of PCAOB inspections on client turnover at firms that are inspected triennially. What we've found is that when our inspection reports find a problem or a deficiency, those firms tend to resign from those issuers. Perhaps they're concerned about their reputation or compliance issues, or they may have been trying to avoid comments in future inspections."
On the other hand, PCAOB inspection reports do not seem to be correlated with issuers' decisions to dismiss auditors. In fact, St. Denis says, "Firms that received at least one comment were significantly more likely to receive new issuer audits in the period following the release of PCAOB findings. Overall, we have some indication that there is a demand for firms that are perceived to provide lower quality audits. Of course, it could also be that there is a demand for lower cost audits, and that there is a correlation between audit fees and audit quality. This is a hypothesis that we will be testing further in the future."
In addition to explaining the inner workings of the ORA, St. Denis also shared with students his thoughts on the role that emerging GAAP issues have to potentially change the industry. Of particular significance to auditors, he says, is the issue of fair value. "This is a real sleeper that could put auditors in a vulnerable position. They will have to understand a lot of the dynamics in the market that aren't obvious."