Cautious optimism for a comprehensive performance audit at TPS

I attended the Toledo Public School Board's Finance Committee Meeting yesterday and left cautiously optimistic about getting a comprehensive, top-to-bottom performance review for the district.

At the direction of Dr. Cecelia Adams, the chairman of the committee, Treasurer Matt Cleland reached out to several of us who have been vocal about the issue and invited us to attend the meeting to hear presentations from three entities under consideration for the performance audit.

The Council of the Great City Schools, the State Auditor and Evergreen Solutions, LLC, were the presenters.

I've detailed my concerns about CGCS and nothing in their comments yesterday changed my mind.

TPS pays dues to belong to CGCS and participates in their board. Clearly, this presents a conflict of interest to hire a group that you belong to as an evaluator of your performance. Doing his presentation via phone, Michael Casserly, the Executive Director, addressed this concern, saying that prior reports they've done are very objective.

"I encourage people to look at any reports done over the years to see that they are extremely candid," he said.

Casserly described their approach as "reviews" of specific areas of function such as budget and finance, transportation, food services, or IT systems. While cost depends upon scope, he estimated roughly $30,000 per review with a 'broad review' typically costing around $100,000.

He emphasized that the costs are only for travel, hotel, meals and other expenses because the individuals used on the different subject teams are already employed in various senior urban school positions.

Casserly said they do not use a template for their reviews but do apply best practices and look at the "difference between why some schools do better than others and why reforms ... aren't getting the traction necessary."

In response to a question about union contracts Casserly replied with a very disturbing answer. "Sometimes we look at collective bargaining agreements if we think it has an impact on the instructional or operational review scope." He added that they will look at contracts "if asked."

He said they don't do follow up after the reviews, but they can refer the district to others who can help with implementation of the recommendations.

Board member Bob Vasquez asked if the Council had worked with other entities. For example, had CGCS done a review of transportation while another entity did a review of budget and finance? Casserly said they had, but emphasized that entities like states who do financial audits are usually looking at compliance with various laws and regulations and that having two groups pursuing the same goal but on two separate paths often takes up staff time.

My impression is that he was referring to a yearly financial audit and not a performance audit like our State Auditor offers.

Casserly explained the "main job is to identify and look at underlying structures and strategies that help or impede the results" the district is trying to achieve. He had no details about cost savings obtained but estimated between $2 million and $20 million in operational efficiencies.

The Auditor of State presentation was more of an overview of the performance audit services they offer. They look at all aspects of the operation with a focus on "skinnying down government." They use Lean Six Sigma and best practices, comparing an entity to public and private services when applicable. They emphasized that what works in one place might not be good in another.

The Auditor's average return is $23 in savings for every $1 spent on the audit. Their PAs cost around $100,000, depending on scope but they do have a revolving loan fund that can advance the district the cost of the audit, so long as it is repaid within one year of the audit completion. Using the LEAP fund, as it's called, allows districts and other entities to have time to implement recommendations and acquire the savings to pay for the audit.

The report they do will note any recommendations that are subject to labor contracts and negotiation so that everyone reading the report will have realistic expectations regarding the implementation of those recommendations.

Evergreen Solutions had not been invited to present. They'd heard about the discussion of a performance audit and contacted the board, asking to present their services. That companies specializing in school performance audits had not been contacted in advance was very disconcerting, but the presentation was top-notch.

Dr. Linda Recio, president of the firm, gave the most professional and comprehensive presentation of the methodology, tools and process they would employ in doing a performance audit.

Their PA team consists of individuals with extensive background in education and prior service in leadership positions in school structures. "We've been where you are," she said, but are now full-time consultants who can be completely objective in evaluating and making recommendations. She said that when they look at school budgets, they don't just bring in a former school treasurer, but a former school treasurer who is also a CPA.

She emphasized their knowledge of various state and federal laws to ensure that all recommendations are in compliance with mandates and requirements.

She brought a copy of the model structure guidebook they use for their PAs as well as a copy of the final report they did for a Virginia district. She said she thought it was important for everyone to see a sample of their work and the product they would receive.

Recio emphasized the constant, web-based interaction with both internal and external stakeholders; their hands-on approach which includes actually riding the bus and eating in the cafeteria; and their interview techniques.

"How do you hold yourself accountable for how you're spending public dollars," she asked, explaining that a comprehensive, top-to-bottom performance audit is key.

She said she would prefer to wait until after the first of the year to start any PA because she didn't think between Thanksgiving and Christmas was a good time to get people to focus on the tasks necessary. She said they could do their comprehensive PA in three months, which is half the time of other estimates.

Recio also said they do not charge extra or refer the district to others to help with the implementation of their recommendations. They provide not just recommendations, but detailed plans and strategies for how to implement their suggestions.

She estimated the cost of a 'the works' audit at "no more than $150,000, probably less" and said the audit of the Virginia school district, similar in size to Toledo, was just under $150,000.

In response to a question she noted that they've done audits of districts at the request of various states and that states have found 85-90% implementation rates. She also said the return on investment for their audits was $193 in savings for every $1 of audit costs.

My impressions:

* Dr. Adams does a very good job running a meeting. She was firm, but kind, in how she praised the progress staff was making on a goal of eliminating "confirming purchase orders" while insisting on further identifying the problems resulting in having them still on the agenda. (These are purchase orders that have been issued AFTER the action or service has been performed and they were a finding on the TPS 2011 audit.)

Her questions to the presenters were thoughtful, detailed and caught a couple of questionable statements that were made.

* Bob Vasquez was clearly thinking 'outside the box' (though he said at one point that he really didn't like that term) as he asked about having the presenters work together - perhaps using one to focus on financial aspects and another to focus on instructional aspects.

* Both Adams and Vasquez clarified that their resolution to pursue a contract with CGCS was not a commitment to hiring them for a performance audit - that a final decision on who will do the audit and the scope of the audit had not yet been finalized. They called on Keith Wilkowski, legal counsel to the board, to verify that was the case.

* Despite saying that they'd done more non-instructional audits than instructional ones, my impression of CGCS is that they are more likely to emphasize and focus on the instructional side of the operation.

My previous concerns about them still stand and I now have more. If I understood correctly, they use current employees in member schools to do the PAs. This pulls those individuals away from their regular duties and, while they are probably good people with good skills, they are more of a peer review rather than independent experts. Would they be more like co-workers than evaluators?

* The State Auditor was my preference prior to this meeting. I do believe they can do an excellent and comprehensive job of evaluating the district (both instructional and operational), but they are limited in their ability to help TPS in the implementation stage. This is not from a lack of desire to see success, but more from the structure of state government and their duties and limitations within that structure.

* Evergreen was the best of the three presentations and gave the best options to the district. They - or a firm like them - is now my preference, though their initiative in reaching out to the district to offer their services moves them to the top of the list.

Recio even had some free advice for the board for after the audit: hold quarterly meetings with the public where you share your progress on each of the recommendations. She said they provide a spreadsheet template boards can use to show the original recommendation, any modifications, percentage completed, savings achieved as well as other items the board might want to track.

I thought this was an excellent idea that would demonstrate how the board is following through and I hope they do that regardless of who is selected to do the PA.

* At one point Vasquez and Adams asked about needing property tax levies in the future. Evergreen noted that they are not a PR firm, but that their clients had successfully used the PA and the implementation of the recommendations in their efforts to educate the public about the finances of the districts. Vasquez wondered if the public would support a levy request in the future.

In response, John Mc Avoy, with the Northwest Ohio Conservative Coalition, told the board members that NWOCC was not opposed to school property tax levies in general, but they want to be sure entities requesting funds truly need them and are already using existing funds in the best possible manner. He shared that he, personally, decided to support the Lake School District levy after seeing their performance audit and the progress they were making as a result.

I told the board that support for future levies was never guaranteed, but that they couldn't spend three or six months doing an audit and then, two months later, ask for a new levy. I told them if they do the top-to-bottom, comprehensive, 'the works' audit, spend a year holding regular meetings showing how they're implementing the recommendations and achieving the targeting savings, and then they find they still need money, even I would probably support a levy.

But the key will be in demonstrating to the public that they are following through.

I also believe that if they do such an audit, they might not need a new levy for an extended period of time.

* Overall, I thought this was a very productive meeting and while I'm cautiously optimistic about the PA, I still have some questions:

- If they want to go with an outside consulting firm (like Evergreen), do they have to issue an RFP and/or go out to bid for the project? I don't think they have to if they use a CGCS or the state auditor.

- Are there board members who are committed to using CGCS and will find a way to do so regardless of better options? I didn't think they were the better choice to begin with and am even more convinced of this now.

- Will the openness I saw from two board members at the finance committee meeting carry over into regular board meetings and move TPS toward a comprehensive performance audit?

- As board president Lisa Sobecki asked previously, will they have the will to implement any recommendations they get?

- Will the TPS unions, especially the teachers and administrators, embrace the process and the outcome?

The ball is not really in the board's court right now - it's in ours. We need to continue to pressure them to stay on this path and to help them make good decisions along the way.

The next board meeting is Tuesday, December 18, at 5:30 p.m. It's in the Board Room at the Thurgood Marshall Building at 420 E. Manhattan Boulevard in Toledo.