After-Action Reviews: The value of taking stockPrint
By Aileen Leventon | Feb 5, 2018
Client reviews of their law firms may be conducted annually, ad hoc, or when a contract for a preferred firm comes to the close of a cycle. Before conducting a law firm review, it is critical to have a baseline: how well does in-house counsel, legal operations, and the department overall function in collaborating with its law firms? That may depend on how well the in-house function is doing in managing itself. Legal Project Management (“LPM”) provides the discipline that enables lawyers to define, demonstrate and deliver value to their organizations by better balancing the scope of work, time and resources allocated to legal work. The last and often overlooked step in the LPM framework, the After-Action Review (“AAR”), examines how well the legal team achieved these goals.
Make or buy? The ongoing question about legal services
A law department is corporate overhead. Make vs. buy—insource or outsource legal work—is a decision reviewed regularly. Offsetting purely numeric calculations are “soft” considerations such as the value of the internal legal team’s business and institutional knowledge, expertise, management skills, focus on governance, and other factors. Consequently, the AAR of a law department’s work should encompass both hard and soft performance indicators. The specific criteria will vary depending upon the priorities of a business client or the nature of the business risk or objective that requires legal support.
The goal of the AAR is to identify how the legal team delivers value overall and, at a matter level, what works well and should be continued and where there is room for improvement in balancing scope of work, time and resources. Underlying every AAR is the need to answer the question posed by management with regard to all business functions — is the organization well-served by its in-house legal team, should it use outside resources, and if law firms or other advisers are involved, what is the optimal mix of inside and outside resources?
Where to begin
An AAR typically is a joint undertaking between a legal team, including in-house and outside lawyers, and the business client. Nevertheless, when first conducting an AAR, law departments should start by focusing on matters where the in-house team is solely responsible for the work—i.e. outside firms and other third parties are minimally or not involved—and conduct the review without involving the business client. Focus on a matter or group of matters that arise regularly so that the findings of the AAR process are immediately applicable to other work. Identify the client goals as they evolved during the course of the matter in relation to the outcomes. Analyze the causes of variation in those matters. This simple start provides the foundation for tackling more complex matters.
How to do an After-Action Review
The level of detail and formality of the review should be commensurate with the importance of the matter and the expectations of corporate management. Take the time. This systematic review is critical to demonstrate the law department’s commitment to the stewardship of corporate resources. The LPM framework provides the structure.
Can you answer five questions?
The LPM framework calls for legal team members to be able to answer five key questions at the launch of a matter and throughout its course, even in the face of changing circumstances, in a manner appropriate to their roles. The AAR assesses, qualitatively and quantitatively, the alignment, gaps or inconsistencies in the team and client’s understanding of:
- WHY we are handling this matter for the business—why is it important to the company’s strategy and operations?
- WHAT legal strategy will accomplish the business goal and what legal work is out of scope?
- WHEN and in what sequence must work be done and what are the uncertainties and dependencies regarding timing?
- HOW do we manage the work, given constraints and requirements such as budget, strategic importance, regulatory and reporting obligations, timing, and risk management protocols?
- WHO is involved in the work and what is each person’s role and responsibility?
The answers to these questions have consequences—positive, negative and neutral—for resource allocation, work plans, scope of work, stakeholders, roles and responsibilities, teamwork, training, communications throughout the project, status reporting and client satisfaction. Not all observations flowing from the AAR will require action; some may require more attention.
Do you have all the facts?
An AAR is a fact-based discussion that requires data. The primary source of data should be the information that grounded the initial planning for the matter and updates to it. Data-gathering processes, checklists, guidelines for outside counsel, precedents, invoice review processes, timelines, staff allocation/management systems and report templates should already be in place to permit this kind of look-back and comparison to the initial plan and similar matters. If the data are unavailable or unreliable or it is difficult to do this kind of analysis, a lesson is quickly learned: action is required to improve information quality.
Analyze the facts
After analyzing the information, address the following issues from an LPM perspective to determine what factors fostered or detracted from efficiency and the client’s view of value:
- What did we intend to accomplish at each point in time and why—what were our goals
and our strategies?
- What did we actually do, who did it and how did we implement our strategy?
- What were the differences between our intended goals, strategies and execution and
what we actually did? Why were there differences? What was the root cause of both distress and success?
- What could we have done to improve the implementation of our plan?
- What were the differences between our approach, the outcome and the client’s expectations?
- What were our successes and how do we repeat them and export lessons learned to other matters?
Look for what worked well and should be replicated as a best practice. “Good work” should result in retained knowledge, new action plans, training opportunities and templates for use in other matters. Watch the tendency to be hypercritical—find the positive. Also watch out for regression to the mean; consensus does not necessarily mean that a common practice is a “best practice.” Strive to raise the bar. Understand how your work product and advice meet a range of client objectives. Ask for candor about what is important to internal constituents—how they define predictability, accountability, risk and value.
Look at what caused tension among the team and with the internal client. Fertile areas for review are those characterized by rework, duplicative effort, misunderstandings, “surprises” and poor communication and the effort to get back on track. Focus on the root cause of variances from plan and the steps taken to recover. Shifting stakeholders may have materially changed the scope of work and strategy. Concern about business reputation or relationships with suppliers and customers may have emerged to eclipse the legal team’s approach and required a dramatic change in plans.
Identify, too, the root cause of problems leading to the need for the legal work. Part of the law department’s mission is to use its knowledge of the business to prevent legal problems from arising in the first place.
What might you find?
A recent AAR revealed significantly different staffing utilization and time lines for closing acquisitions of similar properties depending upon which lawyer handled the matter. The variation primarily resulted from how each lawyer utilized paralegals. Often differences in processes, time lines and cost arise depending upon which lawyer is handling the matter or the experience of the businessperson. Facts, not impressions, are needed to document and address the causes and implications of variations.
Another AAR found inside counsel often gave the business client insufficient notice for document production because of the absence of a sound record-keeping and calendaring practices. Look at what caused tension during a matter and identify steps to alleviate the root causes going forward.
Once the in-house team is familiar with the AAR process and is making improvements and establishing best practices, progress to reviews of matters where outside law firms play a significant role. Law departments that conduct regular self-assessments generally are able to communicate more effectively and learn more from reviews their law firms. And after getting a good handle on the inside-outside relationship and dynamics and determining how the combined legal team will present itself to the client, then bring in the clients.