4 PerspectivesJuly 9, 2009

Advanced Decision Tree Analysis in Litigation: An Interview With Marc Victor, Part II

For advanced decision analysis in litigation, where do we start?  Last week we began to take our series on decision trees to the next level with Part I of our interview with Marc Victor of Litigation Risk Analysis, Inc., who pioneered the use of decision trees in dispute resolution and litigation in the 1970s.  This post is Part II of that two-part interview, in Q & A format.

Marc, people often say that the “inputs” on a decision tree — the probabilities of various outcomes — are imprecise.  One of the the comments to our first post on decision trees put it this way:

The theoretical problem is with the assignment of probabilities and their meaning. Unless you are just goofing around with numbers, the assignment of a probability to an event presupposes that there is a frequency of similar events to count. This is hardly ever true in litigation, unless restricted to something like employment dismissal cases. Even then, I have trouble interpreting the numbers as anything more than subjective probabilities, i.e. just goofing around with numbers.

Is this a fair criticism?

I don’t think so.  First, it’s not a question of probabilities being “precise” or “imprecise” — the idea is for them to be “realistic.”  Second, assignment of a probability does not presuppose “a frequency of similar events to count.”  A probability is simply a reflection of someone’s opinion of the likelihood of success in a particular situation.  Lawyers have always given opinions like these, even in one-of-a-kind cases — they’ve simply used phrases (such as “pretty good chance”) much more often than they’ve used numbers.  But because it’s very easy to show that the phrases are more ambiguous than the percentages, and because centuries-old probability theory tells us how to combine a 60% chance of success on one issue and a 25% chance of success on another issue to determine the overall chance of success (but doesn’t help us to combine a “pretty good chance” on one issue and a “definite possibility” on another), there are tremendous advantages in using the 0 to 100 scale rather than the “no chance” to “sure thing” scale.  I can’t help but think that if the person who wrote the earlier comment heard his or her doctor say “I’m ‘reasonably confident’ you’ll come through this procedure without any complications,” they would immediately ask for clarification:  “What do you mean by ‘reasonably confident’?  95%? 80%? 65%?”  Why should clients expect less from their attorneys?

Anything else?

Definitely.  The commenter suggests there is something wrong with subjective probabilities — that they are nothing more than “goofing around.”  I have a very hard time equating the rigorous process of developing a thorough List of Reasons (as discussed in Part I of this interview), and then expressing an unambiguous opinion regarding the chance of winning versus losing an issue, as “goofing around.”  Well-reasoned “subjective” probabilities are quite helpful — and are exactly what lawyers (and doctors and senior business executives and others) have always been hired for:  “Based on all of your experience and the information you have available to you counselor (or doctor or senior executive), what’s your best guess of my chances?”  And if the advisor has good judgment, then both theory and practice have shown that using his or her subjective probabilities to calculate probability-weighted average values (“expected values”), and then using these average values to make decisions, will lead to better results across the decision-maker’s entire portfolio of problems over time.  And what’s the alternative?  Refuse to give opinions?  How is a client (or patient or board of directors) supposed to choose between alternatives (like settle or go to trial, have the surgery or hope for the best, invest millions or don’t invest) if they are given no sense at all of their expert’s opinion of success or failure for each of the risky alternatives?

I once had an actuary tell me that, because the future is uncertain, his numbers were almost certainly wrong, but he believed they were less wrong than guessing outcomes with no analysis.  Can the same be said for Decision Tree Analysis?

Definitely!  Identifying the underlying uncertainties that will impact your overall results, making reasoned guesses about each of those, and using proven probability theory to combine the pieces has to be better than no analysis at all.  I like to say that because every defendant’s decision whether to pay $X or go to trial (or, if plaintiff, take $X rather than go to trial) NECESSARILY involves thinking about the chances of doing better or worse at trial, it’s best to make those guesses as explicit and unambiguous as possible.  This allows others to better understand your thought process and how you reached your recommendation, and it allows you to explore how sensitive your decision is to each of your underlying judgment calls.  [Editor's Note:  more on sensitivity analysis can be found at pages 12-17 to 12-18 in "Evaluating Legal Risks and Costs with Decision Tree Analysis," which is reprinted available on the articles page at litigationrisk.com; it also appears in the ACC's Successful Partnering Between Inside and Outside Counsel.

By now we have all seen decision trees, which help us visualize the various turning points in a case.  Are there other ways to graphically represent the results of a decision tree?

It's relatively easy to summarize the results of a decision tree in a graph that orders the range of potential results from low to high and shows the relative likelihood of each.  This is illustrated in most of the papers available at litigationrisk.com.  In addition, sensitivity analysis graphs -- that show how the expected value of litigating varies as the probability of success on a particular issue is varied -- are another useful analytical result easily derived from the decision tree.  Some are illustrated on page 12-18 of "Evaluating Legal Risks and Costs with Decision Tree Analysis" [available on the articles page at litigationrisk.com].  In addition, as clients begin to assess the probability of an event occurring, I often present them with a “probability wheel,” which is shown on page 12-10 of the same article, to help them visualize their chances of winning or losing.  This has proven to produce more realistic assessments.

Who are some of the advanced decision tree users out there?

At this point I shouldn’t mention names, but major oil companies, utilities, insurers, financial institutions, and others are repeat users.

Decision Tree Analysis is often associated with defense counsel.  Are plaintiffs’ lawyers using it, too?

If you mean plaintiffs’ personal injury lawyers, probably very few — though the benefits of doing so apply equally to both sides.  But companies (or even government agencies) considering or involved in litigation will use the techniques to be sure they want to bring the case in light of the often steep costs of litigation, as well as to help plan pretrial and settlement strategies.

Besides lawyers and clients, who else out there is using decision trees in litigation?

We’ve already discussed their use by mediators (in Part I of this interview).  Judges in some cases might also need to use decision trees.  I’m not sure if you have seen it, but you might want to read Judge Posner’s opinion in the Reynolds case [Reynolds v. Beneficial National Bank, 288 F.3d 277 (7th Cir. 2002)], which I cite in my paper “The Role of Risk Analysis in Dispute and Litigation Management” [available on the articles page at litigationrisk.com].  In his opinion, Judge Posner reversed a proposed class action settlement, using in part the following analysis [at pages 284-285]:

[T]he judge should have made a greater effort (he made none) to quantify the net expected value of continued litigation to the class, since a settlement for less than that value would not be adequate.  Determining that value would require estimating the range of possible outcomes and ascribing a probability to each point on the range . . ..

After outlining a hypothetical valuation of a litigation and calculating its net expected value, the court continued:

. . . our point is only that the judge made no effort to translate his intuitions about the strength of the plaintiff’s case, the range of possible damages, and the likely duration of the litigation if it was not settled now into numbers that would permit a responsible evaluation of the reasonableness of the settlement.

And as my coauthors and I wrote in the above-cited article, “If clients and circuit judges now expect risk analyses, judges, mediators, shareholders, and the SEC may not be far behind.  Outside counsel had better be ready.”

Is there a criticism of decision trees out there that you feel is unjustified?

Some lawyers will criticize some decision trees as being too complicated.  Assuming the tree was correctly done, but is still complicated, then I like to say that the tree is never more complicated than their real problem (i.e., than the underlying dispute that’s being modeled).  And I ask whether they think they can do a better job keeping track of all the pieces and combining them to form a sound opinion about case value — (1) by doing it in their head or (2) laying it out explicitly in a decision tree.

Many thanks to Marc B. Victor, Esq. for his time and effort in connection with this interview.  And Marc, don’t think it’s the last time we’ll call . . ..

[UPDATE:  For for more on Decision Tree Analysis, Settlement Perspectives' series on decision trees includes:

1. Decision Tree Analysis in Litigation:  The Basics;
   
2. Why Should You Try a Decision Tree in Your Next Dispute?;
3. Advanced Decision Tree Analysis in Litigation:  An Interview with Marc Victor, Part I;
   
4. Advanced Decision Tree Analysis in Litigation:  An Interview With Marc Victor, Part II (this post);
   
5. Decision Trees in Mediation:  A Few Examples; and
6. Avoiding the Limitations of Decision Trees:  A Few Tips from Mediators Who Use Them.]

Categories: Decision Trees,Negotiation

I welcome comments on this or any post. Please feel free to comment in the “Add Your Own Perspective” area below, whether you use your actual name or pseudonym. If you don’t feel comfortable commenting directly on this site, I can easily be reached by email at jd@johndegroote.com.