Alternative exploratory research designs in litigation - part 2
Part 2 of 3
In part 1 of this topic, we introduced the concept of alternative formats for Exploratory Research or “different types of Focus Groups” and detailed the specifics of a traditional Focus Group. In part 2, we describe two alternate Focus Group designs: Graphics Research and Juror Profile Research.
I. Special Designs – Graphics Research
Especially in cases that are fact-intensive, it becomes strategically imperative to formulate graphics with optimal educational and persuasive content. The critical role of well-designed graphics in maximally effective trial strategy has generally become self-evident, although some litigators cling to the belief that hand-drawn charts created on the fly “communicate better” (they do not). In some instances, however, the need to identify precisely how to best present complex data becomes so involved or ambiguous as to necessitate an Exploratory Research design that exclusively investigates demonstrative exhibits and their overall impact.
For example, in the Exxon Valdez litigation, the marine biology issues arising in conjunction with the claims by fisheries, seafood processors, and Native Alaskans generated the production of over 500 graphics by the defense team to communicate all of the issues related to fish and bird populations, water purity, and dissipation of crude oil, to name a few. With nearly a dozen experts having over 40-50 charts and exhibits each, the need to cut down the number of exhibits at trial became a key impediment to forward progress in trial readiness.
In a classic case of information reduction, Graphics Research was undertaken to reduce the number of charts and simultaneously elicit the feedback needed from test respondents to whittle down the stockpile of exhibits to under 100. Added to the need for information reduction, however, was the additional requirement that the trial team “meet the jury where they are” in terms of comprehension and assimilation of technical data. In short, there was a tangible need to have fewer and more persuasive graphics.
The design parameters used in this research, and recommended generally, are as follows:
1) First, recruit a suitable sample of jury-eligible test respondents (30-50) who are representative of the trial venire;
2) Develop a) introductory or “basic” presentations for each side that provide essential information about the case as background; and b) short (2-3 sentence) vignettes to explain the significance of each graphic or demonstrative exhibit;
3) Implementation, consisting of a) giving to respondents the “basic presentations” and subsequently, b) graphic by graphic, providing the explanatory vignettes in conjunction with each chart or exhibit while simultaneously administering c) a measurement device (questionnaire) that gauges their reactions to each chart or exhibit, both in terms of i) clarity and ii) persuasive impact (other parameters may be measured as well, such as “educational value,” “appropriateness” and so on); and d) a short focus session with each graphic (or in clusters of graphics to save time in situations where there are many hundreds of graphics to assess), in which respondents are queried as to their reactions to the individual charts and exhibits.
Data from the these “mini focus sessions” in d) for each exhibit (or cluster) provide the qualitative insight as to how to best modify, refine or re-design each graphic to maximize impact, clarity and overall “juror friendliness” of the exhibit – that is, meeting the jury where they are.
Charts and exhibits developed by experts can often be complex and lacking in “jury friendliness.” By contrast, using Focus Group queries developed to ascertain what jurors need to learn and understand, results obtained from each chart or exhibit can be used to enhance communicativeness, facilitate recall, and maximize overall persuasive content as the charts or exhibits are modified based on the findings from the project. Moreover, the quantitative data from the measurement devices (questionnaires) can be used to rank-order exhibits in terms of their overall lucidity, persuasiveness or according to whichever parameters have been implemented in the measurement instruments. Such rank-ordering generally reveals where the best charts are, what styles and approaches to presenting data are most effective, and what not to do by looking at the bottoms of the lists.
II. Special Designs – Juror Profile Research
Often a case “is what it is” – there are no substantial means of “spinning” the case fact scenario -- and the real challenge is in picking the right jury. The issue of jury selection entails sophisticated scientific inference, as one is necessarily involved in the prediction of behavior: Will this juror vote one way or the other? How strongly will this person advocate a position in deliberations? The need for “sophisticated scientific inference” in turn requires a consideration of a methodological approach to carrying out valid research to accomplish the objectives of prediction.
It is worth mentioning that those who decide to use this type of research are perhaps the most sophisticated of litigation teams who recognize that effective jury selection is much more than “I like this person” or “I don’t like that person.” Prediction of behavior is, in fact, the highest level of scientific achievement, posing substantial challenges even for the most experienced of research psychologists. Accordingly, specific research design objectives must be met in order to obtain information that can be relied upon in the courtroom.
The use of “Focus Group” research in this instance, as Exploratory Research, may be a bit misleading, but the nature of the methodology ends up resembling Focus Group research in the final analysis. The first methodological issue that requires consideration is the need for large samples of test respondents. In order to scientifically delineate valid juror profiles, one first needs to be able to identify the degree of risk associated with various categories. For example, if one needs to be able to assign a degree of risk for, let’s say, Hispanic females versus Hispanic males, it is necessary to have a sample of respondents from each sub-group to determine how that sub-group is likely to vote. When more precision is needed – for example, differentiating married from divorced Hispanic females, and then again for Hispanic males – the number of test respondents required becomes even larger. Without delving into the complex topic of sampling techniques, it suffices to point out that, in this type of research, generally a sample size of test respondents exceeding one hundred is required for the determination of useful juror profiles.
In practice, research projects of this type follow the steps outlined below:
1) First, select a group of about 120-140 respondents comprising representative cross-sections of jury-eligible persons from the venue (i.e., ensuring a representative cross section of ethnic, marital, age, educational and other similar categories);
2) Prepare videotaped presentations for each important party that capture the essential claims and responses in the case; these presentations should normally be about 45-60 minutes each, based on the key issues and evidence involved in the litigation;
3) Schedule multiple Focus Group sessions of about 4-5 hours each, with each one using 20-30 test respondents; thus, using a sample size of 120, four Focus Groups of 30 subjects each or five groups of 24 subjects each would be optimal (while it would in principle be possible to use three Focus Groups of 40 subjects each, it should be remembered that, in order to conduct focus sessions with group discussion, the number of respondents in each group should not be too large);
4) Develop a comprehensive pre-test that includes measurements of any and all relevant pre-existing characteristics (demographics, experiences, beliefs, attitudes and personality variables) that will or could be considered in actual jury selection. Thus, for a toxic case, one might inquire as to hospitalizations, illnesses, past toxic exposure, diet and health behaviors, etc. This pre-test is a comprehensive and labor-intensive amalgamation of measurements that could conceivably be instrumental in differentiating plaintiff and defendant (or low-damage versus high-damage) jurors;
5) Develop a post-test administered after respondents have seen the presentation tapes that assess verdict and damages dispositions (along with any other case-related queries that may be useful);
6) Implementation, consisting of (for each of the multiple 30-40 member Focus Groups used) a) administration of the pre-test in (4) above; b) subjecting test respondents to the presentations in (2); c) administration of the post-test in (5); and d) focus sessions in which test respondents provide their opinions of the case.
7) Data Analysis: Using various statistical correlational techniques, ascertain the existence of significant relationships between the pre-test variables measured in (4) and the verdict and damages outcomes measured in (5) to derive juror profiles.
In this research, the availability of “Focus Group” data – normally, the comments offered by respondents describing their reactions to the case themes and issues – is essentially a by-product of the process of obtaining the juror profiles. That is to say, the real “meat” of the project is the considerable work involved in the steps above leading to the profiles ((4), (5), (6 a-c) and (7)), while the use of focus sessions (6 (d)) is an added “bonus.”
Thus, while the projects appear to be typical Focus Groups on the surface, the real intent is the derivation of reliable juror profiles for use in jury selection through statistical correlational analyses. However, these projects are extremely productive in that they provide both the profiles and the various substantive findings that become available in the utilization of multiple focus sessions.
Juror Confirmation Bias