Why Police Learn from Third-Party Data

By Randall K. Johnson

Introduction

Does lawsuit data collection deter police misconduct lawsuits? One might think so, judging from recent scholarship on police accountability and deterrence.[1] The best of this work argues that police learn from lawsuit data collection, without actually proving the point.[2] While I agree with the premise that law enforcement agencies may learn from better and more complete information, there is little proof that lawsuit data collection deters police misconduct lawsuits.[3] As a result, additional research is necessary in order to support or to deny this claim.

I modeled and tested this claim in a recent paper: Do Police Learn from Lawsuit Data?[4] My paper introduced a new § 1983 dataset[5] in order to determine if lawsuit data collection correlates with better deterrence of published misconduct cases.  This dataset drew on 10,044 cases that were brought against twenty-six U.S. law enforcement agencies.[6] I matched these published cases with police employment data[7] in order to compute officer-to-lawsuit ratios.[8] These computations were done for all twenty-six law enforcement agencies and three separate groups of departments.[9] After comparing these average ratios, at the individual and group levels, I found that departments that consistently gather lawsuit data do not perform better than other law enforcement agencies.[10] This finding indicates that police may not learn from lawsuit data collection.[11] As a result, law enforcement agencies may need to identify a more promising approach. One approach, which is often overlooked by departments, is third-party data collection.

This Essay argues that third-party data collection, particularly of administrative complaints and departmental audit information, holds greater promise than lawsuit data collection. It does so by asserting that third-party data collection is more useful for three reasons. First, third-party data collection may prevent manipulation by individual police officers and law enforcement agencies. Second, it may assure that police behavioral trends are identified. Lastly, third-party data collection may help to deter published § 1983 cases. This Essay, however, only models and tests the final claim.

I.  Methodology

This Essay models and tests one claim: that police may learn from third-party data collection. In doing so, it draws on the same § 1983 dataset that I used to find out if police learn from lawsuit data collection. As in my earlier work, better deterrence is equated with higher officer-to-lawsuit ratios. Less effective deterrence, in contrast, is equated with lower average ratios. By comparing these average ratios, at the individual and group levels,[12] I found a baseline for each subset and another for the entire population. The baselines helped me to determine two things: whether the departments are a part of the same population and are distributed along a normal distribution.

This approach compliments regression analysis in several ways. First, officer-to-lawsuit ratios provide a simple way to test new hypotheses. Second, this approach shows whether lawsuits have been deterred. Third, officer-to-lawsuit ratios account for differences in department size. Finally, this approach captures the effect of changes in litigation strategy such as no-settlement policies.[13]

The preceding analysis indicates that officer-to-lawsuit ratios may be useful, even with a relatively small population.[14] This approach, however, will not be valid when law enforcement agencies do not meet a minimum “size” threshold.[15] The minimum size, at least in this paper, is 330 officers. These departments also must face more than a nominal amount of published § 1983 cases. The failure to meet each requirement means that a department will be excluded from this Essay’s analysis.[16] These two issues, and other potential problems, are dealt with deliberately, with an eye toward avoiding methodological issues.[17]

Within this context, I evaluate a single claim: that law enforcement agencies with greater access to third-party data are, on average, more effective in deterring published § 1983 cases. This claim is evaluated by determining whether law enforcement agencies with greater access to third-party data have higher officer-to-lawsuit ratios than other departments (with less access to third-party data). This finding will substantiate or deny the claim that police may learn from third-party data collection.

II.  Results

As I stated earlier in this Essay, my § 1983 dataset has 10,044 cases. These cases were published by LexisNexis between 2006 and 2012. I restricted these data by year (2006 to 2012), jurisdiction (federal district court), and cause of action (§ 1983). Next, these cases were matched with police employment data in order to compute officer-to-lawsuit ratios for twenty-six law enforcement agencies.  I also used this dataset to compute average ratios for three groups of departments (law enforcement agencies with access to complaint data and audit data, departments without access to third-party data, and a control group, which has access to complaint data or audit data).  These officer-to-lawsuit ratios are given, individually and by department group, in Tables 1, 2, 3, and 4.

As illustrated in Table 2, law enforcement agencies with access to complaint and audit data had an average ratio of sixty-two to one.[18] Departments without access to third-party data,[19] which are described in Table 3, had an officer-to-lawsuit ratio of forty-three to one.

The control group,[20] which is highlighted in Table 4, had an average ratio of fifty-one to one. When these ratios are compared, it is clear that departments with more access to third-party data perform better than others. This finding supports the claim that police learn from third-party data collection.

Conclusion

This Essay demonstrates that law enforcement agencies with greater access to third-party data are, on average, more effective in deterring published § 1983 cases. As a result, police may learn from more third-party data collection. These law enforcement agencies, however, should avoid situations that distort third-party data. For example, third-party data may be less accurate when regulators and police officers share office space.[21] It also may have limited usefulness when data collection is not done in a timely manner or employs substandard procedures.[22] Lastly, third-party data may be less effective when there are costly barriers to reporting police misconduct.[23]

Fortunately, each of these data-collection issues may be overcome by employing solutions that are grounded in practice. Several examples may be found in legal clinics, especially when law students are used to collect and analyze third-party data.[24] Other examples arise in regulatory settings and draw on public resources, staffing, and expertise.[25]  Lastly, additional examples may emerge over time, especially if new legislation calls for more robust third-party data collection.[26]

In summary, it is clear why police learn from third-party data collection. First, it may provide better and more complete information about the underlying causes of misconduct. Second, third-party data collection may be useful for modeling actual police behavior. Lastly, third-party data collection may help departments overcome heuristic biases and other informational failures.

Table 1.  Background Information for Twenty-Six Law Enforcement Agencies

 Table 2. Law Enforcement Agencies with Access to Complaint Data and Departmental Audit Data

Table 3. Law Enforcement Agencies Without Access to Complaint Data or Departmental Audit Data

Table 4. Law Enforcement Agencies with Access to Complaint Data or Departmental Audit Data

Facebook

Twitter

Linkedin

email

Print