Saturday, February 16, 2013

UCPD records first 2013 violent crime - Why did it take so long?

The 2:30 PM "Battery" incident of 2/13/2013, in which an unprovoked male pushed a subject to the ground and fled on foot, marks the first violent crime recorded by the UCPD in the new year. In all that time, UChicago's police department has recorded zero robberies, zero assaults and batteries, and even zero of the dubious "Theft From Person" incidents. Did Hyde Park just have no violent crime? Nope! A simple query of city data shows that the CPD, in roughly the same patrol area as the UCPD, recorded 22 violent incidents. So why does the UCPD have a big fat 0 in that area?

Of course, as an earlier post of mine discussed, the UCPD Incident Reports are far from comprehensive; by its own admission, the Department only records crimes that are directly reported to their phones, not those called in straight to the CPD. In previous years, we always expect to see the UCPD reporting only a fraction of crimes that are reported to the CPD. But 0? That just seems too unusual.

In this piece, I try and answer some key questions about this apparent anomaly. Is the UCPD "juking" the stats, or it is just out of the crime-reporting loop? Is Hyde Park crime really so much better today than in past years? Or is random statistical error to blame for all these differences? Analyzing the last 5 years of CPD and UCPD data, I will offer three takeaway points.
  1. Hyde Park violent crime has actually dropped over the past 5 years!
  2. But crimes reported by the UCPD have also significantly dropped over the past 5 years... 
  3. To explain this decline, we have to turn away from some obvious answers and find some more nuanced ones. As will be shown, the best explanation has nothing to do with crime but everything to do with small changes to the UCPD structure itself. 
(NOTE ABOUT DATES: Why look at just the past 5 years, 2009 through 2013, and not a wider range? The UCPD only has full incident report data from January 2009 through February 2013. Its 2008 incident reports don't start until July, and it has no data whatsoever before that. Because we ultimately want to assess the accuracy of the UCPD data, we need to make sure we are comparing apples to apples.

Additionally, observant readers will notice that the first battery happened on 2/13, but subsequent analysis only looks at the 1/1 through 2/6 range. Why omit the last few days? CPD data has a 7 day backlog, and I don't want to wait until 2/21 to run an analysis that has relevance right now). 

In conducting this analysis, I compared UCPD violent incident reports to CPD violent incident reports in the same period and geographic location. The goal was to find similarities or discrepancies between their numbers. Here is a brief discussion of my methodology: 
  • Violent crime defined broadly
    The UCPD, CPD, and other criminologists like to define "violent crime" as aggravated (armed/deadly force) batteries/assaults, homicides,  robberies, and sexual assaults. Although these crimes are all certainly violent, this definition excludes 2 other categories of crime that need to be included. The first is "simple" battery and assault, incidents in which the offender was unarmed and using non-lethal force (pushes, punches, shoves, angry words, etc.). The second is sexual harassment or other inappropriate sexual advances, such as groping, touching, threats, and so on.
  • CPD Data from CLEAR system
    As usual, all CPD reports are from the City of Chicago Data Portal. The crime dataset being used "reflects reported incidents of crime (with the exception of murders where data exists for each victim) that occurred in the City of Chicago from 2001 to present, minus the most recent seven days. Data is extracted from the Chicago Police Department's CLEAR (Citizen Law Enforcement Analysis and Reporting) system" (Source).
  • UCPD Data from Daily Incident Reports
    All UCPD reports are taken from the Daily Incident page on the UCPD website.
  • No "Domestic" incidents
    There are a lot of reasons for excluding domestic incidents, but two are most relevant for this analysis. For one, domestic incidents are rarely called in to the UCPD. That tends to be a CPD affair and the number show it; in the past 5 years, the UCPD has only reported 2-3 domestic incidents. In the same area, the CPD recorded over 1,000. Clearly the UCPD is not in the business of investigating domestic violence, and it is unfair to count those numbers against them if the victims are just calling 911.
    Second, there are a lot of criminological differences between intimate partner and domestic attacks and "street crime". The UCPD is known mostly for its patrols, and the majority of Chicago domestic crime occurs indoors, not on the streets.
The accompanying map shows the geographic area under consideration. This is an area in which students and UChicago community members live, and most importantly, an area in which the UCPD patrols and places blue light phones. It is unfair to hold UCPD units accountable for crimes that occur outside of this area because that is not where they patrol. From a neighborhood perspective, this area includes all of Hyde Park plus the northernmost part of Woodlawn and the southernmost areas of Kenwood.

As a final note, yes, I know that the UCPD claims to patrol from 63rd to 39th street, but we all know that in practice this is untrue. We can have a separate conversation about their responsibilities to the community and their successes/failures in that regard, but for now, we want to be as conservative as possible. That means holding them responsible only for the area in which most of their units are, an area that is likely to call the UCPD, and an area that is widely considered to be Hyde Park and the University of Chicago.

I cleaned up the CPD and UCPD data to reflect the criteria mentioned above (all violent crimes, no domestic incidents, and only those crimes occurring in the geographic area pictured above).

Having established the methods, I can now go on to the possible explanations for this supposed lack of violent crime. 

The most obvious explanation for the low 2013 numbers is that 2013 is just less less violent. Perhaps Hyde Park has had such a drop in violence that there is no need to call the UCPD any more. Sure, some residents might still need to call the CPD from time to time, but under this theory, violence has dropped so much that the UCPD just has nothing to do around Hyde Park anymore.

How can we confirm or deny this explanation? To start, we need to look at UCPD incident reports over the years to see if 2013 is the continuation of a downward trend. The following graph shows violent incidents reported by the UCPD in the Hyde Park area. Without any statistical analysis, it is pretty clear that, at least according to the UCPD, violent incidents have experienced a massive decline.

That's a promising trend! In fact, if we were to see that trend anywhere else, we would probably just declare that crime has been solved!

Except that's the problem. This trend is TOO promising. If the UCPD Incident Reports were the only measure of Hyde Park violence, we might conclude that area violence has fallen by as much as 100%, or at least 50% if we still want to exclude 2013 as an anomaly. Any critical reader who looks at that graph is going to be extremely suspicious of UCPD data, and I am in full agreement with their skepticism. Recall that UCPD reports don't reflect all incidents; just those called in to the Department.

To get a badly needed second opinion, we need to move beyond the potentially biased UCPD statistics. We need to look at the total violent crime in the area and see if it actually dropped over time. A quick glance at the CPD crime numbers, a much more comprehensive source, show that Hyde Park is indeed a lot safer this year than in previous ones. The drop isn't as outrageous as in the UCPD would have us believe, but it is still clear.

Again, we see a pretty clean drop in violent crime starting in 2010, further confirming the relative safety of our neighborhood and its improving crime conditions. These CPD numbers might give further credence to the idea that  Hyde Park is less violent.

It is thus tempting to write off the 2013 anomaly, and all the other UCPD incident reports, as the natural consequence of declined violence. After all, we know that the UCPD doesn't have all crimes called in by local residents. In a year with less violence (like 2013), there would necessarily be fewer police calls. Similarly, there would be fewer criminals on the street. Given the drop in overall Hyde Park violence, we might expect to see a comparable drop in UCPD incident reporting.

But as you can guess just by looking at the numbers, the drop in UCPD reports is just too steep to be explained away. We need another way of looking at this problem to get some answers.

It is not enough to look at UCPD reports or CPD reports individually. To really figure out what is going on, we need to compare and overlap the two datasets. That is to say, we need to look at the ratio of UCPD reports to the total crime in the area. 

Why are we doing this? We want to look at UCPD reporting behavior over the last 5 years and see if it changed or remained consistent. If it was more or less constant, then 2013 might not be an anomaly; in a year with low violence, we would also expect to see fewer UCPD reports. But if UCPD reporting has changed somehow, then 2013 could not be explained away. 

So let's think about ratios. from January 1 through February 6 of 2013, for every 22 crimes reported to the CPD, there were 0 reported to the UCPD; a 0:22 ratio. We can simplify that ratio to 1/20 to be generous. Thus, if we were to look at January and February in other years, we would also expect to 1:20 ratio (or something similar) of UCPD to CPD incidents. In conducting that analysis, one of two things will happen:
  1. If we found that the ratios were about the same in each year (roughly 1:20), we might conclude that nothing suspicious is happening. In that case, 2013 might just be a really nonviolent year. 
  2. But if we found large differences between ratios (1:20 in 2013 but 1:3 in another year, for example), we will need another explanation as to why 2013 had so few reports.
The graph below illustrates the ratio of violent crimes reported to the UCPD to the total violent crimes in Hyde Park (taken from CPD data). In many respects, this graph just combines the previous two in this article. The pink-shaded part represents those crimes reported by the UCPD. Ratios of UCPD reports to total crime are given above each column. Total crime for each period is given to the right of  columns.

As can be seen from the ratios, our suspicions are confirmed: 2013 is definitely a huge statistical anomaly:

In 2009, of the 49 crimes that occurred in the Hyde Park area, the UCPD reported and responded to 24 of them: A 1:2 ratio to use our terms. That's actually quite commendable and suggests a high level of community involvement. In 2010, 2011, and 2012, that ratio plummeted. We see values at 1:5 and 1:4, showing that the UCPD started reporting less than 25% of all violent crimes in the area. Even so, however, the UCPD was still responding to a fair number of crimes and reporting them in its daily log.

And therein lies the problem: 2013 is still the lowest ratio by a huge margin. 1:4 and 1:5 may not be the best reporting rates, but they aren't nearly as bad as 0:20 (or 1:20, our generous estimate).

A statistical test confirms this anomaly. Running a chi-sqare test on the data, we find that 2013 had significantly fewer reports of UCPD violence (p < .01). How are we to interpret that statistical statement? If UCPD reporting ratios were unchanged in 2013, there would only be a 1% chance that we would have seen 0 crimes reported given that there were 22 total violent incidents in the month. Because that is just so improbable, we have mathematical reason to believe that something else is going on and that UCPD reporting behavior really did change.

This leaves us scratching our heads. What is going on with the UCPD reports? We all understand that the UCPD does not record crimes that are called directly to the CPD, but at least in previous years, we would expect to see the UCPD recording at least 20% of all the incidents in the area. Hyde Park violence did not drop so steeply as to expected 0 reports. So what caused the Department to record 0 in 2013?

Now that we have strong statistical reason to suspect that something else is going in with UCPD reports other than just a reduction in violent crime, we are pressed to come up with plausible explanations. 

Let's start with some of the more obvious explanations. At least, "obvious" in the sense that we are tempted to use them without thinking critically; as will be shown, none of these explanations merit any serious consideration.
  • The UCPD is deliberately removing violent crimes from the Incident Report to improve its numbers and the appearance of UChicago safety!!
    VERDICT: Highly unlikely
To even entertain this notion we need to be wearing some rather heavy tinfoil hats. Stat-juking is certainly common in police departments across the country, but it almost always takes the form of crime reclassifications, not outright omission. Now, there is a separate issue here about whether or not the UCPD is downgrading crimes; "Theft from Person" sounds suspiciously like "Robbery" to me, especially when there is physical contact involved. But in this period of January 1 to February 6 in 2013, there wasn't a single crime in the incident report that could have been violent before being reclassified. So at the very least, there was probably no reclassification going on. As to blatant lies, it would just be too risky for the UCPD to exclude violent crimes from its report, especially if the victim had an internet connection. In this information age, it would be very easy to expose such obvious malfeasance, the consequences of which would be serious for both the UCPD and its parent institution. For those reasons, this explanation seems exceedingly unlikely.
  • The UCPD is no longer responding to emergency calls for violent crime. Instead, the UCPD forwards them on to the CPD.
    VERDICT: Unlikely
Under this theory, the UCPD is actually still receiving violent crime calls even in 2013, but is then deciding not to deploy units to the scene. Instead, they forward the calls to the CPD to keep the crimes off their own ledger. This is a lower tier of malfeasance than the explanation above, but it is still a serious allegation. Thankfully, all evidence points away from this explanation. First of all, there are almost assuredly professional and legal regulations that require the UCPD to respond to violent crime in its jurisdiction. The UCPD would risk disbandment and devastating legal action if it deliberately violated that mandate. Second of all, this assumes a high level of institutional competency by both the UCPD and, especially, the CPD. The CPD barely has enough officers to respond to its own 911 calls in the worst Chicago neighborhoods. They definitely have neither the desire nor manpower to pick up the UCPD slack in one of the safest Chicago community areas. And finally, the UCPD loves going after bad guys. Whenever a violent incident is actually reported, there are at least 4-5 UCPD cars on the scene within a few minutes. Part of this is safety protocol, but part of it is also the excitement in going after the real criminals. For all these reasons, it is quite improbable that the UCPD is just passing the buck to the CPD.

The next two explanations have to deal with the specific details of crime, namely the location of crimes and the actual type of crimes. Unlike the previous theories, these are based on data analysis, not on wild speculation, which gives us some better tools for assessing their likelihood.
  • In 2013 there might have been comparatively fewer violent outdoors, street crimes than in previous years. Because the UCPD primarily responds to outdoor crime, this might account for the dropoff.
    VERDICT: Statistically improbable (p = .8)
When I first thought of this explanation, I was actually quite optimistic. In any given year, there is always some percentage of violent, non-domestic crime that occurs in outdoor area (fights and robberies in streets, alleys, parks, etc.). There is a reciprocal percentage that occurs indoors (fights and robberies in apartments, residences, stores, etc.). Given the public presence of the UCPD, its patrol routes, its blue light phones, and the unarmed security guards on campus, it might be more likely that the UCPD will respond and react to outdoor crime. According to this theory, when violence happens indoors, victims are more likely to call the CPD than the UCPD. As such, if 2013 had significant less street crime than in previous years, this might have accounted for the lower reporting rates.

Unfortunately, 2013 had more or less the same rates of outdoor/indoor violence as previous years. The difference in the distribution between crimes was completely insignificant at any statistical level (p = .8). That is to say, 2013 had the same ratio of outdoor/indoor violence as 2009, 2010, 2011, and 2012. Clearly if we want to explain the drop in reports for 2013, we need to look elsewhere.
  • In 2013, there might have been different types of violent crime (e.g. fewer robberies and more assaults) which accounted for the difference; perhaps the UCPD is only called in on certain crimes (robbery) and not others (assault).
    VERDICT: Statistically improbable (p = .5 and p = .25)
This was another theory that I was initially optimistic about. UCPD officers are probably more likely to be called in on a robbery than on a battery. Similarly, the UCPD might be more likely to drive by and witness a robbery than they are to witness a verbal assault. If 2013 had significantly fewer incidences of one type of crime (like robbery) and comparatively more of another (assault or battery), that difference might account for the decreased UCPD reporting. To check this hypothesis, I ran statistical tests on two slightly different datasets. For the first dataset, I only looked at the differences between the broadest crime classifications (Battery and Robbery). In the second dataset, I analyzed differences between more specific classifications (Aggravated Battery, Simple Battery, Armed Robbery, Strongarm Robbery, etc.).

Both tests showed that there were absolutely no statistical differences between crime in 2013 as compared to previous years. Just looking at the broader crime classifications, 2013 had a roughly identical distribution to all the other years in the sample; roughly 40% of all crimes were batteries, 30% were assaults, and 20% were robberies (with some left over to fill in the gaps). Thus, the difference was totally insignificant (p = .5).

The second test was just disappointing. Even breaking down the crime categories into smaller subcategories (e.g. Robbery into Armed/Unarmed Robbery), there were still no significant differences between the years (p = .25). If anything, of all the years in the sample, 2013 was one of the most average in terms of crime distribution, even if it had comparatively fewer overall violent incidents.
  • Following the UCMC Protest fiasco, the UCPD has been more cautious in answering violet crime calls and/or the community is less trusting of the UCPD and is choosing not to call on them.
    VERDICT: Unlikely
This was another great idea, one suggested by a friend of mine (herself a UChicago graduate and employee of the Hospital). After the highly publicized UCMC arrests on January 29, both the UChicago community and the general Hyde Park/Kenwood/Woodlawn communities were outraged at police treatment of protestors. I have commented on this in the past and I am decidedly less outraged than others, but I do understand how the wider community outcry might have affected UCPD response and reporting behavior. After all, in a community that does not trust the UCPD, victims of a crime will be unlikely to call the University for help. Similarly, UCPD officers might be under instructions to be extra cautious and sensitive in dealings with the community, which could mitigate their involvement in violent arrests.

Yet again, this theory doesn't hold up to scrutiny. The UCMC protests happened on January 29. As such, any consequences of the protests would not have taken into effect until at least the following day, January 30. Although this could have had an effect on the UCPD in the weeks following January 29, it certainly would not have retroactively affected UCPD reports for the entire month of January up until that date. A close look at the numbers confirms this problem; even before the January 29 UCMC protests, the UCPD was still not responding to crimes even though there were plenty of violent incidents in the area. From 2009-2012, just up until January 29, we would expect to see at least a half dozen violent reports. But in 2013, we still saw 0. So although the UCMC protests might have affected UCPD behavior after January 29, it did not decrease its engagement in the weeks before, weeks during which we still see a lack of violent reports.

There are a few other explanations that we could try and parse through, but I am confident that those will also fall to close analysis. Some people might blame the weather (a common crime culprit in Chicago). But if anything, the unseasonably warm January weather should have caused an increase in January crime, and thus an increase in UCPD violent crime responses. The temperatures certainly had that effect across the rest of the city, but it seems that Hyde Park was spared almost all of that violence.

What about the private security guards around campus? Surely their presence has reduced the incidence of violent crime in the area, maybe alleviating the need for the UCPD to respond to incidents. But again, this doesn't really explain the decline. The Allied Barton security personnel are stationed on corners around the University campus and immediate streets, not generally dispersed around the entirety of Hyde Park, south Kenwood, and north Woodlawn. That's a big problem, because the vast majority of Hyde Park violence has historically occurred in areas that are technically off campus; north of 55th Street and south of 61st Street. Because there are no security guards in those areas, they are certainly not leading to a decrease in crime.

All of that apparently leaves us quite stuck. But thankfully, there is one answer that we can turn to. Although it is not supported by any hard numbers or data, the evidence for this comes straight from the UCPD itself.

As is often the case with crime-related phenomena, the explanations are often hidden in seemingly unrelated areas. In this case, the drop in UCPD reports was not cased by changes in violence, crime patterns, blatant malfeasance and misclassification, community mistrust, weather, or any number of other factors. But something did change at the UCPD in the past few months, and its timing and potential effect are just too convenient to ignore: staffing and systems.

Beginning in mid-December, the UCPD underwent a number of seemingly insignificant changes. Together, however, all of these changes could easily combine to explain the drop in 2013 violence reports.

First, during the week of December 3, the Department transitioned to a "new records management software" (2012 UCPD Operations Bulletin 39, 3). Although it is unclear exactly what this record management software was for, "all UCPD personnel [were] notified to attend" a training session on its use. Given that police agencies tend to really keep just two types of records (crimes/arrests and personnel/human resources data), this could definitely have had a huge effect on the way crime is reported at the Department.

Second, to start of the new year, the UCPD has kicked off "Initiative 1" of its new strategic plan, the goal of which is to take "steps to improve our efforts to attract talented staff by strategically targeting our recruitment efforts" (2012 UCPD Operations Bulletin 43, 1). As part of that plan, the UCPD has already inducted a new class of officers to its ranks, with 9 new officers added on December 28 alone (2012 Bulletin 43, 3). New officers always mean new opportunities and challenges for a police department, and the UCPD is no exception. On the one hand, these new young men and women will bring their training and expertise to the force, improving its ability to serve the community. But on the other hand, new officers lack experience in the field. They may also lack training in the systems of the Department (like record management processes). The UCPD undoubtedly understands these challenges and has taken steps to correct them. Those steps themselves, however, might lead to a drop in reported violence. New officers might be deliberately kept out of violent situations during their first months on the job. Older officers might be paired with them as mentors, reducing overall patrols and UCPD response capacity. Or on a more elementary level, new officers might be unaccustomed to entering data into a new system that no one knows how to use. Numbers can easily get lost in that turnover.

Finally, as a supplement to its new record management software, the UCPD hired a new Police Records Manager. Ms. Connie Tsao, a UChicago graduate majoring in Psychology, started her work at the UCPD in December 2012 as an intern before being hired full-time in early 2013 (2013 Operations Bulletin 5, 2). I do not want to question Ms. Tsao's competency or proficiency in the new system; if she was hired in the role she is likely qualified to succeed at it. But given the transitions that the UCPD is going through, it is very possible that some reports (i.e. RECORDS) got lost in the cracks. The UCPD only recently installed its software, so Ms. Taso likely has a lot on her hands in transitioning the Department to a new data system. All I will say is that new data management systems invariably cause the loss of some data, whether in nonprofits, businesses, government agencies, or police departments. The UCPD could be the victim of its own expansion.

All of these reasons combine to create a records-management environment that, although having great future potential, is currently flawed or even dysfunctional.
  1. New record management software in December 2012
    New software is hard to use for anyone! I remember the first time that a group of UChicago students got introduced to the SPSS statistical software; it was like trying to teach nursing home residents how to use iPhones. A comprehensive record management database software is undoubtedly more complicated with much more room for error. UCPD officers are still learning how to use it effectively, and everyone is on the same page together. Through using the new software, UCPD personnel could have incorrectly entered or digitally "lost" some of the violent incidents.
  2. New police officers on the job in January 2013
    The UCPD has hired new officers, and will continue to hire new officers, to keep on the cutting edge of law enforcement. New officers bring a lot of great talents to the department, but they lack experience and require additional training. Given the struggles that the UCPD might be undergoing with its software, this additional burden could make things even more difficult. Moreover, new officers might be deliberately kept out of violent situations and calls to minimize risk. This would also affect violent crime reporting rates.
  3. New records manager in December and January 2012-2013
    The new records manager likely has a lot to deal with in the transition from 2012 to 2013, not to mention the new software that the entire department is learning to use. Governments, universities, and even tech-savvy teenagers have trouble using new software, especially such complicated software as a new database management system. Given the potentially stressful transition between systems, it is possible that incidents were incorrectly entered, registered, or simply not being printed. The technical challenges alone would be large. 
Of course, all of this is purely speculative without actually asking the UCPD (a potential project for another day). Also, none of this is to insult the UCPD or its staff; I have tried to be extremely careful in my choice of language to avoid this.

That all said, these appear to me the likeliest explanations for the lack of violent crime reports in January-February 2013. There is also another explanation that I did not consider here: Crime is totally random and not subject to a lot of pretty patterns and explanations. Although there is truth to that, the precipitous drop in reports that we just saw defies the conventional randonmness and "noise" that we might expect from crime. That forces us to turn other explanations, the best of which is given here.

As 2013 progresses, I will continue to monitor the UCPD Incident Report for violent crimes, keeping in mind the parallel statistics given by the CPD. If things improve, then we might conclude that the UCPD has ironed out its technical and administrative struggles. If not, however, then we might have to dig deeper for additional explanations.

No comments :

Post a Comment