Wilpen L. Gorr
Professor of Public Policy and Information Systems
Wil Gorr has been on the Heinz College faculty since 1985. He serves as the faculty chair of the Master of Science in Public Policy and Management (MSPPM) program and is chair of the Faculty Oversight Committee for the MSPPM program in Washington, D.C. Prior to joining the Heinz College faculty as professor, he was assistant and associate professor at the John Glenn School of Public Affairs, The Ohio State University. He obtained his Ph.D. from the School of Urban and Public Affairs, the Heinz College's predecessor, and obtained his master and bachelor degrees from The Pennsylvania State University. He is a member of the Association of Public Policy and Management and the International Institute of Forecasters.
In 2011, Wil won the Martcia Wade Teaching Award in the Heinz College, Carnegie Mellon University. In 2005, Wil was inducted as a Fellow of the International Institute of Forecasters. He is past editor of the International Journal of Forecasting. He was recipient of the Best Paper Award from the American Information Systems Society for "ServiceNet: An Agent-Based Framework for E-Government" in August 2001, and has received numerous teaching awards from Carnegie Mellon and Ohio State.
Wil's research interests include geographic information systems, predictive models, and management science models applied to public sector problems. His current research includes developing leading indicator forecast models for law enforcement, application of the receiver operating characteristic (ROC) framework to time series analysis and forecasting, and information systems for support of policy and planning in organizations. His publications have appeared in Management Science, Geographical Analysis, The International Journal of Forecasting, Criminology, MIS Quarterly, and other leading journals. He has received funding from the National Institute of Justice, Centers for Disease Control, the National Science Foundation, the Alfred P. Sloan Foundation, and other organizations.
Wil teaches database, geographic information systems, and project courses. He and his colleague, Kristen Kurland, have written three geographic information systems textbooks for ESRI Press.
Gorr, W.L. and Schneider, M., "ROC-Based Model Estimation for Forecasting Large Changes in Demand", International Journal of Forecasting, to appear 2014. (Download the PDF)
Gorr, W.L. and Lee, Y., “Early Warning System for Temporary Crime Hotspots, Journal of Quantitative Criminology, to appear 2014. (Download the PDF)
Gorr, W.L. and M.J. Schneider "Large-Change Forecast Accuracy: Reanalysis of M3-Competition data using receiver operating characteristics analysis," International Journal of Forecasting, 2013, Vol. 29, Issue 2, pp. 274-281. (Download the PDF)
Gorr, W.L., "Forecast Accuracy Measures for Exception Reporting Using Receiver Operating Characteristic Curves," /International Journal of Forecasting/, 2009, Vol. 25, Issue 1, pp. 48-61. (Download the PDF)
Cohen, J., S. Garman, and W. L. Gorr, "Empirical Calibration of Time Series Monitoring Methods Using Receiver Operating Characteristic Curves," /International Journal of Forecasting/, 2009, Vol. 25, Issue 3, pp. 484-497. (Download the PDF)
Cohen, J., W. L. Gorr, and A. Olligschlaeger, "Leading Indicators and Spatial Interactions: A Crime Forecasting Model for Proactive Police Deployment" Geographical Analysis, January 2007, Vol. 39, Issue1, pp. 105-127. (Download the PDF)
Gorr, W.L. and S.A. McKay, "Application of Tracking Signals to Detect Time Series Pattern Changes in Crime Mapping Systems," in F. Wang [ed.] Crime Mapping and Beyond: GIS Applications in Crime Studies, Hershey, PA: Idea Group Publishing, 2005.
Johnson, M., W.L. Gorr, and S. Roehrig, "Location of Elderly Service Facilities," Annals of Operations Research, Vol. 136 (1), January 2005.
Cohen, J., W.L. Gorr, and P. Singh, "Estimating Intervention Effects in Varying Risk Settings: Do Police Raids Reduce Illegal Drug Dealing at Nuisance Bars?," Criminology, Vol. 41 (2), May 2003, pp. 257-292.
Johnson, M., W.L. Gorr, and S. Roehrig, "Location/Allocation/Routing for Home-Delivered Meals Provision: Model and Solution Approaches," International Journal of Industrial Engineering, Special Issue on Facility Layout and Location, Vol. 9 No. 1 (2002), pp. 45 - 56.
Anderson, B.B., A. Bajaj, and W.L. Gorr, "An Estimation of the Relative Effects of External Software Quality Factors on Senior IS Managers' Evaluation of Computing Architectures," Journal of Systems & Software, Vol. 61 (2002), pp. 59-75.
Gorr, W.L., A. Olligschlaeger, and Y. Thompson, "Short-term Forecasting of Crime", International Journal of Forecasting, Vol. 19, No. 4 (2003).
Gorr, W.L., M. Johnson, and S. Roehrig, "Spatial Decision Support System for Home-Delivered Services," Geographical Analysis, Vol. 3(2001), pp. 181-197.
Duncan, G., W.L. Gorr, and J. Szczypula, "Forecasting Analogous Time Series," S. Armstrong [ed.], Forecasting Principles (2001) Kluwer Academic Publishers.
Getis, A., P. Drummey, J. Gartin, W.L. Gorr, K. Harris, P. Rogerson, D.Stoe, and R. Wright, "Geographic Information Science and Crime Analysis," Journal of the Urban and Regional Information Systems Association, Vol. 12, No.2 (2000), pp 7-14.
Geographic Information Systems, Predictive Models, and Management Science Models applied to Public sector problems.
PhD, Operations Research, Carnegie Mellon University
ROC-Based Model Estimation for Forecasting Large Changes in Demand
Forecasting for large changes in demand should benefit from different estimation than that used for estimating mean behavior. We develop a multivariate forecasting model designed for detecting the largest changes across many time series. The model is fit based upon a penalty function that maximizes true positive rates along a relevant false positive rate range and can be used by managers wishing to take action on a small percentage of products likely to change the most in the next time period. We apply the model to a crime dataset and compare results to OLS as the basis for comparisons as well as models that are promising for exceptional demand forecasting such as quantile regression, synthetic data from a Bayesian model, and a power loss model. Using the Partial Area Under the Curve (PAUC) metric, our results show statistical significance, a 35 percent improvement over OLS, and at least a 20 percent improvement over competing methods. We suggest management with an increasing number of products to use our method for forecasting.(Download)
Early Warning System for Crime Hot Spots
Using violent crime data from Pittsburgh, Pennsylvania we investigate the performance of an “early warning system” (EWS) for starting/stopping police deployments to hot spots for crime prevention. We show that (1) even the hottest chronic hot spots are dynamic with months “on” and “off” and (2) temporary hot spots are also targets for prevention. We compare the performance of EWS to constant deployment at chronic hot spots.(Download)
Longitudinal Study of Crime Hot Spots
Longitudinal Study of Crime Hot Spots: Dynamics and Impact on Part 1 Violent Crime(Download)
Using Receiver Operating Characteristic Analysis to Evaluate Large-Change Forecast Accuracy
This paper applies receiver operating characteristics (ROC) analysis to M3 Competition, micro monthly time series for one-month-ahead forecasts. Using the partial area under the curve (PAUC) criterion as a forecast accuracy measure and paired-comparison testing via bootstrapping, we find that complex methods (AutomatANN, Flores-Pearce2, Forecast ProSmart FCS, and Theta) perform best for forecasting large declines in these time series, which tended as a group to decline over time. A regression model of PAUC on a judgmental index for forecast method complexity provides further confirming evidence. We also found that a combination forecast, consisting of the median value of the top three methods, to perform better than the component methods, although not statistically so. The classification of top methods matches that obtained using conventional forecast accuracy methods in the M3 Competition complex
methods forecast these series better than simple ones.
Empirical Calibration of Time Series Monitoring Methods Using Receiver Operating Characteristic Curves
Time series monitoring methods, such as the Brown and Trigg methods, have the purpose of detecting pattern breaks in time series data reliably and in a timely fashion. Traditionally, researchers have used the average run length statistic (ARL) on results from generated signal occurrences in simulated time series data to calibrate and evaluate these methods, with a focus on timeliness of signal detection. This paper investigates the receiver operating characteristic (ROC) framework, well-known in the diagnostic decision making literature, as an alternative to ARL analysis for time series monitoring methods. ROC analysis traditionally uses real data to address the inherent tradeoff in signal detection between the true and false positive rates when varying control limits. We illustrate ROC analysis using time series data on crime at the patrol district level in two cities and use the concept of Pareto frontier ROC curves and reverse functions for methods such as Brown's and Trigg's that have parameters affecting signal-detection performance. We compare the Brown and Trigg methods to three benchmark methods, including one commonly used in practice. The Brown and Trigg methods collapse to the same simple method on the Pareto frontier and dominate the benchmark methods under most conditions. The worst method is the one commonly used in practice.(Download)
Framework for Validating Geographic Profiling
This short paper was prepared for the NIJ Roundtable for Developing an Evaluation Methodology for Geographic Profiling Software (August 10 and 11, 2004) on approaches for validating geographic profiling (GP) methods. The paper presents a framework for validating any GP method or software package using solved serial crimes including data on crime locations and criminal residences or other anchor points (e.g., work location, girl friend’s residence, etc.). Findings from the literature and through analysis include: 1) the appropriate performance measure for GP (that matches policing needs and as extended in this paper) concerns prioritizing relevant areas for investigation; 2) future work should correct the performance measure of GP by excluding irrelevant areas from consideration such as rivers, lakes, cemeteries, etc. (past studies apparently did not do this); 3) additional model parameters may be able to be estimated in empirical studies such as the amount to expand the search area for a serial criminal beyond the minimum rectangle or other boundary enclosing crime sites; and 4) future validation studies for GP should compare alternative models, including simple models for benchmarking, and use holdout samples in a resampling scheme for validating performance. Acknowledgements
Leading Indicators and Spatial Interactions: A Crime Forecasting Model for Proactive Police Deployment
Based on crime attractor and displacement theories of environmental criminology, this paper specifies a leading indicator model for forecasting serious property and violent crimes. The model, intended for support of tactical deployment of police resources, is at the micro-level scale; namely, one-month-ahead forecasts over a grid system of 104 square grid cells 4,000 feet on a side (with approximately 100 blocks per grid cell). The leading indicators are selected lesser crimes and incivilities entering the model in two ways: 1) as time lags within grid cells and 2) time and space lags averaged over contiguous grid cells of observation grid cells. The validation case study uses 1.3 million police records including 16 individual crime types from Pittsburgh, Pennsylvania aggregated over the grid system for a 96 month period ending in December 1998. The study uses the rolling-horizon forecast experimental design with forecasts made over the 36 month period ending in December 1998, yielding 3,774 forecast errors per forecast model.(Download)
Policy Systems: The Integration of Information Technology into Policy Analysis, Planning, and Program Analysis
This paper defines a policy system to be a collection of hardware, software, communication technologies, persons, procedures, protocols, and standards driven by and for the purpose of advancing a public organization’s mission in regard to policy analysis, planning, and program evaluation decisions. While policy systems already exist in practice, they have not been identified and studied as a separate, distinguishable area of information systems. They have components and patterns of use that could benefit governments of all levels in carrying out policy making. This paper proposes principles for building policy systems, identify their components, discuss how they address the complexities of policy making, illustrate them with several examples including a policy system built for a local government agency, and distinguish them from related systems such as management information systems, decision support systems, and collaboratories.(Download)
An Integrated Approach to Developing Human Services Web Portals
A major goal for Human Services Web Portals is to make as much expertise available as possible for clients and their caregivers. The expertise covers three main areas - diagnosing a client’s problem, identifying available resources for solution, and finally, providing assistance to package these resources into a service plan that will serve as a solution for the client’s problem. The main challenge in setting up a human services web portal lies in the nature of diversity and complexity in both the client set and the set of problems clients want to address. Therefore offering prepackaged solutions is not an option. We describe an integrated human services web portal design and provide a phased approach for implementation. Finally, we generalize our design for other domains in which external expertise is required as a component of service delivery.(Download)
Application of Tracking Signals to Detect Time Series Pattern Changes in Crime Mapping Systems
Tracking signals are widely used in industry to monitor inventory and sales demand. These signals automatically and quickly detect departures in product demand, such as step jumps and outliers, from "business-as-usual". This paper explores the application of tracking signals for use in crime mapping to automatically identify areas that are experiencing changes in crime patterns and thus may need police intervention.. Detecting such changes through visual examination of time series plots, while effective, creates too large a work load for crime analysts, easily on the order of 1,000 time series per month for mediumsized cities. It is demonstrated that the so-called smoothed-error-term tracking signal and carry out an exploratory validation on 10 grid cells for Pittsburgh, Pennsylvania. Underlying the tracking signal is an extrapolative forecast that serves as the counterfactual basis of comparison. The approach to validation is based on the assumption that we wish tracking signal behavior to match decisions made by crime analysts on identifying crime pattern changes. Tracking signals are presented in the context of crime early warning systems that provide wide area scanning for crime pattern changes and detailed drill-down maps for crime analysis. Based on preliminary results, the tracking signal is a promising tool for crime analysts.(Download)
Estimation of Crime Seasonality: A Cross-Sectional Extension to Time Series Classical Decomposition
Reliable estimates of crime seasonality are valuable for law enforcement and crime prevention. Seasonality affects many police decisions from long-term reallocation of uniformed officers across precincts to short-term targeting of patrols for hot spots and serial criminals. This paper shows that crime seasonality is a small-scale, neighborhood-level phenomenon. In contrast, the vast literature on crime seasonality has almost exclusively examined crime data aggregations at the city or even larger scales. Spatial heterogeneity of crime seasonality, however, often gives rise to opposing seasonal patterns in different kinds of neighborhoods, canceling out seasonality at the city-wide level. Thus past estimates of crime seasonality have vastly underestimated the magnitude and impact of the phenomenon. This paper presents a model for crime seasonality that extends classical decomposition of time series based on a multivariate, cross-sectional, fixed-effects model. The crux of the model is an interaction of monthly seasonal dummy variables with five factor scores representing the urban ecology as viewed from the perspective of major crime theories. The urban ecology factors, interacted with monthly seasonal dummy variables, provide neighborhood-level seasonality estimates. A polynomial in time and fixed effects dummy variables for spatial units control for large temporal and spatial variations in crime data. Our results require crime mapping for implementation by police including thematic mapping of next month's forecasted crime levels (which are dominated by seasonal variations) by grid cell or neighborhood, thematic mapping of the urban ecology for developing an understanding of underlying causes of crime, and ability to zoom into neighborhoods to study recent crime points.(Download)
Guns and Youth Violence: An Examination of Crime Guns in One City
Firearms are an important factor in violent crimes. Nationally, the percentage of violent offenses that involve use of a firearm closely tracks changes in the supply of newly manufactured pistols (Figure 1). As more pistols became available their use in violent crimes increased. After 1985, firearms were especially implicated in the dramatic rise in juvenile homicide rates, both as victims (Fingerhut, 1993; Fingerhut, et al., 1998) and offenders (Blumstein, 1995). While juvenile rates of homicides by gun surged upward, both adult and nongun juvenile homicide rates remained relatively flat during the same period (Blumstein and Cork, 1996; Cork, 1996). While the link between guns and youth homicides is compelling in aggregate data, very little is known about how gun availability actually affects individual behavior among youth, whether that effect differs between young adults and juveniles, and whether that relationship has changed over time. The research discussed here examines spatial and temporal features of crime guns in one city. The analysis focuses on attributes of crime guns and those who possess them, the geographic sources of those guns, the distribution of crime guns over neighborhoods in a city, and the relationship between the prevalence of crime guns and incidence of violent crimes especially homicides.(Download)
Impact of Police Raids at Nuisance Bars on Illegal Drug Dealing: Estimating Intervention Effects in Varying Risk Settings
This paper examines the effects of police raids at nuisance bars on drug dealing in and around the nuisance bar. We examine effects of both dosage (number of raids) and duration of the intervention, as well as the conditioning effects of land use and population characteristics in shaping the underlying risk levels of drug dealing in the target and surrounding displacement areas. Results indicate that the police intervention does suppress levels of drug dealing during periods of active enforcement, but these effects largely disappear when the intervention is withdrawn. Also, the effects of the intervention are mediated by risk characteristics in target and displacement areas. In general, target areas characterized by higher levels of risk are more resistant to intervention effects than those with lower levels or risk. Risk factors in nearby displacement areas are also significant. Bars with high levels of risk arising from land uses in surrounding areas are easier to treat, while bars with high levels of population-based risk in surrounding displacement areas are harder to treat.(Download)
Point Demand Forecasting
This paper provides geographic information system (GIS) methods and empirical models to forecast point demand for home-delivered goods. A point forecast consists of stops on a street network, including demand at each stop. The purpose of the forecast is to support a network optimization model, based on the traveling salesman problem, to locate one or more new facilities in a region. This paper illustrate the approach with a case study of home-delivered meals (meals ons wheels) in Allegheny County, Pennsylvania.
Assessment of Crime Forecasting Accuracy for Deployment of Police
Crime forecasting is a new area of research, following upon the success of crime mapping for support of tactical deployment of police resources. The major question investigated in this paper is whether it is possible to accurately forecast crime one month ahead at a “smallscale” aggregation, i.e., at the precinct level. In a case study of Pittsburgh, Pennsylvania, we contrast the forecast accuracy of standard, univariate time series models with non-modeling practices commonly used by police. Included is a comparison of seasonality estimates made by precinct versus the city as a whole. As suspected for the small-scale data of this problem, average crime count by precinct and crime type is the major determinant of forecast accuracy. A fixed effects regression model of absolute percent forecast errors shows that such counts need to be on the order of 30 or more to achieve accuracy of 20 percent error or less. A second major result is that practically any model-based forecasting approach is vastly more accurate than current police practices. Thirdly, this is the first empirical paper to investigate crime seasonality at the sub-city level. Our seasonality estimates provide evidence supporting the routine activities theory of crime, but not earlier theories.
Facility Location Model for Home-Delivered Services: Application to the Meals-on-Wheels Program
This paper presents a GIS-based decision support system for the non-profit sector, designed to assist strategic and tactical decision making in the area of home-delivered services such as meals on wheels. Using data collected from existing programs, current and forecasted demographic data, and a series of algorithmic tools, this paper provides a system for evaluating current meals on wheels facilities, and for making facility location decisions that satisfy coverage and equity requirements.(Download)
Organizations in the private sector must do strategic planning over long-term horizons to locate new facilities, plan new products, develop competitive advantages, and so forth. Consequently, long-term forecasts of demand, costs of raw materials, etc. are important in the private sector. There is no such strategic counterpart to police work; consequently, long-term forecasts are of little value to police. Police primarily need short-term forecasts; for example, crime levels one week or one month ahead. Currently, police mostly respond to new crime patterns as they occur. Client-server computing for realtime access to police records and computerized crime mapping have made it possible for police to keep abreast with crime. With short-term forecasting police may be able to get one step ahead of criminals by anticipating and preventing crime. The organization of this paper proceeds first with a description of short-term forecasting models, to provide basic terms and concepts. Next is a discussion of unique features of crime space-time series data, and the need for data pooling to handle small-area model estimation problems. Lastly are a discussion of particular forecasting requirements of police and a summary.(Download)
Forecasting Analogous Time Series
Organizations that use time series forecasting on a regular basis generally forecast many variables, such as demand for many products or services. Within the population of variables forecasted by an organization, we can expect that there will be groups of analogous time series that follow similar, time-based patterns. The co-variation of analogous time series is a largely untapped source of information that can improve forecast accuracy (and explainability). This paper takes the Bayesian pooling approach to drawing information from analogous time series to model and forecast a given time series. Bayesian pooling uses data from analogous time series as multiple observations per time period in a group-level model. It then combines estimated parameters of the group model with conventional time series model parameters, using "shrinkage" weights estimated empirically from the data. Major benefits of this approach are that it 1) minimizes the number of parameters to be estimated (many other pooling approaches suffer from too many parameters to estimate), 2) builds on conventional time series models already familiar to forecasters, and 3) combines time series and cross-sectional perspectives in flexible and effective ways.
Spatio-Temporal Forecasting of Crime: Application of Classical and Neural Network Methods
This paper introduces a new spatio-temporal forecasting methodology that combines artificial neural networks and cellular automata with GIS-based data. The technique, which we refer to as chaotic cellular forecasting (CCF) is similar to spatial adaptive filtering due to Foster and Gorr (1986) and weighted spatial adaptive filtering due to Gorr and Olligschlaeger (1994) in that it uses contiguity relationships and the geographer’s assumption that influence between data points decays with distance. As with spatial adaptive filtering the methodology uses an iterative process to arrive at a solution. Unlike spatial adaptive filtering, however, chaotic forecasting uses a gradient descent method rather than a grid search to find the optimal set of parameters (or, in the case of artificial neural networks, weights). In addition, and most importantly, CCF has the nonlinear and multi-model functional form commonly used in neural net modeling, allowing for increased pattern recognition and accommodation of spatio-temporal heterogeneity. The result is a robust spatio-temporal forecasting method that requires very little model specification, is self - adaptive and performs very well on data sets that exhibit non-traditional statistical behavior.(Download)
Comparative Study of Cross Sectional Methods for Time Series with Structural Changes
Comparative Study of Cross Sectional Methods for Time Series with Structural Changes(Download)