In his paper “Design Performances: How Organizations Inscribe Artifacts to Change Routines”, Dr Glaser presents a case study of how a law enforcement agency designed an algorithm to optimize officers’ patrolling routines, thereby deterring criminal activity. His study reveals the importance of the design process in facilitating change—an aspect that often is overlooked in theory and in practice.
Many organizations use algorithms to improve their performance and achieve strategic objectives. An algorithm specifies the exact sequence of steps necessary to solve a problem without requiring any human intuition or guesswork. Algorithms always work, regardless of the inputs, and may take the form of software, procedures, and/or scripts. For example, Amazon and Netflix use machine learning algorithms to generate automated recommendations about products or services for customers; Proctor & Gamble and other consumer products companies use text analysis algorithms to monitor consumer brand engagement on social media; and shipping companies such as UPS and FedEx use optimization algorithms to minimize delivery costs.
When implementing large-scale strategic change initiatives, organizations tend to focus most of their efforts on pursuing the “right” objectives and selecting the “right” algorithms to achieve them. However, in “Design Performances: How Organizations Inscribe Artifacts to Change Routines”, Dr Vern Glaser, Associate Professor at the University of Alberta, reveals that the process of designing a change initiative may be even more important than defining the right goal or selecting the right algorithm. His case study of a law enforcement agency reveals four aspects of the design process which ultimately determine the effectiveness of a strategic change initiative: identifying essential components of the existing routine; questioning underlying assumptions that shape how routines are performed; redistributing the work to optimize effectiveness; and re-thinking performance metrics. Dr Glaser’s findings have theoretical and practical implications for routine dynamics and organizational strategy.
Designing an algorithm to change organizational routines: the case of Metropol
In his research, Dr Glaser examined how a law enforcement organization in the Western United States (i.e., Metropol, a pseudonym) designed software to change patrolling routines with the objective of deterring criminal activity—specifically, terrorism and fare evasion on public transport. Organizational actors partnered with Algo-Security, a research firm dedicated to developing real-world security solutions, to design a tool that used a game-theoretic algorithm to predict criminal activity and automatically adjust patrolling routines based on those predictions.
The project team had to identify all the components of the existing routine and translate them into knowledge that could be embedded as algorithmic parameters.
Metropol’s existing patrolling routines were determined by schedulers who used an Excel spreadsheet to assign officers to locations in a very complex, labour-intensive process. For example, to make it more difficult for terrorists to realise a successful attack, Metropol responded to suspicious activities by deploying available resources such as patrol units, K-9 units, and plainclothes officers to different locations. Upon their arrival, officers would observe activities, query individuals or look for suspicious materials, such as abandoned backpacks. Likewise, to deter fare evasion, Metropol periodically deployed deputy officers and security assistants to transit stations, trains, and buses to audit passengers for fare payment and issue warnings and citations to passengers without valid fare documentation.
Metropol became interested in randomizing patrolling routines to make it more difficult for criminals to exploit predictable patrolling patterns. However, producing random schedules proved to be challenging because human schedulers tend to follow predictable scheduling habits. Metropol identified game theory as a potential solution to this problem. Game theory employs formal mathematical models to make optimal decisions in scenarios involving high levels of complexity (e.g., which targets to protect from criminals at specific times). To achieve this objective, Metropol partnered with Algo-Security, which had developed game-theoretic algorithms to create randomized patrol schedules for several other law enforcement agencies. The design process involved four core activities: (a) identifying essential components of the existing routine; (b) questioning underlying assumptions; (c) redistributing the work; and (d) rethinking performance metrics.
Identifying essential components of the existing routine
To design a tool that used an algorithm to randomize patrolling routines, the project team had to identify all components of the existing routine (e.g., locations protected by Metropol, the types and number of available security resources, activities that those resources could perform, and the amount of time it takes to perform those activities) and translate them into knowledge that could be embedded as parameters in the algorithmic software application. For example, the algorithm needed to consider tacit knowledge, such as travel time to a patrolling destination, and the relative attractiveness of each location to criminals. Similarly, when patrolling a bus or train station, law enforcement officers could take three different actions: question individuals in the station (15 minutes), observe the station without taking any action (10 minutes), pass through the station en route to another one (10 minutes), or move between different levels of a station (1 minute). It was important to comprehensively identify the core components (i.e., resources deployed, location, travel time, action time) of the routine, and translate these essential components into algorithmic parameters.
Questioning underlying assumptions
During the design process, Metropol and Algo-Security interrogated the key assumptions built into existing routines. The game-theoretic approach to scheduling is based on a fundamental assumption about human behaviour—specifically, that individuals (criminals, in this case) are rational and seek to maximize utility. Team members questioned this assumption as they attempted to define the value of potential targets to criminals, recognizing that these may or may not match their own assessments of value. They also questioned the assumption that all criminals behave similarly. Exposing these assumptions also influenced which actions Metropol would take to deter criminal activity. Ultimately, these conversations prompted Algo-Security and Metropol to develop a new vision for the software that would incorporate knowledge about different types of law enforcement adversaries. Team members also questioned assumptions regarding how patrol units are configured and deployed. For example, Metropol could send a K-9 unit on a counterterrorism patrol, or it could pair a K-9 unit with a plainclothes officer who could surreptitiously observe activities. By questioning the assumptions underlying all aspects of the patrolling routine, members of the project team were able to consider how to use Metropol’s resources most effectively.
Redistributing the work
Designing a scheduling tool that could be repeatedly used by the organization under a variety of contextual circumstances required the project team to think through which individual(s) would perform each task, and how. Metropol and Algo-Security reviewed the roles and responsibilities of each person involved in the patrolling routine and the capabilities of the software to re-allocate those roles and responsibilities. Decisions had to be made regarding the extent to which the activities of each patrolling routine would be specified in the algorithm. For example, an officer might monitor for criminal activity by walking around a train station or by visiting specific platforms in a particular order. Initially, the project team did not embed a high level of specification, enabling officers to exercise their judgement regarding the best way to patrol a particular area. Interestingly, evidence began to reveal that fewer citations were issued when officers followed their intuition rather than the algorithm. Thus, the design team changed the software to specify which tasks officers should perform at each location. Team members also had to consider how they would integrate real-time information into the patrolling routine. For example, a discussion regarding how they would perform a counterterrorism patrolling routine in response to a bomb threat surfaced disagreement about the algorithm’s role and the ability to override the schedule. This sparked an important discussion about the parameters that determine whether officers should follow the directions from the software or use their own judgement.
The routine design process can inspire a wide variety of intended and unintended benefits for an organization.
Rethinking performance metrics
As the design process progressed, team members recognized a need to construct alternate ways of assessing the algorithm’s performance. Clearly, the new algorithm provided a much easier way to randomize patrolling routines. But was it more effective than simply using Excel to randomize the patrol schedule and modifying it based on insights from patrol officers? To settle this debate, the team conducted a 21-day experiment and found that the productivity of officers who followed the schedule produced by the algorithm doubled in terms of the number of fare verifications, warnings, and citations. Moreover, Metropol realised that while patrol coverage metrics are important, it is just as important to track which targets are not being covered. More generally, conversations about the philosophy underlying performance metrics helped Metropol clarify that their objective was to contain criminal activity rather than eliminate it altogether. This shift in focus made it clear that game-theoretic scheduling offers the best overall protection for society.
The Metropol case reveals the importance of the algorithm design process in facilitating organizational change. Although objectives and algorithm selection are important, the design process should not be overlooked, as the routine design process can inspire a wide variety of intended and unintended benefits for an organization. If Metropol and Algo-Security had not invested extensive time and effort in the design process, the change initiative likely would not have been as successful, and may even have failed. To effectively design a new routine, team members must thoughtfully identify all components of the existing routine, question the underlying assumptions, decide how to best redistribute the work, and develop appropriate performance metrics that reflect the overall objective. Perhaps most importantly, designers must enrol key stakeholders in the design process to ensure the new solution is not rejected.
How would you like to further develop your research into how organizations change their practices and culture?
I am continuing to study how organizations use algorithms, analytics, arguments, and analogies to strategically change their practices and culture in a variety of projects.
Based on your research, what one piece of advice would you give to an organization that wishes to change its routines to achieve greater productivity in its operations?
If your organization wants to use analytics and algorithms to change fundamental routines, take a lot of time to think about how you will go about making the change. An effective design process can provide you with an opportunity to elicit tacit knowledge from team members throughout the organization—which will often unearth innovative ideas about how to reconfigure your routines to improve performance.