Abstract

Humanity is experiencing revolutionary changes in the 21st century, including accelerating urbanization, the introduction of disruptive mobility technology services, and new sources of data generated and consumed by urban and mobility processes. However, the environmental, social, and economic sustainability implications of these new mobility services are unclear given the complex nature of urban systems and the multifaceted, contested nature of sustainability goals. In this article, we discuss the concept of urban sustainability observatories that leverage urban experimentation through ongoing data collection and analysis capabilities. The goal is to generate new scientific insights and design effective policies to meet sustainability goals for cities. We outline their functional requirements and related research challenges. We also discuss challenges in building and sustaining these observatories and how university, community, and industry partnerships may establish successful observatories that serve as critical drivers of research, technology transfer, and commercialization.

Keywords: data observatory, sustainability, urban experimentation, geospatial data, mobility data

1. Introduction

Civilization is facing three transformative changes that will profoundly impact humanity and the planet. First, an urban revolution: The world’s urban population became a majority for the first time in history in the early 21st century, and this urbanization trend is accelerating. By the end of this century, the vast majority of the 9–10 billion people on Earth will live in cities (Batty, 2018). The COVID-19 pandemic will impact cities, and while pandemics have occurred throughout history and none has fundamentally altered the trend toward urbanization, COVID-19 has revealed the flaws, tensions, and brittleness of our urban system (Batty, 2020). If human civilization is to survive and thrive in an urban world, we must determine how to create sustainable cities that foster economic growth in an environmentally friendly, socially just, and resilient manner (Basiago, 1998).

Second, we are experiencing a mobility revolution in many cities with the introduction of new mobility technologies and services, including light individual transport (e.g., e-scooters), shared vehicles, microtransit, and eventually connected and autonomous vehicles (CAVs). These technologies will have profound and disruptive effects on cities similar to the development of railroads in the 19th century and highways in the 20th century. Our urban systems are organized through transportation infrastructure; consequently, mobility technologies, services, and practices have profound impacts on urban morphology, landscapes, and ultimately sustainability. We need to ensure that we shape these technologies toward sustainable outcomes and not make the mistakes of the past.

Finally, a data revolution: Smart mobility and related urban data technologies generate massive data about various aspects of cities, including the activities and movements of people in them. These data are critical because cities are complex systems that behave in unpredictable ways that our simple 20th-century models and methods cannot capture adequately (Batty, 2013). These data present an opportunity to inform a new science of urban sustainability that recognizes this complexity and enables better policy and planning decisions during this time of urban transformation (Advisory Committee for Environmental Research and Education, 2018).

In this article, we discuss the concept of an urban sustainability observatory (USO). A USO treats the city as a complex system best understood one event or intervention at a time, and treats sustainability as a crucial but conflicted societal challenge that requires new forms of scientific, policy, and community collaborations (Carrozza, 2015; Funtowicz & Ravetz, 1994; Rittel & Webber, 1973). Ongoing data collection creates timely, always-ready benchmarks that can be leveraged with quasi and natural experimental designs and methods for extracting causal inferences in non-laboratory settings. This can enable new scientific and policy collaborations involving tactical urbanism: a growing trend where communities make local, provisional changes to infrastructures or policies to test impacts and iterate to better solutions before wide deployment; examples include pop-up bus lanes and opening streets for walking and biking on weekends (Silva, 2016). However, monitoring and analysis of interventions and events is not sufficient. Also required are tools and protocols for community-forward coproduction of knowledge and translation of knowledge to policy and practice.

We are calling for a next generation of urban data observatories and cyberinfrastructure that leverage urban experimentation and disruptions to generate new transdisciplinary insights into sustainable urban systems and policy actions. We focus our discussion on mobility due to its centrality to urban sustainability. It is also a domain that is particularly ripe with experimentation, disruption, and challenges. However, our core concepts and challenges are application-agnostic and can be applied to other domains. This article outlines a research agenda by describing the USO functional requirements, scientific and development challenges, and partnerships and relationships that can help build and maintain a USO.

2. Background

2.1. Mobility and Urban Sustainability

A need for new mobility solutions, combined with enabling technologies such as wireless communication and geo-positioning systems, has led to the development and rapid deployment of new technology-enabled mobility services such as vehicle sharing, transportation network companies (TNC) such as Uber and Lyft, micromobility such as e-scooters, and bike sharing systems. While the intent of these new technologies are benefits such as reducing car ownership, lowering emissions, improving safety, and supporting public transit, the evidence so far points in the opposite direction. TNCs are increasing traffic congestion, undermining public transit, and leading to higher energy consumption and emissions (Bliss, 2017, 2019, 2020; Hall et al., 2018).

Introducing disruptive mobility services to cities is a grand, real-world experiment that will impact cities for decades. Even larger impacts are likely with connected, autonomous vehicles (Fagnant & Kockelman, 2015; Stead & Vaddadi, 2019). Similar to other profound technologies such as electric motors computing and phone cameras, people will adapt them in unexpected, synergistic, and deeply transformative ways. A century ago, many thought the automobile was a solution to urban mobility challenges, especially those associated with horses in cities. In some ways they were, but they introduced a broad spectrum of unintended consequences with which we now struggle. A crowded, urbanized planet of 10 billion no longer has these margins for error.

Sustainability is a wicked problem: The overall goal is ill-defined and malleable, with conflicting elements and complex interdependencies (Rittel & Webber, 1973). The most widely accepted definition of sustainability is meeting the needs of current generations without compromising the needs and aspirations of future generations, inclusive across environmental, social, and economic dimensions (Basiago, 1998; Brundtland et al., 1987). How to measure these dimensions, their relative importance, and the acceptability of tradeoffs are highly contested questions, reflecting values, political stances, and perspectives on growth as a problem or solution. For example, weak-sustainability reflects a positive view of growth and technology, and views tradeoffs among natural and human capital as acceptable. The polar opposite is strong sustainability, a stance that is pessimistic about economic and material growth, is skeptical of relying exclusively on technological advancement as a strategy for continued improvements in well-being, and does not accept tradeoffs between human and natural capital (Neumayer, 2003). Between these extremes are myriad other sustainability stances with different values and possible tradeoffs, including recent developments in a weak sustainability plus (WS+) approach that can integrate complex interactions across capital stocks modeling resource allocation decisions (Irwin et al., 2016).

Urban sustainability evaluation depends not only on resolving these conflicting views on acceptable tradeoffs and values, but on dealing with the implications of evaluating these highly dynamic, globally defined criteria at sub-global levels. While the implications for the evolution of well-being over time in any sustainability stance are clearly defined (e.g., non-declining well-being over time), the scale of sustainability evaluation is more ambiguous (Hiremath et al., 2013). Cities do not exist in isolation; they rely on flows of energy, material, and people from other cities and other countries. A city might be growing its economy, providing private and public goods, securing ecosystem health and services, working toward carbon neutrality in production, and distributing the flows from its wealth equitably. This city may seem to be quite sustainable in aggregate; however, this ignores the world outside the boundaries that urban systems rely on that could be quite unsustainable by either strong or weak criteria (Verma & Raghubanshi, 2018). Additionally, it ignores the relative contributions toward urban sustainability that different land uses, capital stocks, public goods, and other characteristics have within the city; these spatially dependent determinants or contributors make a city more or less equitable, prosperous, and resilient depending upon how and where they are embedded within urban systems. Evaluating a given city’s progress toward sustainability requires internalizing these external consequences (Verma & Raghubanshi, 2018) and gauging how well certain areas and subsystems of urban systems contribute to resident well-being over social, economic, and environmental dimensions (e.g., the productivity of space, inclusive of all aspects of human well-being).

2.2. Urban Complexity and Experimentation

The impact of mobility services on sustainability and other goals is an open question, one that will be difficult to answer using 20th-century transportation science, engineering, and planning. Because policy priorities were focused on the automobile and the ease of mobility, data technologies have evolved largely around collecting information about traffic flows or vehicles per unit time. Twentieth-century transportation models also treat travel demand as undifferentiated flow, like water. Consequently, we made traffic congestion worse by trying to build bigger roads, neglecting that, unlike water, we can induce people to travel more (Duranton & Turner, 2011). This historic view fails to see humans as the fundamental driver of multimodal transportation systems, acting to fulfill their multifaceted needs, across heterogenous urban space, interacting with other components of urban social, economic, and physical systems.

Transportation and urban science in the 20th century were dominated by a view of urban systems as a machine, one comprised of simple causal relationships that could be understood in the aggregate and engineered to achieve a desired equilibrium or steady state. That framework is no longer tenable in the 21st century. The impacts of new transportation and communication technologies are accelerating and amplifying urban dynamics. Twenty-first-century cities are complex systems that exhibit intricate feedback loops, path dependency, and emergent behavior. This limits the power of traditional urban science, which uses location as a central organizing principle and views the system as evolving toward a steady state or equilibrium. Instead, the new science of cities views urbanity as a multilevel system of flows and networks that is always in disequilibrium and evolving (Batty, 2013). Ignoring the intricate feedback loops, path dependency, and emergent behaviors that characterize complex urban systems is the reason that well-intended policy interventions can have ineffectual, uneven, and counterintuitive outcomes (Pollock, 2016). While predicting the future of complex urban systems is impossible, it is possible to invent these futures by understanding and shaping the city as an ongoing process (Batty, 2018).

Recognizing the complexity of urban systems, a National Science Foundation consensus report calls for a new convergence science of sustainable urban systems (SUS) that is multiscale, transdisciplinary, and actively advances science-policy-community partnerships. The report calls for the study of urban systems at multiple levels, including sustainability outcomes in a single urban area, multiple cities, and communities to explore interrelationships and identify urban typologies, and supra-aggregations of metropolitan areas to assess the collective impact of urban transformations. The next generation of SUS science requires the development of new data and methods to help us understand the interactions among natural, human-built, and social systems that drive sustainability across multiple scales and the tradeoffs among the environmental, economic, and social dimensions of sustainable urban development. The report also calls for knowledge coproduction among researchers, communities, industry groups, practitioner groups, and governments at multiple levels, leveraging real-world experimentation ongoing in urban areas (Advisory Committee for Environmental Research and Education, 2018).

From a complex system perspective, urban processes are a type of experimentation where the system tests, receives feedback, and adapts, one small step or giant leap at a time, depending on the scale (e.g., a single property versus a large-scale development; see Marohn, 2019). Tactical urbanism is a form of purposeful urban experimentation. Originally emerging from resident-led initiatives, tactical urbanism involves making low-cost, provisional, and sometimes unsanctioned changes to the built environment, such as chair bombing to create social spaces, guerrilla gardening, and intersection ‘repair’ (refurbishing to slow cars and facilitate walking) (Lydon & Garcia, 2015). As more urban planners embrace the perspective of cities as complex systems, they are accepting tactical urbanism as a type of bottom-up, local adaptation that coevolves with top-down planning and interventions. They are also adapting tactical urbanism methods to test impacts of an infrastructural or policy change before wider deployment; examples include pop-up bus lanes and opening streets for walking and biking on weekends. A challenge is how to integrate these local interventions with coherent understanding and change at the urban scale (Silva, 2016).

Evidence-based policy has the potential to make stronger causal claims than cross-sectional studies by leveraging longitudinal data and real-world experiments. However, most policy evaluations tend to focus on a small number of design features with a relatively short duration (Hausmann, 2016). Linking a longitudinal set of measures with the ability to visualize and communicate location-specific ‘snapshots’ could allow for better policy design, propagation of local best practices (Watts, 2011), and reflexive monitoring as policies adjust based on incremental knowledge gained on their outcomes over time (Sanderson, 2002).

2.3. Data Observatories and Opportunistic Science

Observatory science involves open-ended data collection afforded by an advantageous platform or motivated by an imperative situation, such as a hilltop with a clear view of the sky, a pristine rainforest, or a volcano. Observatories have specialized equipment to aid in the data collection and an institutional structure to organize the work and generate science. Observatories have a long history of studying astronomical and terrestrial phenomena such as stars, oceans, climate, ecosystems, and volcanic activity. Some pioneers extended this concept to complex human phenomena such as demography, cities, and public health (e.g., Hemmings & Wilkinson, 2003; Lasswell, 1967; Williams, 1972). Observatories facilitate both scientific discovery (hypothesis generation) and confirmatory science (focused investigation). They can also facilitate direct policy interventions and actions, such as volcano eruption warnings (Miller, 2017). Finally, observatories enable convergent science by serving as critical research infrastructure on which researchers from multiple fields can pursue integrated research that advances each of the disciplines involved while addressing large-scale research questions of significant scientific impact.

The growth of the World Wide Web and cloud computing linking databases, media, services, and people has led to the growth of observatories that enable scientific discoveries about the rapidly exploding universe of online information and its interactions with processes and behaviors in the real world (Hendler, 2013). Similarly, the growth of location-aware and wearable technologies such as GPS-receivers, mobile phones, sensors, and smart infrastructure motivates the development of geographic information observatories (GIOs) that allow researchers to explore the rapidly expanding universe of geo-referenced information, and observe the world via the geospatial data it produces and consumes (Janowicz et al., 2014).

The ability to capture, store, and analyze data on activities, movements, flows, processes, infrastructure, and the environment in basis can allow new types of opportunistic science that enables the coproduction, shared understanding, and application of new knowledge about complex urban systems. Opportunistic science involves the observation of processes as they unfold in specific geographic contexts, over time periods sufficient to expose multiscale temporal dynamics (Liu et al., 2007). It also involves leveraging real-world events via natural and quasi experimental methods to support stronger causality claims. It also involves experimenting and visioning via simulation when real-world experiments are infeasible or we need to go beyond direct empirical evidence (Miller, 2017).

Science gateways support researchers in using cyberinfrastructure to share data and tools of analysis (Barker et al., 2019). Furthermore, researchers have developed user interfaces through cyberinfrastructure to enable research-based decision-making among stakeholders in areas such as environmental management (Biehl et al., 2017; Demir et al., 2018). Yet through the aligned urban, mobility, and data revolutions there is now a growing interest in aggregating urban spatial data that is shared across academic, public, and private sectors. Table 1 summarizes major existing urban observatories and data platforms. Note that we omit here those open-source projects or services that provide only open data-sharing capability as used by many municipalities. We also omit projects focused on a single data source or type and instead include projects that bring together urban data across domains. Academic projects such as the Newcastle Urban Observatory and Array of Things focus on collecting, curating, and publishing sensed and other urban data. The Kinder Institute Urban Data Platform and NYU CUSP Data Hub additionally supply flexible statistical computing environments. Further, their scope is city scale, as opposed to neighborhood or community scale. Corporate products such as the IBM Intelligent Operations Center or Urban Footprint offer robust analytic tools but are ‘enterprise’ platforms geared toward policy decision-making rather than also encompassing research. Populus is similarly a private sector tool designed to support municipalities in decision-making around new forms of mobility but with a focus on efficiency.

Table 1. Existing urban observatories and data platforms.

Urban data centers, networks, and observatories have also been used successfully to form networks for research and public policy. For example, the Australian Urban Research Infrastructure Network (AURIN), a government initiative, supports a collaborative network of researchers and data providers across academic, government, and commercial sectors to address urban policy questions such as equity and access to health care services across the country. The Urban Big Data Centre, at the University of Glasgow, provides national data and supports data-enabled research aimed at exploring a range of issues around public infrastructure in U.K. cities.

While these data observatories and platforms have enabled unprecedented access to urban data and new forms of urban science, we suggest that they fall short of enabling the kinds of knowledge coproduction, partnerships, and shared understanding that are vital for actionable next-generation sustainable urban systems science. Our vision of an urban observatory is certainly complementary to those in Table 1, but one that is aimed at a neighborhood and community, codesigned by researchers working with community stakeholders and the community itself, and aimed at aspects of sustainability that are of specific relevance to the community, are identified by them, and hence come with their buy-in. In addition, the architecture of our observatory is different. We aim for small-scale data collection of multiple types of data, including data collected through human observations. We also include messaging and actuation as part of the requirements of the observatory; in other words, our urban observatory will not only collect and curate data, but also facilitate engagement of the community, through messaging and storytelling, and finally, enable measurement of the effects of policies and direct interventions. Finally, our observatory is intended to support opportunistic science (as explained previously).

In the next section, we describe our vision for creating the next generation of urban data platforms to enable new forms of SUS science for mobility and identify expected challenges.

3. Urban Sustainability Observatories

An urban sustainability observatory (USO) goes beyond current generation urban data observatories and cyberinfrastructure to provide capabilities that are well-suited for scientific discovery, experimentation, and evidence-based policy for complex urban systems where the goal —sustainability—is a wicked problem. A USO is a platform for facilitating partnerships and new ways in which such partnerships can enable public engagement in ways that are practical not only for advancing urban science but also for facilitating exchange of information and co-learning by researchers and community stakeholders, ultimately leading to better decision-making by local communities.

A USO focuses on a community of interest that is geographically bound—generally, an urban area or a metropolitan region. A network of USOs across multiple communities further supports SUS science by enabling comparisons among experiments and best practices in different times and places, exploring interrelationships among networks of cities, and identifying cohort groups for comparative analysis (Advisory Committee for Environmental Research and Education, 2018).

This section discusses the major functionality required to create USOs. This includes: i) integrated geospatial database and services, ii) tools for opportunistic science, iii) capabilities for building knowledge from local events, and iv) embracing of uncertainty and heterogeneous values in constructed knowledge. These functionalities should be embedded within a scalable, open, and collaborative cyberinfrastructure.

3.1. Integrated Geospatial Database and Services

The core of a USO is a database that integrates spatial, temporal, moving objects, and streaming data across the economic, social, and environmental dimensions of sustainability, using georeferences as the common element. Using geography as a central organizing principle for the data and interfaces (via map-based visualizations and queries), the USO provides a common platform for otherwise fragmented and siloed databases on disparate systems. The boundary for the observation domain is also geographic; delineating a meaningful community of interest. For policy relevance, this boundary will typically follow administrative geography, such as political boundaries or the region covered by the local metropolitan planning organization.

A crucial design decision is specifying the sustainability indicators to be collected by the observatory. There are valid reasons to collect all data possible to support discovery and open-ended investigation, but the increasing volume of urban data makes this untenable. The selection of indicators should balance: i) science—reflects the best settled science on urban sustainability, ii) locality—reflects local issues, interests, and policy levers, and iii) comparability— consistent with principles for constructing sustainability indicator frameworks (Bell & Morse, 2012; Singh et al., 2009; Verma & Raghubanshi, 2018) and established frameworks (e.g., International Organization of Standardization [ISO], 2018; Lynch et al., 2011; United Nations, 2007) to allow benchmarking.

The data in a USO should be curated and ready-to-use: cleaned, documented, co-registered, or related based on georeferencing. Data curation is likely to be a human-centric rather than machine-centric process, at least for the immediate future, although artificial intelligence (AI) and machine learning techniques can support the human curators. A USO also benefits from a data concierge service for linking and organizing data, and supporting analysis, mapping, and visualization. User-friendly interfaces and guided procedures that draw from a knowledge base could help guide these processes, similar to commercial geographic information system (GIS) software. For more complicated processes that defy simple rules, the USO could apply machine learning and AI methods to gain knowledge from other users’ interactions. This could involve a rating or feedback system to asses processing and analytical outcomes—a difficult, multidimensional task that could include users’ subjective ratings.

3.2. Tools for Opportunistic Science

A powerful benefit of a USO is always having measurements ready when planned interventions or unplanned disruptions occur, such as a change in bus service in a corridor, a micromobility company deploying scooters on city streets, an extreme storm flooding an urban highway, or a global pandemic leading to shutdowns and travel restrictions. The continuous, open-ended data collection facilitated by a USO allows timely baselines that can help determine the causal effects of these interventions and events.

Although lab-based, randomized control trials (RCTs) are the scientific gold standard for inferring causality, randomly assigning participants to case (treatment) or control groups is not feasible or ethical in non-lab settings or situations. Quasi and natural experimental techniques use statistical controls to replicate RCTs in empirical settings with little control over assignment to case and controls (Remler & Van Ryzin, 2015). The growing availability of data makes these designs feasible in new domains such as ecology (Butsic et al., 2017), public health (Craig et al., 2017), and mobility and urban science. Examples include bicycle education programs (Goodman et al., 2016), air quality status and housing values (Chay & Greenstone, 2005), impacts of new public transit on neighborhoods (Cao & Fan, 2012; Diao et al., 2017; Pilgram & West, 2018), lower speed limits and road trauma (Grundy et al., 2009), and road fatalities after public sector budget cuts (De Angelo & Hansen, 2014). Challenges include scalability, dealing with varying data quality and missing data, unknown sampling frameworks and representation biases, nonparametric methods, and methods that account for spatial dependency and heterogeneity; for example, spatial difference-in-differences methods (Diao et al., 2017).

3.3. Building Knowledge Beyond Local Events

A limitation on extracting scientific knowledge from tactical urbanism and other urban experiments is that these are often local and provisional. A major challenge is how to translate these small-scale, ephemeral impacts to knowledge that can inform large-scale and enduring changes in a city. For example, how do we use the knowledge gained from a small set of pop-up bus lane experiments to understand the sustainability impacts of a community-wide bus rapid transit network? If we view each experiment as a geospatial data point, this is a type of data imputation problem involving spatial, temporal, and moving objects data, multidimensional outcomes and often problems where details of the built environment matter—for example, safety, energy and emissions, and active transportation. Also, network effects matter: There is a qualitative difference between a bike lane and a bike network. Recent research in advancing imputation methods for spatial, temporal, and moving objects data are occurring (e.g., Yang et al., 2018; Yao et al., 2020). This is an exceptionally challenging frontier for these methods.

Urban digital twins (Mohammadi & Taylor, 2017) have several uses in a USO. The ability to generate hypothetical futures supports exploratory scenario planning for dealing with uncertainty about the future and anchoring plans in stakeholder values (Avin & Goodspeed, 2020; Goodspeed, 2020). Digital twins also allow inference of behavioral and other hidden processes, and conduct sensitivity analysis to understand data limitations. They can also help identify locations, times, and modes for interventions and experiments, at least to a first approximation, by simulating impacts.

3.4. Embracing Uncertainty and Heterogeneous Values

Societal problems such as climate change, conservation, and sustainability share the double curse of wicked problems embedded within complex systems. Dissatisfaction with the contributions of traditional science to progress with solutions to these urgent problems has led to calls for a post-normal science (PNS) that jettisons scientific certainty and value neutrality to embrace uncertainty about the future and contested values (Carrozza, 2015; Funtowicz & Ravetz, 1994, 2020; Ravetz, 2006). In this paradigm, a scientist’s role in decision-making shifts from predicting the future to giving stakeholders an appreciation of how the future may unfold. This involves understanding the possible future states, the conditions under which these futures may occur, the tradeoffs that the different states represent, the actions that can help adaptation to different states, and the degree of confidence in this knowledge (Kay et al., 1999). PNS is a powerful critique, and its principles can serve as a policy framework (e.g., Petersen et al., 2011), but it has not been explored as a design standard for user interaction with data and systems.

A possible strategy for integrating heterogenous values is visual exploration and analytical reasoning via sustainability lenses. A sustainability lens is defined over a set of indicators spanning the three dimensions of sustainability. These can be based on existing sustainability indicator frameworks. Users can also create bespoke lenses that reflect their values and tradeoffs. The stances that underlie these lenses can be elicited as part of the lens construction process and made transparent and shareable. These indicators can be mapped and visualized independently or combined into overall index value using multicriteria analysis (MCA) methods that capture acceptable and unacceptable tradeoffs among sustainability indicators (Gan et al., 2017; Miller et al., 2013). Capabilities for going beyond mapping to support spatial reasoning with indicators are improving (Allain et al., 2017; Boggia et al., 2018), but are still lagging.

The storymap concept developed and implemented by the GIS software company ESRI is a good conceptual foundation for integrating qualitative and quantitative data. This involves interactive maps with georeferenced text, figures, and multimedia (Cope et al., 2018). Design and evaluation of geospatial and map-based, mixed methods in participatory decision-making is a domain that is just beginning to be explored (Brown et al., 2017; Shay et al., 2016).

3.5. Scalable, Open, and Collaborative

The other requirements described in this section may appear daunting: ingesting, processing, and archiving massive amounts of data and enabling convergent science and shared decision-making across diverse stakeholders based on these data and coupled high-resolution simulations. They appear daunting because they are daunting. The USO framework dovetails with the CyberGIS framework that integrates geographic information science, cyberinfrastructure, and e-science. This framework calls for a spatiotemporal data platform that is scalable, open, distributed, user-centric, service-oriented, and community-driven in its design and operation (Wang, 2016; Wang et al., 2013). There have been some forays into CyberGIS designs for participatory decision-making (Nyerges et al., 2013), citizen science (Yin et al., 2017), and crucial, time-critical applications such as disaster management and flooding (Huang et al., 2015; Liu et al., 2016). A USO is another frontier application domain for CyberGIS.

4. Data Challenges

This section discusses some the data challenges surrounding USOs. These challenges include data standards, biases, governance, and privacy. These are common data challenges across many platforms, but they are risky deal-breakers in community applications and partnerships, and they deserve special attention.

4.1. Data Standards

Mobility and urban data are fragmented, with organizations operating within the same community disagreeing about basic semantics, particularly across transportation modes, which are traditionally siloed in planning and operations. A famous, perhaps apocryphal example, is a highway and rail agency disagreeing about the definition of a highway-railroad crossing and unable to share data. Even seemingly simple mobility concepts such as a ‘trip’ can have varying definitions depending on the questions asked. Also, public sector agencies have traditionally developed in-house definitions, measures, and formats.

As data become more plentiful—and crucial—to mobility science and planning, there are growing efforts to create standards. There is a long-standing effort to create standards with household travel surveys, the data resource used most often to model and forecast demand as well as understand behavior (Stopher et al., 2006). General Transit Feed Specification (GTFS) is a very successful standard allowing public transit agencies to publish their transit data in a format that can be utilized to support a variety of software applications. Other examples include the Mobility Data Collaborative for developing mobility data sharing frameworks and the Mobility Data Specification (MDS) developed by the Los Angeles Department of Transportation for exchanging data between mobility service providers and the city. Data models and standards for stationary urban entities are more mature; for example, the Open Geospatial Consortium is developing for geospatial and 3D urban data, including building information models and digital twins (Milner & Liberman, 2019). These efforts must continue as the mobility data universe continues to expand and new players enter the mobility arena.

4.2. Data Biases

Measured data can have biases and measurement errors, but also important are biases that arise from a lack of data collection, especially data that we do not or cannot easily obtain. To date, smart city and smart mobility data efforts tend to focus on ‘low-hanging fruit,’ that is, data that can be easily measured or sensed, such as infrastructure, mobility patterns, and other physical properties of the system. The use of new, novel sensing technologies and sensor fabrics (Sridharan et al., 2010) should be explored to collect new types of data and to bridge gaps in data. Next, difficult to measure and complex social concepts such as equity, social capital, neighborhood stability, and well-being have not received as much attention. As discussed above, the value and effectiveness of qualitative and unstructured mobility data is underexplored; this can provide deeper understanding of the conditions superficially illuminated by quantitative data. Finally, there is a need to be cognizant of historically marginalized individuals, neighborhoods, and communities. These entities are vulnerable to transportation deprivation, and have been historically underrepresented, undercounted, and ignored in traditional planning processes. This is particularly the case for data collected via smart phones, vehicle GPS devises, and web-based instruments. Further, information about trips not taken may be critical to understanding mobility needs of these populations, but these cannot be observed directly.

4.3. Data Governance

In the past, transportation data were largely generated from public investments or processes. Cities, states, or other public jurisdictions obtained information from publicly administered surveys or traffic counts. In the last 30 years, they have invested in sensors in major facilities on the transportation network to detect traffic flows. The data revolution has brought forward a wealth of transportation data now generated from location information in cell phones, smart phones, and associated applications, as well as supply and demand information from shared mobility platforms, such as ride hailing services, scooters, and bike sharing. These data provide information on a scale and scope not previously possible. However, unlike data collected by public agencies, these data are held by private companies, are considered proprietary, and are sometimes monetized as a commercial product. Thus, building USOs that incorporate these new sources of big data will require data-sharing agreements between public agencies, private firms, and universities. Today, these agreements are brokered between cities and firms or individual researchers and firms and, thus, vary considerably from place to place. This hampers the construction of a USO, and unevenness in data availability limits comparisons across cities even within the same geographic regions. This raises fundamental governance issues with USOs—who has control of and access to public and privately held resources?

4.4. Privacy

Embedded within data governance are issues of privacy and trust. Building trust with the community and ensuring privacy are key components to success. This includes communication with the public about what the data are, how they can be used, and protocols for guaranteeing good stewardship. New approaches to community engagement are necessary to use community wisdom in more effective, genuine, and transparent ways. Coproduction of knowledge, citizen science collaborations, and community-led initiatives are products of trusting relationships. Managing data privacy is critical but complex, requiring accounting for continually changing technology, different perceptions of the meaning of privacy, and managing historic distrust of public agencies. Cities are starting to address these issues, for example, the City of Portland has recently developed and passed a set of Data Privacy Principles, but there are still many unanswered questions (see Khatoun & Zeadally, 2017).

5. Creating Successful USOs via Partnerships

Partnerships are a promising way to build, maintain, and apply USOs to help shape cities and transportation toward more sustainable outcomes. Public officials and other stakeholders are grounded in the challenges and opportunities in the community, but often lack capacity and expertise to leverage the new forms of data and computational techniques that enable USOs. Universities are repositories of knowledge and diverse skills with faculty, staff, and students who are working on cutting-edge techniques and solutions but are often at a level of abstraction removed from the realism, intricacies, and complexities of the community. The private sector is the innovation and commercialization engine for new digital technologies, and controls massive amounts of digital information on individual trips, payments, and demands for their services. In this section, we discuss some of the ways we can create productive USO partnerships engaging universities, communities, and the private sector.

Establishing long-term collaborative research, education and translational pipelines. A steady flow of collaborative activities centered on the USO, and for which the USO serves as essential infrastructure, is its lifeblood. The primary activity must be research, because the resultant knowledge creation will drive education, outreach, translation, and commercialization. In particular, a USO enables convergent research: integrated research across multiple fields that advances each of the disciplines. For example, convergent research in energy sustainability may require advances in nonintrusive sensing (say, of thermal fields and vibrations within households) that can help build personalized profiles of energy use within homes. Such research may also require novel machine learning techniques that can build accurate models from sparse amounts of data, and reduce bias in providing explanations for the insights these models generated. Within the same research stream, researchers in behavioral psychology may study personalized incentive models, while political scientists may study the role of government.

Key to success in research and development around a USO is the establishment of active mechanisms to identify novel research questions, form teams of researchers and students to address these questions, make available the resources needed for the projects (computation, meetings spaces, courses, collaborative tools, etc.) and, certainly, funds to support this activity (see Ramanathan et al., 2013, for a comprehensive example). Essentially, what is needed is a research engine for the USO that propels the activities that the USO can then support.

Business models. Defining the financial and business model for data observatories is a key concern, since this sets the foundation for funding, long-term viability, data ownership and access, and community engagement. Private interests—companies such as Google, Apple, telecommunications providers, and so on—have assembled proprietary, large-scale ‘big’ data around mobility that offer advantages over traditional transportation sources, such as household travel surveys and single modal data streams, such as traffic counts or transit feeds. They offer the advantages of much larger temporal, spatial, and population samples, which allow for longitudinal study and natural experiments. However, they often have a much narrower focus and their data are proprietary, documentation is opaque, have use and access restrictions, and are expensive. The question of how a data observatory, comprising multiple data sources and serving multiple stakeholders, is funded also raises questions of access if a fee-for-service model needs to be part of the financial model.

There are important considerations about where a data observatory resides and who owns, maintains, and disseminates data from this resource. Since we envision USOs as a public asset—one that provides non-excludable benefits to a large group of users—there is a case for data observatories to be owned and controlled by public entities. However, public officials are likely to be constrained by internal capacity and skills to take on such responsibilities; whereas support from and partnerships with universities is a potentially fruitful way forward. Further, a data observatory formed with the objective of sustainable mobility will have complex data needs from a variety of sources. How does it intersect with other data collection efforts and their ownership, control, and business models? There is also a need for ways to encourage (or mandate) data sharing from private entities that benefit from public investments and operate on public right of way.

University-community partnerships. To create and sustain a successful partnership, institutions and agencies require long-term commitments of resources, time, and personnel. Current models for community–university partnerships tend to be informal arrangements based upon individual relationships or project-specific efforts that are vulnerable to dissolution with personnel changes or at the conclusion of a grant or project. Further, universities and city agencies are complex organizations with very different hierarchies, operating principles, and needs. There are a few examples of successful commitments to these partnerships (Hennepin County-University of Minnesota; Chattanooga-University of Tennessee at Chattanooga). In these cases, common elements include: a staff liaison embedded in the university (or vice versa); financial commitments to fund projects of mutual interest; engagement with students and faculty; and regular meetings to set agenda, share progress, and create ideas. Extension faculty and staff in Land Grant Universities can also be highly effective in serving as trusted liaisons between the university and community partners.

Traditional promotion and tenure processes rarely provide adequate encouragement or incentives for faculty to build partnerships with communities and engage in civic-led, transdisciplinary research. If partnerships are to be meaningful and sustained, faculty need to get due credit for these activities, which currently fit outside the norm of expected activities for tenure-track faculty. Changing this paradigm will require buy-in from the larger academy, not just individual institutions, as it relates to guidelines for peer-review publications and promotion packages.

Workforce education and capacity building. Given the wealth of new data emerging from sensors, new mobility providers, and other technologies, staff of public agencies are struggling to keep place and need new skillsets with respect to database management, data science/analytics, and visualization techniques, in addition to specific knowledge about transportation and land use planning, engineering, environmental science, and so on. Universities are in a unique position to redesign degree programs, teaching modules, and workshops to help provide the education and training for the future and current workforce of public agencies. Partnerships can help inform professional education programs by improving responsiveness of university programs to labor needs, in particular the need for accelerating diversity and inclusion. Further, the broader public needs help in understanding their rights, how their data are used, and what the possibilities are for their communities to participate, collect, and use data themselves.

Technology transfer and dissemination of knowledge. One benefit of creating a network of community–university partnerships, beyond any individual city or institution, is broader and faster dissemination of knowledge and findings from research, use cases, innovative technologies, and policy approaches. While the context of urban mobility issues differs from location to location, there are many common themes, including the need to understand rapidly evolving technologies and their sustainable impacts. There are several existing networks for cities (e.g., National Association of City Transportation Officials, National League of Cities), universities (e.g., Association for the Advancement of Sustainability in Higher Education, Council of University Transportation Centers), and both (Transportation Research Board, MetroLab). Creating research networks around themes helps to build specific collaborations motivated around a common issue to advance the sharing of information and lead to smarter implementation.

Public agencies are often challenged by technical questions or issues that are out of the scope of staff expertise as well as consultant services. In addition, these issues are often subject to political debate and opinion. Faculty experts across disciplines can provide objective analysis and policy advising to help navigate many short-term investments/decisions that have long-term consequences. Further, most cities don’t have the capacity to conduct their own policy evaluation. Jointly developed research questions and agreed-upon approaches can help foster trust and encourage debate based upon evidence and science, rather than starting from subjective positions.

6. Conclusion

At this revolutionary time, USOs provide an incredible opportunity to take on the challenges of sustainable urban mobility armed with new scientific knowledge, mutually beneficial partnerships, and an engaged citizenry. We present a compelling and somewhat optimistic vision of how data can be harnessed to advance this agenda. However, the development and maintenance of these USOs will require time as well as significant investments of financial and intellectual resources. Taking on such an enterprise will benefit from the aid of a national network of invested partners to share best practices, develop standards, and contribute to a new science of sustainable urban systems.

Acknowledgments

This article is based on a workshop on Exploring a Research Network of Urban Sustainability Observatories via Data-Enabled University-Community Partnerships held in Columbus, Ohio, USA, July 15–16, 2019. We thank the participants for their time, engagement and ideas. We also thank Adam Porr, Michael Rayo, and Ningchuan Xiao (The Ohio State University) for their helpful thoughts and ideas.

Disclosure Statement

An award from the National Science Foundation (BCS-1929927) supported this research.

References

Advisory Committee for Environmental Research and Education. (2018). Sustainable urban systems: Articulating a long-term convergence research agenda. A Report from the NSF Advisory Committee for Environmental Research and Education. Sustainable Urban Systems Subcommittee, National Science Foundation. https://www.nsf.gov/ere/ereweb/ac-ere/sustainable-urban-systems.pdf

Allain, S., Plumecocq, G., & Leenhardt, D. (2017). How do multi-criteria assessments address landscape-level problems? A review of studies and practices. Ecological Economics, 136, 282–295. https://doi.org/10.1016/j.ecolecon.2017.02.011

Avin, U., & Goodspeed, R. (2020). Using exploratory scenarios in planning practice: A spectrum of approaches. Journal of the American Planning Association, 86(4), 403–416. https://doi.org/10.1080/01944363.2020.1746688

Barker, M., Olabarriaga, S.D., Wilkins-Diehr, N., Gesing, S., Katz, D.S., Shahand, S., Henwood, S., Glatard, T., Jeffery, K., Corrie, B., & Treloar, A. (2019). The global impact of science gateways, virtual research environments and virtual laboratories. Future Generation Computer Systems, 95, 240–248. https://doi.org/10.1016/j.future.2018.12.026

Basiago, A. D. (1998). Economic, social, and environmental sustainability in development theory and urban planning practice. Environmentalist, 19(2), 145–161. https://doi.org/10.1023/A:1006697118620

Batty, M. (2013). The new science of cities. MIT Press.

Batty, M. (2018). Inventing future cities. MIT Press.

Batty, M. (2020). The coronavirus crisis: What will the post-pandemic city look like? Environment and Planning B: Urban Analytics and City Science, 47(4), 547–552. https://doi.org/10.1177/2399808320926912

Bell, S., & Morse, S. (2012). Sustainability indicators: Measuring the immeasurable? Routledge.

Biehl, L. L., Zhao, L., Song, C. X., & Panza, C. G. (2017). Cyberinfrastructure for the collaborative development of U2U decision support tools. Climate Risk Management, 15, 90–108. https://doi.org/10.1016/j.crm.2016.10.003

Bliss, L. (2017, February 24). What's behind declining transit ridership nationwide? CityLab. https://www.citylab.com/transportation/2017/02/whats-behind-declining-transit-ridership-nationwide/517701/

Bliss, L. (2019, August 5). How much traffic do Uber and Lyft cause? CityLab. https://www.citylab.com/transportation/2019/08/uber-lyft-traffic-congestion-ride-hailing-cities-drivers-vmt/595393/

Bliss, L. (2020, February 25). Ride-hailing isn’t really green. CityLab. https://www.citylab.com/environment/2020/02/uber-lyft-pollution-data-carbon-emissions-ride-hailing-study/607063/

Boggia, A., Massei, G., Pace, E., Rocchi, L., Paolotti, L., & Attard, M. (2018). Spatial multicriteria analysis for sustainability assessment: A new model for decision making. Land Use Policy, 71, 281–292. https://doi.org/10.1016/j.landusepol.2017.11.036

Brown, G., Strickland-Munro, J., Kobryn, H., & Moore, S. A. (2017). Mixed methods participatory GIS: An evaluation of the validity of qualitative and quantitative mapping methods. Applied Geography, 79, 53–166. https://doi.org/10.1016/j.apgeog.2016.12.015

Brundtland, G. H., Khalid, M., Agnelli, S., Al-Athel, S., & Chidzero, B. (1987). Report of the World Commission on Environment and Development: Our common future. United Nations.

Butsic, V., Lewis, D. J., Radeloff, V. C., Baumann, M., & Kuemmerle, T. (2017). Quasi-experimental methods enable stronger inferences from observational data in ecology. Basic and Applied Ecology, 19, 1–10. https://doi.org/10.1016/j.baae.2017.01.005

Cao, X., & Fan, Y. (2012). Exploring the influences of density on travel behavior using propensity score matching. Environment and Planning B: Planning and Design, 39(3), 459–470. https://doi.org/10.1068/b36168

Carrozza, C. (2015). Democratizing expertise and environmental governance: Different approaches to the politics of science and their relevance for policy analysis. Journal of Environmental Policy & Planning, 17(1), 108–126. https://doi.org/10.1080/1523908X.2014.914894

Chay, K. Y., & Greenstone, M. (2005). Does air quality matter? Evidence from the housing market. Journal of Political Economy, 113(2), 376–424. https://doi.org/10.1086/427462

Cope, M. P., Mikhailova, E. A., Post, C. J., Schlautman, M. A., & Carbajales-Dale, P. (2018). Developing and evaluating an ESRI story map as an educational tool. Natural Sciences Education, 47(1), 1–9. https://doi.org/10.4195/nse2018.04.0008

Craig, P., Katikireddi, S. V., Leyland, A., & Popham, F. (2017). Natural experiments: An overview of methods, approaches, and contributions to public health intervention research. Annual Review of Public Health, 38, 39–56. https://doi.org/10.1146/annurev-publhealth-031816-044327

DeAngelo, G., & Hansen, B. (2014). Life and death in the fast lane: Police enforcement and traffic fatalities. American Economic Journal: Economic Policy, 6(2), 231–257. https://doi.org/10.1257/pol.6.2.231

Demir, I., Yildirim, E., Sermet, Y., & Sit, M. A. (2018). FLOODSS: Iowa flood information system as a generalized flood cyberinfrastructure. International Journal of River Basin Management, 16(3), 393–400. https://doi.org/10.1080/15715124.2017.1411927

Diao, M., Leonard, D., & Sing, T. F. (2017). Spatial-difference-in-differences models for impact of new mass rapid transit line on private housing values. Regional Science and Urban Economics, 67, 64–77. https://doi.org/10.1016/j.regsciurbeco.2017.08.006

Duranton, G., & Turner, M. A. (2011). The fundamental law of road congestion: Evidence from US cities. American Economic Review, 101(6), 2616–2652. https://doi.org/10.1257/aer.101.6.2616

Fagnant, D. J., & Kockelman, K. (2015). Preparing a nation for autonomous vehicles: Opportunities, barriers and policy recommendations. Transportation Research Part A: Policy and Practice, 77, 167–181. https://doi.org/10.1016/j.tra.2015.04.003

Funtowicz, S. O., & Ravetz, J. R. (1994). Uncertainty, complexity and post‐normal science. Environmental Toxicology and Chemistry, 13(12), 1881–1885. https://doi.org/10.1002/etc.5620131203

Funtowicz, S., & Ravetz, J. (2020). Post-normal science: How does it resonate with the world of today? In V. Šucha, & M. Sienkiewicz (Eds.), Science for policy handbook (pp. 14–18). Elsevier. https://doi.org/10.1016/B978-0-12-822596-7.00002-4

Gan, X., Fernandez, I. C., Guo, J., Wilson, M., Zhao, Y., Zhou, B., & Wu, J. (2017). When to use what: Methods for weighting and aggregating sustainability indicators. Ecological Indicators, 81, 491–502. https://doi.org/10.1016/j.ecolind.2017.05.068

Goodman, A., van Sluijs, E. M., & Ogilvie, D. (2016). Impact of offering cycle training in schools upon cycling behaviour: A natural experimental study. International Journal of Behavioral Nutrition and Physical Activity, 13, Article 34. https://doi.org/10.1186/s12966-016-0356-z

Goodspeed, R. (2020). Scenario planning for cities and regions: Managing and envisioning uncertain futures. Lincoln Institute of Land Policy.

Grundy, C., Steinbach, R., Edwards, P., Green, J., Armstrong, B., & Wilkinson, P. (2009). Effect of 20 mph traffic speed zones on road injuries in London, 1986–2006: Controlled interrupted time series analysis. BMJ, 339, Article b4469. https://doi.org/10.1136/bmj.b4469

Hall, J. D., Palsson, C., & Price, J. (2018). Is Uber a substitute or complement for public transit? Journal of Urban Economics, 108, 36–50. https://doi.org/10.1016/j.jue.2018.09.003

Hausmann, R. (2016, February 25). The problem with evidence-based policies. Project Syndicate. https://www.project-syndicate.org/commentary/evidence-based-policy-problems-by-ricardo-hausmann-2016-02

Hemmings, J., & Wilkinson, J. (2003). What is a public health observatory? Journal of Epidemiology and Community Health, 57(5), 324–326. https://doi.org/10.1136/jech.57.5.324

Hendler, J. (2013). Broad data: Exploring the emerging web of data. Big Data, 1(1), 18–20. https://doi.org/10.1089/big.2013.1506

Hiremath, R. B., Balachandra, P., Kumar, B., Bansode, S. S., & Murali, J. (2013). Indicator-based urban sustainability—A review. Energy for Sustainable Development, 17(6), 555–563. https://doi.org/10.1016/j.esd.2013.08.004

Huang, Q., Cervone, G., Jing, D., & Chang, C. (2015). DisasterMapper: A CyberGIS framework for disaster management using social media data. In Proceedings of the 4th International ACM SIGSPATIAL Workshop on Analytics for Big Geospatial Data (pp. 1–6). https://doi.org/10.1145/2835185.2835189

Irwin, E. G., Gopalakrishnan, S., & Randall, A. (2016). Welfare, wealth, and sustainability. Annual Review of Resource Economics, 8, 77–98. https://doi.org/10.1146/annurev-resource-100815-095351

International Organization of Standardization. (2018). ISO 37120: 2018 sustainable development of communities—Indicators for city services and quality of life. https://www.iso.org/obp/ui#iso:std:iso:37120:ed-2:v1:en

Janowicz, K., Adams, B., McKenzie, G., & Kauppinen, T. (2014). Towards geographic information observatories. In Proceedings of the Workshop on Geographic Information Observatories 2014, co-located with the 8th International Conference on Geographic Information Science (pp. 1–5). http://ceur-ws.org/Vol-1273/

Kay, J. J., Regier, H. A., Boyle, M., & Francis, G. (1999). An ecosystem approach for sustainability: Addressing the challenge of complexity. Futures, 31(7), 721–742. https://doi.org/10.1016/S0016-3287(99)00029-4

Khatoun, R., & Zeadally, S. (2017). Cybersecurity and privacy solutions in smart cities. IEEE Communications Magazine, 55(3), 51–59. https://doi.org/10.1109/MCOM.2017.1600297CM

Lasswell, H. D. (1967, August 5). Do we need social observatories? The Saturday Review, 49–52.

Liu, J., Dietz, T., Carpenter, S. R., Alberti, M., Folke, C., Moran, E., Pell, A. N., Deadman, P., Kratz, T., Lubchenco, J., & Ostrom E. (2007). Complexity of coupled human and natural systems. Science, 317(5844), 1513–1516. https://www.doi.org/10.1126/science.1144004

Liu, Y. Y., Maidment, D. R., Tarboton, D. G., Zheng, X., Yildirim, A., Sazib, N. S., & Wang, S. (2016). A CyberGIS approach to generating high-resolution height above nearest drainage (HAND) raster for national flood mapping. In Third International Conference on CyberGIS and Geospatial Data Science (Vol. 1), Urbana, Illinois. https://doi.org/10.13140/RG.2.2.24234.41925/1

Lydon, M., & Garcia, A. (2015). Tactical urbanism: Short-term action for long-term change. Island Press.

Lynch, A. J., Andreason, S., Eisenman, T. S., Robinson, J., & Steif, K. (2011). Sustainable urban development indicators for the United States: Report to the Office of International and Philanthropic Innovation, Office of Policy Development and Research, U.S. Department of Housing and Urban Development. Penn Institute for Urban Research White Paper Series on Sustainable Urban Development. http://works.bepress.com/theodore-eisenman2/8/

Marohn, C. L. (2019). Strong towns: A bottom-up revolution to rebuild American prosperity. John Wiley & Sons.

Miller, H. J. (2017). Geographic information observatories and opportunistic GIScience. Progress in Human Geography, 41(4), 489–500. https://doi.org/10.1177/0309132517710741

Miller, H. J., Witlox, F., & Tribby, C. P. (2013). Developing context-sensitive livability indicators for transportation planning: A measurement framework. Journal of Transport Geography, 26, 51–64. https://doi.org/10.1016/j.jtrangeo.2012.08.007

Milner, G., & Liberman, J. (2019, June 4). OGC using data standards for urban planning. Geospatial World. https://www.geospatialworld.net/article/ogc-is-developing-and-applying-data-standards-for-urban-planning/

Mohammadi, N., & Taylor, J. E. (2017). Smart city digital twins. In 2017 IEEE Symposium Series on Computational Intelligence (pp. 1–5). IEEE. https://doi.org/10.1109/SSCI.2017.8285439

Neumayer, E. (2003). Weak versus strong sustainability: Exploring the limits of two opposing paradigms. Edward Elgar, Northampton.

Nyerges, T. L., Roderick, M. J., & Avraam, M. (2013). CyberGIS design considerations for structured participation in collaborative problem solving. International Journal of Geographical Information Science, 27(11), 2146–2159. https://doi.org/10.1080/13658816.2013.770516

Petersen, A. C., Cath, A., Hage, M., Kunseler, E., & van der Sluijs, J. P. (2011). Post-normal science in practice at the Netherlands Environmental Assessment Agency. Science, Technology, and Human Values, 36(3), 362–388. https://doi.org/10.1177/0162243910385797

Pilgram, C. A., & West, S. E. (2018). Fading premiums: The effect of light rail on residential property values in Minneapolis, Minnesota. Regional Science and Urban Economics, 69, 1–10. https://doi.org/10.1016/j.regsciurbeco.2017.12.008

Pollock, K. (2016). Urban physics. Nature, 531(7594 Suppl. 1), S64–S66. https://doi.org/10.1038/531S64a

Ramanathan, J., Ramnath, R., Herold, M., & Wierwille, B. (2013). An agile translation process for complex innovations: An Industry/University Cooperative Research Center case study. In 2003 IEEE Frontiers in Education Conference (FIE), 1532–1538. IEEE. https://doi.org/10.1109/FIE.2013.6685094

Ravetz, J. R. (2006). Post-normal science and the complexity of transitions towards sustainability. Ecological Complexity, 3(4), 275–284. https://doi.org/10.1016/j.ecocom.2007.02.001

Remler, D. K., & Van Ryzin, G. G. (2015). Research methods in practice: Strategies for description and causation (2nd ed.). SAGE.

Rittel, H. W., & Webber, M. M. (1973). Dilemmas in a general theory of planning. Policy Sciences, 4(2), 155-169. https://doi.org/10.1007/BF01405730

Sanderson, I. (2002). Evaluation, policy learning and evidence-based policy making. Public Administration, 80(1), 1–22. https://doi.org/10.1111/1467-9299.00292

Shay, E., Combs, T. S., Findley, D., Kolosna, C., Madeley, M., & Salvesen, D. (2016). Identifying transportation disadvantage: Mixed-methods analysis combining GIS mapping with qualitative data. Transport Policy, 48, 129–138. https://doi.org/10.1016/j.tranpol.2016.03.002

Silva, P. (2016). Tactical urbanism: Towards an evolutionary cities’ approach? Environment and Planning B: Planning and Design, 43(6), 1040–1051. https://doi.org/10.1177/0265813516657340

Singh, R. K., Murty, H. R., Gupta, S. K., & Dikshit, A. K. (2009). An overview of sustainability assessment methodologies. Ecological Indicators, 9(2), 189–212. https://doi.org/10.1016/j.ecolind.2008.05.011

Sridharan, M., Zeng, W., Leal, W., Ju, X., Ramnath, R., Zhang, H., & Arora, A. (2010). From Kansei to KanseiGenie: Architecture of federated, programmable wireless sensor fabrics. In Magedanz T., Gavras A., Thanh N. H., & Chase J. S. (Eds.), Lecture notes of the institute for computer sciences, social informatics and telecommunications engineering: Vol. 46. TridentCom 2010: Testbeds and research infrastructures. Development of networks and communities (pp. 155–165). Springer. https://doi.org/10.1007/978-3-642-17851-1_12

Stanley, K., Bell, S., Kreuger, L. K., Bhowmik, P., Shojaati, N., Elliott, A., & Osgood, N. D. (2016). Opportunistic natural experiments using digital telemetry: A transit disruption case study. International Journal of Geographical Information Science, 30(9), 1853–1872. https://doi.org/10.1080/13658816.2016.1145224

Stead, D., & Vaddadi, B. (2019). Automated vehicles and how they may affect urban form: A review of recent scenario studies. Cities, 92, 125–133. https://doi.org/10.1016/j.cities.2019.03.020

Stopher, P., Wilmot, C., Stecher, C., & Alsnih, R. (2006). Household travel surveys: Proposed standards and guidelines. In P. Stopher, & C. Stecher (Eds.), Travel survey methods (pp. 19–74). Emerald Group Publishing Limited. https://doi.org/10.1108/9780080464015-002

United Nations. (2007). Indicators of sustainable development: Guidelines and methodologies (3rd ed).

Verma, P., & Raghubanshi, A. S. (2018). Urban sustainability indicators: Challenges and opportunities. Ecological Indicators, 93, 282–291. https://doi.org/10.1016/j.ecolind.2018.05.007

Wang, S. (2016). CyberGIS and spatial data science. GeoJournal, 81(6), 965–968. https://doi.org/10.1007/s10708-016-9740-0

Wang, S., Anselin, L., Bhaduri, B., Crosby, C., Goodchild, M. F., Liu, Y., & Nyerges, T. L. (2013). CyberGIS software: A synthetic review and integration roadmap. International Journal of Geographical Information Science, 27(11), 2122–2145. https://doi.org/10.1080/13658816.2013.776049

Watts, D. J. (2011). Everything is obvious—Once you know the answer. Crown Business.

Williams, L. A. (1972). The urban observatory approach: A decade of conceptualization and experimentation. Urban Affairs Review, 8(1), 5–20.

Yang, H., Yang, J., Han, L. D., Liu, X., Pu, L., Chin, S. M., & Hwang, H. L. (2018). A Kriging based spatiotemporal approach for traffic volume data imputation. PLOS One, 13(4), Article e0195957. https://doi.org/10.1371/journal.pone.0195957

Yao, X., Gao, Y., Zhu, D., Manley, E., Wang, J., & Liu, Y. (2020). Spatial origin-destination flow imputation using graph convolutional networks. IEEE Transactions on Intelligent Transportation Systems. Advance online publication. https://doi.org/10.1109/TITS.2020.3003310

Yin, J., Gao, Y., & Wang, S. (2017). CyberGIS-enabled urban sensing from volunteered citizen participation using mobile devices. In P. Thakuriah, N. Tilahun, & M. Zellner (Eds.), Seeing cities through Big Data (pp. 83–96). Springer, Cham. https://doi.org/10.1007/978-3-319-40902-3_5

©2021 by Harvey Miller, Kelly Clifton, Gulsah Akar, Kristin Tufte, Sathya Gopalakrishnan, John MacArthur, Elena Irwin, Rajiv Ramnath, and Jonathan Stiles. This article is licensed under a Creative Commons Attribution (CC BY 4.0) International license, except where otherwise indicated with respect to particular material included in the article.