Data Quality

Data must be timely and accurate and cover most of the corridor for the model to generate meaningful results. Data quality affects network modeling and calibration, estimation of origin-destination trip patterns, evaluation of response plans, and more. The quality of the work thus depends directly on the quality of the data:

  • Missing data reduces situational awareness and the ability to locate routes with available capacity.
  • Bad data can lead to bad management decisions and worse traffic.

Ensuring data quality is therefore an ongoing requirement for an ICM system. Key aspects to consider include:

Basic detector health Data accuracy
  • Do we know and agree where the sensors are?
  • Are the sensors labeled and configured correctly (no sensors on the wrong side of the freeway, no HOV sensors mistaken for mainline sensors, etc.)?
  • Do the sensors capture a full cross section of traffic flow?
  • Are the sensors turned on and communicating data regularly?
  • Is the data provided by a detector trustworthy?
  • When we compare traffic flowing into and out of each section of freeway, do the numbers make sense (flow balance)?
  • Are data consistent with traffic engineering expectations?

The process the Connected Corridors team used to address these questions is described briefly in Processing Received Loop Data for the ICM System.

To help monitor data quality, the Connected Corridors team developed a web-based visualization tool for tracking sensor health, both on the freeway and surface streets:

Each row in the table represents one week; each column represents one sensor category. The values show the percentage of loops providing data for that week, calculated as the number of received detector-days of data divided by total possible detector-days of data.

For freeway data, for instance, the tool tracks average weekly sensor availability from Caltrans' Performance Measurement System (PeMS). Every week, the tool automatically updates the aggregated status of each sensor category along I-210, I-605, and SR-134 within the corridor pilot area, as well as a section of I-10.

Arterial data is available on the other tabs in the tool, and the team is working to expand the tool to eventually include all arterial sensor health data within the corridor.

Additional information on data quality can be found in the AMS Phase 2 presentation.