Feature extraction and clustering analysis of highway congestion

Classification of congestion patterns is important in many areas in traffic planning and management, ranging from policy appraisal, database design, to prediction and real-time control. One of the key constraints in applying machine learning techniques for classification is the availability of sufficient data (traffic patterns) with clear and undisputed labels, e.g. traffic pattern X or Y. The challenge is that labelling traffic patterns (e.g. combinations of congested and freely flow areas over time and space) is highly subjective. In our view this means that assessment of how well algorithms label the data should also include a qualitative component that focuses on what the found patterns really mean for traffic flow operations and applications. In this study, we investigate the application of clustering analysis to obtain labels automatically from the data, where we indeed first qualitatively assess how meaningful the found labels are, and subsequently test quantitatively how well the labels separate the resulting feature space. By transforming traffic measurements (speeds) into (colored) images, two different approaches are proposed to extract the features of a large number of traffic patterns for clustering: point-based and area-based. The point-based approach is widely applied in the image processing literature, and explores local interest points in images (i.e. where large changes occur in color intensity); whereas a new area-based approach combines domain knowledge with Watershed segmentation to partition the images into different spatial-temporal segments from which domain specific features, such as wide moving jam patterns, are extracted. The results show that the Watershed segmentation separates the traffic (congestion) patterns into more meaningful and separable classes, comparable to those that have been proposed in the literature. Since there is no ground-truth set of labels, the quantitative assessment tests how well both methods are able to separate the respective feature spaces they construct for the (large) database of traffic patterns. We argue that the more crisp this separation is; the better the labelling has turned out. For this quantitative comparison we train a multinomial classifier that maps unseen patterns to the labels discovered by each of the two labeling approaches. The most important result is that the classifier using the area-based feature vector achieves the highest average levels of confidence in its decisions to classify patterns, implying a highly separable feature vector space. We argue this is good news! Not only does the combination of image processing (Watershed) and domain knowledge (traffic flow characteristics) lead to meaningful labels that can be automatically retrieved from large databases of data; this method also leads to a more efficient separation of the resulting feature space. Our next endeavor is to further refine and use this method to develop a search engine for the (rapidly growing) 200 TB historical database of traffic data hosted by the Dutch National Datawarehouse (NDW).