| Title | A System for the Acquisition and Analysis of Image Sequences to Model Longitudinal Driving Behavior |
|---|---|
| Author | Karimi Nejadasl, F. |
| Thesis advisor | Menenti, M.; Hoogendoorn, S.P. |
| Degree grantor | TU Delft, Delft University of Technology |
| Originator | TU Delft |
| Date | 2010-02-05 |
| Language | English |
| Type | doctoral thesis |
| Abstract | Traffic causes important problems in many societies. Considerable amounts of energy, money and time are wasted in traffic jams and even more important are car accidents and casualties in traffic (more than 40,000 deaths per year in the USA (Hitti (2005)) and 791 deaths in 2007 in the Netherlands (van Verkeer en Waterstaat (2008))). To alleviate the problems, traffic is continuously and intensively controlled and studied by authorities and researchers. The results of ongoing research on traffic lead to the development of new infrastructure facilities and rules. The behavior of drivers in different traffic situations, however, is still not sufficiently known. The driving behavior consists in the reaction of a driver to the action of surrounding vehicles. The stimuli can be the distance of the follower from the leader and the speed of the follower and leaders when the leaders change their behavior such as in changing lane and braking. As a result, the response of the follower to these actions may be predictable with stimuli-response models. This can be used, for example, to regulate traffic by advising a safe distance which the driver should keep when leading vehicles change their behavior. The findings on the driving dynamics can be used in dynamic traffic management and in advanced driver assistance systems such as adaptive cruise control. Dynamic traffic management controls the efficient use of infrastructure and predicts the effect of future changes on the infrastructure. Thus it allows more effective management and providing better traffic information such as in ramp metering, incident management, travel information (travel time, route recommendation). To get insight into drivers' behavior, it is, therefore, indispensable to reconstruct extended trajectories of observed vehicles for a long stretch of a freeway for a long time at high frequency under the same or nearly the same conditions, while satisfying positioning and speed accuracy requirements. Current systems for measuring the trajectories of each individual vehicle at any time are not sufficiently mature. This thesis describes a system which was designed and implemented for the collection of many long and detailed vehicle trajectories with the same or nearly the same conditions of weather, traffic, road and surroundings to improve stimuli-response models (longitudinal driving behavior). We evaluated possible platforms and sensors and a helicopter was chosen due to its widespread use for many applications, its flexibility and hovering properties. A camera with the resolution 1392x1040 and frame rate 15 was selected to record a sequence of images. The image sequence contains locations of vehicles as a function of time. We established that for our purpose the helicopter should hover over a freeway while looking in a nadir direction, with the camera mounted horizontally below the helicopter. The helicopter height is the main platform parameter affecting the characteristics of the recorded data: The increasing height implies to observe a longer stretch of a road is observed at higher flight altitude, but at the price of a larger ground sampling distance (GSD) value. For the camera the combination of CCD elements and the field of view (FOV) determines the GSD at a given height. A larger frame rate provides more details about vehicle trajectories. The system configuration, i.e. the set of camera and platform parameters, is determined taking also into account the ground slope and meet the given requirements. The proposed system acquires image sequences from a helicopter hovering over freeways. These image sequences contain both camera and vehicle motion. The vehicle motion is, therefore, contaminated by the camera motion. Current methods to remove the camera motion from an entire image sequence, i.e. performing image-sequence stabilization, require the determination of the scene position, which is a complicated and computationally expensive task. We therefore decided to develop a new procedure to stabilize the entire image sequence. We investigated in which cases corresponding pixels projected from a scene object to images in a sequence are related by the camera motion only without requiring the scene position. This relation is a so-called projective or homography. As a result, the relation between corresponding fixed points of two consecutive images and corresponding road area points in any two images is expressed by a homography. By knowing this homography, every given point in one image is transformed to the corresponding point in the other image. Only few reliable corresponding points are needed to establish each homography relation. Because of having a very large number of images from different areas, the automatic and robust extraction of corresponding points is required. The difficulties in finding correspondences are due to having pixels of moving objects, changing illumination conditions, image noise, existence of repeated patterns, scene elements with sparse structure, a large transformation which changes the perspective and scene occlusions. The robust registration methods are, therefore, investigated, designed and developed to handle these difficulties. Two different groups of methods were investigated for automatic and robust registration of two consecutive images. These methods were feature based and featureless. Two feature-based methods are introduced: KLT-RANSAC and SIFT-RANSAC with the difference being the procedure to find corresponding points between two consecutive images. In KLT, points are first extracted in one image and then tracked in the second image. In SIFT, the points are extracted from both images and then they are connected. These corresponding points are used to estimate parameters of the homography model between two consecutive images by RANSAC, which is a robust parameter estimator. In the featureless method, a DE-based registration procedure is designed and developed. Without explicitly extracting any corresponding points, this method uses all the image pixels. Parameters of the homography model between two consecutive images are obtained by minimizing the difference between one image and the transformed second image. We found out that the registration of the consecutive images is robust by any of the registration methods that we used. In contrast, the registration of the arbitrary images in the whole image area or only on the road area is not robust due to having a large number of wrong corresponding points. Consequently, a new framework was designed to stabilize the image sequence on the road area. This framework includes two steps: registration of two consecutive images and stabilization of the entire image sequence of the road area. After registration of two consecutive images, the registration of the entire sequence of images of the road area is done. The homography between two consecutive images is taken equal to the homography corresponding to the road surface. A product of this homography and the homography corresponding to the road surface between the previous image and a reference image (here the first image in the sequence), provides another homography, which registers the current image to the reference image on the road area. Such registration contains errors from both described homographies. As a result, the road areas are registered approximately due to small errors in the two homographies. By identifying the points in the road area of the reference image only, these points are matched in the transformed current image just around a very small search area. These corresponding points are used to calculate the homography parameters between the reference image and the transformed current image. The product of this homography by the approximate homography gives the homography corresponding to the road area of the current image and reference image. The same procedure is applied iteratively to the entire sequence. As a result, the road area in image sequence is registered. The image sequence is stabilized on the road area in an automatic, robust and reliable way, independently of differences in image content, traffic and environmental conditions. Besides, the new framework provides an error free sequential registration, without requiring computationally expensive bundle adjustment. The procedure is fast as well. No assumption is made on the camera motion or on the scene. After removing the camera motion from the entire sequence, vehicles are identified and tracked. The vehicles were identified by first distinguishing between background and vehicle pixel for each image by analyzing the gray-level temporal profile of each pixel. Background pixels are assumed to be discriminated by the higher frequency in a gray-level histogram. After identifying the vehicle and background pixels, clusters of vehicle pixels are grouped and indicated as a "blob". A blob and each pixel of it are then tracked using the optical flow method. These displacements are contaminated by errors which are removed by analyzing the histogram of the displacements. From the histogram, the most frequent value corresponds to the real vehicle displacement, which is the same for all pixels of a vehicle (a rigid object). The positions of the vehicles are updated after tracking them. The vehicles are identified in each image to detect new vehicles and update the boundaries of all vehicles. The vehicle extraction method detects many similar vehicles which contain only few details and with different motion, particularly vehicles having low speed and low contrast. Both vehicle extraction and tracking methods are sequential and only require very small amount of memory. The results are reliable due to the temporal connectivity in the vehicle extraction and spatio-temporal connectivity in the tracking method. Trajectories obtained with the system described above can be used to get insights into drivers' behavior towards improved predictive models. The system can be used for dynamic traffic management using the real time trajectory extraction by a hardware implementation of the procedure. If the real time extraction of trajectories is not an issue, trajectories can be improved by analyzing them and then give a feed back to the vehicle identification system. Integration of the results from spatio-temporal analysis can further improve the identification system. |
| Publication | http://resolver.tudelft.nl/uuid:0de8efcb-1c1c-4a16-8014-c27c... |
| Persistent Identifier | urn:NBN:nl:ui:24-uuid:0de8efcb-1c1c-4a16-8014-c27c64eb3dd1 |
| Metadata | XML |
Go to page top
Go back to contents
Go back to site navigation