Automatic extraction of road seeds from high-resolution aerial images

This article presents an automatic methodology for extraction of road seeds from high-resolution aerial images. The method is based on a set of four road objects and another set of connection rules among road objects. Each road object is a local representation of an approximately straight road fragment and its construction is based on a combination of polygons describing all relevant image edges, according to some rules embodying road knowledge. Each one of the road seeds is composed by a sequence of connected road objects, in which each sequence of this type can be geometrically structured as a chain of contiguous quadrilaterals. Experiments carried out with high-resolution aerial images showed that the proposed methodology is very promising in extracting road seeds. This article presents the fundamentals of the method and the experimental results, as well.


INTRODUCTION
Road extraction is of fundamental importance in the context of spatial data capturing and updating for GIS (Geographic Information Systems) applications.Substantial work on road extraction has been accomplished since the 70's in computer vision and digital photogrammetry, with pioneering works by, e.g., Bajcsy and Tavakoli (1976) and Quam (1978).At times, the use of the term 'extraction' is vague, invoking varied meaning among a diverse image analysis community.In this context, the task of road extraction is related to two subtasks, i.e.: recognition and delineation.By convention, road extraction algorithm is categorized according to the extent to which it addresses either subtask, thereby im-plying the relative level of automation (Doucette et al. 2001).Usually road extraction methods, which in principle do not need human interaction, are categorized as automatic and the opposite as semi-automatic.Thus, automatic methods address both road extraction subtasks and semi-automatic methods address only the geometric delineation of the roads, leaving the high-level decisions (i.e., the recognition) to a human operator, who uses his natural skill to set the meaning to the object 'road'.
With regard to semi-automatic methods, probably some existing ones can already be used into operational workflows.Semi-automatic approaches may be divided into two broad categories.The first includes road-following approaches, in which the road is sequentially traced by using only local road information (McKeown and Denlinger 1988, Vosselman and de Knecht 1995, Dal-Poz andSilva 2002).These approaches are usually initialized by two close seed points on the road, being one a starting point and another a point to define the road direction.The second category includes active contour models (Kass et al. 1987, Neuenschwander et al. 1997, Grüen and Li 1997, Agouris et al. 2000), simulated annealing (Trinder et al. 2000), and dynamic programming optimization (Merlet andZerubia 1996, Grüen andLi 1997), in which some type of simultaneous curve fitting is used.Usually, these approaches are initialized by a few seed points describing coarsely the road.
Until now, fully automatic methods for road extraction seem to be faraway from a mature state and, consequently, no such operational system is expected to be available in a near future.Fully automatic methods attempt to completely circumvent human intervention during the extraction process, in which two basic steps can be identified: 1. Road finding or road seed extraction.The whole area of interest is tested for local road property, and road segments (i.e., road seeds) are found (Zlotnick and Carnine 1993).Usually, the local road properties tested are geometric (e.g.: roads are smooth) and radiometric (e.g.: roads are usually lighter than the background) in sense.The results of this step are the road seeds or a fragmented road network.
2. Road network completion.This requires a skilful integration of contextual information (i.e., relations between roads and other objects like trees and buildings) and other a priori road knowledge into the road extraction methodology (Baumgartner et al. 1999).These highlevel information are used in strategies to automatically complete the road network, including the ones to bridge gaps between road seeds and to extract road crossings.At the end of the road extraction process, the goal is to obtain a complete road network, both in geometrical and topological senses.
This paper only addresses the first phase of process for fully automatic road network extraction.The motivation is the fundamental importance of road seeds for the subsequent phase.The potential success of road network completion strategies is significantly determined by the quality of the results of the first phase.This paper is organized in four sections.Section 2 presents the proposed methodology for automatic road seed extraction, which is essentially based on radiometric and geometric road constraints.Preliminary results are presented and discussed in Section 3. Conclusion and future perspectives are provided in Section 4.

Overview
We propose a methodology for road seed extraction which is based on a set of four road objects and another set of connection rules among them.Each road object is a local representation of an approximately straight road fragment and its building is based on a combination of polygons describing all relevant image edges.The polygons representing image edges are extracted using well-known image processing algorithms as, e.g., the Canny edge detection method followed by an edge linking algorithm and the split and merge algorithm (Parker 1997).After the polygon extraction some steps are needed to build road objects.Firstly, two polygons that are candidates for representing edges of a same road are selected.Now taking one polygon as reference, the algorithm tries to combine their straight-line segments to the ones of other polygon.Two straight-line segments of the different polygons are accepted to form a road object when: 1 -only two endpoints of them are orthogonally projected onto opposite straight-line segment, whose results are shown as circles in figure 1; 2 -they are close to parallel and the distance between them is compatible to the road width (figure 1); and 3 -the region between them satisfies road knowledge as, for example, road surface is usually homogeneous, road surface has good contrast to its adjacent areas, road width does not change much, road edges are anti-parallels etc.In other words, as stated before a road object is a representation for an approximately straight road fragment.Moreover, road objects can be classified in four types, but additional details are given in the following sections.
The road objects are sequentially connected to each other according to a rule set, allowing a road seed to be formed (Figure 1).In the following, the extraction of road objects and the way they are combined to construct road seeds are described with enough details.

Extraction of road objects
The road objects are defined using straight-line segments belonging to two different polygons with characteristics that are compatible to a road.
Figure 2 shows four road objects found in any road seed.In the building of a road object, by convention the inferior straight-line segment is called base and the superior one is called candidate.For each road object case, the endpoints of both straightline segments (base and candidate) are orthogonally projected from one to each other, resulting only in two points projected between endpoints.For example, in the figure 2(a) the endpoints of the candidate straight-line segment are projected into two points of the base straight-line segment.The opposite occurs with case 2 (figure 2(b)).In relation to cases 3 and 4, as respectively illustrated in figures 2(c) and 2(d), only one endpoint of a straight-line segment is projected between endpoints of other straight-line segment, and vice-versa.In all cases, two end points belonging to the base and/or candidate straight-line segment and two projected endpoints are combined to build quadrilaterals very similar in shape to rectangles.Each road object gives rise to a quadrilateral, being each one identified as crosshatched area in the figure 2. The axis of each quadrilateral coincides with a short road centerline.
The building of the four road objects is based on a rule set constructed from a priori road knowledge.The main rules used to identify and build road objects are described below: 1. Anti-parallelism rule: According to this rule, two image gradient vectors taken at two opposite road edge points, and belonging to the same road cross-section, are in approximately opposite directions.Beside this, they are approximately orthogonal to the road edges.This also means that if the road edges are approximated by polygons, the image gradient vectors, computed at edge pixels fitted to each straightline segment (of a polygon), are approximately parallel to each other.Thus, a compact and effective representation for the image gradient vectors, computed for each straight-line segment, is the mean image gradient vector.This mean vector embodies an important road property (i.e., the anti-parallelism) and its compactness facilitates the image analysis process for road seed extraction.The mean image gradient vectors for the base and candidate straightline segments are anti-parallels.By the antiparallelism rule, two straight-line segments, base and candidate, would be compatible to a  The order of application of the rules presented above is important, mainly when the base and candidate straight-line segments are incompatible.It can avoid, in most cases, the verification of all rules for road object construction.The first rule to be applied is the sixth as it allows parts or whole polygons potentially not related with road objects to be eliminated.The next rule to be applied is the fifth, avoiding the use of another set of rules in the case this rule is not satisfied.In the following, the order of rule to be applied is the 2 nd rule, the 1 st rule, the 3 rd rule, and the 4 th rule.A road object will be accepted if all rules are satisfied.Figure 4 demonstrates how the road seed, shown in figure 1, is decomposed using the four road objects defined above.From the right to the left, the following cases can be identified: first, second, fourth, and again the last one.The inverse problem, i.e., the reconstruction of the road seed using the road objects, is described in the following section.

Road Segment Extraction by Grouping Road Objects
As described above, the road objects are constructed by combining the base and candidate straight-line segments, which in turn belong to polygons representing all relevant image edges.Each road object is a local representation for the longest straight segment of a road seed.Thus, the problem we have in hands is how to connect the road objects to form the road seeds.
Figure 5 shows the possible connections to the left and to the right between the road objects.Figure 5(a) shows that 1 st case road object can connect to the left with the 2 nd and 3 rd cases and to the right with the 2 nd and 4 th cases.The 2 nd case road object (figure 5(b)) can connect to the left with the 1 st and 4 th cases and to the left with the 1 st and 3 rd cases.Note that the 3 rd and 4 th cases (figures 5(c) and 5(d), respectively) can connect themselves to both, left and right cases.
In order to construct a road seed by combining road objects, two polygons are selected and their straight-line segments are combined two-by-two and the resulting road objects are connected sequentially.The advantage of using the connection rules is that the construction of any new road object is limited to one or two cases (figure 5).The great problem of the polygon combination is the large search space if no heuristic is used.For high-resolution images, an efficient way for drastically reducing the search space is to use strategies based on the space scale, which allow the elimination of most part of previously extracted polygons (Baumgartner et al. 1999).
Figure 6 shows the combination of polygons representing the edges of a same road.One of the polygons being combined is labeled base and another one is labeled candidate.As also shown in figure 6, the base polygon can be compatible with more than one candidate polygon.It means that a base polygon needs to be combined with candidate polygons until the whole base polygon is combined.Assuming that every combination gives true results, parts or whole polygons (base or candidate) are removed from the search space after they are combined.
The application of the methodology described above to the illustrative example of figure 6 would allow to obtain the result shown in figure 7.
Supposing that the base polygon is combined from the right to the left (figure 7(a)), two road objects are constructed by combining the base polygon with the first candidate polygon.The connection be- tween these two road objects generates a first road seed (figure 7(b)), which in turn can be decomposed into three consecutive quadrilaterals.The vertices of these quadrilaterals allow the definition of a short road seed centerline.Now the combination of the base polygon with the second candidate polygon generates the second road seed constituted by only one quadrilateral.The straight-line segment of the base polygon not integrating to both road seeds extracted, may be a useful information for further analyses of the reasons for the missing road seed.It is an evidence that both extracted road seeds are related anyway.For example, there could be a ''T'' or ''Y'' road crossing.Therefore, everything providing information on road seed extraction problem should be preserved for further use in automatic completion strategies of the road network.

Candidate polygons
Finally, figure 8 shows the final result obtained from the application of the road seed extraction method to the image presented in figure 3(a).The polygons corresponding to the road seeds extracted are overlaid on the image as black lines.

EXPERIMENTAL RESULTS
In order to evaluate the potential of the methodology for road seed extraction, three experiments with real image data are carried out.As this methodology is suitable for road seed extraction from medium-and high-resolution images of rural scenes, one image of medium-resolution and two of high-resolution are used.Baumgartner et al. (1999) classify the image resolution in context of road extraction strategies into three categories: 1 -low-resolution images, i.e., pixel footprint greater than 2m; 2 -mediumresolution images, i.e., pixel footprint ranging from 0.7m to 2m; and 3 -high-resolution images, i.e., pixel footprint lesser than 0.7m.
The first experiment is carried out with a medium-resolution image (500 × 500 pixels), in which the main roads manifest as ribbons with 6pixel width.Figure 9 shows this image and the results overlaid on it.Parts of polygons used to construct the road seeds and the centerlines of road seeds are overlaid on the input image.In this experiment, an average road width of 6 pixels is used to set up the road objects, meaning that only main road seeds are expected to be extracted.The results obtained can be considered close to the expected one.Most fails are expected due to the missing edges and other anomalies along both roads.For these places and especially for the road crossing no road objects can be constructed and, as a result, a fragmented road network, composed by isolate road seeds, is extracted.Two false negatives pointed out by short arrows are unexpected as they could be extracted.Probably, the fail closer to the road crossing is resulted from the low contrast around that road region.Two isolate and short road seeds are extracted along secondary roads, which are circled on the input image.These road seeds could be considered false positive, as the road objects are set up to the pattern of the main roads.In general, the method allows long road seeds, along main roads, to be extracted, facilitating the posterior automatic completion of the main road network.In terms of completeness, about 80% of the main road network are extracted.
The second experiment (figure 10) is carried out with a high-resolution image (498 × 535 pixels), in which the roads manifest as wide ribbons with 40pixel width.Figure 10 shows that the results reflect those theoretically expected.In fact, due to the incompatibility with any road objects, the road crossing and few road parts, perturbed by illumination posts, are not extracted.These fails are needed to be treated by specific strategies embody other types of road knowledge, like the ones based on context and scale space (Baumgartner et al. 1999).Note that the vegetation edges adjacent to road edges do not cause false positives, showing the robustness of the method in these situations.The road crossing is not extracted as no road object can be constructed for that road region.As in previous experiment, the completeness is also about 80%.
The third experiment (figure 11) is also carried out with a high-resolution image (272 × 1168 pixels), but with lower resolution when compared to the resolution of the image used in the previous experiment.The test image of current experiment shows a road with mean road width of 20 pixels.The results obtained by the road seed extraction methodology are shown to be very satisfactory for the extraction of the two straight segments of the road, as long road seeds are recognized and delineated, es-  pecially the left one (which is approximately three times longer than the other road seed).These results are obtained as the road edges are well-defined along those regions.The opposite occurs with the curved segment of the road along which road edge anomalies are presented.As a result, many short road seeds are extracted in this road region, bringing additional difficulties for further reconstruction of that curve road segment.The completeness is about 80% as well.

CONCLUSIONS AND FUTURE PERSPECTIVES
This paper presented an automatic method for the road seed extraction from medium-and highresolution images of rural scenes.The innovation in the proposed methodology is the way the road objects and the connection rules between them are defined.
With a purpose of evaluating the method's potential in extracting road seeds, three experiments were conducted using three test images: two of high-resolution and one of medium-resolution.In all cases the results obtained could be considered satisfactory.In most cases, they are in accordance to the ones theoretically expected.Some little disconnections are expected, as the edge detection is sensible to the irregularities along the road margins.As a result, no road objects can be constructed for such road parts, giving rise to the missing road segments.Also due to incompatibility with any road objects, road crossings have not been extracted by the proposed methodology.Some false positives have been also expected.Despite these theoretical expected fails, the methodology was able to extract An Acad Bras Cienc (2005) 77 (3) AUTOMATIC EXTRACTION OF ROAD SEEDS FROM AERIAL IMAGES 519 about 80% of the road network.The automatic road network completion methodologies will be the focus of our future researches, whose basic input will be the road seeds extracted by the proposed methodology.
road fragment if it were satisfied.As shown in the figure 3, which is an illustrative example, several steps are necessary to compute the average image gradient vectors, i.e.: edge detection (figure 3(b)), vectorization or edge linking (figure 3(c)), and polygonization (by using e.g. the splitting and merging algorithm (Jain et al. 1995)) and computation of the average image gradient vectors for all straight-line segments of the polygons (figure 3(d)).

Fig. 8 -
Fig. 8 -Extraction of a road seed from a real image.