Volume 18 Number 1
January 2021
Article Contents
Yun-Shui Zheng, Yan-Wei Jin, Yu Dong. Rail Detection Based on LSD and the Least Square Curve Fitting. International Journal of Automation and Computing, 2021, 18(1): 85-95. doi: 10.1007/s11633-020-1241-4
Cite as: Yun-Shui Zheng, Yan-Wei Jin, Yu Dong. Rail Detection Based on LSD and the Least Square Curve Fitting. International Journal of Automation and Computing, 2021, 18(1): 85-95. doi: 10.1007/s11633-020-1241-4

Rail Detection Based on LSD and the Least Square Curve Fitting

Author Biography:
  • Yun-Shui Zheng received the B. Sc. and M. Sc. degrees in traffic information engineering & control from Lanzhou Jiaotong University, China in 1994 and 2008, respectively. In 2008, he was a faculty member at Lanzhou Jiaotong University. Currently, he is a professor of Automatic Control Department, Lanzhou Jiaotong University, and a researcher of Automation Research Institute, Lanzhou Jiaotong University. He has won a first prize and two third prizes in the national multimedia software competition. He participated in the compilation of one textbook and published many papers. Now, he is in charge of a science and technology support project of Gansu Provincial Science and Technology. His research interests include the application of modern traffic information technology, the research of new generation centralized control system, the reliability research of high-speed railway signal equipment, the research and application of multimedia and virtual reality technology. E-mail: 2197565962@qq.com (Corresponding author) ORCID iD: 0000-0001-7971-2470

    Yan-Wei Jin received the B. Sc. degree in traffic information engineering & control from Lanzhou Jiaotong University, China in 2018. Currently, he is a master student in traffic information engineering & control of Lanzhou Jiaotong University. His research interests include railway safety, sensor application and image processing. E-mail: 1179618854@qq.com ORCID iD: 0000-0003-0961-2583

    Yu Dong received the B. Sc. degree in traffic signal & control from Lanzhou Jiaotong University, China in 1985. In 1985, he was a faculty member at Lanzhou Jiaotong University. Currently, he is a professor of Automatic Control Department, Lanzhou Jiaotong University. He participated in the research project and won a second prize, a third prize and one second prize of teaching achievements in Gansu Province. He edited two textbooks, participated in writing two textbooks, published more than twenty academic papers. He presided over a scientific and technological research project of Gansu Province, and now undertakes a project of National Nature Science Foundation of China (NSFC). His research interest is rail transit transportation automation. E-mail: 563576120@qq.com ORCID iD: 0000-0002-7644-5482

  • Received: 2020-02-21
  • Accepted: 2020-06-11
  • Published Online: 2020-08-10
通讯作者: 陈斌, bchen63@163.com
  • 1. 

    沈阳化工大学材料科学与工程学院 沈阳 110142

  1. 本站搜索
  2. 百度学术搜索
  3. 万方数据库搜索
  4. CNKI搜索

Figures (13)  / Tables (5)


Abstract Views (53) PDF downloads (17) Citations (0)

Rail Detection Based on LSD and the Least Square Curve Fitting

Abstract: It is necessary to rely on the rail gauge to determine whether the object beside the track will affect train operation safety or not. A convenient and fast method based on line segment detector (LSD) and the least square curve fitting to identify the rail in the image is proposed in this paper. The image in front of the train can be obtained through the camera on-board. After preprocessing, it will be divided equally along the longitudinal axis. Utilizing the characteristics of the LSD algorithm, the edges are approximated into multiple line segments. After screening the terminals of the line segments, it can generate the mathematical model of the rail in the image based on the least square. Experiments show that the algorithm in this paper can fit the rail curve accurately and has good applicability and robustness.

Yun-Shui Zheng, Yan-Wei Jin, Yu Dong. Rail Detection Based on LSD and the Least Square Curve Fitting. International Journal of Automation and Computing, 2021, 18(1): 85-95. doi: 10.1007/s11633-020-1241-4
Citation: Yun-Shui Zheng, Yan-Wei Jin, Yu Dong. Rail Detection Based on LSD and the Least Square Curve Fitting. International Journal of Automation and Computing, 2021, 18(1): 85-95. doi: 10.1007/s11633-020-1241-4
    • The closed environment of railway reduces the occurrence of accidents to a certain extent while uncontrollable factors such as line patrol workers and natural disasters still bring great hidden dangers to it. The International Union of Railways (UIC) released the accident safety reports of 30 railway companies in Europe, Asia and the Middle East in 2019, which pointed out that 90% of railway accidents are caused by the third parties trespassing on railway lines. Therefore, it is the necessary to detect foreign objects in real time because of its perniciousness. The traditional static monitoring cannot meet the demand of complex operation environments and all-weather high-speed operation at present, so it is necessary to monitor in real time[13]. Dynamic monitoring needs to build and update safety clearance based on rail structure in real time[4]. Therefore, the rail is an important reference for judging whether an object has violated the limit or not, which makes rail modeling necessary.

      There are two methods of rail detection based on computer vision at the moment: feature detection and model-based detection[5]. The former has good accuracy, but it is easy to interfere with, and its real-time performance is insufficient. The latter uses the rail features to fit the model in the image, which is simple, has strong anti-interference ability and good real-time performance.

      The Bresenham algorithm is used to describe straight rail with a mathematical model while it is difficult to apply to curved rail[6].

      References [79] introduced a piecewise curve model for rail detection. Hough transform and Bezier spline curve were used Dong and Guo[7] to get the mathematical model of the track. Firstly, based on the edge feature of the rail in the video image, the algorithm was used to recognize the position of the track by improved Hough transform, ascertain the vanishing point boundary and complete the calibration of the nearby and distant scene. After that, the algorithm used a linear model to calibrate the straight track in the nearby area. In the distant area, feature points were extracted by searching the Hu invariant moments. Then, the feature points were used to fit the B spline curve with the least square method. Finally, based on the scene switching model, the track line was tracked automatically. However, Hu invariant moment cannot maintain good invariance to affine transformation, so it cannot accurately detect the curve for camera jitter and pitch angle offset.

      Jiang and Dong[8] presented an algorithm based on Hough transform and Catmull-Rom spline curve. The method here uses improved Canny operator for edge detection first. Then Hough transform was used to detect the straight track and the vanishing boundary and feature points of curved rail were searched by the bifurcation window. Finally, the track model was generated by these feature points. In fact, due to the imaging difference between the left and right track lines in the image, the actual switch point of straight and curved track is not in the same horizontal, so there are errors in the mathematical description of the switch point in this algorithm. The method of document [9] is similar to the above.

      Radon transform was used for rail identification in reference [10] while it is fixed-point static monitoring which is not suitable for continuous real-time occasions.

      In this paper, a rail detection algorithm based on LSD and the least square curve fitting is proposed. Through image segmentation and line detection, feature points of the track are obtained, then the least square method is used for rail curve fitting to get the rail model in the image.

    • The algorithm steps and theoretical description of this paper are shown in Fig. 1. The rail recognition algorithm is based on the video frame. Firstly, preprocess the image frame, weaken the noise and enhance the edge features. Then the image is segmented optimally to make the segments smaller. After that the line segment detector (LSD) is used to extract the edge lines of the image and obtain the coordinate representation of the line segments. Finally, the endpoint coordinates of the line segments are screened, background interference points are eliminated, and the least square is carried out for the discrete points after screening to get the final rail identification results.

      Figure 1.  Structure of the rail detection algorithm

    • The rail has significant edge features. In this paper, edge features[1115] are used as rail feature points for curve fitting through edge detection of image.

    • It will introduce zero-mean-value Gaussian noise into the image due to electromagnetic interference when the camera works in a strong electromagnetic environment, and the locomotive jitter will cause image motion blur. Therefore, it is necessary to filter and deblur (inverse filter) the original image for image processing using machine vision. Preprocessing improves the image quality, reduces the data quantity of image processing, and improves the real-time performance of the algorithm. As shown in Fig. 2, Fig. 2 (a) is the original image, Fig. 2 (b) is the grayscale processing image, Fig. 2 (c) is the Gaussian filtering, and Fig. 2 (d) is the result of inverse filtering.

      Figure 2.  Initialize preprocessing

    • Replacing curve with a straight line is a classical simplified calculation method. However, due to the accuracy of approximation, the deviation between the final calculation result and the actual curve is large, so multi straight line is selected to replace curve in this paper. The rail presents a smooth curve in the image. To build the rail model, it is necessary to collect as many feature points of the rail as possible. In this paper, the optimal bisection is used to segment the image horizontally, which can give us more approximate lines and obtain more feature points of the rail. The schematic diagram of curve segmentation is shown in Fig. 3.

      Figure 3.  Schematic diagram of curve division

      Fig. 3 (a) is a schematic diagram of line detection directly, and the line segment l1 is used as an approximation of the curve L. Fig. 3 (b) is a schematic diagram of detection after image segmentation, with curve L replaced by l1, l2, l3. Compared Fig. 3 (a) with Fig. 3 (b), Fig. 3 (b) has more information about the original curve, which makes it possible to improve the fitting accuracy.

      In order to describe the quality of image segmentation, the evaluation criterion Σhi is introduced, where h is the maximum vertical distance between the point on the curve AB and the line AB, and it represents the error of the straight approximation of the curve[16]. Comparison of different divisions are shown in Table 1.

      GroupSegmentationsScattersFeature pointsTime (ms)Error (pixel)Index evaluation

      Table 1.  Comparison of different divisions

      It can be known from Table 1 that an increase in the number of equal divisions will lead to an increase in the linear detection time but reduce the error. Obviously, the two indicators are opposite. We choose to normalize them to select the most appropriate number of equal divisions. In order to simplify the complexity of data processing and improve the real-time performance of the algorithm, linear transformation (dispersion standardization) is used to process the “Time” and “Error”, and both of them are mapped in the [0,1] interval. The normalized calculation method is defined by

      ${x^*} = \frac{{x - {x_{\min }}}}{{{x_{\max }} - {x_{\min }}}}$


      where x* is the result of normalization, xmin and xmax represent the minimum and maximum values in the data respectively, and x represents the current data to be normalized. "Index evaluation" in Table 1 is the sum of processing time and error normalization results. The larger the sum is, the greater the processing time or error is.

      When the segmentation interval is less than 10 pixels, it cannot detect the effective edge of the image accurately because of the threshold limit. As shown in Groups 5–7 in Table 1, the number of scattered points of rail curve detected by increasing the segmentation number is not enough to represent the track, so only the first four groups of data are compared and evaluated. To sum up, this paper divides the image into 15 parts.

    • LSD was proposed by Rafael et al.[17] in 2010. It is based on pixel gradient to achieve edge detection. By traversing the gradient and pseudo sorting the gradient, the pixels with larger gradient are regarded as edges. The pixels whose gradient amplitude is greater than the threshold and whose gradient direction is within the tolerance of error τ (τ = π / 8) are divided into the same approximate rectangular area, and the inertial spindle of the rectangular area is selected as the detected line segment.

      The algorithm will recognize the intersection lines as multiple line segments. Because the algorithm sets the “Status” variable for each pixel, when a pixel is contained by a rectangular box, its “Status” variable will be set to “Used”, which means that this pixel can only belong to the current rectangular. In other words, each pixel can only belong to a straight line. For the intersection of two lines, overlapping pixels can only belong to one of them, so another one must be divided into two. Because the algorithm is based on gradient detection, the intersection point always has a small one, which means the intersection point will not be recognized as an edge point. Therefore, the intersecting lines are often divided into four segments. Region growth is used to obtain a line-support region. Because of occlusion and local blur, the long line segment will be divided into multiple short segments.

      Utilizing the characteristic of LSD described above, the line detection of the above-mentioned bisection image is carried out, and a number of line segments are obtained. The rail model is obtained by curve fitting using the terminals of the lines. The high speed and dynamic background environment require high real-time performance of the algorithm. It is verified that the processing time of LSD is about 40 ms/frame. The flow chart of LSD line detection algorithm is shown in Fig. 4.

      Figure 4.  Process of LSD algorithm

      The results of line detection performed on the preprocessed image directly is shown in Fig. 5 (a) and the detection after image segmentation is shown in Fig. 5 (b). The test image of this paper is 400 × 360 pixels, and the rail curve is distributed longitudinally in the image, so we choose to segment the image horizontally with equal intervals. In order to filter out the detection interference of the segmentation lines and considering the inclination angle of the long-term curved rail, only the line segments with slope k ≤ − 0.1 and k ≥ 0.1 are retained as the effective ones. The distribution of these endpoints in the image coordinate is shown in Fig. 6 and part of them are listed in Table 2.


      Table 2.  Data of line detection (Partial data)

      Figure 5.  Results of line detection with LSD

      Figure 6.  Feature point extraction

    • After the above processing, the feature points of the rail, the edge of the track bed and the contact line column of the image edge are obtained. In order to avoid the scatter interference in curve fitting, it is necessary to filter the points other than the rail curve, i.e., to keep the feature points of rail curve, and then use this information for curve fitting[18].

    • Rail is actually a straight line or a smooth curve. In the feature point extraction stage, the image is segmented. The actual curve can be approximately replaced by multiple line segments. According to the imaging characteristics of the rail, the region search is carried out.

      The pixel coordinate origin is located in the upper left corner of the image. In the camera shot image, the close-up curve is located at the lower part of the image. The statistical analysis of the shot image data shows that the starting point of the rail close-up in the image is relatively fixed, and the starting point coordinates are located in [xl1, yl1], [xl2, yl2], [xl3, yl3], [xl4, yl4] (Left); [xr1, yr1], [xr2, yr2], [xr3, yr3], [xr4, yr4] (Right). (The starting coordinates are [85, 350], [85, 360], [120, 350], [120, 360] (Left), [200, 350], [200, 360], [235, 350], [235, 360] (Right) in this paper for passenger dedicated line.)

      Drawing on the idea of Kalman filtering, this paper uses the current selection box (i−th) and the last selection box ((i-1)−th) to predict the next selection box((i+1)−th), and then searches for the feature points in the box. The process is repeated until the search termination condition is met and the search is stopped[19]. The initial selection box is set to a×b (35×10 in this paper for passenger dedicated line) pixels. Due to the small lateral distance between the feature points of the prospective rail and the edge of the track bed, choosing a fixed size selection box will generate a lot of interference values. Consequently, this article stipulates that the selection box is kept the same size as the previous when it moves along the reconstructed line (vi-1-vi), otherwise it changes to (35−i) × 10, where i is the cumulative number of the selection box to move along the reconstructed line.

      The feature point filtering process is as follows:

      Step 1. Take the arithmetic mean of the feature point coordinates of the two starting areas, and use the coordinate average as the starting point V0. The starting point coordinate is calculated as shown in (2).

      $\left\{ {\begin{aligned} & {{{{x}}_{{{v0}}}} = \frac{1}{N}\sum\nolimits_1^N {{{{x}}_{{i}}}} } \\ & {{{{y}}_{{{v0}}}} = \frac{1}{N}\sum\nolimits_1^N {{{{y}}_{{i}}}} } \end{aligned}} \right.$


      where xi and yi are the abscissa and ordinate of the feature points contained in the starting area respectively, and N is the total number of feature points in the area.

      Step 2. Translate the selection box vertically upward by b units. V1 is obtained by processing the data of the second selection box in Step 1.

      Step 3. Move the selection box with V0-V1 as the prediction direction, and repeat the averaging process to obtain V2, V3,···. When the number of feature points in the selection box is greater than or equal to 2, the included feature point data is arithmetically averaged to obtain Vi. Otherwise, move the selection box vertically upwards until the number of feature points in the area is greater than or equal to 2.

      Step 4. Take the reconstructed line Vi-1Vi as the (i+1)−th movement direction of the selection box. In this cycle, the termination condition is that the number of vertical movements of the selection box M ≥ 3, at this time, take the result of the last movement of the box in the direction of the reconstructed line as the end.

      Step 5. Save the effective feature points in the moving process of the selection box, and then complete the screening of rail feature points. The region search process is shown in Fig. 7.

      Figure 7.  Process of region search

      Rail has four edges in the image, i.e., the two edges on the top of the rail contacting the wheel and the two edges on the bottom of the rail, which will be detected in line detection, thus four feature points will be generated in the same ordinate range. In order to reduce the loss of data in the filtering process and avoid a large number of interference points in the algorithm, the threshold value of termination algorithm is set as M = 3. In this paper, we choose to segment the image 15 equally, with a segmentation interval of 24 pixels. In the region search process with a step of 10 pixels, three boxes must contain a set of feature points. Otherwise, we think that the search is over, the data filtering process is shown in Fig. 8, and the filtering results are shown in Fig. 9.

      Figure 8.  Schematic diagram of feature point filtering

      Figure 9.  Results of feature point filtering

    • According to the above process, the feature points of rail are non-linearly distributed in the image. Because of the difference of edge features in the process of line detection, the final feature points are randomly distributed along the actual rail curve, so that the fitting results based on spline curve are locally smooth but overall uneven. The curve fitting based on the least square method is to find the best matching function with the discrete data by minimizing the square sum of the errors[20, 21]. This method can overcome the disadvantages of the spline curve fitting and get the overall smooth curve fitting results, which is more in line with the actual imaging characteristics of the rail curve. Therefore, this paper uses the least square method to fit the rail curve to get the rail mathematical model in the image.

      The principle of least square method is as follows:

      The polynomial P (x)=a0+a1x+a2x2+···+anxn−1 is obtained by the discrete points (xi, yi), which satisfies the requirement that the sum of the squares of the distance from each point to the curve is minimized.

      Define fitting polynomials as (3).

      $y = {a_0} + {a_1}x + {a_2}{x^2} + \cdots + {a_k}{x^k}.$


      The sum of the distances from each point to this curve, i.e. the sum of the squares of the deviations, is shown in (4).

      ${R^2} = {\sum\limits_{i = 1}^n {[y_i - ({a_0} + {a_1}{x_i} + {a_2}{x_i}^2 + \cdots + {a_k}{x_i}^k)]} ^2}$


      where R2 is the minimum sum of squares, xi is the abscissa of the discrete point and yi is the corresponding ordinate.

      In order to obtain the coefficients ai of the polynomial, the derivative of (4) is calculated for each coefficient ai, and the relationship between coefficient matrix and discrete points is shown in (5).

      $\begin{split} & \left[ {\begin{array}{*{20}{c}} n&{\sum\nolimits_{i = 1}^n {{x_i}} }& \cdots &{\sum\nolimits_{i = 1}^n {{x_i}^k} } \\ {\sum\nolimits_{i = 1}^n {{x_i}} }&{{{\sum\nolimits_{i = 1}^n {{x_i}} }^2}}& \cdots &{{{\sum\nolimits_{i = 1}^n {{x_i}} }^{k + 1}}} \\ \vdots & \vdots & \ddots & \vdots \\ {{{\sum\nolimits_{i = 1}^n {{x_i}} }^k}}&{{{\sum\nolimits_{i = 1}^n {{x_i}} }^{k + 1}}}& \cdots &{{{\sum\nolimits_{i = 1}^n {{x_i}} }^{2k}}} \end{array}} \right]\left[ {\begin{aligned} {{a_0}} \\ {a{}_1} \\ \vdots \; \\ {{a_k}} \end{aligned}} \right] = \\ &\left[ {\begin{array}{*{20}{c}} {\sum\nolimits_{i = 1}^n {{y_i}} } \\ {\sum\nolimits_{i = 1}^n {{x_i}{y_i}} } \\ \vdots \\ {\sum\nolimits_{i = 1}^n {{x_i}^k{y_i}} } \end{array}} \right]\\[-35pt]\end{split}$


      where n is the total number of the points, xi and yi are the abscissa and ordinate of discrete points.

      Formula (6) is obtained by simplifying (5).

      $\left[ {\begin{array}{*{20}{c}} 1&{{x_1}}& \cdots &{{x_1}^k} \\ 1&{{x_2}}& \cdots &{{x_2}^k} \\ \vdots & \vdots & \ddots & \vdots \\ 1&{{x_n}}& \cdots &{{x_n}^k} \end{array}} \right]\left[ {\begin{array}{*{20}{c}} {{a_0}} \\ {{a_1}} \\ \vdots \\ {{a_k}} \end{array}} \right] = \left[ {\begin{array}{*{20}{c}} {{y_1}} \\ {{y_2}} \\ \vdots \\ {{y_n}} \end{array}} \right]$


      where n is the total number of the points, xi and yi are the abscissa and ordinate of discrete points.

      The coefficient matrix is defined as A, the abscissa matrix of each discrete point is X, and the ordinate matrix is Y. Formula (6) can be written in (7).

      $X \times A = Y.$


      Then we can compute the matrix A by (8), and the complete polynomial can be obtained.

      $A = {X^{ - 1}} \times Y.$


      In this paper, the cubic polynomial is used to build the model, which is shown in (9). The fitting result of the test picture is shown in Fig. 10.

      Figure 10.  Result of curve fitting

      In order to improve the efficiency of curve fitting, the feature points in the process of feature point selection are preprocessed. In the process of region search, the effective selection box contains at least three feature points. The average of feature points in the same selection box is taken to get the reconstructed feature point sequence. As shown in Fig. 10, these feature points are obtained by averaging the feature points screened in Fig. 9.

      $\left\{\begin{aligned} {P_L}\left( x \right) =\; & 0.000\,358\,28{x^3} - 0.127{x^2} + 11.427\,8x + \\ & 134.462\,2 \quad \left( {102.6 \le x \le 164} \right) \\ {P_R}\left( x \right) =\; & - 0.000\,014\,305{x^3} + 0.012\,1{x^2} - 3.125\,5x + \\ & 434.528\,7 \quad\left( {178.3 \le x \le 219.5} \right). \end{aligned} \right.$

    • After using convolution neural network (CNN) to locate and recognize the object in the image, it is necessary to judge whether the object invades or not. Firstly, according to the standard gauge and the railway safety clearance, the safety gauge is obtained. Then the edge coordinates of the object obtained by positioning are used to determine whether it is within the safety gauge.

      According to the standard of railway rolling stock gauge (GB146.2–83), the standard gauge is 1435 mm, and the safety clearance is 2440 × 2 mm. The safety clearance calibration diagram is shown in Fig. 11.

      Figure 11.  Safety clearance calibration diagram

      According to the rail detection result, the expressions of rail curve in the image are yl (x) and yr (x). By setting yl (x) = yr (x) = y (pi), the rail point xA and xB at the object to be measured can be obtained. The safety clearance points xC, xD under the same ordinate are obtained from the standard gauge and safety clearance, as shown in (10) and (11).

      ${x_c} = {x_m} - 244\;0 \times \frac{{{x_B} - {x_A}}}{{143\;5}}$


      ${x_D} = {x_m} + 244\;0 \times \frac{{{x_B} - {x_A}}}{{143\;5}}.$



      $\left\{ {\begin{aligned} & {\frac{{{x_B} - {x_A}}}{{143\;5}} = \frac{{{x_D} - {x_C}}}{{488\;0}}} \\ & {{x_m} = \frac{1}{2}({x_A} + {x_B})} \\ & {{y_l}(x) = {y_r}(x)} \;. \end{aligned}} \right.$

      The schematic diagram of object violation judgment is shown in Fig. 12.

      Figure 12.  Schematic diagram of limit invasion judgment

      The judgment process of object invasion limit is as follows:

      Step 1. Read the vertex coordinates p1 (x1, y1), p2 (x2, y2), p3 (x3, y3), p4 (x4, y4) of the envelope box in the image coordinates.

      Step 2. Substitute y1, y2, y3, y4 into the rail curve model yl (x), yr (x) to get xA and xB.

      Step 3. Calculate the limit abscissa (x1C, x1D), (x2C, x2D), (x3C, x3D), (x4C, x4D) corresponding to each vertex.

      Step 4. Judge whether the condition [(x1Cx1x1D) ∪(x2Cx2x2D)∪(x3Cx3x3D)∪(x4Cx4x4D)∪(xixiC, xmDxm)] (where i=1, 3, m=2, 4) is satisfied. The object will be regarded as an object of invasion if it is satisfied, otherwise it is safe.

      Set the obstacle artificially (different distance away from the center line of the rail surface), identify and locate the object in the image through CNN, as shown in Fig. 12. The partial test result is shown in Table 3.

      NumberObjectSet distance (m)Whether it is set as
      foreign matter
      Calculated distance (m)Whether it is judged as
      foreign matter
      Judgment result

      Table 3.  Experimental results

      It can be seen from Table 3 that the error between the real distance and calculated by the image is small. The smoother the edge of the object is, the smaller the error is. Because the carton is on the edge of the safety limit and the error exists, the judgment result of the 5th group is opposite to the actual. In practical applications, it is acceptable to classify it as foreign matter invading the limit when an object is at the edge of the safety limit. The rail curve identified by the rail detection algorithm can provide an accurate reference for the object invasion through the test.

    • The rail detection algorithm proposed in this paper consists of three modules: initialization preprocessing, LSD line detection and curve fitting. The whole algorithm is a serial structure, i.e., each step needs to take on the processing results of the previous steps. In the actual processing, if it is failed to get the lines, the next frame will be read for next cycle, and then the whole algorithm flow will be executed again.

      The procedures of the rail identification algorithm are as follows:

      Step 1. Receive the video image transmitted by the camera, and then read each frame of image.

      Step 2. Perform graying, denoising and inverse filtering on the image to reduce the amount of processed data, eliminate motion blur and save the images sequentially.

      Step 3. Divide the image in the vertical axis by N / n, where N is the vertical length of the image, and n is the common divisor of N greater than 10 pixels, choose the optimal number of equal divisions, and set the texture thickness θ of the dividing line.

      Step 4. Perform line segment detection on the segmented image using LSD, read the terminals coordinates (pixel coordinates) of the line segments with slopes k ≤ −0.1 and k ≥ 0.1 and save them.

      Step 5. Make statistics on the starting area of the image rails, screen the starting points in the fixed area ([xl1, yl1], [xl2, yl2], [xl3, yl3], [xl4, yl4] (Left); [xr1, yr1], [xr2, yr2], [xr3, yr3], [xr4, yr4] (Right)) and take arithmetic average for them in this area.

      Step 6. The average of the starting points acts as the initial point of the region search, and all of the feature points are filtered by region search as described in Section 3.1. Then the feature point sets belonging to the left and right rails are obtained.

      Step 7. Divide Pi and Pj that satisfy |${yP_i} -{yP_j}$| ≤ d (θ d ≤ 2θ) into the same set, and perform the average processing on the points in the same box to obtain the reconstructed rail feature point set (TL={P0,P1,···,PN-1}, TR={P0,P1,···,PM-1}).

      Step 8. Take the reconstruction feature points as the control points of the least square method, and the mathematical model of the fitting curve is generated.

      Step 9. Read the next frame of image and repeat the process of Steps 2–9.

      The parameters of the algorithm are summarized in Table 4.

      ParametersLess than the optimal valueOptimal value in this paperGreater than the optimal value
      Segmentation interval (N/n)Weaken the time performance and the feature points are insufficient.360/15The feature points are insufficient and a larger error will be existed.
      Texture thickness of
      separator (θ)
      1It cannot be greater than half of the split interval.
      Length of selection box (a)Important feature points will be lost.35A large number of interference feature points will be introduced.
      Width of selection box (b)The feature point filtering will be terminated prematurely.10A large number of interference feature points will be introduced or bring an error to the result.
      Decrement of the “aA large number of interference feature points will be introduced.1The feature point filtering will be terminated prematurely.
      Threshold NUnable to control the search direction and result in error eventually.2The feature point filtering will be terminated prematurely without an ideal result.
      Threshold MThe feature point filtering will be terminated prematurely without an ideal result.3A number of interference feature points will be introduced and result in error eventually.

      Table 4.  Performance summary of the algorithm parameters

    • In this paper, some image data collected by CRH380 camera on-board of a passenger dedicated line and an oil dedicated line are selected. Among them, the scene of the passenger dedicated line is rich while the oil dedicated line is single. The emphasis is on the image data of the operation between the passenger dedicated line for testing. The experimental hardware environment is based on Win10, CPU core i5-5200u (2.2GHz), RAM 4 GB, and software environment is based on VS2015 (with OpenCV configured) and Matlab 2018b. A total of 231 images were tested, including 182 for passenger dedicated line and 49 for oil dedicated line. Some of the test results are shown in Fig. 13. The white line Fig. 13 is the drawing of the polynomial obtained after rail detection in the original image coordinates. The statistical results are compared with some literature algorithms as shown in Table 5.

      AlgorithmsApplication scenariosAccuracy (%)Time (ms)
      Reference [7]Continuous detection in dynamic background85.85
      Reference [8]Continuous detection in dynamic background89.642
      Reference [9]Continuous detection in dynamic background
      Reference [10]Continuous detection in static background
      Proposed methodContinuous detection in dynamic background90.9139.2

      Table 5.  Comparison of different algorithms

      Figure 13.  Results of experimental verification

      The imaging range of the on-board camera is about 350 meters. In this sight distance range, the actual rail is either in a straight line, or in a curve, or in a combination of a straight line and a curve. In order to standardize the algorithm, they are all regarded as curves, and the cubic polynomial is used for curve fitting. When there is only a straight track in the image, the high-order coefficient of the curve is very small, making the fitting result approximate to a straight line, as shown in Fig. 13 (b). When there are both curves and straight lines in the image, the cubic polynomial obtained can still describe the curve relatively.

      According to the statistical analysis of the experimental results, 21 frames of the rail curve are not detected accurately, including 16 frames of the passenger dedicated line and 5 frames of the oil dedicated line, with an accuracy of 90.91%. The average processing time of each frame is 39.2 ms. The average accuracy of reference [7] is 85.85%, and the time performance of the algorithm is not counted. The average accuracy in [8] is 89.6%, and the single frame processing time is 42 ms. There is no introduction to the real-time and accuracy of the algorithm in [9, 10]. All algorithms can realize continuous monitoring under dynamic background, except for document 10. The comparison between them shows that the method in this paper has better performance.

      Because of the dependence of the algorithm on the edge feature in the image, when there is physical occlusion, insufficient exposure or overexposure, there will be some blocks without texture in the image, so that it fails to detect the edge and generate the complete curve model in the image. As shown in Fig. 13 (c), in the inner side of the tunnel, the rail detection in the figure is short because of the insufficient light and the weak feature of the prospective edge, so the prospective edge cannot be detected. As shown in Fig. 13 (d), due to the overexposure of the tunnel exit, the white spots with high pixel value appear in the image, covering the rail edge, so the prospective rail cannot be detected.

    • 1) A rail detection algorithm based on LSD and the least square is proposed in this paper. The experiment shows that this method has better accuracy and real-time performance.

      2) Based on the region search, the rail feature points are selected, and the rail imaging characteristics are considered comprehensively, which makes the selection process simpler and improves the real-time and accuracy of the algorithm.

      3) The result of testing in different line scenarios shows that the algorithm is robust and has good application value.

    • This work was supported by National Natural Science Foundation of China (No. 61763023).

Reference (21)



    DownLoad:  Full-Size Img  PowerPoint