Loading repository data…
Loading repository data…
mohamedameen-io / repository
In this project, I used Python and OpenCV to detect lane lines on the road. I developed a processing pipeline that works on a series of individual images, and applied the result to a video stream.
I'll explain each step in details below.
We have 6 test images. I wrote a function called list_images() that shows all the test images we're working on using matplotlib.
def list_images(images, cols = 2, rows = 5, cmap=None):
"""
Display a list of images in a single figure with matplotlib.
Parameters:
images: List of np.arrays compatible with plt.imshow.
cols (Default = 2): Number of columns in the figure.
rows (Default = 5): Number of rows in the figure.
cmap (Default = None): Used to display gray images.
"""
plt.figure(figsize=(10, 11))
for i, image in enumerate(images):
plt.subplot(rows, cols, i+1)
#Use gray scale color map if there is only one channel
cmap = 'gray' if len(image.shape) == 2 else cmap
plt.imshow(image, cmap = cmap)
plt.xticks([])
plt.yticks([])
plt.tight_layout(pad=0, h_pad=0, w_pad=0)
plt.show()
Lane lines in the test images are in white and yellow. We need to choose the most suitable color space, that clearly highlights the lane lines. I applied color selection to the original RGB images, HSV images, and HSL images, and found out that using HSL will be the best color space to use.
def HSL_color_selection(image):
"""
Apply color selection to the HSL images to blackout everything except for white and yellow lane lines.
Parameters:
image: An np.array compatible with plt.imshow.
"""
#Convert the input image to HSL
converted_image = convert_hsl(image)
#White color mask
lower_threshold = np.uint8([0, 200, 0])
upper_threshold = np.uint8([255, 255, 255])
white_mask = cv2.inRange(converted_image, lower_threshold, upper_threshold)
#Yellow color mask
lower_threshold = np.uint8([10, 0, 100])
upper_threshold = np.uint8([40, 255, 255])
yellow_mask = cv2.inRange(converted_image, lower_threshold, upper_threshold)
#Combine white and yellow masks
mask = cv2.bitwise_or(white_mask, yellow_mask)
masked_image = cv2.bitwise_and(image, image, mask = mask)
return masked_image
We need to detect edges in the images to be able to correctly detect lane lines. The Canny edge detector is an edge detection operator that uses a multi-stage algorithm to detect a wide range of edges in images. The Canny algorithm involves the following steps:
We're interested in the area facing the camera, where the lane lines are found. So, we'll apply region masking to cut out everything else.
def region_selection(image):
"""
Determine and cut the region of interest in the input image.
Parameters:
image: An np.array compatible with plt.imshow.
"""
mask = np.zeros_like(image)
#Defining a 3 channel or 1 channel color to fill the mask with depending on the input image
if len(image.shape) > 2:
channel_count = image.shape[2]
ignore_mask_color = (255,) * channel_count
else:
ignore_mask_color = 255
#We could have used fixed numbers as the vertices of the polygon,
#but they will not be applicable to images with different dimesnions.
rows, cols = image.shape[:2]
bottom_left = [cols * 0.1, rows * 0.95]
top_left = [cols * 0.4, rows * 0.6]
bottom_right = [cols * 0.9, rows * 0.95]
top_right = [cols * 0.6, rows * 0.6]
vertices = np.array([[bottom_left, top_left, top_right, bottom_right]], dtype=np.int32)
cv2.fillPoly(mask, vertices, ignore_mask_color)
masked_image = cv2.bitwise_and(image, mask)
return masked_image
The Hough transform is a technique which can be used to isolate features of a particular shape within an image. I'll use it to detected the lane lines in selected_region_images.
def hough_transform(image):
"""
Determine and cut the region of interest in the input image.
Parameters:
image: The output of a Canny transform.
"""
rho = 1 #Distance resolution of the accumulator in pixels.
theta = np.pi/180 #Angle resolution of the accumulator in radians.
threshold = 20 #Only lines that are greater than threshold will be returned.
minLineLength = 20 #Line segments shorter than that are rejected.
maxLineGap = 300 #Maximum allowed gap between points on the same line to link them
return cv2.HoughLinesP(image, rho = rho, theta = theta, threshold = threshold,
minLineLength = minLineLength, maxLineGap = maxLineGap)
We have multiple lines detected for each lane line. We need to average all these lines and draw a single line for each lane line. We also need to extrapolate the lane lines to cover the full lane line length.
def average_slope_intercept(lines):
"""
Find the slope and intercept of the left and right lanes of each image.
Parameters:
lines: The output lines from Hough Transform.
"""
left_lines = [] #(slope, intercept)
left_weights = [] #(length,)
right_lines = [] #(slope, intercept)
right_weights = [] #(length,)
for line in lines:
for x1, y1, x2, y2 in line:
if x1 == x2:
continue
slope = (y2 - y1) / (x2 - x1)
intercept = y1 - (slope * x1)
length = np.sqrt(((y2 - y1) ** 2) + ((x2 - x1) ** 2))
if slope < 0:
left_lines.append((slope, intercept))
left_weights.append((length))
else:
right_lines.append((slope, intercept))
right_weights.append((length))
left_lane = np.dot(left_weights, left_lines) / np.sum(left_weights) if len(left_weights) > 0 else None
right_lane = np.dot(right_weights, right_lines) / np.sum(right_weights) if len(right_weights) > 0 else None
return left_lane, right_lane
def pixel_points(y1, y2, line):
"""
Converts the slope and intercept of each line into pixel points.
Parameters:
y1: y-value of the line's starting point.
y2: y-value of the line's end point.
line: The slope and intercept of the line.
"""
if line is None:
return None
slope, intercept = line
x1 = int((y1 - intercept)/slope)
x2 = int((y2 - intercept)/slope)
y1 = int(y1)
y2 = int(y2)
return ((x1, y1), (x2, y2))
def lane_lines(image, lines):
"""
Create full lenght lines from pixel points.
Parameters:
image: The input test image.
lines: The output lines from Hough Transform.
"""
left_lane, right_lane = average_slope_intercept(lines)
y1 = image.shape[0]
y2 = y1 * 0.6
left_line = pixel_points(y1, y2, left_lane)
right_line = pixel_points(y1, y2, right_lane)
return left_line, right_line
def draw_lane_lines(image, lines, color=[255, 0, 0], thickness=12):
"""
Draw lines onto the input image.
Parameters:
image: The input test image.
lines: The output lines from Hough Transform.
color (Default = red): Line color.
thickness (Default = 12): Line thickness.
"""
line_image = np.zeros_like(image)
for line in lines:
if line is not None:
cv2.line(line_image, *line, color, thickness)
return cv2.addWeighted(image, 1.0, line_image, 1.0, 0.0)
Now, we'll use the above functions to detect lane lines from a video stream. The video inputs are in test_videos folder. The video outputs are generated in output_videos folder.
def frame_processor(image):
"""
Process the input frame to detect lane lines.
Parameters:
image: Single video frame.
"""
color_select = HSL_color_selection(image)
gray = gray_scale(color_select)
smooth = gaussian_smoothing(gray)
edges = canny_detector(smooth)
region = region_selection(edges)
hough = hough_transform(region)
result = draw_lane_lines(image, lane_lines(image, hough))
return result
def process_video(test_video, output_video):
"""
Read input video stream and produce a video file with detected lane lines.
Parameters:
test_video: Input video.
output_video: A video file with detected lane lines.
"""
input_video = VideoFileClip(os.path.join('test_videos', test_video), audio=False)
processed = input_video.fl_image(frame_processor)
processed.write_videofile(os.path.join('output_videos', output_video), audio=False)
The project succeeded in detecting the lane lines clearly in the video streams. This project is intended to only detect (mostly) straight lines. Detecting curved lane line is behind the scope of this work.