scikit-surgeryimage¶
Data Acquisition¶
Timestamped Video Source¶
Module for video source acquisition. Classes capture data from a video source into a numpy array.
-
class
sksurgeryimage.acquire.video_source.
TimestampedVideoSource
(source_num_or_file, dims=None)[source]¶ Bases:
object
Capture and store data from camera/file source. Augments the cv2.VideoCapture() to provide passing of camera dimensions in constructor, and storage of frame data.
-
class
sksurgeryimage.acquire.video_source.
VideoSourceWrapper
[source]¶ Bases:
object
Wrapper for multiple TimestampedVideoSource objects.
-
add_camera
(camera_number, dims=None)[source]¶ Create VideoCapture object from camera and add it to the list of sources.
Parameters: - camera_number – integer camera number
- dims – (width, height) as integer numbers of pixels
-
add_file
(filename, dims=None)[source]¶ Create videoCapture object from file and add it to the list of sources.
Parameters: - filename – a string containing a valid file path
- dims – (width, height) as integer numbers of pixels
-
Stereo Video Source¶
Module for stereo video source acquisition.
-
class
sksurgeryimage.acquire.stereo_video.
StereoVideo
(layout, channels, dims=None)[source]¶ Bases:
object
Provides a convenient object to manage various stereo input styles. Developed firstly for laparoscopic surgery, but broadly applicable to any stereo setup using our TimestampedVideoSource and VideoSourceWrapper.
Design Principles:
- Fail early, throwing exceptions for all errors.
- Works with or without camera parameters.
- If no camera parameters, calling get_undistorted() or get_rectified() is an Error.
-
get_rectified
()[source]¶ Returns the 2 channels, rectified, as a list of images.
Returns: list of images Raises: ValueError, TypeError - if camera parameters are not set.
-
get_undistorted
()[source]¶ Returns the 2 channels, undistorted, as a list of images.
Returns: list of images Raises: ValueError - if you haven’t already provided camera parameters
-
set_extrinsic_parameters
(rotation, translation, dims)[source]¶ Sets the stereo extrinsic parameters.
Parameters: - rotation – 3x3 numpy array representing rotation matrix.
- translation – 3x1 numpy array representing translation vector.
- dims – new image size for rectification
Raises: ValueError, TypeError
Video Writing¶
Write stream of frames to file using OpenCV
-
class
sksurgeryimage.acquire.video_writer.
ThreadedTimestampedVideoWriter
(filename, fps=25, width=640, height=480, codec='MJPG')[source]¶ Bases:
sksurgeryimage.acquire.video_writer.TimestampedVideoWriter
TimestampedVideoWriter that can be run in a thread. Uses Queue.Queue() to store data, which is thread safe.
Frames will be processed as they are added to the queue:
threaded_vw = ThreadedTimestampedVideoWriter(file, fps, w, h) threaded_vw.start()
threaded_vw.add_to_queue(frame, timestamp) threaded_vw.add_to_queue(frame, timestamp) threaded_vw.add_to_queue(frame, timestamp)
threaded_vw.stop()
-
write_frame
(frame, timestamp=None)[source]¶ Add a frame and a timestamp to the queue for writing. Named for consistency with the non-threaded version. Actual writing to disk is done by write_frame_to_disk() :param frame: Image frame :type frame: numpy array :param timestamp: Frame timestamp :type timestamp: datetime.datetime object
-
-
class
sksurgeryimage.acquire.video_writer.
TimestampedVideoWriter
(filename, fps=25, width=640, height=480, codec='MJPG')[source]¶ Bases:
sksurgeryimage.acquire.video_writer.VideoWriter
Class to write images and timestamps to disk, inherits from VideoWriter.
Parameters: - fps – Frames per second to save to disk.
- filename – Filename to save output video to. Timestamp file is “filename + ‘timestamps’”
-
class
sksurgeryimage.acquire.video_writer.
VideoWriter
(filename, fps=25, width=640, height=480, codec='MJPG')[source]¶ Bases:
object
Class to write images to disk using cv2.VideoWriter.
Parameters: - fps – Frames per second to save to disk.
- filename – Filename to save output video to.
- width – width of input frame
- height – height of input frame
Calibration Tools¶
Point Detector¶
Base class for a PointDetector.
e.g. Chessboard corners, SIFT points, Charuco points etc.
-
class
sksurgeryimage.calibration.point_detector.
PointDetector
(scale=(1, 1), camera_intrinsics=None, distortion_coefficients=None)[source]¶ Bases:
object
Class to detect points in a 2D video image.
These point detectors are often used to detect points for camera calibration. However, it would also be possible for some subclasses to utilise camera intrinsics and distortion coefficients in order to improve the point detection process itself. It would be up to the derived class to decide how to use them, if at all.
Parameters: - scale – tuple (x scale, y scale) to scale up/down the image
- camera_intrinsics – [3x3] camera matrix
- distortion_coefficients – [1xn] distortion coefficients
-
get_camera_parameters
()[source]¶ Returns a copy of the camera matrix, and distortion coefficients. Throws RuntimeError if either are None.
Returns: [3x3], [1xn] matrices
-
get_model_points
()[source]¶ Derived classes should override this, to detector returns the complete model of 3D points. e.g. for a chessboard this would be all the corners in chessboard coordinates (e.g. z=0).
By design, this can return an ndarray with zero rows, if the detector does not support 3D coordinates.
Returns: [Nx3] numpy ndarray representing model points.
Chessboard Detector¶
Chessboard implementation of PointDetector.
-
class
sksurgeryimage.calibration.chessboard_point_detector.
ChessboardPointDetector
(number_of_corners, square_size_in_mm, scale=(1, 1))[source]¶ Bases:
sksurgeryimage.calibration.point_detector.PointDetector
Class to detect chessboard points in a 2D grey scale video image.
ArUco Point Detector¶
ArUco implementation of PointDetector.
-
class
sksurgeryimage.calibration.aruco_point_detector.
ArucoPointDetector
(dictionary, parameters, model, scale=(1, 1))[source]¶ Bases:
sksurgeryimage.calibration.point_detector.PointDetector
Class to detect ArUco points in a 2D grey scale video image.
Note: For ArUco points, these don’t have to be on a regular grid. If you provide a ‘model’ which is a map of id : 3D point, the function _internal_get_points will provide the corresponding 3D points of those points that were detected.
-
sksurgeryimage.calibration.aruco_point_detector.
get_intersect
(a_1, a_2, b_1, b_2)[source]¶ Returns the point of intersection of the lines passing through a2,a1 and b2,b1.
See https://stackoverflow.com/questions/3252194/numpy-and-line-intersections
Parameters: - a_1 – [x, y] a point on the first line
- a_2 – [x, y] another point on the first line
- b_1 – [x, y] a point on the second line
- b_2 – [x, y] another point on the second line
ChArUco Point Detector¶
ChArUco implementation of PointDetector.
-
class
sksurgeryimage.calibration.charuco_point_detector.
CharucoPointDetector
(dictionary, number_of_squares, size, scale=(1, 1), camera_matrix=None, distortion_coefficients=None, filtering=False)[source]¶ Bases:
sksurgeryimage.calibration.point_detector.PointDetector
Class to detect ChArUco points in a 2D video image.
ChArUco & Chesboard Detector¶
ChArUco + Chessboard implementation of PointDetector.
-
class
sksurgeryimage.calibration.charuco_plus_chessboard_point_detector.
CharucoPlusChessboardPointDetector
(reference_image, minimum_number_of_points=50, scale=(1, 1), number_of_charuco_squares=(19, 26), size_of_charuco_squares=(5, 4), dictionary=< cv2.aruco.Dictionary 0x7fd13e381dd0>, camera_matrix=None, distortion_coeff=None, charuco_filtering=False, use_chessboard_inset=True, number_of_chessboard_squares=(9, 14), chessboard_square_size=3, chessboard_id_offset=500, error_if_no_chessboard=True, error_if_no_charuco=False)[source]¶ Bases:
sksurgeryimage.calibration.point_detector.PointDetector
Class to detect ChArUco points and Chessboard points in a 2D grey scale video image.
Dotty Grid Point Detector¶
Dotty Grid implementation of PointDetector.
-
class
sksurgeryimage.calibration.dotty_grid_point_detector.
DottyGridPointDetector
(model_points, list_of_indexes, camera_intrinsics, distortion_coefficients, scale=(1, 1), reference_image_size=None, rms=30, gaussian_sigma=5, threshold_window_size=151, threshold_offset=20, min_area=50, max_area=50000, dot_detector_params=None)[source]¶ Bases:
sksurgeryimage.calibration.point_detector.PointDetector
Class to detect a grid of dots in a 2D grey scale video image.
More specifically, a grid of dots with 4 larger dots at known locations.
-
sksurgeryimage.calibration.dotty_grid_point_detector.
get_model_points
(dots_rows_columns: (<class 'int'>, <class 'int'>), pixels_per_mm: int, dot_separation: float) → numpy.ndarray[source]¶ Generate the expected locations of dots in the pattern, in pixel space.
Parameters: - dots_rows_columns ([int, int]) – Number of rows, number of columns
- pixels_per_mm (int) – Pixels per mm
- dot_separation (float) – Distance between dots in mm
Returns: array pf point info - [id, x_pix, y_pix, x_mm, y_mm, z_mm]
Return type: np.ndarray
ChArUco Helper Functions¶
Functions to support camera calibration using ChArUco chessboard markers.
-
sksurgeryimage.calibration.charuco.
detect_charuco_points
(dictionary, board, image, camera_matrix=None, distortion_coefficients=None, filtering=False)[source]¶ Extracts ChArUco points. If you can provide camera matrices, it may be more accurate.
Parameters: - dictionary – aruco dictionary definition
- board – aruco board definition
- image – grey scale image in which to search
- camera_matrix – if specified, the 3x3 camera intrinsic matrix
- distortion_coefficients – if specified, the distortion coefficients
- filtering – if True, filter out wrongly detected markers
Returns: marker_corners, marker_ids, chessboard_corners, chessboard_ids
-
sksurgeryimage.calibration.charuco.
draw_charuco_corners
(image, chessboard_corners, chessboard_ids)[source]¶ Function to draw chessboard corners on an image.
Parameters: - image – input image
- chessboard_corners – from detect_charuco_points
- chessboard_ids – from detect_charuco_points
Returns: new image with corners marked
-
sksurgeryimage.calibration.charuco.
erase_charuco_markers
(image, marker_corners)[source]¶ Method to automatically blank out ChArUco markers, leaving an image that looks like it contains just a chessboard, rather than ChArUco board. It does this by drawing a plain white polygon, with vertices defined by the tag detection process. So, on a synthetic image, this works perfectly. On a real image, due to blurring or other artefacts such as combing, there may be some residual.
Parameters: - image – image containing a view of a ChArUco board.
- marker_corners – detected corners
Returns: edited image
-
sksurgeryimage.calibration.charuco.
filter_out_wrong_markers
(marker_corners, marker_ids, board)[source]¶ Filters out markers that were mis-labelled. For each inner corner on the ChArUco board, if both neighbouring markers are detected, look at the projected positions of this corner using the perspective transformations obtained form the two markers. If the two positions are not close (further than 20 pixels away), then at least one of the markers is mis-labelled but we won’t know which one. Remove both markers.
Parameters: - marker_corners – marker corners detected by OpenCV
- marker_ids – ids of markers detected
- board – charuco board definition
Returns: marker_corners, marker_ids
-
sksurgeryimage.calibration.charuco.
make_charuco_board
(dictionary, number_of_squares, size, image_size)[source]¶ Generates a ChArUco pattern.
Don’t forget to select an image size that is a nice multiple of the square size in millimetres, to avoid any interpolation artefacts. You should check the resultant image has only 2 values, [0|255], and nothing interpolated between these two numbers.
Parameters: - dictionary – aruco dictionary definition
- number_of_squares – tuple of (number in x, number in y)
- size – tuple of (size of chessboard square, size of internal tag), mm.
- image_size – tuple of (image width, image height), pixels.
Returns: image, board
-
sksurgeryimage.calibration.charuco.
make_charuco_with_chessboard
(dictionary=< cv2.aruco.Dictionary 0x7fd13e4e2dd0>, charuco_squares=(19, 26), charuco_size=(5, 4), pixels_per_millimetre=10, chessboard_squares=(9, 14), chessboard_size=3, chessboard_border=0.7)[source]¶ Helper function to make an image of a calibration target combining ChArUco markers and a chessboard. It’s up to the caller to work out a nice number of pixels per millimetre, so that the resultant image is correctly scaled.
Defaults are as used in SmartLiver project. Not also, that we compute the image and coordinates in portrait, but it’s used in landscape.
Parameters: - dictionary – ChArUco dictionary
- charuco_squares – tuple of (squares in x, squares in y)
- charuco_size – tuple of (external size, internal tag size) in mm
- pixels_per_millimetre – which determines size of eventual image.
- chessboard_squares – tuple of (squares in x, squares in y)
- chessboard_size – size of chessboard squares in mm
- chessboard_border – border round chessboard, as fraction of square
Returns: calibration image
Utilities¶
Camera Utilities¶
Functions to check cameras.
Video Interlacing Functions¶
Functions to support deinterlacing, reinterlacing and vertical destacking of 2D video frames.
-
sksurgeryimage.processing.interlace.
deinterlace_to_new
(interlaced)[source]¶ Takes the interlaced image, and splits into two new images of even_rows and odd_rows.
Returns: even_rows, odd_rows images
-
sksurgeryimage.processing.interlace.
deinterlace_to_preallocated
(interlaced, even_rows, odd_rows)[source]¶ Deinterlaces the interlaced image into even_rows and odd_rows images, which must be pre-allocated, and the correct size.
-
sksurgeryimage.processing.interlace.
deinterlace_to_view
(interlaced)[source]¶ Takes the interlaced image, and returns two new views of even_rows and odd_rows.
Returns: even_rows, odd_rows images
-
sksurgeryimage.processing.interlace.
interlace_to_new
(even_rows, odd_rows)[source]¶ Interlaces even_rows and odd_rows images into a new output image.
-
sksurgeryimage.processing.interlace.
interlace_to_preallocated
(even_rows, odd_rows, interlaced)[source]¶ Interlaces even_rows and odd_rows images into the interlaced image, where all inputs must be pre-allocated to the correct size.
-
sksurgeryimage.processing.interlace.
split_stacked_to_new
(stacked)[source]¶ Takes the input stacked image, and extracts the top and bottom half.
Useful if you have stereo 1080x1920 inputs, into an AJA Hi5-3D which stacks them vertically into 2 frames of 540x1920 in the same image.
Returns: top_half and bottom_half images
-
sksurgeryimage.processing.interlace.
split_stacked_to_preallocated
(stacked, top, bottom)[source]¶ Splits a vertically stacked image, extracting the top and bottom halves, assuming images are the right size and pre-allocated.
Useful if you have stereo 1080x1920 inputs, into an AJA Hi5-3D which stacks them vertically into 2 frames of 540x1920 in the same image.
-
sksurgeryimage.processing.interlace.
split_stacked_to_view
(stacked)[source]¶ Takes the input stacked image, and returns views that refer to the top and bottom half.
Returns: top_half, bottom_half images
-
sksurgeryimage.processing.interlace.
stack_to_new
(left, right)[source]¶ Vertically stack left and right array into single output array. Left and right images should have the same dimensions.
Parameters: - left (numpy array) – left image
- right (numpy array.) – right image
-
sksurgeryimage.processing.interlace.
validate_interlaced_image_sizes
(even_rows, odd_rows, interlaced)[source]¶ Validates the sizes of the even_rows, odd_rows and interlaced images.
- Inputs must all be numpy images.
- Inputs must all have the same number of columns.
- Inputs must all have an even number of rows.
- even_rows and odd_rows must have the same number of rows.
- even_rows and odd_rows must have half the number of rows as interlaced.
Morphological Operators¶
Functions to support morphological operators.
In many cases, these will just be convenience wrappers around OpenCV functions.
-
sksurgeryimage.processing.morphological_operators.
dilate_with_cross
(src, dst=None, size=3, iterations=1)[source]¶ Dilates an image with a cross element. OpenCV supports both grey scale and RGB erosion.
Parameters: - src – source image
- dst – if provided, an image of the same size as src
- size – size of structuring element
- iterations – number of iterations
Returns: the eroded image
-
sksurgeryimage.processing.morphological_operators.
erode_with_cross
(src, dst=None, size=3, iterations=1)[source]¶ Erodes an image with a cross element. OpenCV supports both grey scale and RGB erosion.
Parameters: - src – source image
- dst – if provided, an image of the same size as src
- size – size of structuring element
- iterations – number of iterations
Returns: the eroded image
Image Cropper¶
Misc¶
Various utilities, like preparing overlay text.
-
sksurgeryimage.utilities.utilities.
are_similar
(image0, image1, threshold=0.995, metric=5, mean_threshold=0.005)[source]¶ Compares two images to see if they are similar.
Parameters: - image0 (image0,) – The images
- threshold – The numerical threshold to use, default 0.995
- method – The comparison metric, default normalised cross correlation, cv2.TM_CCOEFF_NORMED
- mean_threshold – Also compare the mean values of each array, return false if absolute difference of image means divided by the average of both images is greater than the mean_threshold, if less than zero this test will be skipped
Returns: True if the metric is greater than the threshols, false otherwise or if the images are not the same dimensions or type
-
sksurgeryimage.utilities.utilities.
image_means_are_similar
(image0, image1, threshold=0.005)[source]¶ Compares two images to see if they have similar mean pixel values
Parameters: - image0 (image0,) – The images
- threshold – The mean value threshold to use. return false if absolute difference of image means divided by the average of both images is greater than the mean_threshold.
Returns: false if absolute difference of image means divided by the average of both images is greater than the mean_threshold, true otherwise or if threshold is less than zero.
-
sksurgeryimage.utilities.utilities.
noisy_image
(image, mean=0, stddev=(50, 5, 5))[source]¶ Creates a noise image, based on the dimensions of the passed image. param: the image to define size and channels of output returns: a noisy image
-
sksurgeryimage.utilities.utilities.
prepare_cv2_text_overlay
(overlay_text, frame, text_scale=1)[source]¶ Return settings for text overlay on a cv2 frame.
A class for making a natty WEISS logo
-
class
sksurgeryimage.utilities.weisslogo.
WeissLogo
(image_size=331.0)[source]¶ Bases:
object
Creates a WEISS logo and passes a copy on request
-
sksurgeryimage.utilities.weisslogo.
circle
(img, center, radius, color[, thickness[, lineType[, shift]]]) → img¶ . @brief Draws a circle. . . The function cv::circle draws a simple or filled circle with a given center and radius. . @param img Image where the circle is drawn. . @param center Center of the circle. . @param radius Radius of the circle. . @param color Circle color. . @param thickness Thickness of the circle outline, if positive. Negative values, like #FILLED, . mean that a filled circle is to be drawn. . @param lineType Type of the circle boundary. See #LineTypes . @param shift Number of fractional bits in the coordinates of the center and in the radius value.
-
sksurgeryimage.utilities.weisslogo.
fillConvexPoly
(img, points, color[, lineType[, shift]]) → img¶ . @brief Fills a convex polygon. . . The function cv::fillConvexPoly draws a filled convex polygon. This function is much faster than the . function #fillPoly . It can fill not only convex polygons but any monotonic polygon without . self-intersections, that is, a polygon whose contour intersects every horizontal line (scan line) . twice at the most (though, its top-most and/or the bottom edge could be horizontal). . . @param img Image. . @param points Polygon vertices. . @param color Polygon color. . @param lineType Type of the polygon boundaries. See #LineTypes . @param shift Number of fractional bits in the vertex coordinates.