Birdwatcher API Documentation

Video input from file

This module contains classes and functions to work with video files. It depends on FFmpeg.

class birdwatcher.VideoFileStream(filepath, streamnumber=0)

Video stream from file.

This class can read video frames from a file.

Parameters:
  • filepath (str of pathlib.Path) – Path to videofile.
  • streamnumber (int, optional) – Video stream number to use as input. Often there is just one video stream present in a video file (default=0), but if there are more you can use this parameter to specify which one you want.

Examples

>>> import birdwatcher as bw
>>> vfs = bw.VideoFileStream('zebrafinchrecording.mp4')
>>> frames = vfs.iter_frames() # create frame iterator
>>> frames.togray().tovideo('zebrafinchrecording_gray.mp4')
avgframerate

Average frame rate of video stream, as reported in the metadata of the video file.

count_frames(threads=8, ffprobepath='ffprobe')

Count the number of frames in video file stream.

This can be necessary as the number of frames reported in the video file metadata may not be accurate. This method requires decoding the whole video stream and may take a lot of time. Use the nframes property if you trust the video file metadata and want fast results.

Parameters:
  • threads (int, default=8) – The number of threads you want to devote to decoding.
  • ffprobepath (str or Path, default='ffprobe') –
Returns:

The number of frames in video file.
Return type:

int
duration

Duration of video stream in seconds, as reported in the metadata of the video file.

extract_audio(outputpath=None, overwrite=False)

Extract audio as 24-bit pcm wav file.

Parameters:
  • outputpath (str or pathlib.Path, optional) – Filename and path to write audio to. The default is None, which means the same name as the video file is used, but then with ‘.wav’ extension.
  • overwrite (bool, default=False) – Overwrite if audio file exists or not.
formatmetadata

Metadata of video file format as provided by ffprobe.

frameheight

height in pixels of frames in video stream.

framesize

tuple (frame width, frame height) in pixels in video stream.

framewidth

Width in pixels of frames in video stream.

get_frame(framenumber, color=True, ffmpegpath='ffmpeg')

Get frame specified by frame sequence number.

Note that this can take a lot of processing because the video has to be decoded up to that number. Specifying by time is more efficient (see: get_frameat method.

Parameters:
  • framenumber (int) – Get the frame framenumber from the video stream.
  • color (bool, default=True) – Read as a color frame (3 dimensional) or as a gray frame (2 dimensional).
  • ffmpegpath (str or pathlib.Path, optional) – Path to ffmpeg executable. Default is ffmpeg, which means it should be in the system path.
Returns:

The frame at the specified framenumber.
Return type:

numpy ndarray

Example

>>> import birdwatcher as bw
>>> vfs = bw.testvideosmall()
>>> frame = vfs.get_frame(500)
get_frameat(time, color=True, ffmpegpath='ffmpeg')

Get frame at specified time.

Parameters:
  • time (str) – Get frame at a time point in the video file. You can use two different time unit formats: sexagesimal (HOURS:MM:SS.MILLISECONDS, as in 01:23:45.678), or in seconds.
  • color (bool, default=True) – Read as a color frame (2 dimensional) or as a gray frame (3 dimensional).
  • ffmpegpath (str or pathlib.Path, optional) – Path to ffmpeg executable. Default is ffmpeg, which means it should be in the system path.
Returns:

The frame at the specified time.
Return type:

numpy ndarray

Example

>>> import birdwatcher as bw
>>> vfs = bw.testvideosmall()
>>> frame = vfs.get_frameat('5.05') # at 5 sec and 50 msec
>>> frame = vfs.get_frameat('00:00:05.05') # same thing
get_info()

Provides a dictionary will all kinds of video info.

Much of it is provided by ffprobe.

Returns:
Return type:Dictionary with info.
iter_frames(startat=None, nframes=None, color=True, ffmpegpath='ffmpeg', reportprogress=False)

Iterate over frames in video.

Parameters:
  • startat (str, optional) – If specified, start at this time point in the video file. You can use two different time unit formats: sexagesimal (HOURS:MM:SS.MILLISECONDS, as in 01:23:45.678), or in seconds.
  • nframes (int, optional) – Read a specified number of frames.
  • color (bool, default=True) – Read as a color frame (3 dimensional) or as a gray frame (2 dimensional). Color conversions occur through ffmpeg.
  • ffmpegpath (str or pathlib.Path, optional) – Path to ffmpeg executable. Default is ffmpeg, which means it should be in the system path.
  • reportprogress (bool, default=False) –
Yields:

Frames – Iterator that generates numpy array frames (height x width x color channel).
nframes

Number of frames in video stream as reported in the metadata of the video file. Note that this may not be accurate. Use count_frames to measure the actual number (may take a lot of time).

show(startat=None, nframes=None, framerate=None)

Shows frames in a video window.

The frames of a VideoFileStream are displayed in a seperate window. Press ‘q’ to quit the video before the end.

Parameters:startat (str, optional) – If specified, start at this time point in the video file. You can use two different time unit formats: sexagesimal (HOURS:MM:SS.MILLISECONDS, as in 01:23:45.678), or in seconds.
nframes : int, optional
Read a specified number of frames.
framerate : int, optional
The default framerate is None, which means that the average frame rate of video stream, as reported in the metadata, is used.
streammetadata

Metadata of video stream as provided by ffprobe.

birdwatcher.testvideosmall()

A 20-s video of a zebra finch for testing purposes.

Returns:An instance of Birdwatcher’s VideoFileSteam class.
Return type:VideoFileStream

Frame processing

Frames is a central class in Birdwatcher. It is an iterable that yields frames and many of its methods return another Frames object. Since there are quite a few methods, we’ll list property and method names first, and then provide detailed info below that.

Frames properties

  • dtype
  • frameheight
  • framewidth
  • nchannels

Frames methods

  • absdiff_frame
  • add_weighted
  • apply_backgroundsegmenter
  • blur
  • calc_meanframe
  • crop
  • draw_circles
  • draw_framenumbers
  • draw_rectangles
  • draw_text
  • find_contours
  • find_nonzero
  • get_info
  • morphologyex
  • peek_frame
  • resize
  • resizebyfactor
  • show
  • threshold
  • tocolor
  • togray
  • tovideo

This module contains classes for generating image frames and general processing functionality such as measurement, drawing of labels/text and saving as video files.

Many methods mirror functions from OpenCV. Docstrings are provided but it is a good idea to look at OpenCV’s documentation and examples if you want to understand the parameters in more depth.

class birdwatcher.Frames(frames, processingdata=None)

An iterator of video frames with useful methods.

This is a main base class in Birdwatcher, as many functions and methods return this type and take it as input. Many methods of Frames objects return new Frames objects, but some of them generate final output, such as a video file or a measurement.

Parameters:frames (iterable) – This can be anything that is iterable and that produces image frames. A numpy array, a VideoFileStream or another Frames object.

Examples

>>> import birdwatcher as bw
>>> frames = bw.FramesNoise(250, height=720, width=1280)
>>> frames = frames.draw_framenumbers()
>>> frames.tovideo('noisewithframenumbers.mp4', framerate=25)
>>> # next example based on input from video file
>>> vfs = bw.VideoFileStream('zebrafinchrecording.mp4')
>>> frames = vfs.iter_frames() # create Frames object
>>> # more concise expression
>>> frames.blur(ksize=(3,3)).togray().tovideo('zf_blurgray.mp4')
absdiff_frame(frame)

Subtract static image frame from frame iterator.

Parameters:frame (ndarray frame) – Fixed image frame that will be subtracted from each frame of the frame iterator.
Yields:Frames – Iterator that generates absolute difference frames.
add_weighted(alpha, frames, beta, gamma=0)

Calculates the weighted sum of frames from self and the frames of the object specified by the frames parameter.

Parameters:
  • alpha (float) – Weight of the frames of self.
  • frames (frame iterator) – The other source of input frames.
  • beta (float) – Weight of the frames of the other frame iterator, specified by the frames parameter.
  • gamma (float, optional) – Scalar added to each sum.
Yields:

Frames – Iterator that generates summed image frames.
apply_backgroundsegmenter(bgs, fgmask=None, learningRate=-1.0, roi=None, nroi=None)

Compute foreground masks based on input sequence of frames.

Parameters:
  • bgs (Subclass of BaseBackgroundSubtractor) – Instance of one of Birdwatcher’s BackgroundSubtractor classes, such as BackgroundSubtractorMOG2.
  • fgmask (numpy array image, optional) – The output foreground mask as an 8-bit binary image.
  • learningRate (float, optional) – The value between 0 and 1 that indicates how fast the background model is learnt. The default negative parameter value (-1.0) makes the algorithm to use some automatically chosen learning rate. 0 means that the background model is not updated at all, 1 means that the background model is completely reinitialized from the last frame.
  • roi ((int, int, int, int), optional) – Region of interest. Only look at this rectangular region. h1, h2, w1, w2.
  • nroi ((int, int, int, int), optional) – Not region of interest. Exclude this rectangular region. h1, h2, w1, w2.
Yields:

Frames – Iterator that generates foreground masks.
blur(ksize, anchor=(-1, -1), borderType=4)

Blurs frames using the normalized box filter.

Parameters:
  • ksize ((int, int)) – Kernel size. Tuple of integers.
  • anchor ((int, int), optional) – Anchor point. Default value (-1,-1) means that the anchor is at the kernel center.
  • borderType (int, default=cv.BORDER_DEFAULT) – Border mode used to extrapolate pixels outside of the image.
Yields:

Frames – Iterator that generates blurred frames.

Examples

>>> import birdwatcher as bw
>>> frames = bw.FramesNoise(250, height=720, width=1280)
>>> frames = frames.blur(ksize=(10,10))
>>> frames.tovideo('noiseblurred.mp4', framerate=25)
crop(h1, h2, w1, w2)

Crops frames to a smaller size.

Parameters:
  • h1 (int) – Top pixel rows.
  • h2 (int) – Bottom pixel row.
  • w1 (int) – Left pixel column.
  • w2 (int) – Right pixel colum.
Yields:

Frames – Iterator that generates cropped frames.
draw_circles(centers, radius=6, color=(255, 100, 0), thickness=2, linetype=16, shift=0)

Draws circles on frames.

Centers should be an iterable that has a length that corresponds to the number of frames.

Parameters:
  • centers (iterable) – Iterable that generate center coordinates (x, y) of the circles
  • radius (int, default=6) – Radius of circle.
  • color (tuple of ints, optional) – Color of circle (BGR). The default (255, 100, 0) color is blue.
  • thickness (int, default=2) – Line thickness.
  • linetype (int, default=cv2.LINE_AA) – OpenCV line type of circle boundary.
  • shift (int, default=0) – Number of fractional bits in the coordinates of the center and in the radius value.
Yields:

Frames – Iterator that generates frames with circles.
draw_framenumbers(startat=0, org=(2, 25), fontface=0, fontscale=1, color=(200, 200, 200), thickness=2, linetype=16)

Draws the frame number on frames.

Parameters:
  • startat (int, optional) – The number to start counting at.
  • org ((int, int), optional) – A tuple of ints (horizontal coordinate value, vertical coordinate value) indicates where to draw the framenumbers. The default (2, 25) draws numbers in the top left corner of the image.
  • fontface (OpenCV font type, default=cv.FONT_HERSHEY_SIMPLEX) –
  • fontscale (float, optional) – Font scale factor that is multiplied by the font-specific base size.
  • color ((int, int, int), optional) – Font color (BGR). The default (200, 200, 200) color is gray.
  • thickness (int, default=2) – Line thickness.
  • linetype (int, default=cv2.LINE_AA) – OpenCV line type.
Yields:

Frames – Iterator that generates frames with frame numbers.
draw_rectangles(points, color=(255, 100, 0), thickness=2, linetype=16, shift=0)

Draws rectangles on frames.

Points should be an iterable that has a length that corresponds to the number of frames.

Parameters:
  • points (iterable) – Iterable that generates sequences of rectangle corners ((x1, y1), (x2, y2)) per frame, where the coordinates specify opposite corners (e.g. top-left and bottom-right).
  • color (tuple of ints, optional) – Color of rectangle (BGR). The default (255, 100, 0) color is blue.
  • thickness (int, default=2) – Line thickness.
  • linetype (int, default=cv2.LINE_AA) – OpenCV line type of rectangle boundary.
  • shift (int, default=0) – Number of fractional bits in the point coordinates.
Yields:

Frames – Iterator that generates frames with rectangles.
draw_text(textiterator, org=(2, 25), fontface=0, fontscale=1, color=(200, 200, 200), thickness=2, linetype=16)

Draws text on frames.

Parameters:
  • textiterator (iterable) – Something that you can iterate over and that produces text for each frame.
  • org ((int, int), optional) – A tuple of ints (horizontal coordinate value, vertical coordinate value) indicates where to draw the text. The default (2, 25) draws text in the top left corner of the image.
  • fontface (OpenCV font type, default=cv.FONT_HERSHEY_SIMPLEX) –
  • fontscale (float, optional) – Font scale factor that is multiplied by the font-specific base size.
  • color ((int, int, int), optional) – Font color (BGR). The default color (200, 200, 200) is gray.
  • thickness (int, default=2) – Line thickness.
  • linetype (int, default=cv2.LINE_AA) – OpenCV line type.
Yields:

Frames – Iterator that generates frames with text.
edge_detection(minval=80, maxval=150)

Finds edges (boundaries) in frames.

Only works on gray frames! Blur frames before applying edge detection for optimal results. Edges are defined by sudden changes in pixel intensity.

Parameters:
  • minval (str, optional) – Lower threshold for finding smaller edges.
  • maxval (str, optional) – Higher threshold to determine segments of strong edges.
Yields:

Frames – Iterator that generates frames with edges.
find_contours(retrmode='tree', apprmethod='simple', offset=(0, 0))

Finds contours in frames.

Contours can only be performed on gray frames. Use threshold or edge detection before applying contours for optimal results.

Parameters:
  • retrmode (str, optional) –
  • apprmethod (str, optional) –
  • offset ((int, int), optional) –
Yields:

Generator – Iterator that generates tuples (contours, hierarchy), with contours as a tuple of arrays, and hierarchy as an array denoting the parent-child relationship between contours.
find_nonzero()

Yields the locations of non-zero pixels.

If the frame is a color frame, non-zero means that a pixel does not have the value (0,0,0).

Yields:Iterator that generates shape (N, 2) arrays, where N is the number – of non-zero pixels.
morphologyex(morphtype='open', kernelsize=2, iterations=1)

Performs advanced morphological transformations on frames.

Can perform advanced morphological transformations using an erosion and dilation as basic operations.

In case of multi-channel images, each channel is processed independently.

Parameters:
  • morphtype ({'open', 'erode', 'dilate, 'close', 'gradient', 'tophat',) –
  • 'blackhat'} – Type of transformation. Default is ‘open’, which is an erosion followed by a dilation.
  • kernelsize (int, default=2) – Size of kernel in 1 dimension.
  • iterations (int, default=1) – Number of times erosion and dilation are applied.
Yields:

Frames – Iterator that generates transformed image frames.
peek_frame()

Returns first frame without removing it.

Returns:The first frame.
Return type:numpy ndarray
resize(dsize, interpolation='linear')

Resizes frames.

Parameters:
  • dsize (tuple) – Destination size (width, height).
  • interpolation ({'linear', 'nearest', 'area', 'cubic', 'lanczos4'},) – optional Interpolation method: linear - a bilinear interpolation (used by default). nearest - a nearest-neighbor interpolation. area - resampling using pixel area relation. It may be a preferred method for image decimation, as it gives moire-free results. But when the image is zoomed, it is similar to the nearest method. cubic - a bicubic interpolation over 4x4 pixel neighborhood. lanczos4 - a Lanczos interpolation over 8x8 pixel neighborhood.
Yields:

Frames – Iterator that generates resized frames.
resizebyfactor(fx, fy, interpolation='linear')

Resizes frames by a specified factor.

Parameters:
  • fx (float) – Scale factor along the horizontal axis.
  • fy (float) – Scale factor along the vertical axis.
  • interpolation ({'linear', 'nearest', 'area', 'cubic', 'lanczos4'},) – optional Interpolation method: linear - a bilinear interpolation (used by default). nearest - a nearest-neighbor interpolation. area - resampling using pixel area relation. It may be a preferred method for image decimation, as it gives moire-free results. But when the image is zoomed, it is similar to the nearest method. cubic - a bicubic interpolation over 4x4 pixel neighborhood. lanczos4 - a Lanczos interpolation over 8x8 pixel neighborhood.
Yields:

Frames – Iterator that generates resized frames.
save_nonzero(filepath, metadata, ignore_firstnframes=10, overwrite=True)

Save nonzero pixel coordinates (i.e. foreground) as Coordinate Arrays object.

Parameters:
  • filepath (str) – Name of the filepath that should be written to.
  • metadata (dict, optional) –
  • ignore_firstnframes (int, default=10) – Do not provide coordinates for the first n frames. These often have a lot of false positives.
  • overwrite (bool, default=True) – Overwrite existing CoordinateArrays or not.
Returns:
Return type:

CoordinateArrays
show(framerate=25)

Shows frames in a video window.

Iterates through Frames and displaying each frame in a seperate window. Press ‘q’ to quit the video before the end.

Parameters:framerate (int, default=25) –

Notes

Frames iterator is (partly) empty after using ‘show’.

threshold(thresh, maxval=255, threshtype='tozero')

Thresholds frames at value thresh.

Parameters:
  • thresh (int) – Threshold value.
  • maxval (int, default=255) – Maximum value to use with the THRESH_BINARY and THRESH_BINARY_INV thresholding types.
  • threshtype ({'tozero', 'tozero_inv', 'binary', 'binary_inv', 'trunc',) – ‘mask’, ‘otsu’, ‘triangle’}, optional Thresholding type. The default is ‘tozero’, which means that everything below thresh will be set to zero. See doc OpenCV.
Yields:

Frames – Iterator that generates thresholded frames.
tocolor()

Converts gray frames to color frames using OpenCV.

Yields:Frames – Iterator that generates color frames.
togray()

Converts color frames to gray frames using OpenCV.

Yields:Frames – Iterator that generates gray frames.
tovideo(filepath, framerate, crf=23, scale=None, format='mp4', codec='libx264', pixfmt='yuv420p', ffmpegpath='ffmpeg')

Writes frames to video file.

Parameters:
  • filepath (str) – Name of the videofilepath that should be written to.
  • framerate (int) – framerate of video in frames per second.
  • crf (int, default=23) – Value determines quality of video. The default 23 is good quality. Use 17 for high quality.
  • scale (tuple, optional) – (width, height). The default (None) does not change width and height.
  • format (str, default='mp4') – ffmpeg video format.
  • codec (str, default='libx264') – ffmpeg video codec.
  • pixfmt (str, default='yuv420p') – ffmpeg pixel format.
  • ffmpegpath (str or pathlib.Path, optional) – Path to ffmpeg executable. Default is ffmpeg, which means it should be in the system path.

Notes

See ffmpeg documentation for more information.

class birdwatcher.FramesColor(nframes, height, width, color=(0, 0, 0), dtype='uint8')

An iterator that yields color frames.

This class inherits from Frames, and hence has all its methods.

class birdwatcher.FramesGray(nframes, height, width, value=0, dtype='uint8')

An iterator that yields gray frames.

This class inherits from Frames, and hence has all its methods.

Background subtraction

This module provides background subtractors. Background subtraction is a major preprocessing step in many computer vision applications. It extracts the moving foreground from static background.

All classes are based on OpenCV’s background subtraction algorithms. They can be used for example in movement detection. Note that OpenCV’s API to parameters of these algorithms is inconsistent. Sometimes parameters can be provided at instantiation, sometimes you can change them with a method. In Birdwatcher you can only provide parameters to background subtractor objects at instantiation. The parameter names follow those of OpenCV to avoid confusion.

MOG2

class birdwatcher.BackgroundSubtractorMOG2(**kwargs)

Wraps OpenCV’s BackgroundSubtractorMOG2 class. Parameter names follow those in OpenCV.

Parameters:
  • History (int, default=3) – Length of the history.
  • ComplexityReductionThreshold (float, default=0.5) – This parameter defines the number of samples needed to accept to prove the component exists. CT=0.05 is a default value for all the samples. By setting CT=0 you get an algorithm very similar to the standard Stauffer&Grimson algorithm.
  • BackgroundRatio (float, default=0.1) – If a foreground pixel keeps semi-constant value for about backgroundRatio*history frames, it’s considered background and added to the model as a center of a new component. It corresponds to TB parameter in the paper.
  • NMixtures (int, default=7) – The number of gaussian components in the background model.
  • VarInit (int, default=15) – The initial variance of each gaussian component.
  • VarMin (int, default=10) – The minimum variance of each gaussian component.
  • VarMax (int, default=75) – The maximum variance of each gaussian component.
  • VarThreshold (int, default=70) – The variance threshold for the pixel-model match. The main threshold on the squared Mahalanobis distance to decide if the sample is well described by the background model or not. Related to Cthr from the paper.
  • VarThresholdGen (int, default=9) – The variance threshold for the pixel-model match used for new mixture component generation. Threshold for the squared Mahalanobis distance that helps decide when a sample is close to the existing components (corresponds to Tg in the paper). If a pixel is not close to any component, it is considered foreground or added as a new component. 3 sigma => Tg=3*3=9 is default. A smaller Tg value generates more components. A higher Tg value may result in a small number of components but they can grow too large.
  • DetectShadows (bool, default=False) – If true, the algorithm detects shadows and marks them.
  • ShadowThreshold (float, default=0.5) – A shadow is detected if pixel is a darker version of the background. The shadow threshold is a threshold defining how much darker the shadow can be. 0.5 means that if a pixel is more than twice darker then it is not shadow.
  • ShadowValue (int, default=0) – Shadow value is the value used to mark shadows in the foreground mask. Value 0 in the mask always means background, 255 means foreground.
apply(frame, fgmask=None, learningRate=-1.0)

Computes a foreground mask.

Parameters:
  • frame (numpy array image) – Next video frame.
  • fgmask (numpy array image, optional) – The output foreground mask as an 8-bit binary image.
  • learningRate (float, optional) – The value between 0 and 1 that indicates how fast the background model is learnt. The default negative parameter value (-1.0) makes the algorithm to use some automatically chosen learning rate. 0 means that the background model is not updated at all, 1 means that the background model is completely reinitialized from the last frame.
Returns:

The output foreground mask as an 8-bit image.
Return type:

image frame
get_params()

Get parameters of the background subtraction algorithm.

Returns:Dictionary with algorithm parameters.
Return type:dict

KNN

class birdwatcher.BackgroundSubtractorKNN(**kwargs)

Wraps OpenCV’s BackgroundSubtractorKNN class. Parameter names follow those in OpenCV.

Parameters:
  • History (int, default=5) – Length of the history.
  • kNNSamples (int, default=10) – The number of neighbours, the k in the kNN. K is the number of samples that need to be within dist2Threshold in order to decide that that pixel is matching the kNN background model.
  • NSamples (int, default=6) – The number of data samples in the background model.
  • Dist2Threshold (float, default=500) – Threshold on the squared distance between the pixel and the sample to decide whether a pixel is close to that sample. This parameter does not affect the background update.
  • DetectShadows (bool, default=False) – If true, the algorithm detects shadows and marks them.
  • ShadowThreshold (float, default=0.5) – A shadow is detected if pixel is a darker version of the background. The shadow threshold is a threshold defining how much darker the shadow can be. 0.5 means that if a pixel is more than twice darker then it is not shadow.
  • ShadowValue (int, default=127) – Shadow value is the value used to mark shadows in the foreground mask. Value 0 in the mask always means background, 255 means foreground.
apply(frame, fgmask=None, learningRate=-1.0)

Computes a foreground mask.

Parameters:
  • frame (numpy array image) – Next video frame.
  • fgmask (numpy array image, optional) – The output foreground mask as an 8-bit binary image.
  • learningRate (float, optional) – The value between 0 and 1 that indicates how fast the background model is learnt. The default negative parameter value (-1.0) makes the algorithm to use some automatically chosen learning rate. 0 means that the background model is not updated at all, 1 means that the background model is completely reinitialized from the last frame.
Returns:

The output foreground mask as an 8-bit image.
Return type:

image frame
get_params()

Get parameters of the background subtraction algorithm.

Returns:Dictionary with algorithm parameters.
Return type:dict

LSBP

class birdwatcher.BackgroundSubtractorLSBP(**kwargs)

Wraps OpenCV’s BackgroundSubtractorLSBP class. Parameter names follow those in OpenCV.

Parameters:
  • mc (int, default=0) – Whether to use camera motion compensation.
  • nSamples (int, default=20) – Number of samples to maintain at each point of the frame.
  • LSBPRadius (int, default=16) – LSBP descriptor radius.
  • Tlower (float, default=2.0) – Lower bound for T-values. See [103] for details.
  • Tupper (float, default=32.0) – Upper bound for T-values. See [103] for details.
  • Tinc (float, default=1.0) – Increase step for T-values.
  • Tdec (float, default=0.05) – Decrease step for T-values.
  • Rscale (float, default=10.0) – Scale coefficient for threshold values.
  • Rincdec (float, default=0.005) – Increase/Decrease step for threshold values.
  • noiseRemovalThresholdFacBG (float, default=0.0004) – Strength of the noise removal for background points.
  • noiseRemovalThresholdFacFG (float, default=0.0008) – Strength of the noise removal for foreground points.
  • LSBPthreshold (int, default=8) – Threshold for LSBP binary string.
  • minCount (int, default=2) – Minimal number of matches for sample to be considered as foreground.
apply(frame, fgmask=None, learningRate=-1.0)

Computes a foreground mask.

Parameters:
  • frame (numpy array image) – Next video frame.
  • fgmask (numpy array image, optional) – The output foreground mask as an 8-bit binary image.
  • learningRate (float, optional) – The value between 0 and 1 that indicates how fast the background model is learnt. The default negative parameter value (-1.0) makes the algorithm to use some automatically chosen learning rate. 0 means that the background model is not updated at all, 1 means that the background model is completely reinitialized from the last frame.
Returns:

The output foreground mask as an 8-bit image.
Return type:

image frame
get_params()

Get parameters of the background subtraction algorithm.

Returns:Dictionary with algorithm parameters.
Return type:dict

Coordinate Arrays

This module provides objects and functions for coordinate data. Coordinates are pixel positions (x,y) in a Frame. Coordinate Arrays are a convenient way of storing output from image algorithms that determine if a pixel is categorized as something (e.g. crossing some threshold). Information is memory-mapped from disk, because data can easily become very large and will not fit in RAM memory. Since the number of pixels per frame may be variable, we use Ragged Arrays from the python library Darr to store them. This is not the most disk-space efficient way of doing this (no compression), but it is fast and the data can easily be read in any scientific computing environement (Python, Matlab, R, Mathematica, etc.) Coordinate files can be archived in compressed form (lzma) to save disk space.

class birdwatcher.CoordinateArrays(path, accessmode='r')

A disk-based data type to store frame coordinates of consecutive frames.

Maximum for frame width and height is 65535.

Parameters:
  • path (str or pathlib.Path) – Path to disk-based coordinate array directory.
  • accessmode ({'r', 'r+'}, default 'r') – File access mode of the darr data. r means read-only, r+ means read-write. w does not exist. To create new coordinate arrays, potentially overwriting an other one, use the create_coordarray functions.
accessmode

Data access mode of metadata, {‘r’, ‘r+’}.

append(array)

Append array-like objects to the ragged array.

The shape of the data and the darr must be compliant. The length of its first axis may vary, but if the are more axes, these should have the same lengths as all other subarrays (which is the atom of the raged array). When appending data repeatedly it is more efficient to use iterappend.

Parameters:array (array-like object) – This can be a numpy array, a sequence that can be converted into a numpy array.
Returns:
Return type:None
archive(filepath=None, compressiontype='xz', overwrite=False)

Archive ragged array data into a single compressed file.

Parameters:
  • filepath (str) – Name of the archive. In None, it will be derived from the data’s path name.
  • compressiontype (str) – One of ‘xz’, ‘gz’, or ‘bz2’, corresponding to the gzip, bz2 and lzma compression algorithms supported by the Python standard library.
  • overwrite ((True, False), optional) – Overwrites existing archive if it exists. Default is False.
Returns:

The path of the created archive
Return type:

pathlib.Path

Notes

See the tarfile library for more info on archiving formats

atom

Dimensions of the non-variable axes of the arrays.

copy(path, dtype=None, accessmode='r', overwrite=False)

Copy darr to a different path, potentially changing its dtype.

The copying is performed in chunks to avoid RAM memory overflow for very large darr arrays.

Parameters:
  • path (str or pathlib.Path) –
  • dtype (<dtype, None>) – Numpy data type of the copy. Default is None, which corresponds to the dtype of the darr to be copied.
  • accessmode ({'r', 'r+'}, default 'r') – File access mode of the darr data of the returned Darr object. r means read-only, r+ means read-write.
  • overwrite ((True, False), optional) – Overwrites existing darr data if it exists. Note that a darr path is a directory. If that directory contains additional files, these will not be removed and an OSError is raised. Default is False.
Returns:

copy of the darr array
Return type:

Array
datadir

Data directory object with many useful methods, such as writing information to text or json files, archiving all data, calculating checksums etc.

dtype

Numpy data type of the array values.

get_coordcount(startframeno=0, endframeno=None)

Get the number of coordinates present per frame.

Parameters:
  • startframeno (int, optional) – Defaults to the beginning of the coordinate array.
  • endframeno (int, optional) – Defaults to the end of the coordinate array.
Returns:

Sequence of numbers, each with a coordinate count.
Return type:

Numpy Array
get_coordmean(startframeno=0, endframeno=None)

Get the mean of the coordinates per frame.

Parameters:
  • startframeno (int, optional) – Defaults to the beginning of the coordinate array.
  • endframeno (int, optional) – Defaults to the end of the coordinate array.
Returns:

Sequence of numbers, each with a coordinate mean.
Return type:

Numpy Array
get_coordmedian(startframeno=0, endframeno=None)

Get the median of the coordinates per frame.

Parameters:
  • startframeno (int, optional) – Defaults to the beginning of the coordinate array.
  • endframeno (int, optional) – Defaults to the end of the coordinate array.
Returns:

Sequence of numbers, each with a coordinate median.
Return type:

Numpy Array
get_frame(frameno, nchannels=None, dtype='uint8', value=1)

Get a frame based on a sequence number in the coordinate array.

Parameters:
  • frameno (int) – The sequence number of the frame to get.
  • nchannels (int, optional) – The number of color channels in the frame. Default None leads to no color dimension, just a 2D frame with gray values.
  • dtype (numpy dtype, default='uint8') – Dtype of the returned frame.
  • value (int, default=1) – The value to set the present coordinates with.
Returns:
Return type:

Numpy array
iter_arrays(startindex=0, endindex=None, stepsize=1, accessmode=None)

Iterate over ragged array yielding subarrays.

startindex: <int, None>
Start index value. Default is None, which means to start at the beginning.
endindex: <int, None>
End index value. Default is None, which means to end at the end.
stepsize: <int, None>
Size of the shift per iteration across the first axis. Default is None, which means that stepsize equals chunklen.
iter_frames(startframe=0, endframe=None, stepsize=1, nchannels=None, dtype='uint8', value=1)

Iterate over coordinate array and produce frames.

Parameters:
  • startframe (int, optional) – If specified, start iteration at this frame number.
  • endfrom (int, optional) – Frame number to end iteration at. Default is None, which is to the end.
  • stepsize (int, optional) – Step sizes. Defaults to 1, but if you want to skip frames, you can use this parameter.
  • nchannels (int, optional) – The number of color channels in the frame. Default is None which leads to no color dimension, just a 2D frame with gray values.
  • dtype (numpy dtype, default='uint8') – Dtype of the returned frame.
  • value (int, default=1) – The value to set the present coordinates with.
Yields:

Frames – Iterator that produces video frames based on the coordinates.
iterappend(arrayiterable)

Iteratively append data from a data iterable.

The iterable has to yield array-like objects compliant with darr. The length of first dimension of these objects may be different, but the length of other dimensions, if any, has to be the same.

Parameters:arrayiterable (an iterable that yield array-like objects) –
Returns:
Return type:None
mb

Storage size in megabytes of the ragged array.

metadata

Dictionary of meta data.

narrays

number of subarrays in the RaggedArray.

path

File system path to array data

readcode(language, abspath=False, basepath=None)

Generate code to read the array in a different language.

Note that this does not include reading the metadata, which is just based on a text file in JSON format.

Parameters:
  • language (str) – One of the languages that are supported. Choose from: ‘matlab’, ‘numpymemmap’, ‘R’.
  • abspath (bool) – Should the paths to the data files be absolute or not? Default: True.
  • basepath (str or pathlib.Path or None) – Path relative to which the binary array data file should be provided. Default: None.

Example

>>> import darr
>>> a = darr.asraggedarray('test.darr', [[1],[2,3],[4,5,6],[7,8,9,10]], overwrite=True)
>>> print(a.readcode('matlab'))
fileid = fopen('indices/arrayvalues.bin');
i = fread(fileid, [2, 4], '*int64', 'ieee-le');
fclose(fileid);
fileid = fopen('values/arrayvalues.bin');
v = fread(fileid, 10, '*int32', 'ieee-le');
fclose(fileid);
% example to read third subarray
startindex = i(1,3) + 1;  % matlab starts counting from 1
endindex = i(2,3);  % matlab has inclusive end index
a = v(startindex:endindex);
readcodelanguages

Tuple of the languages that the readcode method can produce reading code for. Code in these languages is also included in the README.txt file that is stored as part of the array .

show(startframe=0, endframe=None, stepsize=1, framerate=None, draw_framenumbers=True)

Shows coordinates frames in a video window.

Turns each coordinate array into a frame and then plays video.

Parameters:
  • startframe (int, optional) – If specified, start iteration at this frame number.
  • endfrom (int, optional) – Frame number to end iteration at. Default is None, which is to the end.
  • stepsize (int, optional) – Step sizes. Defaults to 1, but if you want to skip frames, you can use this parameter.
  • framerate (int, optional) – framerate of video in frames per second. If None, will look for avgframerate in metadata.
  • draw_framenumbers (bool, default=True) – Should I draw frame numbers yes or no?
size

Total number of values in the ragged array.

tovideo(filepath, startframe=0, endframe=None, stepsize=1, framerate=None, crf=17, scale=None, format='mp4', codec='libx264', pixfmt='yuv420p', ffmpegpath='ffmpeg')

Writes frames based on coordinate info to a video file.

Parameters:
  • filepath (str) – Name of the videofilepath that should be written to.
  • startframe (int, optional) – If specified, start iteration at this frame number.
  • endfrom (int, optional) – Frame number to end iteration at. Default is None, which is to the end.
  • stepsize (int, optional) – Step sizes. Defaults to 1, but if you want to skip frames, you can use this parameter.
  • framerate (int, optional) – framerate of video in frames per second. If None, will look for avgframerate in metadata.
  • crf (int, default=17) – Value determines quality of video. The default 17 is high quality. Use 23 for good quality.
  • scale (tuple, optional) – (width, height). The default (None) does not change width and height.
  • format (str, default='mp4') – ffmpeg video format.
  • codec (str, default='libx264') – ffmpeg video codec.
  • pixfmt (str, default='yuv420p') – ffmpeg pixel format.
  • ffmpegpath (str or pathlib.Path, optional) – Path to ffmpeg executable. Default is ffmpeg, which means it should be in the system path.

Notes

See ffmpeg documentation for more information.

birdwatcher.coordinatearrays.open_archivedcoordinatedata(path, temppath=None)

A context manager that temporarily decompresses coordinate data to work with coordinate array.

Parameters:

path (str) – Path to the archive.
Yields:
  • Context manager to work with temporarily uncompressed coordinate
  • array.

Examples

>>> with open_archivedcoordinatedata('coord.tar.xz') as coords:
        # do stuff with coordinate array
birdwatcher.coordinatearrays.create_coordarray(path, framewidth, frameheight, metadata=None, overwrite=True)

Creates an empty Coordinate Arrays object.

Parameters:
  • path (str) – Path to disk-based coordinate array directory that should be written to.
  • framewidth (int) – Width in pixels.
  • frameheight (int) – Height in pixels.
  • metadata (dict, optional) –
  • overwrite (bool, default=True) – Overwrite existing CoordinateArrays or not.
Returns:
Return type:

CoordinateArrays

Movement detection

Movement detection contains top-level functionality. The classes, methods and functions provided in these submodules, are written to help the user get the most out of Birdwatcher. Also see the notebooks for examples how to use it and how to find the optimal parameter settings for movement detection.

birdwatcher.movementdetection.batch_detect_movement(vfs_list, settings=None, startat=None, nframes=None, roi=None, nroi=None, bgs_type=<class 'birdwatcher.backgroundsubtraction.BackgroundSubtractorMOG2'>, analysispath='.', ignore_firstnframes=10, overwrite=False, resultvideo=False, nprocesses=6)

The reason for having a special batch function, instead of just applying functions in a loop, is that compression of coordinate results takes a long time and is single-threaded. We therefore do this in parallel. Use the nprocesses parameter to specify the number of cores devoted to this.

birdwatcher.movementdetection.detect_movement(vfs, settings=None, startat=None, nframes=None, roi=None, nroi=None, bgs_type=<class 'birdwatcher.backgroundsubtraction.BackgroundSubtractorMOG2'>, analysispath='.', ignore_firstnframes=10, overwrite=False, resultvideo=False)

Detects movement based on a background subtraction algorithm.

High-level function to perform movement detection with default parameters, but also the option to modify many parameter settings.

Parameters:
  • vfs (VideoFileStream) – A Birdwatcher VideoFileStream object.
  • settings ({dict, dict}, optional) – Dictionary with two dictionaries. One ‘bgs_params’ with the parameter settings from the BackgroundSubtractor and another ‘processing’ dictionary with settings for applying color, resizebyfactor, blur and morphologyex manipulations. If None, the default settings of the BackgroundSubtractor are used on grey color frames, including morphological transformation to reduce noise.
  • startat (str, optional) – If specified, start at this time point in the video file. You can use two different time unit formats: sexagesimal (HOURS:MM:SS.MILLISECONDS, as in 01:23:45.678), or in seconds.
  • nframes (int, optional) – Read a specified number of frames.
  • roi ((int, int, int, int), optional) – Region of interest. Only look at this rectangular region. h1, h2, w1, w2.
  • nroi ((int, int, int, int), optional) – Not region of interest. Exclude this rectangular region. h1, h2, w1, w2.
  • bgs_type (BackgroundSubtractor, default=bw.BackgroundSubtractorMOG2) – This can be any of the BackgroundSubtractors in Birdwatcher, e.g. BackgroundSubtractorMOG2, BackgroundSubtractorKNN, BackgroundSubtractorLSBP.
  • analysispath (Path or str, optional) – Where to write results to. The default writes to the current working directory.
  • ignore_firstnframes (int, default=10) – Do not provide coordinates for the first n frames. These often have a lot of false positives.
  • overwrite (bool, default=False) – Overwrite results or not.
  • resultvideo (bool, default=False) – Automatically generate a video with results, yes or no.
Returns:

These are Darr arrays that are disk-based.
Return type:

tuple of arrays (coordinates, coordinate count, coordinate mean)
birdwatcher.movementdetection.apply_settings(vfs, settings, startat=None, nframes=None, roi=None, nroi=None, bgs_type=<class 'birdwatcher.backgroundsubtraction.BackgroundSubtractorMOG2'>)

Applies movement detection based on various parameter settings.

The background subtractor should be provided as a parameter.

Parameters:
  • vfs (VideoFileStream) – A Birdwatcher VideoFileStream object.
  • settings (dict) – Dictionary with parameter settings from the BackgroundSubtractor and settings for applying color, resizebyfactor, blur and morphologyex manipulations.
  • startat (str, optional) – If specified, start at this time point in the video file. You can use two different time unit formats: sexagesimal (HOURS:MM:SS.MILLISECONDS, as in 01:23:45.678), or in seconds.
  • nframes (int, optional) – Read a specified number of frames.
  • roi ((int, int, int, int), optional) – Region of interest. Only look at this rectangular region. h1, h2, w1, w2.
  • nroi ((int, int, int, int), optional) – Not region of interest. Exclude this rectangular region. h1, h2, w1, w2.
  • bgs_type (BackgroundSubtractor, default=bw.BackgroundSubtractorMOG2) – This can be any of the BackgroundSubtractors in Birdwatcher, e.g. BackgroundSubtractorMOG2, BackgroundSubtractorKNN, BackgroundSubtractorLSBP.
Yields:

Frames – Iterator that generates numpy array frames (height x width x color channel).
birdwatcher.movementdetection.create_movementvideo(vfs, coords, startat=None, nframes=None, videofilepath=None, draw_mean=True, draw_framenumbers=(2, 25), crf=17, scale=None)

Create a nice video from the original video with movement detection results superimposed.

Parameters:
  • vfs (VideoFileStream) – A Birdwatcher VideoFileStream object
  • coords (CoordinateArrays) – CoordinateArrays object with movement results.
  • startat (str, optional) – If specified, start at this time point in the video file. You can use two different time unit formats: sexagesimal (HOURS:MM:SS.MILLISECONDS, as in 01:23:45.678), or in seconds.
  • nframes (int, optional) – Read a specified number of frames.
  • videofilepath (Path or str, optional) – Output path. If None, writes to filepath of videofilestream.
  • draw_mean (bool, default=True) – Draw the mean of the coordinates per frame, or not.
  • draw_framenumbers (tuple, optional) – Draw frame numbers. A tuple of ints indicates where to draw them. The default (2, 25) draws numbers in the top left corner. To remove the framenumbers use None.
  • crf (int, default=17) – Quality factor output video for ffmpeg. The default 17 is high quality. Use 23 for good quality.
  • scale (tuple, optional) – (width, height). The default (None) does not change width and height.
Returns:

Videofilestream object of the output video.
Return type:

VideoFileStream

Parameter selection

class birdwatcher.movementdetection.ParameterSelection(df, videofilepath, bgs_type, startat, nframes, roi, nroi, path=None)

A Pandas dataframe with movement detection results of various parameter settings associated with a (fragment of a) Videofilestream.

batch_plot_parameters(default_values, overwrite=False)

Saves multiple figures with subplots of all combinations of parameters.

The figures are saved in a folder ‘figures’ in the same directory as the associated ParameterSelection file. Multiple rounds of plotting with different default values can be easily saved in new folders with a number added to the foldername.

Parameters:
  • default_values (dict) – All parameters that are tested with multiple settings, should be added to a dictionary with each parameter as key, and the default as value.
  • overwrite (bool, default=False) – If False, an integer number (1,2,3,etc.) will be added as suffix to the foldername, if the filepath already exists.
draw_multiple_circles(all_settings, radius=60, thickness=2)

Returns a Frames object with circles on the videofragment.

It is possible to plot multiple circles on the videofragment to see the results from different parameter settings.

Parameters:
  • settings (dict) – All parameters that are tested with multiple settings, should be added to a dictionary with each parameter as key, and the value(s) that should be superimposed on the video added as list.
  • radius (int, default=60) – Radius of circle.
  • thickness (int, default=2) – Line thickness.
Returns:

Iterator that generates frames with multiple circles, and a Pandas DataFrame with the settings for each color of the circles.
Return type:

Frames, DataFrame
get_parameters(selection='multi_only')

Returns the parameter settings used for movement detection.

Parameters:selection ({'all', multi_only'}) – Specify which selection of parameters is returned: all : returns all parameters and their settings. multi_only : returns only parameters for which multiple values have been used to run movement detection.
Returns:With parameters as keys, and each value contains a list of the settings used for movement detection.
Return type:dict
get_videofragment()

Returns video fragment as Frames.

NOTE: the whole frames are returned. If a region of interest (roi or nroi) is specified, this is not visible in the videofragment.

plot_parameters(rows, cols, default_values)

Returns a figure from seaborn with subplots.

Usefull to look at different location detection results from two parameters with various values.

Parameters:
  • rows (str) – One parameter tested with multiple values.
  • cols (str) – A second parameter tested with multiple values.
  • default_values (dict) – All parameters that are tested with multiple settings, should be added to a dictionary with each parameter as key, and the default as value.
Returns:

A seaborn object managing multiple subplots.
Return type:

FacedGrid
save_parameters(path, foldername=None, overwrite=False)

Save results of all parameter settings as .csv file.

Often several rounds of parameter selection per videofragment will be done with different parameter settings. For this, the same foldername could be used, in which case a number is added automatically as suffix to display the round.

Parameters:
  • path (str) – Path to disk-based directory that should be written to.
  • foldername (str, optional) – Name of the folder the data should be written to.
  • overwrite (bool, default=False) – If False, an integer number (1,2,3,etc.) will be added as suffix to the foldername, if the filepath already exists.
birdwatcher.movementdetection.apply_all_parameters(vfs, all_settings, startat=None, nframes=None, roi=None, nroi=None, bgs_type=<class 'birdwatcher.backgroundsubtraction.BackgroundSubtractorMOG2'>, reportprogress=50)

Run movement detection with each set of parameters.

Parameters:
  • vfs (VideoFileStream) – A Birdwatcher VideoFileStream object.
  • all_settings ({dict, dict}) – Dictionary with two dictionaries. One ‘bgs_params’ with the parameter settings from the BackgroundSubtractor and another ‘processing’ dictionary with settings for applying color, resizebyfactor, blur and morphologyex manipulations.
  • startat (str, optional) – If specified, start at this time point in the video file. You can use two different time unit formats: sexagesimal (HOURS:MM:SS.MILLISECONDS, as in 01:23:45.678), or in seconds.
  • nframes (int, optional) – Read a specified number of frames.
  • roi ((int, int, int, int), optional) – Region of interest. Only look at this rectangular region. h1, h2, w1, w2.
  • nroi ((int, int, int, int), optional) – Not region of interest. Exclude this rectangular region. h1, h2, w1, w2.
  • bgs_type (BackgroundSubtractor) – This can be any of the BackgroundSubtractors in Birdwatcher, e.g. BackgroundSubtractorMOG2, BackgroundSubtractorKNN, BackgroundSubtractorLSBP.
  • reportprogress (int or bool, default=50) – The input integer represents how often the progress of applying each combination of settings is printed. Use False, to turn off reportprogress.
birdwatcher.movementdetection.load_parameterselection(path)

Load a parameterselection.csv file.

Parameters:path (str) – Name of the directory where parameterselection.csv is saved.
Returns:
Return type:ParameterSelection

Plotting

This module provides convenient functions for plotting video-related data.

These are based on Matplotlib. If you want to display things automatically in Jupyter, use %matplotlib inline

birdwatcher.plotting.imshow_frame(frame, fig=None, ax=None, figsize=None, cmap=None, draw_rectangle=None)

Create an matplotlib image plot of the frame.

Parameters:
  • frame (numpy array image) – Video frame.
  • fig (matplotlib Figure object, optional) – Provide if you already have a figure in which the frame should be plotted.
  • ax (matplotlib Axes object, optional) – Provide if you already have an axes in which the frame should be plotted.
  • figsize ((float, float), optional) – Width, height in inches. If not provided, defaults to rcParams[“figure.figsize”] = [6.4, 4.8].
  • draw_rectangle ((int, int, int, int), optional) – Draw a rectangle on image. h1, h2, w1, w2. Origin is left top.
Returns:

fig, ax – The matplotlib Figure and Axes objects.
Return type:

matplotlib.figure.Figure, matplotlib.axes.Axes

Utils

birdwatcher.utils.product_dict(**kwargs)

Generates a Cartesian product of dictionary values.

birdwatcher.utils.tempdir(dirname='.', keep=False, report=False)

Yields a temporary directory which is removed when context is closed.

birdwatcher.utils.derive_filepath(filepath, append_string='', suffix=None, path=None)

Generate a file path based on the name and potentially path of the input file path.

Parameters:
  • filepath (str of pathlib.Path) – Path to file.
  • append_string (str, optional) – String to append to file name stem.
  • suffix (str, optional) – File extension to use. If None, the same as video file.
  • path (str or pathlib.Path, optional) – Path to use. If None, use same path as video file.
Returns:

Path derived from video file path.
Return type:

pathlib.Path
birdwatcher.utils.print_dirstructure(dirpath)

Prints the hierarchical structure of directories, starting at dirpath

Parameters:dirpath (str or Path) – The top-level directory to start at.
birdwatcher.utils.walk_paths(dirpath, extension='.*')

Walks recursively over contents of dirpath and yield contents as pathlib Path objects, potentially based on their extension.

Parameters:
  • dirpath (str or Path) – The top-level directory to start at.
  • extension (str, optional) – Filter on this extension. The default includes all extensions.