Detect action

The detect action is the primary detection action in RES. It analyzes camera streams using YOLO-based models to detect objects, track them across frames with stable identities, and monitor how long they remain in specific zones.

​

What it does

The detect action provides a complete detection and tracking pipeline:

1. Captures frames from one or more RTSP camera streams using the optimized StreamManager
2. Runs inference using configured YOLO models (TensorRT or PyTorch)
3. Filters detections based on allowed classes, detection zones, and exclusion zones
4. Tracks objects across frames using BoTSORT with re-identification support
5. Monitors zone dwell time calculating how long each tracked object stays in a zone
6. Saves evidence including detection crops, annotated frames, and video clips

​

Requirements

​

Optional settings

​

Operating modes

The detect action operates in two distinct modes depending on your configuration.

​

Standard detection mode

When enable_tracking_and_zone_dwell is set to false, the action runs in standard detection mode:

• Processes frames until a detection is found or timeout is reached
• Returns immediately when objects matching your criteria are detected
• Saves a single detection image and video clip
• Best for: Alert-based rules where you need to know if something appeared

​

Continuous tracking mode

When enable_tracking_and_zone_dwell is set to true, the action runs continuously:

• Processes frames indefinitely (set max_runtime_seconds to 0)
• Tracks objects across frames with stable identities
• Calculates zone dwell time for each tracked object
• Yields results continuously for downstream edge conditions
• Best for: Monitoring scenarios where you need to track presence duration

​

Multi-stream support

The detect action can process multiple camera streams simultaneously: When using multi-stream mode, each stream must have at least one detection or exclusion zone configured. Streams without zones are automatically released and skipped.

​

Detection zones

Zones define regions of interest in each camera frame. Coordinates are specified as percentages (0-100) of the frame dimensions.

​

Zone types

​

Zone filtering logic

1. If a detection falls inside any exclusion zone, it’s discarded
2. If detection zones are configured, the detection must be inside at least one detection zone
3. If no detection zones are configured, all non-excluded detections are valid

​

Object tracking deep dive

When tracking is enabled, the detect action uses BoTSORT (Bag of Tricks for SORT) with re-identification to maintain stable object identities across frames.

​

How tracking works

​

Track ID format

Track IDs are constructed to be globally unique: This format ensures that even when processing multiple streams with multiple models, every tracked object has a unique identifier.

​

Re-identification (ReID)

BoTSORT includes appearance-based re-identification:

• Extracts visual features from each detected object
• Compares features when associating detections to tracks
• Helps recover tracks after occlusions or brief disappearances
• Particularly effective for person tracking

​

Track lifecycle

​

Zone dwell time

When tracking is enabled, the ZoneDwellEngineStable calculates how long each tracked object remains in detection zones.

​

How dwell time works

​

Stable IDs

The zone dwell engine assigns human-readable stable IDs to tracked objects:

​

Zone events

The engine logs zone entry and exit events to the database:

​

Image-from-previous-actions mode

The detect action can also run detection on images from previous workflow actions instead of live streams.

​

When this mode activates

• No streams are configured in the action detail, AND
• Previous actions have attachments in their results

​

Supported attachment formats

This mode is useful for cascading detection workflows where you want to run a secondary model on crops from an initial detection.

​

Result format

​

During processing (tracking mode)

While the action is running in tracking mode, it yields results with:

​

On completion

​

Possible errors

​

Performance considerations

​

Memory management

The detect action implements several optimizations to manage memory during long-running tracking sessions:

​

Practical examples

​

Example 1: Detecting unauthorized access

Scenario: You want to detect when people enter a restricted area during business hours. Configuration:

• Set up one detection zone covering the restricted area
• Configure a person detection model with appropriate confidence threshold
• Set max_runtime_seconds to 60 for quick detection
• Leave tracking disabled since you only need to know if someone entered

Expected behavior: The action captures frames and returns as soon as a person is detected in the zone. A detection image and video clip are saved for review.

​

Example 2: Monitoring queue wait times

Scenario: You want to track how long customers wait in a queue area. Configuration:

• Draw a detection zone covering the queue area
• Enable enable_tracking_and_zone_dwell
• Set max_runtime_seconds to 0 for continuous operation
• Configure edge conditions to trigger when dwell time exceeds threshold

Expected behavior: The action continuously tracks people in the queue, assigning stable IDs and calculating dwell time. Detection labels show person ID:A1 05:23 indicating person A1 has been waiting 5 minutes 23 seconds. Edge conditions can trigger violations when wait times exceed limits.

​

Example 3: Multi-camera perimeter monitoring

Scenario: You want to monitor multiple entry points simultaneously. Configuration:

• Add multiple streams to the action detail
• Configure detection zones for each stream’s entry point
• Enable tracking to maintain identities across frames
• Set appropriate confidence thresholds for outdoor conditions

Expected behavior: All streams are processed in parallel. Each stream has independent tracking with globally unique IDs. Detections from any stream can trigger the workflow’s edge conditions.