Integration

Performance and integration considerations to integrate the Celantur SDK effectively.

Pipeline Efficiency: Encoding/Decoding

To achieve real-time throughput, minimizing overhead is essential.

Avoid Redundant Encoding/Decoding

The SDK interface expects a cv::Mat of individual frames. Integrating directly at this level avoids the overhead of repeated image encoding/decoding, which is critical for maintaining performance.

Identify Bottlenecks

• Decoding/Encoding: Becomes a significant bottleneck with very large images (e.g., 70+ MP).

• Detection: Often the primary bottleneck for small to medium resolution images.

• Blurring: Generally the fastest stage of the pipeline.

Recommendations

If the system is feeding the SDK fast enough (e.g., via optimized decoding like turbojpeg), it is possible to reach 100% GPU utilization with a single processor.

CPU vs. GPU Performance (TensorRT/OpenVINO)

The SDK supports diverse deployment environments, offering specific performance pathways.

GPU (TensorRT) Optimization

• TensorRT requires a hardware-specific engine file (.plan). This process is dependent on the specific GPU type, driver version, and TensorRT version.

• Performance depends on input tensor profiles (e.g., fixed vs. dynamic input sizes). Optimizing for fixed input sizes or well-defined profiles typically yields better performance than dynamic ranges.

• GPU memory usage may ramp up until it reaches a stable ceiling; this ceiling is generally determined by the maximum input resolution processed.

CPU (OpenVINO/ONNX)

• CPU inference is available via OpenVINO or ONNX and serves as a robust alternative where GPU acceleration is not present or feasible.

• Scaling CPU performance often requires external parallelism. Handling multiple video feeds simultaneously is recommended for maximizing 24-core CPU utilization.

Scaling Options

Strategic scaling ensures efficient use of infrastructure without unnecessary overhead.

Single Processor vs. Multiple Pods

If one processor can successfully utilize 100% of the assigned hardware resources (CPU/GPU) when fed images fast enough, deploying multiple pods/containers on the same machine often provides no performance improvement.

Horizontal Scaling

• For video processing, implement external parallelism by running multiple concurrent video feeds through the SDK.

• Horizontal scaling (multiple containers/nodes) is effective, but it is advised to monitor whether the input feeding stage (decoding) is keeping pace with the detection stage to prevent under-utilization.

Model Selection & Tuning

• Model Variants: Models are provided in various profiles (default, small) and optimized for different hardware targets (e.g., FP32). Selecting the right model size involves balancing accuracy requirements with the hardware resource ceiling.

• Detection Thresholds: Users can set detection thresholds to control the sensitivity of the model. High thresholds may miss objects, while low thresholds can lead to false positives (artifacts).

• Continuous Improvement: In large-scale projects, performance can be optimized by testing against reference datasets. Providing proprietary imagery for model training can significantly reduce false negatives.

Integration Architecture

Selective Redaction

The SDK allows filtering detections (based on bounding boxes or object types) before the blurring stage, enabling workflows where specific objects (e.g., one person in a frame) can be kept visible while others are anonymized, or detections from 3rd party sources can be ingested.