Segmentation Masks and Metadata
Last updated
Last updated
Celantur container can generate two different segmentation masks and metadata per processed image:
Binary Segmentation
Instance Segmentation
It's activated with the --save-mask {all, instance, binary} parameter. The segmentation is saved as a PNG file.
The binary segmentation mask consist of two colors:
Background is black
Anonymized segments are white
The file will be saved as image-name_bin_mask.png.
The instance segmentation mask consists of multiple colors. The RGB color values are used to differentiate to individual instances/objects.
The R (red) channel encodes the object type:
License plate: 64
Person: 128
Face: 192
Vehicle: 255
The G (green) channel encodes individual instances/objects.
The B (blue) channel is 0.
E.g. [192, 85, 0] is a face.
The file will be saved as image-name_ins_maks.png.
By adding the optional --mask-scale {0-100} (CLI) or /v1/file/1/instance-mask?mask-scale={0-100} (Container API) parameter, mask files will be scaled down by the specified ratio.
Metadata about detected instances/objects are stored in the corresponding image-name.json file.
{
"id": "image-name.jpg",
"detections": [
{
"id": 0,
"parent_image": "image-name.jpg",
"offset": [
1560,
744
],
"bbox": [
1957,
744,
3003,
1855
],
"type": 103,
"score": 0.9993754029273987,
"is_anonymised": true,
"type_label": "face",
"color": null
},
"size": [
3456,
5184
],
"duration": 1.8533296539999355,
"filename": "image-name.jpg",
"folder": null
}Metadata is generated for individual frames and provided as a range of mulitple frames.
Batch/Stream mode:
The information is stored in filename-[startframe]-[endframe].json in the output directory. The maximum number of frames covered by a single file is 500.
[
{
"id": 0,
"detections": [
{
"id": 0,
"parent_image": 0,
"offset": [
3,
4
],
"bbox": [
3,
4,
201,
171
],
"type": 103,
"score": 0.8267934918403625,
"is_anonymised": true,
"type_label": "face",
"color": null
}
],
"size": [
178,
320
],
"duration": 0.35388135999892256
},
{
"id": 1,
"detections": [
{
"id": 0,
"parent_image": 1,
"offset": [
0,
6
],
"bbox": [
0,
20,
137,
175
],
"type": 103,
"score": 0.8945223689079285,
"is_anonymised": true,
"type_label": "face",
"color": null
}
],
"size": [
178,
320
],
"duration": 0.2979645720006374
},
...
]id
The id of the image (file name) or video frame (sequential number)
detections
size
Size of the image or frame in [width, height]
duration
Duration of processing (inference and anonymization) an image or video frame. Does not include IO, e.g. read/write from hard drive.
filename
Name of the file
folder
Name of the folder (relative to root input folder)
detections attributeid
The id of the detection, a sequential number starting from 0.
parent_image
The name of the image or the id of the video frame the detected instance/object was found on
offset
The offset of the detection's bounding box from the upper left corner of the image (x/y coordinates in pixels)
bbox
The coordinates of the detection's bounding box (x1, y1, x2, y2)
score
The detection's confidence score. States how confident the model is about the detection being a specific label (see type_label) between 0.0 and 1.0.
is_anonymized
Specifies whether the detection was anonymized (or detected with method = detect).
type_label
The detection's label assigned by the model. E.g. face, license plates, etc.
type
Numerical representation of the type_label.
color
The detections color (RGB) in the instance segmentation mask.
duration
Processing duration for a video frame (only for videos).
REST API mode: The infromation can be retrieved via the endpoint.
List of detections, see
