Batch and Stream mode
Celantur Container batch and stream mode allow you to anonymize images and videos stored on the local file system.
Last updated
Celantur Container batch and stream mode allow you to anonymize images and videos stored on the local file system.
Last updated
The following folders are necessary:
$ROOT
├── input/
├── licensing/
├── log/
└── output/
input
: Where the original files are stored (including subfolders)
output
: Folder for the anonymized images/videos. If a file exists with the same as in input folder, then processing of the file is skipped.
log
: Folder containing the logs.
licensing
: Folder containing the license key. Copy the license file license
into this folder as licensing/license
.
Starting the Container in Batch mode triggers a processing of all files in the input
folder. After files have been processed, the Container is shut down.
./celantur.sh -a face -a license-plate -f whole
Stream mode keeps the Container continuously checking the input
folder for new files that have to be processed. The delay between checks can be specified in seconds. The Container needs to be shut down manually.
./celantur.sh --stream -a face -a license-plate -f whole
By default, Celantur Container processes images. Video processing has to be specified by adding the --video
parameter.
Images: ./celantur.sh -a face -a license-plate -f whole
Videos: ./celantur.sh --video -a face -a license-plate -f whole
You can edit ./celantur.sh <parameters>
to modify the behavior of Celantur Container. Experienced Linux user can modify the script celantur.sh
--anonymise {face, license-plate, person, vehicle}
-a {face, license-plate, person, vehicle}
REQUIRED
Specify objects to be anonymized.
--format FORMAT
-f FORMAT
REQUIRED
Specify image format, to control how an image should be processed. Default: whole
--help
-h
Prints help message with all parameters and exits.
--bbox
Anonymizes objects with bounding boxes
--debug
Enables debug mode. Bounding boxes of detected objects are displayed on images.
--score
Prints the certainty scores of detected objects on images, if debug mode is enabled.
--cpu-mode
--method {blur, pixelate, blacken, detect}
Choose anonymization method (default: blur)
--metrics <filename.json>
File name for metrics (default: metrics.json)
--quality [1..100]
Quality of output (JPEG) image (default: 90)
--overwrite
Overwrite existing files in output folder.
--stream [seconds]
Streaming mode (Wait time in seconds).
--server <host:port>
--server-array <host:port>
--video
Anonymize videos.
--model segmentatio-omnium
Specifies which models shall be used. `segmentatio-omnium` model is significantly faster than the default model (without --model ...
paramter).
--no-object-tracking
--keep-bit-rate [bool]
Specifies whether the bitrate of the anonymized video should remain very close to the one of the original video.
Enabling keep-bit-rate
will ensure a nearly identically file size of the processed video, at the cost of 10% to 20% higher processing time.
Default: false
--person-threshold [0..1]
Threshold for whole person detection.
--vehicle-threshold [0..1]
Threshold for vehicle detection.
--face-threshold [0..1]
Threshold for face detection.
--license-plate-threshold [0..1]
Threshold for license plate detection.
--save-mask {all, instance, binary}
--mask-scale {0..100}
--log-level {CRITICAL,ERROR,WARNING,INFO, DETAIL,DEBUG}
Specifies which levels of logs will be printed out.
For more arguments, check ./celantur.sh --help
.
By specifying image format settings, Celantur Container can achieve improved anonymization results and apply anonymization only to a specific region of an image.
--format '{"number":[nx,ny],"overlap":[ox,oy],"section":[x1,y1,x2,y2]}'
In certain cases, its beneficial to process high-resolution imagery in tiles, instead of the whole image. Small, distant objects are more likely to be detected.
--format '{"number":[nx,ny],"overlap":[ox,oy]}'
nx
Number of horizontal tiles
ny
Number of vertical tiles
ox
Horizontal overlap between tiles in pixels
oy
Vertical overlap between tiles in pixels
Apply anonymization only to a specified rectangular section of the image. This can result in faster processing time.
--format '{"number":[3,2],"overlap":[ox,oy],"section":[x1,y1,x2,y2]}'
x1, y1
Top left coordinate of the section
x2, y2
Bottom right coordinate of the section
Alternatively, you can use the option --format
to choose the resolution of the input images, eg. --format pano:8000
for an image resolution of 8000x4000.
Use --format whole
for all formats not listed below, or if the input images have different resolutions.
pano:4096
4096x2048
pano:5400
5400x2700
pano:5640
5640x2816
pano:7060
7060x3530
pano:7680
7680x3840
pano:8000
8000x4000
pano:7680
7680x3840
pano:8000
8000x4000
pano:8192
8192x4096
pano:11000
11000x5500
whole
default, valid for all resolutions
You can use symbolic links (see ln for reference) for the input and output folders, e.g. in the folder with the celantur.sh
script:
# set "input" as a symbolic link to an external drive
ln -s /path/to/external/drive/input-images input
# set "output" as a symbolic link to an external drive
ln -s /path/to/external/drive/output-images output
Yes, this is possible and can lead to a higher throughput when you process images, by having several container instances working in parallel. Please make sure that your system has enough resources available.
Run the following command as many times as how many container instances you want to start:
celantur.sh --detach
The --detach
flag is supported starting from version 22.06.3.
Note that this will start container in the detached mode and you will not be able to observe the text output. To inspect which containers are currently running use docker ps
, which will also print the container IDs. To inspect log outputs of a particular container, use docker logs <container_id>
. Finally, to stop the container use docker kill <container_id> && docker rm <container_id>
Problem: [Errno 13] Permission denied: '/path/to/file'
In Docker, if you mount a directory to Docker that does not exist, Docker creates the folder as root. Inside Docker container, the user (with UID 1000) cannot write in the directory.
Solution: Create the folders output
and log
before you mount them and ensure that the file owner has UID 1000.
mkdir log output
sudo chown 1000:1000 log output
Alternatively, give everyone write permission to log
and output
:
mkdir log output
chmod 777 log output
Problem: [Errno 30] Read-only file system: '/path/to/file'
Solution: The folder is mounted in Docker as read-only. Remove the appendix :ro
from the -v
parameter.
Yes, metadata (e.g. EXIF, IPTC and XMP information, ICC color profile) is retained when creating the anonymized version of an image.
Celantur Container attempts to copy over all stream/track and container metadata to the anonymized video.
The following image formats are supported:
.jpg
/ .jpeg
.png
.tif
/ .tiff
.bmp
.jfif
Codecs are essential for encoding and decoding audio, video, or other data streams.
Celantur Container provides support (i.e. continuous testing) for there three codecs:
h265
h264
mpeg
The following list of codecs (excluding the aforementioned), are not continuously tested:
amv AMV Video
apng APNG (Animated Portable Network Graphics) image
asv1 ASUS V1
asv2 ASUS V2
avrp Avid 1:1 10-bit RGB Packer
avui Avid Meridien Uncompressed
ayuv Uncompressed packed MS 4:4:4:4
bitpacked Bitpacked
cfhd GoPro CineForm HD
cinepak Cinepak
cljr Cirrus Logic AccuPak
dirac Dirac (encoders: vc2 )
dnxhd VC3/DNxHD
dvvideo DV (Digital Video)
ffv1 FFmpeg video codec #1
ffvhuff Huffyuv FFmpeg variant
fits FITS (Flexible Image Transport System)
flashsv Flash Screen Video v1
flashsv2 Flash Screen Video v2
flv1 FLV / Sorenson Spark / Sorenson H.263 (Flash Video) (decoders: flv ) (encoders: flv )
h261 H.261
h263 H.263 / H.263-1996, H.263+ / H.263-1998 / H.263 version 2 (decoders: h263 h263_v4l2m2m ) (encoders: h263 h263_v4l2m2m )
h263p H.263+ / H.263-1998 / H.263 version 2
h264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (decoders: h264 h264_v4l2m2m h264_cuvid ) (encoders: libx264 libx264rgb h264_nvenc h264_v4l2m2m h264_vaapi )
hdr HDR (Radiance RGBE format) image
hevc H.265 / HEVC (High Efficiency Video Coding) (decoders: hevc hevc_v4l2m2m hevc_cuvid ) (encoders: libx265 hevc_nvenc hevc_v4l2m2m hevc_vaapi )
huffyuv HuffYUV
magicyuv MagicYUV video
mjpeg Motion JPEG (decoders: mjpeg mjpeg_cuvid ) (encoders: mjpeg mjpeg_vaapi )
mpeg1video MPEG-1 video (decoders: mpeg1video mpeg1_v4l2m2m mpeg1_cuvid )
mpeg2video MPEG-2 video (decoders: mpeg2video mpegvideo mpeg2_v4l2m2m mpeg2_cuvid ) (encoders: mpeg2video mpeg2_vaapi )
mpeg4 MPEG-4 part 2 (decoders: mpeg4 mpeg4_v4l2m2m mpeg4_cuvid ) (encoders: mpeg4 mpeg4_v4l2m2m )
msmpeg4v2 MPEG-4 part 2 Microsoft variant version 2
msmpeg4v3 MPEG-4 part 2 Microsoft variant version 3 (decoders: msmpeg4 ) (encoders: msmpeg4 )
msvideo1 Microsoft Video 1
prores Apple ProRes (iCodec Pro) (encoders: prores prores_aw prores_ks )
qtrle QuickTime Animation (RLE) video
r10k AJA Kona 10-bit RGB Codec
r210 Uncompressed RGB 10-bit
rawvideo raw video
roq id RoQ video (decoders: roqvideo ) (encoders: roqvideo )
rpza QuickTime video (RPZA)
rv10 RealVideo 1.0
rv20 RealVideo 2.0
smc QuickTime Graphics (SMC)
snow Snow
speedhq NewTek SpeedHQ
svq1 Sorenson Vector Quantizer 1 / Sorenson Video 1 / SVQ1
utvideo Ut Video
v210 Uncompressed 4:2:2 10-bit
v308 Uncompressed packed 4:4:4
v408 Uncompressed packed QT 4:4:4:4
v410 Uncompressed 4:4:4 10-bit
vnull Null video codec
vp8 On2 VP8 (decoders: vp8 vp8_v4l2m2m vp8_cuvid ) (encoders: vp8_v4l2m2m vp8_vaapi )
vp9 Google VP9 (decoders: vp9 vp9_v4l2m2m vp9_cuvid ) (encoders: vp9_vaapi )
wmv1 Windows Media Video 7
wmv2 Windows Media Video 8
y41p Uncompressed YUV 4:1:1 12-bit
yuv4 Uncompressed packed 4:2:0
zlib LCL (LossLess Codec Library) ZLIB
zmbv Zip Motion Blocks Video
Please see Anonymization of ROS2 .mcap files
Yes, by starting multiple instances/processes of Celantur Container.
To specify the GPU which Celantur Container utilizes, you need to adapt the celantur.sh
script. Instead of:
[[ ${PARAMETERS} != *"--cpu-mode"* ]] && GPU="--gpus
all
"
change it to
[[ ${PARAMETERS} != *"--cpu-mode"* ]] && GPU="--gpus
device=<GPU-UUID>
"
You find the GPU UUID with nvidia-smi -L
. Result is GPU-xxxx-xxx-xxx-xxxx-xxxxxxx
Yes, Celantur Container can be used on multi GPU systems. Multiple GPU UUIDs can be specified in a license file.
Celantur applies anonymization on the raw pixel data. JPEG compression is applied when saving an anonymized image as a JPEG, resulting in a different file size. The resulting JPEG file size can be influenced by changing the quality
parameter (default: 90).
for improved anonymization results and of the image can be specified here as well.
See for reference or check usage with ./celantur.sh -h
.
Disables GPU utilization. See
Disables object tracking in videos, whic is enabled by default. See
Generate binary and instance segmentation mask files per processed images. See .
Scales binary and instance segmentation mask files down by the specified ratio. See .