> For the complete documentation index, see [llms.txt](https://doc.celantur.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://doc.celantur.com/sdk/benchmarks.md).

# Benchmarks

## RTX 4090 24 GB, AMD Ryzen 9 7900X

Hardware specification:

* AMD Ryzen 9 7900X 12-Core Processor
* G.SKILL Flare X5 64 GB RAM (2x32)
* MSI MAG B760 TOMAHAWK WIFI DDR5 Mainboard, Intel
* Asus GeForce RTX 4090 24 GB
* 2 TB Samsung SSD 980 PRO, PCIe 4.0 NVMe M.2

### Images (GPU, TensorRT)

* Measured with Celantur SDK v1.5.1.
* Processing time of 100 images using GPU.
* One SDK process running (higher throuput by parallelization possible).
* Includes software ramp-up time (end-to-end).

{% tabs %}
{% tab title="TensorRT (1280)" %}

<table><thead><tr><th width="178">Image Resolution (Tiling)</th><th width="127">E2E Throughput (imgs/s)</th><th width="128">Processing Time (100 images)</th><th>VRAM</th><th>RAM</th></tr></thead><tbody><tr><td>1 MP (1 Tile)</td><td>36</td><td>2.8 s</td><td>682 MB</td><td>490 MB</td></tr><tr><td>2 MP (1 Tile)</td><td>34</td><td>3 s</td><td>682 MB</td><td>490 MB</td></tr><tr><td>8 MP (2 Tiles)</td><td>11</td><td>9.1 s</td><td>682 MB</td><td>490 MB</td></tr><tr><td>72 MP (4 Tiles)</td><td>8.8</td><td>11.3 s</td><td>682 MB</td><td>655 MB</td></tr></tbody></table>
{% endtab %}

{% tab title="TensorRT (640)" %}

<table><thead><tr><th width="173">Image Resolution (Tiling)</th><th width="130">E2E Throughput (imgs/s)</th><th width="124">Processing Time (100 images)</th><th>VRAM</th><th>RAM</th></tr></thead><tbody><tr><td>1 MP (1 Tile)</td><td>121</td><td>0.83 s</td><td>554 MB</td><td>484 MB</td></tr><tr><td>2 MP (1 Tile)</td><td>101</td><td>0.99 s</td><td>554 MB</td><td>484 MB</td></tr><tr><td>8 MP (2 Tiles)</td><td>17</td><td>5.88 s</td><td>554 MB</td><td>484 MB</td></tr><tr><td>72 MP (4 Tiles)</td><td>10</td><td>10 s</td><td>554 MB</td><td>662 MB</td></tr></tbody></table>
{% endtab %}
{% endtabs %}

### Images (CPU Mode)

* Measured with Celantur Edge SDK v1.5.1.
* Processing time of 100 images using CPU (OpenVINO).
* One SDK process running (higher throuput by parallelization possible).
* Includes software ramp-up time (end-to-end).

{% tabs %}
{% tab title="OpenVINO (1280)" %}

<table><thead><tr><th width="173">Image Resolution (Tiling)</th><th width="130">E2E Throughput (imgs/s)</th><th width="124">Processing Time (100 images)</th><th>CPU Util.</th><th>RAM</th></tr></thead><tbody><tr><td>1 MP (1 Tile)</td><td>7</td><td>14.29 s</td><td>86%</td><td>670 MB</td></tr><tr><td>2 MP (1 Tile)</td><td>7</td><td>14.29 s</td><td>86%</td><td>670 MB</td></tr><tr><td>8 MP (2 Tiles)</td><td>3</td><td>33.33 s</td><td>75%</td><td>670 MB</td></tr><tr><td>72 MP (4 Tiles)</td><td>1.65</td><td>60.61 s</td><td>57%</td><td>878 MB</td></tr></tbody></table>
{% endtab %}

{% tab title="OpenVINO (640)" %}

<table><thead><tr><th width="173">Image Resolution (Tiling)</th><th width="130">E2E Throughput (imgs/s)</th><th width="124">Processing Time (100 images)</th><th>CPU Util.</th><th>RAM</th></tr></thead><tbody><tr><td>1 MP (1 Tile)</td><td>27</td><td>3.7 s</td><td>81%</td><td>655 MB</td></tr><tr><td>2 MP (1 Tile)</td><td>24</td><td>4.17 s</td><td>68%</td><td>655 MB</td></tr><tr><td>8 MP (2 Tiles)</td><td>8</td><td>12.5 s</td><td>56%</td><td>655 MB</td></tr><tr><td>72 MP (4 Tiles)</td><td>5</td><td>20 s</td><td>31%</td><td>673 MB</td></tr></tbody></table>
{% endtab %}

{% tab title="ONNX (1280)" %}

<table><thead><tr><th width="173">Image Resolution (Tiling)</th><th width="130">E2E Throughput (imgs/s)</th><th width="124">Processing Time (100 images)</th><th>CPU Util.</th><th>RAM</th></tr></thead><tbody><tr><td>1 MP (1 Tile)</td><td>1.6</td><td>62 s</td><td>96%</td><td>1,4 GB</td></tr><tr><td>2 MP (1 Tile)</td><td>1.5</td><td>66 s</td><td>95%</td><td>1,3 GB</td></tr><tr><td>8 MP (2 Tiles)</td><td>0.75</td><td>133 s</td><td>92%</td><td>1,5 GB</td></tr><tr><td>72 MP (4 Tiles)</td><td>0.4</td><td>250 s</td><td>84%</td><td>1,7 GB</td></tr></tbody></table>
{% endtab %}
{% endtabs %}

## Nvidia Jetson AGX Orin Developer Kit

### Images

Measured with Celantur Edge SDK v1.0.0.

Processing time of 100 images on Nvidia Jetson AGX Orin Developer Kit. Includes software ramp-up time (end-to-end).

<table><thead><tr><th width="217">Image Resolution / Tiling</th><th width="220">Processing Time (100 images)</th><th>Memory Utilization</th></tr></thead><tbody><tr><td>1 MP (1 Tile)</td><td>3.3 s</td><td>686 MB VRAM, &#x3C; 500 MB RAM</td></tr><tr><td>2 MP (1 Tile)</td><td>3.3 s</td><td>686 MB VRAM, &#x3C; 500 MB RAM</td></tr><tr><td>8 MP (2 Tiles)</td><td>7.6 s</td><td>686 MB VRAM, &#x3C; 500 MB RAM</td></tr></tbody></table>