Documenting speed-gains of TensorRT models

[geroldmeisinger]

Documenting speed gains of different models, batch sizes and resource usage across different systems:

\```
Model: 
Sampler+Scheduler: 
Speed-gain:
Batch-size:
VRAM conversion:
VRAM inference:
OS:
GPU:
CUDA version:
Driver version:
\```

on Linux:
use watch nvidia-smi to get version and resource usage.
cat /proc/version or cat /etc/*-release to get your distribution.

[geroldmeisinger]

Model: sd3_medium.safetensors
Sampler+Scheduler: dpmpp_2m sgm_uniform
Speed Gain: +50-70%
Batch-size: 1, 2, 8
VRAM conversion: ~16GB
VRAM inference: ~16GB
OS: Debian Linux 12 (bookworm)
GPU: NVIDIA GeForce RTX 4060 Ti
CUDA version: 12.3
Driver version: 545.23.08

• SD3 converted with [Request] Example workflows for SD3 models. #28 (comment)
• I was not able to build a batch 4 version
• I get a lot of black images, with batch=8 I wasn't able to produce any image (see all SD3 engines generate black images randomly (almost all of the time) #25)

[DaveScream]

RTX 3060 12GB
Windows 10 x64
ComfyUI
Cuda Toolkit 12.5
Python 3.11

sampler: euler
scheduler: normal

sd1.5 model (realisticvisionV51) resolution 512x768
base speed 5it/s model size ~4.1 gb
tensorrt static speed 8.2it/s model size ~1.7gb 64% speed increase
tensorrt dynamic speed 7.9-8it/s model size ~1.7 gb 60% speed increase

sdxl model 768x1024
base speed 1.6it/s model size 6.5gb
tensorrt static 3it/s model size 4.8gb