If you can see a lag between the frame coming off the camera and the box drawn on screen during the final pre-flight test, the problem is usually not the model's accuracy — it is how the model is being run. In our case YOLO11m runs at 15-20 FPS on a Jetson Orin NX 16GB doing full-frame inference at 640 pixels. That is enough for target detection from 20 m altitude, but the same pipeline also has to decode the 1080p RTSP stream from the SIYI A8 mini, let FastMosaic accumulate frames, and service pymavlink telemetry. Every millisecond we waste on the GPU gets taken out of something else.

A .pt file trained in PyTorch is really just a recipe: every layer runs separately, under the supervision of the Python interpreter, on general-purpose kernels. On a desktop RTX card you never notice what that costs. On a module living inside a 10-25 W power envelope, it comes straight back at you as FPS.

This post covers the most direct way to gain speed without touching the model at all: .engine files compiled with NVIDIA TensorRT. And in particular the rule that gets broken most often here — an engine file is not a portable model format.

What TensorRT actually does

TensorRT is an inference compiler. It takes a trained network graph and produces an execution plan tailored to the target GPU. The core techniques NVIDIA lists are:

• Layer and tensor fusion: consecutive operations such as Conv + BatchNorm + activation get merged into a single GPU kernel. Every fusion means one less kernel launch and one less round trip to memory.

• Precision calibration: holding weights and intermediate results in narrower numeric formats such as FP16 or INT8. Tensor Cores are markedly faster in those formats.

• Kernel auto-tuning: for each layer, the fastest candidate CUDA kernel is chosen based on real measurements taken on that specific card.

• Dynamic tensor memory: memory for intermediate tensors is reused across layers whose lifetimes do not overlap.

The third item is where the critical rule later in this post comes from. TensorRT does not guess; it measures on the target GPU and then decides.

FP32, FP16, INT8: trading accuracy for speed

The Orin NX 16GB numbers in the Ultralytics Jetson guide show that trade-off clearly. The table below is for YOLO26n, the reference model used in the documentation, measured with a 640-pixel input — not our YOLO11m, but the ratios between formats read the same way (as of 2026):

There are three ratios worth reading here. Even TensorRT FP32 is roughly 2x faster than PyTorch — so the very first step pays off without lowering precision at all. Moving to FP16 adds about another 1.7x on top of that, and in this table it does no measurable damage to mAP. INT8, meanwhile, buys only ~1.2x over FP16 while pushing mAP50-95 from 0.479 down to 0.449 — a loss of roughly 3 points.

We went with FP16. In a detection pipeline trying to pick up small targets from 20 m, 3 points of mAP is not a concession worth making for 0.6 ms.

The .pt → ONNX → .engine pipeline

The Ultralytics exporter uses ONNX as an intermediate graph, which is what the opset argument applies to. A single command handles it:

# Run this ON the Jetson itself
yolo export model=best.pt format=engine imgsz=640 quantize=16 batch=1 workspace=4

# Run this ON the Jetson itself
yolo export model=best.pt format=engine imgsz=640 quantize=16 batch=1 workspace=4

The same thing from Python:

from ultralytics import YOLO

model = YOLO("best.pt")
model.export(format="engine", imgsz=640, quantize=16, batch=1)

# The inference side does not change at all
trt = YOLO("best.engine")
results = trt("frame.jpg", conf=0.25)

from ultralytics import YOLO

model = YOLO("best.pt")
model.export(format="engine", imgsz=640, quantize=16, batch=1)

# The inference side does not change at all
trt = YOLO("best.engine")
results = trt("frame.jpg", conf=0.25)

One important API change: the old half=True and int8=True flags have moved to quantize=16 and quantize=8. The old ones are still accepted, but they emit a deprecation warning (as of 2026). The arguments you will reach for most often:

The critical rule: the engine has to be built on the target device

The Ultralytics documentation says it outright: TensorRT profiles and tunes the engine on the GPU it is built on, so .engine should not be treated as a portable model format. The reasoning on NVIDIA's side is even sharper — by default, a serialized plan file is locked to three things:

1. The TensorRT version it was built with. That is recorded inside the plan and checked at load time.

2. The GPU's compute capability. Major and minor versions are written into the plan and verified against the card when it loads.

3. The platform. The operating system and CPU architecture are fixed too; a plan built on an x86 desktop will not open on an ARM-based Jetson.

NVIDIA does offer flags to relax these constraints: VERSION_COMPATIBLE, kAMPERE_PLUS (all architectures from Ampere onward) and kSAME_COMPUTE_CAPABILITY. All of them can cost performance compared with a device-specific engine. The practical takeaway does not change: build on the card that is going to fly, keep .engine files out of version control, and don't try to produce them in CI and ship them to the field.

Fixed or dynamic input size

By default the engine is built with a fixed input: imgsz and batch are baked into the plan. dynamic=True opens a variable-size profile; for INT8 exports it is already enabled by default.

Keep in mind that the batch argument specifies a maximum: smaller batches are accepted, larger ones throw an error.

imgsz has to match the inference pipeline

This is one we learned the hard way. We train the model at imgsz=1280, but in flight we process the full frame at 640. If you build the engine at 1280 because that's the training value, then when the inference pipeline sends 640 you either get a shape mismatch error or a silent rescale — and the speed you measured is not the speed you actually get.

The rule is simple: the engine's imgsz must be the size of the tensor you actually feed it in flight. Don't confuse it with the training resolution. If you also tile at a different resolution, either build a separate engine per size or use dynamic=True and pick the profile range deliberately.

Calibration data, if you're moving to INT8

INT8 works through post-training quantization, and it needs real data to learn the activation ranges. NVIDIA recommends at least 500 representative calibration images. On the Ultralytics side you do this by pointing the data argument at a dataset configuration; fraction lets you take a subset.

yolo export model=best.pt format=engine imgsz=640 quantize=8 \
     data=suas_aerial.yaml fraction=0.5 batch=1

yolo export model=best.pt format=engine imgsz=640 quantize=8 \
     data=suas_aerial.yaml fraction=0.5 batch=1

The word "representative" is doing the heavy lifting here. If your calibration set is made of close-up photos scraped off the web, the activation ranges of small targets shot from 20 m never show up, and field performance comes out worse than your validation metric suggests. Calibration produces a .cache file; delete it whenever the dataset or the batch size changes noticeably, otherwise the old ranges get reused.

Common pitfalls

• Building on the desktop and copying to the Jetson. The most common mistake. You get either a load failure or an architecture mismatch — neither is a surprise you want on the morning of a flight.

• Not rebuilding the engine after a JetPack upgrade. JetPack brings its own TensorRT and CUDA versions with it; an old plan may not open under the new runtime.

• Confusing the training imgsz with the export imgsz. You end up optimizing without knowing which resolution the FPS you measured belongs to.

• Benchmarking without setting the power mode. Numbers taken without running sudo nvpmodel -m 0 and sudo jetson_clocks don't show the board's real ceiling. Watching temperature and clock frequency with jtop helps too.

• Assuming DLA is free speed. The Orin NX 16GB has 2 DLA cores running at 614 MHz, but DLA primarily targets throughput and energy efficiency; it is under no obligation to lower latency relative to the GPU. On top of that, DLA is not supported in TensorRT 11.0 — DLA export requires TensorRT 10.x (as of 2026).

• Switching to INT8 without a validation run. Always re-measure the quantized engine on your own validation set. The mAP50 of 0.950 on our aerial-only validation set belongs to the FP32 model; assuming the same number after INT8 would be wrong.

Practical summary

1. Always build the engine on the Jetson that is going to fly. Keep the .engine file out of the repo and put the build step in your setup script.

2. Make FP16 (quantize=16) your default. In the reference benchmarks it delivers ~3x the speed of PyTorch with no measurable accuracy loss.

3. Match the imgsz and batch values in the export exactly to the tensor your inference pipeline actually sends — not to the training resolution.

4. Only consider INT8 if your frame budget is genuinely tight; calibrate with at least 500 representative images, and decide by measuring the mAP loss on your validation set.

5. Take every speed measurement after nvpmodel -m 0 + jetson_clocks, once the board has warmed up. Whenever the JetPack or Ultralytics version changes, rebuild the engine and repeat the measurement.