Export

Exports a trained model to an optimized inference format.

Exports are written to a fixed location inside the model directory:

<model_dir>/
|-- exports_onnx/
|   `-- <checkpoint_stem>_<onnx_precision>.onnx
`-- exports_trt/
    `-- <checkpoint_stem>_<onnx_precision>/
        `-- trt_metadata.json

The export path is not configurable. litpose predict –runtime onnx or –runtime tensorrt reads from these same locations.

TensorRT builds its engine from an existing ONNX export rather than re-tracing the model – run –runtime onnx first for the same –onnx-precision.

usage: litpose export <model_dir> [OPTIONS]

Positional Arguments

model_dir

path to a model directory

Named Arguments

--runtime

Possible choices: onnx, tensorrt

inference format to export to. Default: onnx.

Default: 'onnx'

--onnx-precision

Possible choices: fp16, fp32

weight precision baked into the exported file. Unlike litpose predict –precision, this changes the exported weights themselves rather than the precision of a forward pass. For –runtime tensorrt, selects which existing ONNX export to build the engine from. Default: fp16.

Default: 'fp16'

--max-batch-size

only used with –runtime tensorrt. Upper bound of the dynamic-shape batch profile the engine is built for; inference at a batch size above this fails. Default: 8.

Default: 8

--opt-batch-size

only used with –runtime tensorrt. Batch size the engine is optimized for; defaults to –max-batch-size. Correctness holds for any batch size in [1, max-batch-size], but performance is best near this value.