Inference

The command litpose predict is used to run model inference on new data.

Inference on new videos

The model_dir argument is the path to the model outputted by litpose train.

To predict on one or more video files:

litpose predict <model_dir> <video_file1> <video_file2> ...

To predict on a folder of video files:

litpose predict <model_dir> <video_files_dir>

The litpose predict command saves frame-by-frame predictions and confidences as a CSV file, unsupervised losses in CSV file per loss type. By default it also generates videos annotated with predictions, a feature which can be disabled using the --skip_viz flag.

For the full list of options, see the CLI reference: Predict.

Note

Videos must be mp4 files that use the h.264 codec; see more information in the FAQs.

Inference on new images

Lightning pose also supports inference on images, as well as computing pixel error against new labeled images. This is useful for evaluating a model on out-of-distribution data to see how well the model generalizes.

Currently it’s required to create a CSV file similar to the one used for training labeled frames. Once you have a CSV file, run:

litpose predict <model_dir> <csv_file>

Output location

Video and image predictions are saved within the model directory. For a detailed reference on the model directory structure, see Model Directory Structure.

Briefly, video predictions are saved to:

<model_dir>/
└── video_preds/
    β”œβ”€β”€ <video_filename>.csv              (predictions)
    β”œβ”€β”€ <video_filename>_<metric>.csv     (losses)
    └── labeled_videos/
        └── <video_filename>_labeled.mp4

Image predictions are saved to:

<model_dir>/
└── image_preds/
    └── <image_dirname | csv_filename | timestamp>/
        β”œβ”€β”€ predictions.csv
        β”œβ”€β”€ predictions_<metric>.csv      (losses)
        └── <image_filename>_labeled.png

Inference on sample dataset

The lightning pose repo includes a sample dataset (see Training on sample dataset). The sample video file is located in the git repo at data/mirror-mouse-example/videos. Thus, to run inference on a model trained on the sample dataset, run from the lightning-pose directory (make sure you have activated your conda environment):

litpose predict <model_dir> data/mirror-mouse-example/videos