TemporalLoss

class lightning_pose.losses.losses.TemporalLoss[source]

Bases: Loss

Penalize temporal differences for each target.

Motion model: x_t = x_(t-1) + e_t, e_t ~ N(0, s)

Attributes Summary

loss_name

Methods Summary

__call__(keypoints_pred[,Β confidences,Β stage])

Compute the temporal loss for a batch of predicted keypoints.

compute_loss(predictions)

Compute per-keypoint L2 temporal differences between consecutive frames.

rectify_epsilon(loss)

Rectify supporting a list of epsilons, one per bodypart.

remove_nans(loss,Β confidences)

Zero out temporal difference losses where either neighboring frame is low-confidence.

Attributes Documentation

loss_name: str = 'temporal'

Methods Documentation

__call__(keypoints_pred: Float[Tensor, 'batch two_x_num_keypoints'], confidences: Float[Tensor, 'batch num_keypoints'] | None = None, stage: Literal['train', 'val', 'test'] | None = None, **kwargs: Any) → tuple[Float[Tensor, ''], list[dict]][source]

Compute the temporal loss for a batch of predicted keypoints.

Parameters:

  • keypoints_pred – predicted (x, y) keypoints of shape (batch, 2*num_keypoints).

  • confidences – per-frame confidence scores; if provided, low-confidence frames are masked out.

  • stage – training stage for logging.

  • **kwargs – ignored extra keyword arguments.

Returns:

Tuple of scalar loss and list of logging dicts.

compute_loss(predictions: Float[Tensor, 'batch two_x_num_keypoints']) → Float[Tensor, 'batch_minus_one num_keypoints'][source]

Compute per-keypoint L2 temporal differences between consecutive frames.

Parameters:

predictions – predicted (x, y) keypoints of shape (batch, 2*num_keypoints).

Returns:

L2 norm of frame-to-frame differences, shape (batch-1, num_keypoints).

rectify_epsilon(loss: Float[Tensor, 'batch_minus_one num_keypoints']) → Float[Tensor, 'batch_minus_one num_keypoints'][source]

Rectify supporting a list of epsilons, one per bodypart. Not implemented in Loss class, because shapes of broadcasting may vary

remove_nans(loss: Float[Tensor, 'batch_minus_one num_keypoints'], confidences: Float[Tensor, 'batch num_keypoints']) → Float[Tensor, 'batch_minus_one num_keypoints'][source]

Zero out temporal difference losses where either neighboring frame is low-confidence.

Parameters:

  • loss – temporal difference losses of shape (batch-1, num_keypoints).

  • confidences – per-frame confidence scores of shape (batch, num_keypoints).

Returns:

Loss tensor with entries zeroed where confidence falls below self.prob_threshold.

__init__(data_module: BaseDataModule | UnlabeledDataModule | None = None, epsilon: float | list[float] = 0.0, prob_threshold: float = 0.0, log_weight: float = 0.0, **kwargs: Any) → None[source]

Initialize TemporalLoss.

Parameters:

  • data_module – data module providing access to datasets; passed to the parent class.

  • epsilon – loss values below this threshold are zeroed out. May be a scalar or a list with one value per keypoint.

  • prob_threshold – predictions whose confidence is below this value are excluded from the loss computation.

  • log_weight – final weight in front of the loss term in the objective function is computed as 1.0 / (2.0 * exp(log_weight)).

__new__(**kwargs)