Official repo for PhysioLatent: Editing physiological signals in videos using latent representations

Note: The codebase was adapted from rPPG-toolbox

Please prepare the environment according to the requirements.txt file.

We adopt PURE as our training set, MMPD, UBFC-rPPG, and PURE as our test set.

Please refer to rPPG toolbox for the data preparation process.

Our model is build upon the latent space of 3D Causal VAE from CogVideoX. Hence, we need to download their pretrained weights for the 3D Causal VAE from their official Hugging Face repo.

After you have downloaded both the diffusion_pytorch_model.safetensors and the config.json, place them in ./vae.

You can find the config files under ./configs/train_configs To train from scratch, please use PURE_UBFC_rPPG_AnonyPulse.yaml.

Please set the TOOLBOX_MODE to train_and_test, and change the VAE: MODEL_PATH: to the correct path on your device.

Also, please change the DATA_PATH and CACHED_DATA_PATH in TRAIN: and VALID: section to their correct versions. In our experiment, we use PURE as the training set, and UBFC_rPPG as the validation set.

Please refer to our manuscript for the hyper parameters in the config file.

You can find the config files under ./configs/train_configs

To test with pretrained models, please use XXXX_AnonyPulse.yaml, in which XXXX stands for the corresponding test set, i.e. MMPD, PURE and UBFC-rPPG.

Please set the TOOLBOX_MODE to only_test, and change the VAE: MODEL_PATH: and INFERENCE: MODEL_PATH: to the correct path on your device.

Also, please change the DATA_PATH and CACHED_DATA_PATH in TEST: section to their correct versions.