Question about running PhysFormer #376
Description
Hello, I am very sorry that I have a new problem. I reprocessed the file according to your instructions, but found the error again today:
Traceback (most recent call last):
File "main.py", line 313, in
train_and_test(config, data_loader_dict)
File "main.py", line 86, in train_and_test
model_trainer.train(data_loader_dict)
File "G:\rPPG-Toolbox-main\neural_methods\trainer\PhysFormerTrainer.py", line 111, in train
rPPG, _, _, _ = self.model(data, gra_sharp)
File "D:\ProgramData\envs\pytorch\lib\site-packages\torch\nn\modules\module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "D:\ProgramData\envs\pytorch\lib\site-packages\torch\nn\parallel\data_parallel.py", line 166, in forward
return self.module(*inputs[0], **kwargs[0])
File "D:\ProgramData\envs\pytorch\lib\site-packages\torch\nn\modules\module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "G:\rPPG-Toolbox-main\neural_methods\model\PhysFormer.py", line 302, in forward
features_last = Trans_features3.transpose(1, 2).view(b, self.dim, t//4, 4, 4) # [B, 64, 40, 4, 4]
RuntimeError: shape '[4, 96, 40, 4, 4]' is invalid for input of size 61440
I changed the value of DYNAMIC_DETECTION_FREQUENCY in test from 32 to 30 because I saw physformer use 30 here in another test. At the same time, I set the size of the picture to 72*72 and the length to 160, and it seems that I have not modified other configurations. My yaml file is configured as:
BASE: ['']
TOOLBOX_MODE: "train_and_test" # "train_and_test" or "only_test"
TRAIN:
BATCH_SIZE: 4
EPOCHS: 30
LR: 1e-4
MODEL_FILE_NAME: UBFC-rPPG_UBFC-rPPG_UBFC-PHYS_physformer
PLOT_LOSSES_AND_LR: True
DATA:
FS: 30
DATASET: UBFC-rPPG
DO_PREPROCESS: false # if first time, should be true
DATA_FORMAT: NCDHW
DATA_PATH: "G:/data_rPPG/UBFCrPPG" # Raw dataset path, need to be updated
CACHED_PATH: "F:/PreprocessingDataset" # Processed dataset save path, need to be updated
EXP_DATA_NAME: ""
BEGIN: 0.0
END: 0.8
PREPROCESS:
DATA_TYPE: ['DiffNormalized']
DATA_AUG: ['None'] # 'None' or 'Motion' is supported, used if the data path points to an augmented dataset or requires augmentation
LABEL_TYPE: DiffNormalized
DO_CHUNK: True
CHUNK_LENGTH: 160
CROP_FACE:
DO_CROP_FACE: True
BACKEND: 'HC' # HC for Haar Cascade, RF for RetinaFace
USE_LARGE_FACE_BOX: True
LARGE_BOX_COEF: 1.5
DETECTION:
DO_DYNAMIC_DETECTION: False
DYNAMIC_DETECTION_FREQUENCY : 30
USE_MEDIAN_FACE_BOX: False # This should be used ONLY if dynamic detection is used
RESIZE:
H: 72
W: 72
VALID:
DATA:
FS: 30
DATASET: UBFC-rPPG
DO_PREPROCESS: false # if first time, should be true
DATA_FORMAT: NCDHW
DATA_PATH: "G:/data_rPPG/UBFCrPPG" # Raw dataset path, need to be updated
CACHED_PATH: "F:/PreprocessingDataset" # Processed dataset save path, need to be updated
EXP_DATA_NAME: ""
BEGIN: 0.8
END: 1.0
PREPROCESS:
DATA_TYPE: ['DiffNormalized']
DATA_AUG: ['None'] # 'None' or 'Motion' is supported, used if the data path points to an augmented dataset or requires augmentation
LABEL_TYPE: DiffNormalized
DO_CHUNK: True
CHUNK_LENGTH: 160
CROP_FACE:
DO_CROP_FACE: True
BACKEND: 'HC' # HC for Haar Cascade, RF for RetinaFace
USE_LARGE_FACE_BOX: True
LARGE_BOX_COEF: 1.5
DETECTION:
DO_DYNAMIC_DETECTION: False
DYNAMIC_DETECTION_FREQUENCY : 30
USE_MEDIAN_FACE_BOX: False # This should be used ONLY if dynamic detection is used
RESIZE:
H: 72
W: 72
TEST:
METRICS: ['MAE', 'RMSE', 'MAPE', 'Pearson', 'SNR', 'BA']
USE_LAST_EPOCH: False # to use provided validation dataset to find the best epoch, should be false
DATA:
FILTERING:
USE_EXCLUSION_LIST: True
EXCLUSION_LIST: [
's3_T1', 's8_T1', 's9_T1', 's26_T1', 's28_T1', 's30_T1', 's31_T1', 's32_T1',
's33_T1', 's40_T1', 's52_T1', 's53_T1', 's54_T1', 's56_T1', 's1_T2', 's4_T2',
's6_T2', 's8_T2', 's9_T2', 's11_T2', 's12_T2', 's13_T2', 's14_T2', 's19_T2',
's21_T2', 's22_T2', 's25_T2', 's26_T2', 's27_T2', 's28_T2', 's31_T2', 's32_T2',
's33_T2', 's35_T2', 's38_T2', 's39_T2', 's41_T2', 's42_T2', 's45_T2', 's47_T2',
's48_T2', 's52_T2', 's53_T2', 's55_T2', 's5_T3', 's8_T3', 's9_T3', 's10_T3',
's13_T3', 's14_T3', 's17_T3', 's22_T3', 's25_T3', 's26_T3', 's28_T3', 's30_T3',
's32_T3', 's33_T3', 's35_T3', 's37_T3', 's40_T3', 's47_T3', 's48_T3', 's49_T3',
's50_T3', 's52_T3', 's53_T3']
SELECT_TASKS: True
TASK_LIST: ['T1', 'T2', 'T3']
FS: 35
DATASET: UBFC-PHYS
DO_PREPROCESS: false # if first time, should be true
DATA_FORMAT: NCDHW
DATA_PATH: "G:/data_rPPG/UBFCPhys" # Raw dataset path, need to be updated
CACHED_PATH: "F:/PreprocessingDataset" # Processed dataset save path, need to be updated
EXP_DATA_NAME: ""
BEGIN: 0.0
END: 1.0
PREPROCESS:
DATA_TYPE: ['DiffNormalized'] #if use physnet, should be DiffNormalized
LABEL_TYPE: DiffNormalized
DO_CHUNK: True
CHUNK_LENGTH: 160
CROP_FACE:
DO_CROP_FACE: True
BACKEND: 'HC' # HC for Haar Cascade, RF for RetinaFace
USE_LARGE_FACE_BOX: True
LARGE_BOX_COEF: 1.5
DETECTION:
DO_DYNAMIC_DETECTION: False
DYNAMIC_DETECTION_FREQUENCY : 30
USE_MEDIAN_FACE_BOX: False # This should be used ONLY if dynamic detection is used
RESIZE:
H: 72
W: 72
DEVICE: cuda:0
NUM_OF_GPU_TRAIN: 1
LOG:
PATH: runs/exp
MODEL:
DROP_RATE: 0.1
NAME: PhysFormer
PHYSFORMER: # Probably need to update these in some meaningful way
PATCH_SIZE: 4
DIM: 96
FF_DIM: 144
NUM_HEADS: 4
NUM_LAYERS: 12
THETA: 0.7
INFERENCE:
BATCH_SIZE: 4
EVALUATION_METHOD: "FFT" # "FFT" or "peak detection"
EVALUATION_WINDOW:
USE_SMALLER_WINDOW: False # Change this if you'd like an evaluation window smaller than the test video length
WINDOW_SIZE: 30 # In seconds
MODEL_PATH: ""
Looking forward to your reply, thank you very much!