| | distributed init (rank 1): env://, gpu 1 | |
| | distributed init (rank 0): env://, gpu 0 | |
| [00:11:45.613004] job dir: /mnt/localDisk2/wgj/FSFM-3C/codespace/fsfm-3c/finuetune/cross_dataset_DfD | |
| [00:11:45.613201] Namespace(aa='rand-m9-mstd0.5-inc1', | |
| accum_iter=1, | |
| apply_simple_augment=True, | |
| batch_size=32, | |
| blr=0.00025, | |
| clip_grad=None, | |
| color_jitter=None, | |
| cutmix=1.0, | |
| cutmix_minmax=None, | |
| data_path=['../../../datasets/finetune_datasets/deepfakes_detection/FaceForensics/32_frames/DS_FF++_all_cls/c23'], | |
| dataset_abs_path=None, | |
| delimiter_in_spilt=' ', | |
| device='cuda', | |
| dist_backend='nccl', | |
| dist_eval=True, | |
| dist_on_itp=False, | |
| dist_url='env://', | |
| distributed=True, | |
| drop_path=0.1, | |
| epochs=10, | |
| eval=False, | |
| finetune='../../pretrain/checkpoint/pretrained_models/VF2_ViT-B/checkpoint-400.pth', | |
| global_pool=True, | |
| gpu=0, | |
| input_size=224, | |
| layer_decay=0.65, | |
| local_rank=0, | |
| log_dir='./checkpoint/finetuned_models/FF++_c23_32frames', | |
| lr=None, | |
| min_lr=1e-06, | |
| mixup=0.8, | |
| mixup_mode='batch', | |
| mixup_prob=1.0, | |
| mixup_switch_prob=0.5, | |
| model='vit_base_patch16', | |
| nb_classes=2, | |
| normalize_from_IMN=False, | |
| num_workers=10, | |
| output_dir='./checkpoint/finetuned_models/FF++_c23_32frames', | |
| pin_mem=True, | |
| rank=0, | |
| recount=1, | |
| remode='pixel', | |
| reprob=0.25, | |
| resplit=False, | |
| resume='', | |
| seed=0, | |
| smoothing=0.1, | |
| start_epoch=0, | |
| train_split=None, | |
| val_split=None, | |
| warmup_epochs=5, | |
| weight_decay=0.05, | |
| world_size=2) | |
| [00:11:46.254157] Dataset ImageFolder | |
| Number of datapoints: 184185 | |
| Root location: ../../../datasets/finetune_datasets/deepfakes_detection/FaceForensics/32_frames/DS_FF++_all_cls/c23/train | |
| StandardTransform | |
| Transform: Compose( | |
| RandomResizedCropAndInterpolation(size=(224, 224), scale=(0.08, 1.0), ratio=(0.75, 1.3333), interpolation=PIL.Image.BILINEAR) | |
| RandomHorizontalFlip(p=0.5) | |
| <timm.data.auto_augment.RandAugment object at 0x7fe87b1f0b50> | |
| ToTensor() | |
| Normalize(mean=tensor([0.5482, 0.4234, 0.3655]), std=tensor([0.2789, 0.2439, 0.2349])) | |
| <timm.data.random_erasing.RandomErasing object at 0x7fe87b1f0f10> | |
| ) | |
| [00:11:46.396924] Dataset ImageFolder | |
| Number of datapoints: 35796 | |
| Root location: ../../../datasets/finetune_datasets/deepfakes_detection/FaceForensics/32_frames/DS_FF++_all_cls/c23/val | |
| StandardTransform | |
| Transform: Compose( | |
| Resize(size=256, interpolation=bicubic, max_size=None, antialias=None) | |
| CenterCrop(size=(224, 224)) | |
| ToTensor() | |
| Normalize(mean=[0.5482207536697388, 0.42340534925460815, 0.3654651641845703], std=[0.2789176106452942, 0.2438540756702423, 0.23493893444538116]) | |
| ) | |
| [00:11:46.397062] len(dataset_train):184185 | |
| [00:11:46.397075] len(dataset_val):35796 | |
| [00:11:46.397128] Sampler_train = <torch.utils.data.distributed.DistributedSampler object at 0x7fe87b2490d0> | |
| [00:11:46.397188] [INFO]log dir: %./checkpoint/finetuned_models/FF++_c23_32frames | |
| [00:11:46.399425] Mixup is activated! | |
| [00:11:48.577181] Load pre-trained checkpoint from: ../../pretrain/checkpoint/pretrained_models/VF2_ViT-B/checkpoint-400.pth | |
| [00:11:48.625108] _IncompatibleKeys(missing_keys=['head.weight', 'head.bias', 'fc_norm.weight', 'fc_norm.bias'], unexpected_keys=['mask_token', 'rep_decoder_pos_embed', 'decoder_pos_embed', 'norm.weight', 'norm.bias', 'projector.projection_head.0.weight', 'projector.projection_head.0.bias', 'projector.projection_head.1.weight', 'projector.projection_head.1.bias', 'projector.projection_head.3.weight', 'projector.projection_head.3.bias', 'predictor.projection_head.0.weight', 'predictor.projection_head.0.bias', 'predictor.projection_head.1.weight', 'predictor.projection_head.1.bias', 'predictor.projection_head.3.weight', 'predictor.projection_head.3.bias', 'rep_decoder_embed.weight', 'rep_decoder_embed.bias', 'rep_decoder_blocks.0.norm1.weight', 'rep_decoder_blocks.0.norm1.bias', 'rep_decoder_blocks.0.attn.qkv.weight', 'rep_decoder_blocks.0.attn.qkv.bias', 'rep_decoder_blocks.0.attn.proj.weight', 'rep_decoder_blocks.0.attn.proj.bias', 'rep_decoder_blocks.0.norm2.weight', 'rep_decoder_blocks.0.norm2.bias', 'rep_decoder_blocks.0.mlp.fc1.weight', 'rep_decoder_blocks.0.mlp.fc1.bias', 'rep_decoder_blocks.0.mlp.fc2.weight', 'rep_decoder_blocks.0.mlp.fc2.bias', 'rep_decoder_blocks.1.norm1.weight', 'rep_decoder_blocks.1.norm1.bias', 'rep_decoder_blocks.1.attn.qkv.weight', 'rep_decoder_blocks.1.attn.qkv.bias', 'rep_decoder_blocks.1.attn.proj.weight', 'rep_decoder_blocks.1.attn.proj.bias', 'rep_decoder_blocks.1.norm2.weight', 'rep_decoder_blocks.1.norm2.bias', 'rep_decoder_blocks.1.mlp.fc1.weight', 'rep_decoder_blocks.1.mlp.fc1.bias', 'rep_decoder_blocks.1.mlp.fc2.weight', 'rep_decoder_blocks.1.mlp.fc2.bias', 'rep_decoder_norm.weight', 'rep_decoder_norm.bias', 'rep_decoder_pred.weight', 'rep_decoder_pred.bias', 'decoder_embed.weight', 'decoder_embed.bias', 'decoder_blocks.0.norm1.weight', 'decoder_blocks.0.norm1.bias', 'decoder_blocks.0.attn.qkv.weight', 'decoder_blocks.0.attn.qkv.bias', 'decoder_blocks.0.attn.proj.weight', 'decoder_blocks.0.attn.proj.bias', 'decoder_blocks.0.norm2.weight', 'decoder_blocks.0.norm2.bias', 'decoder_blocks.0.mlp.fc1.weight', 'decoder_blocks.0.mlp.fc1.bias', 'decoder_blocks.0.mlp.fc2.weight', 'decoder_blocks.0.mlp.fc2.bias', 'decoder_blocks.1.norm1.weight', 'decoder_blocks.1.norm1.bias', 'decoder_blocks.1.attn.qkv.weight', 'decoder_blocks.1.attn.qkv.bias', 'decoder_blocks.1.attn.proj.weight', 'decoder_blocks.1.attn.proj.bias', 'decoder_blocks.1.norm2.weight', 'decoder_blocks.1.norm2.bias', 'decoder_blocks.1.mlp.fc1.weight', 'decoder_blocks.1.mlp.fc1.bias', 'decoder_blocks.1.mlp.fc2.weight', 'decoder_blocks.1.mlp.fc2.bias', 'decoder_blocks.2.norm1.weight', 'decoder_blocks.2.norm1.bias', 'decoder_blocks.2.attn.qkv.weight', 'decoder_blocks.2.attn.qkv.bias', 'decoder_blocks.2.attn.proj.weight', 'decoder_blocks.2.attn.proj.bias', 'decoder_blocks.2.norm2.weight', 'decoder_blocks.2.norm2.bias', 'decoder_blocks.2.mlp.fc1.weight', 'decoder_blocks.2.mlp.fc1.bias', 'decoder_blocks.2.mlp.fc2.weight', 'decoder_blocks.2.mlp.fc2.bias', 'decoder_blocks.3.norm1.weight', 'decoder_blocks.3.norm1.bias', 'decoder_blocks.3.attn.qkv.weight', 'decoder_blocks.3.attn.qkv.bias', 'decoder_blocks.3.attn.proj.weight', 'decoder_blocks.3.attn.proj.bias', 'decoder_blocks.3.norm2.weight', 'decoder_blocks.3.norm2.bias', 'decoder_blocks.3.mlp.fc1.weight', 'decoder_blocks.3.mlp.fc1.bias', 'decoder_blocks.3.mlp.fc2.weight', 'decoder_blocks.3.mlp.fc2.bias', 'decoder_blocks.4.norm1.weight', 'decoder_blocks.4.norm1.bias', 'decoder_blocks.4.attn.qkv.weight', 'decoder_blocks.4.attn.qkv.bias', 'decoder_blocks.4.attn.proj.weight', 'decoder_blocks.4.attn.proj.bias', 'decoder_blocks.4.norm2.weight', 'decoder_blocks.4.norm2.bias', 'decoder_blocks.4.mlp.fc1.weight', 'decoder_blocks.4.mlp.fc1.bias', 'decoder_blocks.4.mlp.fc2.weight', 'decoder_blocks.4.mlp.fc2.bias', 'decoder_blocks.5.norm1.weight', 'decoder_blocks.5.norm1.bias', 'decoder_blocks.5.attn.qkv.weight', 'decoder_blocks.5.attn.qkv.bias', 'decoder_blocks.5.attn.proj.weight', 'decoder_blocks.5.attn.proj.bias', 'decoder_blocks.5.norm2.weight', 'decoder_blocks.5.norm2.bias', 'decoder_blocks.5.mlp.fc1.weight', 'decoder_blocks.5.mlp.fc1.bias', 'decoder_blocks.5.mlp.fc2.weight', 'decoder_blocks.5.mlp.fc2.bias', 'decoder_blocks.6.norm1.weight', 'decoder_blocks.6.norm1.bias', 'decoder_blocks.6.attn.qkv.weight', 'decoder_blocks.6.attn.qkv.bias', 'decoder_blocks.6.attn.proj.weight', 'decoder_blocks.6.attn.proj.bias', 'decoder_blocks.6.norm2.weight', 'decoder_blocks.6.norm2.bias', 'decoder_blocks.6.mlp.fc1.weight', 'decoder_blocks.6.mlp.fc1.bias', 'decoder_blocks.6.mlp.fc2.weight', 'decoder_blocks.6.mlp.fc2.bias', 'decoder_blocks.7.norm1.weight', 'decoder_blocks.7.norm1.bias', 'decoder_blocks.7.attn.qkv.weight', 'decoder_blocks.7.attn.qkv.bias', 'decoder_blocks.7.attn.proj.weight', 'decoder_blocks.7.attn.proj.bias', 'decoder_blocks.7.norm2.weight', 'decoder_blocks.7.norm2.bias', 'decoder_blocks.7.mlp.fc1.weight', 'decoder_blocks.7.mlp.fc1.bias', 'decoder_blocks.7.mlp.fc2.weight', 'decoder_blocks.7.mlp.fc2.bias', 'decoder_norm.weight', 'decoder_norm.bias', 'decoder_pred.weight', 'decoder_pred.bias']) | |
| [00:11:49.520224] ========================================================================================== | |
| Layer (type:depth-idx) Output Shape Param # | |
| ========================================================================================== | |
| ├─PatchEmbed: 1-1 [-1, 196, 768] -- | |
| | └─Conv2d: 2-1 [-1, 768, 14, 14] 590,592 | |
| ├─Dropout: 1-2 [-1, 197, 768] -- | |
| ├─ModuleList: 1 [] -- | |
| | └─Block: 2-2 [-1, 197, 768] -- | |
| | | └─LayerNorm: 3-1 [-1, 197, 768] 1,536 | |
| | | └─Attention: 3-2 [-1, 197, 768] 2,362,368 | |
| | | └─Identity: 3-3 [-1, 197, 768] -- | |
| | | └─LayerNorm: 3-4 [-1, 197, 768] 1,536 | |
| | | └─Mlp: 3-5 [-1, 197, 768] 4,722,432 | |
| | | └─Identity: 3-6 [-1, 197, 768] -- | |
| | └─Block: 2-3 [-1, 197, 768] -- | |
| | | └─LayerNorm: 3-7 [-1, 197, 768] 1,536 | |
| | | └─Attention: 3-8 [-1, 197, 768] 2,362,368 | |
| | | └─DropPath: 3-9 [-1, 197, 768] -- | |
| | | └─LayerNorm: 3-10 [-1, 197, 768] 1,536 | |
| | | └─Mlp: 3-11 [-1, 197, 768] 4,722,432 | |
| | | └─DropPath: 3-12 [-1, 197, 768] -- | |
| | └─Block: 2-4 [-1, 197, 768] -- | |
| | | └─LayerNorm: 3-13 [-1, 197, 768] 1,536 | |
| | | └─Attention: 3-14 [-1, 197, 768] 2,362,368 | |
| | | └─DropPath: 3-15 [-1, 197, 768] -- | |
| | | └─LayerNorm: 3-16 [-1, 197, 768] 1,536 | |
| | | └─Mlp: 3-17 [-1, 197, 768] 4,722,432 | |
| | | └─DropPath: 3-18 [-1, 197, 768] -- | |
| | └─Block: 2-5 [-1, 197, 768] -- | |
| | | └─LayerNorm: 3-19 [-1, 197, 768] 1,536 | |
| | | └─Attention: 3-20 [-1, 197, 768] 2,362,368 | |
| | | └─DropPath: 3-21 [-1, 197, 768] -- | |
| | | └─LayerNorm: 3-22 [-1, 197, 768] 1,536 | |
| | | └─Mlp: 3-23 [-1, 197, 768] 4,722,432 | |
| | | └─DropPath: 3-24 [-1, 197, 768] -- | |
| | └─Block: 2-6 [-1, 197, 768] -- | |
| | | └─LayerNorm: 3-25 [-1, 197, 768] 1,536 | |
| | | └─Attention: 3-26 [-1, 197, 768] 2,362,368 | |
| | | └─DropPath: 3-27 [-1, 197, 768] -- | |
| | | └─LayerNorm: 3-28 [-1, 197, 768] 1,536 | |
| | | └─Mlp: 3-29 [-1, 197, 768] 4,722,432 | |
| | | └─DropPath: 3-30 [-1, 197, 768] -- | |
| | └─Block: 2-7 [-1, 197, 768] -- | |
| | | └─LayerNorm: 3-31 [-1, 197, 768] 1,536 | |
| | | └─Attention: 3-32 [-1, 197, 768] 2,362,368 | |
| | | └─DropPath: 3-33 [-1, 197, 768] -- | |
| | | └─LayerNorm: 3-34 [-1, 197, 768] 1,536 | |
| | | └─Mlp: 3-35 [-1, 197, 768] 4,722,432 | |
| | | └─DropPath: 3-36 [-1, 197, 768] -- | |
| | └─Block: 2-8 [-1, 197, 768] -- | |
| | | └─LayerNorm: 3-37 [-1, 197, 768] 1,536 | |
| | | └─Attention: 3-38 [-1, 197, 768] 2,362,368 | |
| | | └─DropPath: 3-39 [-1, 197, 768] -- | |
| | | └─LayerNorm: 3-40 [-1, 197, 768] 1,536 | |
| | | └─Mlp: 3-41 [-1, 197, 768] 4,722,432 | |
| | | └─DropPath: 3-42 [-1, 197, 768] -- | |
| | └─Block: 2-9 [-1, 197, 768] -- | |
| | | └─LayerNorm: 3-43 [-1, 197, 768] 1,536 | |
| | | └─Attention: 3-44 [-1, 197, 768] 2,362,368 | |
| | | └─DropPath: 3-45 [-1, 197, 768] -- | |
| | | └─LayerNorm: 3-46 [-1, 197, 768] 1,536 | |
| | | └─Mlp: 3-47 [-1, 197, 768] 4,722,432 | |
| | | └─DropPath: 3-48 [-1, 197, 768] -- | |
| | └─Block: 2-10 [-1, 197, 768] -- | |
| | | └─LayerNorm: 3-49 [-1, 197, 768] 1,536 | |
| | | └─Attention: 3-50 [-1, 197, 768] 2,362,368 | |
| | | └─DropPath: 3-51 [-1, 197, 768] -- | |
| | | └─LayerNorm: 3-52 [-1, 197, 768] 1,536 | |
| | | └─Mlp: 3-53 [-1, 197, 768] 4,722,432 | |
| | | └─DropPath: 3-54 [-1, 197, 768] -- | |
| | └─Block: 2-11 [-1, 197, 768] -- | |
| | | └─LayerNorm: 3-55 [-1, 197, 768] 1,536 | |
| | | └─Attention: 3-56 [-1, 197, 768] 2,362,368 | |
| | | └─DropPath: 3-57 [-1, 197, 768] -- | |
| | | └─LayerNorm: 3-58 [-1, 197, 768] 1,536 | |
| | | └─Mlp: 3-59 [-1, 197, 768] 4,722,432 | |
| | | └─DropPath: 3-60 [-1, 197, 768] -- | |
| | └─Block: 2-12 [-1, 197, 768] -- | |
| | | └─LayerNorm: 3-61 [-1, 197, 768] 1,536 | |
| | | └─Attention: 3-62 [-1, 197, 768] 2,362,368 | |
| | | └─DropPath: 3-63 [-1, 197, 768] -- | |
| | | └─LayerNorm: 3-64 [-1, 197, 768] 1,536 | |
| | | └─Mlp: 3-65 [-1, 197, 768] 4,722,432 | |
| | | └─DropPath: 3-66 [-1, 197, 768] -- | |
| | └─Block: 2-13 [-1, 197, 768] -- | |
| | | └─LayerNorm: 3-67 [-1, 197, 768] 1,536 | |
| | | └─Attention: 3-68 [-1, 197, 768] 2,362,368 | |
| | | └─DropPath: 3-69 [-1, 197, 768] -- | |
| | | └─LayerNorm: 3-70 [-1, 197, 768] 1,536 | |
| | | └─Mlp: 3-71 [-1, 197, 768] 4,722,432 | |
| | | └─DropPath: 3-72 [-1, 197, 768] -- | |
| ├─LayerNorm: 1-3 [-1, 768] 1,536 | |
| ├─Linear: 1-4 [-1, 2] 1,538 | |
| ========================================================================================== | |
| Total params: 85,648,130 | |
| Trainable params: 85,648,130 | |
| Non-trainable params: 0 | |
| Total mult-adds (M): 371.04 | |
| ========================================================================================== | |
| Input size (MB): 0.57 | |
| Forward/backward pass size (MB): 153.52 | |
| Params size (MB): 326.72 | |
| Estimated Total Size (MB): 480.82 | |
| ========================================================================================== | |
| [00:11:49.520972] ========================================================================================== | |
| Layer (type:depth-idx) Output Shape Param # | |
| ========================================================================================== | |
| ├─PatchEmbed: 1-1 [-1, 196, 768] -- | |
| | └─Conv2d: 2-1 [-1, 768, 14, 14] 590,592 | |
| ├─Dropout: 1-2 [-1, 197, 768] -- | |
| ├─ModuleList: 1 [] -- | |
| | └─Block: 2-2 [-1, 197, 768] -- | |
| | | └─LayerNorm: 3-1 [-1, 197, 768] 1,536 | |
| | | └─Attention: 3-2 [-1, 197, 768] 2,362,368 | |
| | | └─Identity: 3-3 [-1, 197, 768] -- | |
| | | └─LayerNorm: 3-4 [-1, 197, 768] 1,536 | |
| | | └─Mlp: 3-5 [-1, 197, 768] 4,722,432 | |
| | | └─Identity: 3-6 [-1, 197, 768] -- | |
| | └─Block: 2-3 [-1, 197, 768] -- | |
| | | └─LayerNorm: 3-7 [-1, 197, 768] 1,536 | |
| | | └─Attention: 3-8 [-1, 197, 768] 2,362,368 | |
| | | └─DropPath: 3-9 [-1, 197, 768] -- | |
| | | └─LayerNorm: 3-10 [-1, 197, 768] 1,536 | |
| | | └─Mlp: 3-11 [-1, 197, 768] 4,722,432 | |
| | | └─DropPath: 3-12 [-1, 197, 768] -- | |
| | └─Block: 2-4 [-1, 197, 768] -- | |
| | | └─LayerNorm: 3-13 [-1, 197, 768] 1,536 | |
| | | └─Attention: 3-14 [-1, 197, 768] 2,362,368 | |
| | | └─DropPath: 3-15 [-1, 197, 768] -- | |
| | | └─LayerNorm: 3-16 [-1, 197, 768] 1,536 | |
| | | └─Mlp: 3-17 [-1, 197, 768] 4,722,432 | |
| | | └─DropPath: 3-18 [-1, 197, 768] -- | |
| | └─Block: 2-5 [-1, 197, 768] -- | |
| | | └─LayerNorm: 3-19 [-1, 197, 768] 1,536 | |
| | | └─Attention: 3-20 [-1, 197, 768] 2,362,368 | |
| | | └─DropPath: 3-21 [-1, 197, 768] -- | |
| | | └─LayerNorm: 3-22 [-1, 197, 768] 1,536 | |
| | | └─Mlp: 3-23 [-1, 197, 768] 4,722,432 | |
| | | └─DropPath: 3-24 [-1, 197, 768] -- | |
| | └─Block: 2-6 [-1, 197, 768] -- | |
| | | └─LayerNorm: 3-25 [-1, 197, 768] 1,536 | |
| | | └─Attention: 3-26 [-1, 197, 768] 2,362,368 | |
| | | └─DropPath: 3-27 [-1, 197, 768] -- | |
| | | └─LayerNorm: 3-28 [-1, 197, 768] 1,536 | |
| | | └─Mlp: 3-29 [-1, 197, 768] 4,722,432 | |
| | | └─DropPath: 3-30 [-1, 197, 768] -- | |
| | └─Block: 2-7 [-1, 197, 768] -- | |
| | | └─LayerNorm: 3-31 [-1, 197, 768] 1,536 | |
| | | └─Attention: 3-32 [-1, 197, 768] 2,362,368 | |
| | | └─DropPath: 3-33 [-1, 197, 768] -- | |
| | | └─LayerNorm: 3-34 [-1, 197, 768] 1,536 | |
| | | └─Mlp: 3-35 [-1, 197, 768] 4,722,432 | |
| | | └─DropPath: 3-36 [-1, 197, 768] -- | |
| | └─Block: 2-8 [-1, 197, 768] -- | |
| | | └─LayerNorm: 3-37 [-1, 197, 768] 1,536 | |
| | | └─Attention: 3-38 [-1, 197, 768] 2,362,368 | |
| | | └─DropPath: 3-39 [-1, 197, 768] -- | |
| | | └─LayerNorm: 3-40 [-1, 197, 768] 1,536 | |
| | | └─Mlp: 3-41 [-1, 197, 768] 4,722,432 | |
| | | └─DropPath: 3-42 [-1, 197, 768] -- | |
| | └─Block: 2-9 [-1, 197, 768] -- | |
| | | └─LayerNorm: 3-43 [-1, 197, 768] 1,536 | |
| | | └─Attention: 3-44 [-1, 197, 768] 2,362,368 | |
| | | └─DropPath: 3-45 [-1, 197, 768] -- | |
| | | └─LayerNorm: 3-46 [-1, 197, 768] 1,536 | |
| | | └─Mlp: 3-47 [-1, 197, 768] 4,722,432 | |
| | | └─DropPath: 3-48 [-1, 197, 768] -- | |
| | └─Block: 2-10 [-1, 197, 768] -- | |
| | | └─LayerNorm: 3-49 [-1, 197, 768] 1,536 | |
| | | └─Attention: 3-50 [-1, 197, 768] 2,362,368 | |
| | | └─DropPath: 3-51 [-1, 197, 768] -- | |
| | | └─LayerNorm: 3-52 [-1, 197, 768] 1,536 | |
| | | └─Mlp: 3-53 [-1, 197, 768] 4,722,432 | |
| | | └─DropPath: 3-54 [-1, 197, 768] -- | |
| | └─Block: 2-11 [-1, 197, 768] -- | |
| | | └─LayerNorm: 3-55 [-1, 197, 768] 1,536 | |
| | | └─Attention: 3-56 [-1, 197, 768] 2,362,368 | |
| | | └─DropPath: 3-57 [-1, 197, 768] -- | |
| | | └─LayerNorm: 3-58 [-1, 197, 768] 1,536 | |
| | | └─Mlp: 3-59 [-1, 197, 768] 4,722,432 | |
| | | └─DropPath: 3-60 [-1, 197, 768] -- | |
| | └─Block: 2-12 [-1, 197, 768] -- | |
| | | └─LayerNorm: 3-61 [-1, 197, 768] 1,536 | |
| | | └─Attention: 3-62 [-1, 197, 768] 2,362,368 | |
| | | └─DropPath: 3-63 [-1, 197, 768] -- | |
| | | └─LayerNorm: 3-64 [-1, 197, 768] 1,536 | |
| | | └─Mlp: 3-65 [-1, 197, 768] 4,722,432 | |
| | | └─DropPath: 3-66 [-1, 197, 768] -- | |
| | └─Block: 2-13 [-1, 197, 768] -- | |
| | | └─LayerNorm: 3-67 [-1, 197, 768] 1,536 | |
| | | └─Attention: 3-68 [-1, 197, 768] 2,362,368 | |
| | | └─DropPath: 3-69 [-1, 197, 768] -- | |
| | | └─LayerNorm: 3-70 [-1, 197, 768] 1,536 | |
| | | └─Mlp: 3-71 [-1, 197, 768] 4,722,432 | |
| | | └─DropPath: 3-72 [-1, 197, 768] -- | |
| ├─LayerNorm: 1-3 [-1, 768] 1,536 | |
| ├─Linear: 1-4 [-1, 2] 1,538 | |
| ========================================================================================== | |
| Total params: 85,648,130 | |
| Trainable params: 85,648,130 | |
| Non-trainable params: 0 | |
| Total mult-adds (M): 371.04 | |
| ========================================================================================== | |
| Input size (MB): 0.57 | |
| Forward/backward pass size (MB): 153.52 | |
| Params size (MB): 326.72 | |
| Estimated Total Size (MB): 480.82 | |
| ========================================================================================== | |
| [00:11:49.522410] Model = VisionTransformer( | |
| (patch_embed): PatchEmbed( | |
| (proj): Conv2d(3, 768, kernel_size=(16, 16), stride=(16, 16)) | |
| ) | |
| (pos_drop): Dropout(p=0.0, inplace=False) | |
| (blocks): ModuleList( | |
| (0): Block( | |
| (norm1): LayerNorm((768,), eps=1e-06, elementwise_affine=True) | |
| (attn): Attention( | |
| (qkv): Linear(in_features=768, out_features=2304, bias=True) | |
| (attn_drop): Dropout(p=0.0, inplace=False) | |
| (proj): Linear(in_features=768, out_features=768, bias=True) | |
| (proj_drop): Dropout(p=0.0, inplace=False) | |
| ) | |
| (drop_path): Identity() | |
| (norm2): LayerNorm((768,), eps=1e-06, elementwise_affine=True) | |
| (mlp): Mlp( | |
| (fc1): Linear(in_features=768, out_features=3072, bias=True) | |
| (act): GELU(approximate='none') | |
| (fc2): Linear(in_features=3072, out_features=768, bias=True) | |
| (drop): Dropout(p=0.0, inplace=False) | |
| ) | |
| ) | |
| (1): Block( | |
| (norm1): LayerNorm((768,), eps=1e-06, elementwise_affine=True) | |
| (attn): Attention( | |
| (qkv): Linear(in_features=768, out_features=2304, bias=True) | |
| (attn_drop): Dropout(p=0.0, inplace=False) | |
| (proj): Linear(in_features=768, out_features=768, bias=True) | |
| (proj_drop): Dropout(p=0.0, inplace=False) | |
| ) | |
| (drop_path): DropPath() | |
| (norm2): LayerNorm((768,), eps=1e-06, elementwise_affine=True) | |
| (mlp): Mlp( | |
| (fc1): Linear(in_features=768, out_features=3072, bias=True) | |
| (act): GELU(approximate='none') | |
| (fc2): Linear(in_features=3072, out_features=768, bias=True) | |
| (drop): Dropout(p=0.0, inplace=False) | |
| ) | |
| ) | |
| (2): Block( | |
| (norm1): LayerNorm((768,), eps=1e-06, elementwise_affine=True) | |
| (attn): Attention( | |
| (qkv): Linear(in_features=768, out_features=2304, bias=True) | |
| (attn_drop): Dropout(p=0.0, inplace=False) | |
| (proj): Linear(in_features=768, out_features=768, bias=True) | |
| (proj_drop): Dropout(p=0.0, inplace=False) | |
| ) | |
| (drop_path): DropPath() | |
| (norm2): LayerNorm((768,), eps=1e-06, elementwise_affine=True) | |
| (mlp): Mlp( | |
| (fc1): Linear(in_features=768, out_features=3072, bias=True) | |
| (act): GELU(approximate='none') | |
| (fc2): Linear(in_features=3072, out_features=768, bias=True) | |
| (drop): Dropout(p=0.0, inplace=False) | |
| ) | |
| ) | |
| (3): Block( | |
| (norm1): LayerNorm((768,), eps=1e-06, elementwise_affine=True) | |
| (attn): Attention( | |
| (qkv): Linear(in_features=768, out_features=2304, bias=True) | |
| (attn_drop): Dropout(p=0.0, inplace=False) | |
| (proj): Linear(in_features=768, out_features=768, bias=True) | |
| (proj_drop): Dropout(p=0.0, inplace=False) | |
| ) | |
| (drop_path): DropPath() | |
| (norm2): LayerNorm((768,), eps=1e-06, elementwise_affine=True) | |
| (mlp): Mlp( | |
| (fc1): Linear(in_features=768, out_features=3072, bias=True) | |
| (act): GELU(approximate='none') | |
| (fc2): Linear(in_features=3072, out_features=768, bias=True) | |
| (drop): Dropout(p=0.0, inplace=False) | |
| ) | |
| ) | |
| (4): Block( | |
| (norm1): LayerNorm((768,), eps=1e-06, elementwise_affine=True) | |
| (attn): Attention( | |
| (qkv): Linear(in_features=768, out_features=2304, bias=True) | |
| (attn_drop): Dropout(p=0.0, inplace=False) | |
| (proj): Linear(in_features=768, out_features=768, bias=True) | |
| (proj_drop): Dropout(p=0.0, inplace=False) | |
| ) | |
| (drop_path): DropPath() | |
| (norm2): LayerNorm((768,), eps=1e-06, elementwise_affine=True) | |
| (mlp): Mlp( | |
| (fc1): Linear(in_features=768, out_features=3072, bias=True) | |
| (act): GELU(approximate='none') | |
| (fc2): Linear(in_features=3072, out_features=768, bias=True) | |
| (drop): Dropout(p=0.0, inplace=False) | |
| ) | |
| ) | |
| (5): Block( | |
| (norm1): LayerNorm((768,), eps=1e-06, elementwise_affine=True) | |
| (attn): Attention( | |
| (qkv): Linear(in_features=768, out_features=2304, bias=True) | |
| (attn_drop): Dropout(p=0.0, inplace=False) | |
| (proj): Linear(in_features=768, out_features=768, bias=True) | |
| (proj_drop): Dropout(p=0.0, inplace=False) | |
| ) | |
| (drop_path): DropPath() | |
| (norm2): LayerNorm((768,), eps=1e-06, elementwise_affine=True) | |
| (mlp): Mlp( | |
| (fc1): Linear(in_features=768, out_features=3072, bias=True) | |
| (act): GELU(approximate='none') | |
| (fc2): Linear(in_features=3072, out_features=768, bias=True) | |
| (drop): Dropout(p=0.0, inplace=False) | |
| ) | |
| ) | |
| (6): Block( | |
| (norm1): LayerNorm((768,), eps=1e-06, elementwise_affine=True) | |
| (attn): Attention( | |
| (qkv): Linear(in_features=768, out_features=2304, bias=True) | |
| (attn_drop): Dropout(p=0.0, inplace=False) | |
| (proj): Linear(in_features=768, out_features=768, bias=True) | |
| (proj_drop): Dropout(p=0.0, inplace=False) | |
| ) | |
| (drop_path): DropPath() | |
| (norm2): LayerNorm((768,), eps=1e-06, elementwise_affine=True) | |
| (mlp): Mlp( | |
| (fc1): Linear(in_features=768, out_features=3072, bias=True) | |
| (act): GELU(approximate='none') | |
| (fc2): Linear(in_features=3072, out_features=768, bias=True) | |
| (drop): Dropout(p=0.0, inplace=False) | |
| ) | |
| ) | |
| (7): Block( | |
| (norm1): LayerNorm((768,), eps=1e-06, elementwise_affine=True) | |
| (attn): Attention( | |
| (qkv): Linear(in_features=768, out_features=2304, bias=True) | |
| (attn_drop): Dropout(p=0.0, inplace=False) | |
| (proj): Linear(in_features=768, out_features=768, bias=True) | |
| (proj_drop): Dropout(p=0.0, inplace=False) | |
| ) | |
| (drop_path): DropPath() | |
| (norm2): LayerNorm((768,), eps=1e-06, elementwise_affine=True) | |
| (mlp): Mlp( | |
| (fc1): Linear(in_features=768, out_features=3072, bias=True) | |
| (act): GELU(approximate='none') | |
| (fc2): Linear(in_features=3072, out_features=768, bias=True) | |
| (drop): Dropout(p=0.0, inplace=False) | |
| ) | |
| ) | |
| (8): Block( | |
| (norm1): LayerNorm((768,), eps=1e-06, elementwise_affine=True) | |
| (attn): Attention( | |
| (qkv): Linear(in_features=768, out_features=2304, bias=True) | |
| (attn_drop): Dropout(p=0.0, inplace=False) | |
| (proj): Linear(in_features=768, out_features=768, bias=True) | |
| (proj_drop): Dropout(p=0.0, inplace=False) | |
| ) | |
| (drop_path): DropPath() | |
| (norm2): LayerNorm((768,), eps=1e-06, elementwise_affine=True) | |
| (mlp): Mlp( | |
| (fc1): Linear(in_features=768, out_features=3072, bias=True) | |
| (act): GELU(approximate='none') | |
| (fc2): Linear(in_features=3072, out_features=768, bias=True) | |
| (drop): Dropout(p=0.0, inplace=False) | |
| ) | |
| ) | |
| (9): Block( | |
| (norm1): LayerNorm((768,), eps=1e-06, elementwise_affine=True) | |
| (attn): Attention( | |
| (qkv): Linear(in_features=768, out_features=2304, bias=True) | |
| (attn_drop): Dropout(p=0.0, inplace=False) | |
| (proj): Linear(in_features=768, out_features=768, bias=True) | |
| (proj_drop): Dropout(p=0.0, inplace=False) | |
| ) | |
| (drop_path): DropPath() | |
| (norm2): LayerNorm((768,), eps=1e-06, elementwise_affine=True) | |
| (mlp): Mlp( | |
| (fc1): Linear(in_features=768, out_features=3072, bias=True) | |
| (act): GELU(approximate='none') | |
| (fc2): Linear(in_features=3072, out_features=768, bias=True) | |
| (drop): Dropout(p=0.0, inplace=False) | |
| ) | |
| ) | |
| (10): Block( | |
| (norm1): LayerNorm((768,), eps=1e-06, elementwise_affine=True) | |
| (attn): Attention( | |
| (qkv): Linear(in_features=768, out_features=2304, bias=True) | |
| (attn_drop): Dropout(p=0.0, inplace=False) | |
| (proj): Linear(in_features=768, out_features=768, bias=True) | |
| (proj_drop): Dropout(p=0.0, inplace=False) | |
| ) | |
| (drop_path): DropPath() | |
| (norm2): LayerNorm((768,), eps=1e-06, elementwise_affine=True) | |
| (mlp): Mlp( | |
| (fc1): Linear(in_features=768, out_features=3072, bias=True) | |
| (act): GELU(approximate='none') | |
| (fc2): Linear(in_features=3072, out_features=768, bias=True) | |
| (drop): Dropout(p=0.0, inplace=False) | |
| ) | |
| ) | |
| (11): Block( | |
| (norm1): LayerNorm((768,), eps=1e-06, elementwise_affine=True) | |
| (attn): Attention( | |
| (qkv): Linear(in_features=768, out_features=2304, bias=True) | |
| (attn_drop): Dropout(p=0.0, inplace=False) | |
| (proj): Linear(in_features=768, out_features=768, bias=True) | |
| (proj_drop): Dropout(p=0.0, inplace=False) | |
| ) | |
| (drop_path): DropPath() | |
| (norm2): LayerNorm((768,), eps=1e-06, elementwise_affine=True) | |
| (mlp): Mlp( | |
| (fc1): Linear(in_features=768, out_features=3072, bias=True) | |
| (act): GELU(approximate='none') | |
| (fc2): Linear(in_features=3072, out_features=768, bias=True) | |
| (drop): Dropout(p=0.0, inplace=False) | |
| ) | |
| ) | |
| ) | |
| (head): Linear(in_features=768, out_features=2, bias=True) | |
| (fc_norm): LayerNorm((768,), eps=1e-06, elementwise_affine=True) | |
| ) | |
| [00:11:49.522449] number of params (M): 85.80 | |
| [00:11:49.522461] base lr: 2.50e-04 | |
| [00:11:49.522467] actual lr: 6.25e-05 | |
| [00:11:49.522473] accumulate grad iterations: 1 | |
| [00:11:49.522478] effective batch size: 64 | |
| [00:11:49.545046] criterion = SoftTargetCrossEntropy() | |
| [00:11:49.545066] Start training for 10 epochs | |
| [00:11:49.545569] log_dir: ./checkpoint/finetuned_models/FF++_c23_32frames | |
| [00:11:51.899172] Epoch: [0] [ 0/2877] eta: 1:52:46 lr: 0.000000 loss: 0.6932 (0.6932) time: 2.3519 data: 1.3011 max mem: 4308 | |
| [00:12:04.594530] Epoch: [0] [ 100/2877] eta: 0:06:53 lr: 0.000000 loss: 0.6931 (0.6931) time: 0.1239 data: 0.0002 max mem: 5308 | |
| [00:12:20.440679] Epoch: [0] [ 200/2877] eta: 0:06:51 lr: 0.000001 loss: 0.6931 (0.6931) time: 0.1741 data: 0.0002 max mem: 5308 | |
| [00:12:37.025239] Epoch: [0] [ 300/2877] eta: 0:06:46 lr: 0.000001 loss: 0.6931 (0.6931) time: 0.1455 data: 0.0002 max mem: 5308 | |
| [00:12:49.527202] Epoch: [0] [ 400/2877] eta: 0:06:10 lr: 0.000002 loss: 0.6931 (0.6931) time: 0.1251 data: 0.0001 max mem: 5308 | |
| [00:13:02.109948] Epoch: [0] [ 500/2877] eta: 0:05:44 lr: 0.000002 loss: 0.6930 (0.6931) time: 0.1270 data: 0.0001 max mem: 5308 | |
| [00:13:15.334534] Epoch: [0] [ 600/2877] eta: 0:05:24 lr: 0.000003 loss: 0.6933 (0.6931) time: 0.1395 data: 0.0130 max mem: 5308 | |
| [00:13:30.904207] Epoch: [0] [ 700/2877] eta: 0:05:14 lr: 0.000003 loss: 0.6929 (0.6931) time: 0.1626 data: 0.0274 max mem: 5308 | |
| [00:13:50.869446] Epoch: [0] [ 800/2877] eta: 0:05:14 lr: 0.000003 loss: 0.6929 (0.6931) time: 0.1847 data: 0.0243 max mem: 5308 | |
| [00:14:14.161536] Epoch: [0] [ 900/2877] eta: 0:05:17 lr: 0.000004 loss: 0.6925 (0.6930) time: 0.2663 data: 0.0408 max mem: 5308 | |
| [00:14:40.170706] Epoch: [0] [1000/2877] eta: 0:05:19 lr: 0.000004 loss: 0.6929 (0.6930) time: 0.2551 data: 0.1299 max mem: 5308 | |
| [00:14:56.609183] Epoch: [0] [1100/2877] eta: 0:05:01 lr: 0.000005 loss: 0.6929 (0.6930) time: 0.1248 data: 0.0001 max mem: 5308 | |
| [00:15:09.075570] Epoch: [0] [1200/2877] eta: 0:04:38 lr: 0.000005 loss: 0.6928 (0.6930) time: 0.1237 data: 0.0001 max mem: 5308 | |
| [00:15:21.451969] Epoch: [0] [1300/2877] eta: 0:04:16 lr: 0.000006 loss: 0.6941 (0.6930) time: 0.1234 data: 0.0001 max mem: 5308 | |
| [00:15:33.841172] Epoch: [0] [1400/2877] eta: 0:03:56 lr: 0.000006 loss: 0.6927 (0.6929) time: 0.1237 data: 0.0001 max mem: 5308 | |
| [00:15:46.285320] Epoch: [0] [1500/2877] eta: 0:03:37 lr: 0.000007 loss: 0.6940 (0.6929) time: 0.1257 data: 0.0001 max mem: 5308 | |
| [00:15:58.837447] Epoch: [0] [1600/2877] eta: 0:03:18 lr: 0.000007 loss: 0.6919 (0.6928) time: 0.1241 data: 0.0001 max mem: 5308 | |
| [00:16:11.474941] Epoch: [0] [1700/2877] eta: 0:03:01 lr: 0.000007 loss: 0.6922 (0.6927) time: 0.1263 data: 0.0001 max mem: 5308 | |
| [00:16:24.014093] Epoch: [0] [1800/2877] eta: 0:02:44 lr: 0.000008 loss: 0.6916 (0.6926) time: 0.1253 data: 0.0001 max mem: 5308 | |
| [00:16:36.591833] Epoch: [0] [1900/2877] eta: 0:02:27 lr: 0.000008 loss: 0.6898 (0.6926) time: 0.1258 data: 0.0001 max mem: 5308 | |
| [00:16:49.142189] Epoch: [0] [2000/2877] eta: 0:02:11 lr: 0.000009 loss: 0.6908 (0.6925) time: 0.1257 data: 0.0001 max mem: 5308 | |
| [00:17:01.683380] Epoch: [0] [2100/2877] eta: 0:01:55 lr: 0.000009 loss: 0.6919 (0.6924) time: 0.1256 data: 0.0001 max mem: 5308 | |
| [00:17:14.221816] Epoch: [0] [2200/2877] eta: 0:01:39 lr: 0.000010 loss: 0.6892 (0.6922) time: 0.1261 data: 0.0001 max mem: 5308 | |
| [00:17:26.879099] Epoch: [0] [2300/2877] eta: 0:01:24 lr: 0.000010 loss: 0.6893 (0.6921) time: 0.1261 data: 0.0001 max mem: 5308 | |
| [00:17:39.526825] Epoch: [0] [2400/2877] eta: 0:01:09 lr: 0.000010 loss: 0.6832 (0.6919) time: 0.1270 data: 0.0001 max mem: 5308 | |
| [00:17:52.192418] Epoch: [0] [2500/2877] eta: 0:00:54 lr: 0.000011 loss: 0.6892 (0.6918) time: 0.1266 data: 0.0001 max mem: 5308 | |
| [00:18:04.890664] Epoch: [0] [2600/2877] eta: 0:00:39 lr: 0.000011 loss: 0.6890 (0.6917) time: 0.1266 data: 0.0001 max mem: 5308 | |
| [00:18:17.615457] Epoch: [0] [2700/2877] eta: 0:00:25 lr: 0.000012 loss: 0.6835 (0.6915) time: 0.1278 data: 0.0001 max mem: 5308 | |
| [00:18:30.331686] Epoch: [0] [2800/2877] eta: 0:00:11 lr: 0.000012 loss: 0.6845 (0.6914) time: 0.1271 data: 0.0001 max mem: 5308 | |
| [00:18:39.930631] Epoch: [0] [2876/2877] eta: 0:00:00 lr: 0.000012 loss: 0.6867 (0.6913) time: 0.1241 data: 0.0002 max mem: 5308 | |
| [00:18:40.170254] Epoch: [0] Total time: 0:06:50 (0.1427 s / it) | |
| [00:18:40.171931] Averaged stats: lr: 0.000012 loss: 0.6867 (0.6914) | |
| [00:18:41.473801] Test: [ 0/560] eta: 0:12:07 loss: 0.6874 (0.6874) auc: 65.8824 (65.8824) time: 1.2992 data: 1.2655 max mem: 5308 | |
| [00:18:41.772698] Test: [ 10/560] eta: 0:01:19 loss: 0.6699 (0.6707) auc: 74.2915 (75.0876) time: 0.1452 data: 0.1152 max mem: 5308 | |
| [00:18:42.073526] Test: [ 20/560] eta: 0:00:48 loss: 0.6682 (0.6698) auc: 80.7359 (79.9045) time: 0.0299 data: 0.0002 max mem: 5308 | |
| [00:18:42.374284] Test: [ 30/560] eta: 0:00:37 loss: 0.6613 (0.6669) auc: 84.3254 (81.4501) time: 0.0300 data: 0.0002 max mem: 5308 | |
| [00:18:42.676708] Test: [ 40/560] eta: 0:00:31 loss: 0.6635 (0.6669) auc: 82.5397 (80.9105) time: 0.0301 data: 0.0002 max mem: 5308 | |
| [00:18:42.976864] Test: [ 50/560] eta: 0:00:27 loss: 0.6705 (0.6675) auc: 82.0833 (81.5161) time: 0.0300 data: 0.0002 max mem: 5308 | |
| [00:18:43.277360] Test: [ 60/560] eta: 0:00:25 loss: 0.6636 (0.6668) auc: 81.8182 (81.1055) time: 0.0300 data: 0.0002 max mem: 5308 | |
| [00:18:43.577965] Test: [ 70/560] eta: 0:00:23 loss: 0.6630 (0.6662) auc: 81.8182 (81.3349) time: 0.0300 data: 0.0002 max mem: 5308 | |
| [00:18:43.875192] Test: [ 80/560] eta: 0:00:21 loss: 0.6582 (0.6662) auc: 81.8182 (81.1841) time: 0.0298 data: 0.0002 max mem: 5308 | |
| [00:18:44.170818] Test: [ 90/560] eta: 0:00:20 loss: 0.6635 (0.6661) auc: 81.3725 (81.0894) time: 0.0296 data: 0.0002 max mem: 5308 | |
| [00:18:44.467284] Test: [100/560] eta: 0:00:19 loss: 0.6651 (0.6666) auc: 80.2734 (80.9505) time: 0.0295 data: 0.0002 max mem: 5308 | |
| [00:18:44.764258] Test: [110/560] eta: 0:00:18 loss: 0.6591 (0.6660) auc: 80.7692 (81.1747) time: 0.0296 data: 0.0002 max mem: 5308 | |
| [00:18:45.060902] Test: [120/560] eta: 0:00:17 loss: 0.6592 (0.6666) auc: 82.3413 (80.8463) time: 0.0296 data: 0.0002 max mem: 5308 | |
| [00:18:45.356528] Test: [130/560] eta: 0:00:16 loss: 0.6714 (0.6671) auc: 80.5556 (80.7944) time: 0.0295 data: 0.0001 max mem: 5308 | |
| [00:18:45.652653] Test: [140/560] eta: 0:00:16 loss: 0.6726 (0.6678) auc: 81.3492 (80.7031) time: 0.0295 data: 0.0001 max mem: 5308 | |
| [00:18:45.949327] Test: [150/560] eta: 0:00:15 loss: 0.6670 (0.6674) auc: 80.0000 (80.6251) time: 0.0296 data: 0.0002 max mem: 5308 | |
| [00:18:46.248467] Test: [160/560] eta: 0:00:15 loss: 0.6597 (0.6674) auc: 80.0000 (80.6803) time: 0.0297 data: 0.0002 max mem: 5308 | |
| [00:18:46.549540] Test: [170/560] eta: 0:00:14 loss: 0.6586 (0.6673) auc: 80.4545 (80.6816) time: 0.0299 data: 0.0002 max mem: 5308 | |
| [00:18:46.851266] Test: [180/560] eta: 0:00:13 loss: 0.6663 (0.6679) auc: 78.5425 (80.5652) time: 0.0300 data: 0.0002 max mem: 5308 | |
| [00:18:47.151121] Test: [190/560] eta: 0:00:13 loss: 0.6724 (0.6679) auc: 79.5455 (80.5298) time: 0.0300 data: 0.0002 max mem: 5308 | |
| [00:18:47.447445] Test: [200/560] eta: 0:00:12 loss: 0.6635 (0.6675) auc: 81.7460 (80.6410) time: 0.0297 data: 0.0002 max mem: 5308 | |
| [00:18:47.743063] Test: [210/560] eta: 0:00:12 loss: 0.6644 (0.6676) auc: 80.4167 (80.5790) time: 0.0295 data: 0.0001 max mem: 5308 | |
| [00:18:48.038026] Test: [220/560] eta: 0:00:12 loss: 0.6683 (0.6677) auc: 77.5000 (80.5544) time: 0.0295 data: 0.0001 max mem: 5308 | |
| [00:18:48.333415] Test: [230/560] eta: 0:00:11 loss: 0.6665 (0.6677) auc: 81.7460 (80.5966) time: 0.0295 data: 0.0001 max mem: 5308 | |
| [00:18:48.629234] Test: [240/560] eta: 0:00:11 loss: 0.6651 (0.6679) auc: 81.9444 (80.6255) time: 0.0295 data: 0.0001 max mem: 5308 | |
| [00:18:48.925216] Test: [250/560] eta: 0:00:10 loss: 0.6668 (0.6677) auc: 83.1349 (80.8067) time: 0.0295 data: 0.0001 max mem: 5308 | |
| [00:18:49.220282] Test: [260/560] eta: 0:00:10 loss: 0.6521 (0.6672) auc: 85.2941 (80.9234) time: 0.0295 data: 0.0001 max mem: 5308 | |
| [00:18:49.516073] Test: [270/560] eta: 0:00:09 loss: 0.6680 (0.6676) auc: 78.9683 (80.6967) time: 0.0295 data: 0.0001 max mem: 5308 | |
| [00:18:49.811347] Test: [280/560] eta: 0:00:09 loss: 0.6686 (0.6676) auc: 78.9683 (80.7285) time: 0.0295 data: 0.0001 max mem: 5308 | |
| [00:18:50.107257] Test: [290/560] eta: 0:00:09 loss: 0.6686 (0.6681) auc: 80.3571 (80.7003) time: 0.0295 data: 0.0001 max mem: 5308 | |
| [00:18:50.403149] Test: [300/560] eta: 0:00:08 loss: 0.6693 (0.6678) auc: 82.9960 (80.8058) time: 0.0295 data: 0.0001 max mem: 5308 | |
| [00:18:50.698366] Test: [310/560] eta: 0:00:08 loss: 0.6672 (0.6678) auc: 82.7451 (80.7535) time: 0.0295 data: 0.0001 max mem: 5308 | |
| [00:18:50.994423] Test: [320/560] eta: 0:00:08 loss: 0.6692 (0.6679) auc: 76.9841 (80.7093) time: 0.0295 data: 0.0001 max mem: 5308 | |
| [00:18:51.287278] Test: [330/560] eta: 0:00:07 loss: 0.6665 (0.6678) auc: 77.7083 (80.6883) time: 0.0294 data: 0.0001 max mem: 5308 | |
| [00:18:51.580685] Test: [340/560] eta: 0:00:07 loss: 0.6671 (0.6679) auc: 81.9608 (80.8021) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [00:18:51.873285] Test: [350/560] eta: 0:00:06 loss: 0.6654 (0.6677) auc: 85.1190 (80.8910) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:18:52.166221] Test: [360/560] eta: 0:00:06 loss: 0.6701 (0.6679) auc: 83.1349 (80.8605) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:18:52.459588] Test: [370/560] eta: 0:00:06 loss: 0.6742 (0.6681) auc: 79.9595 (80.8488) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:18:52.752341] Test: [380/560] eta: 0:00:05 loss: 0.6709 (0.6681) auc: 80.3922 (80.9071) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [00:18:53.044689] Test: [390/560] eta: 0:00:05 loss: 0.6762 (0.6683) auc: 77.7778 (80.7482) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:18:53.338479] Test: [400/560] eta: 0:00:05 loss: 0.6762 (0.6684) auc: 77.1255 (80.7119) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:18:53.630876] Test: [410/560] eta: 0:00:04 loss: 0.6733 (0.6684) auc: 76.1905 (80.5823) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:18:53.924019] Test: [420/560] eta: 0:00:04 loss: 0.6726 (0.6683) auc: 76.1719 (80.5809) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:18:54.216672] Test: [430/560] eta: 0:00:04 loss: 0.6609 (0.6682) auc: 80.8594 (80.6182) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:18:54.509161] Test: [440/560] eta: 0:00:03 loss: 0.6609 (0.6683) auc: 80.1932 (80.5704) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:18:54.802314] Test: [450/560] eta: 0:00:03 loss: 0.6699 (0.6683) auc: 79.3651 (80.5627) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:18:55.095265] Test: [460/560] eta: 0:00:03 loss: 0.6682 (0.6683) auc: 79.7571 (80.5176) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:18:55.389055] Test: [470/560] eta: 0:00:02 loss: 0.6690 (0.6683) auc: 79.3651 (80.4494) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [00:18:55.682219] Test: [480/560] eta: 0:00:02 loss: 0.6668 (0.6683) auc: 79.3522 (80.4269) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [00:18:55.976229] Test: [490/560] eta: 0:00:02 loss: 0.6645 (0.6683) auc: 81.5686 (80.4755) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [00:18:56.269202] Test: [500/560] eta: 0:00:01 loss: 0.6645 (0.6683) auc: 81.7814 (80.4898) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [00:18:56.562795] Test: [510/560] eta: 0:00:01 loss: 0.6654 (0.6684) auc: 82.0346 (80.4497) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [00:18:56.856295] Test: [520/560] eta: 0:00:01 loss: 0.6642 (0.6684) auc: 79.7619 (80.4525) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [00:18:57.149276] Test: [530/560] eta: 0:00:00 loss: 0.6773 (0.6687) auc: 77.7056 (80.3659) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [00:18:57.443038] Test: [540/560] eta: 0:00:00 loss: 0.6773 (0.6688) auc: 77.7056 (80.3787) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [00:18:57.732315] Test: [550/560] eta: 0:00:00 loss: 0.6687 (0.6689) auc: 84.7222 (80.4725) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:18:58.065617] Test: [559/560] eta: 0:00:00 loss: 0.6658 (0.6688) auc: 85.5159 (80.5176) time: 0.0325 data: 0.0001 max mem: 5308 | |
| [00:18:58.228000] Test: Total time: 0:00:18 (0.0322 s / it) | |
| [00:18:58.229571] * Auc 80.462 loss 0.669 | |
| [00:18:58.229814] AUC of the network on the 35796 val images: 80.46% | |
| [00:18:58.229856] Max auc: 80.46% | |
| [00:18:58.229889] Save model with min_val_loss at epoch: 0 | |
| [00:18:59.113485] log_dir: ./checkpoint/finetuned_models/FF++_c23_32frames | |
| [00:18:59.996545] Epoch: [1] [ 0/2877] eta: 0:42:17 lr: 0.000013 loss: 0.6833 (0.6833) time: 0.8819 data: 0.7462 max mem: 5308 | |
| [00:19:12.521249] Epoch: [1] [ 100/2877] eta: 0:06:08 lr: 0.000013 loss: 0.6868 (0.6877) time: 0.1254 data: 0.0001 max mem: 5308 | |
| [00:19:25.054578] Epoch: [1] [ 200/2877] eta: 0:05:45 lr: 0.000013 loss: 0.6849 (0.6860) time: 0.1253 data: 0.0001 max mem: 5308 | |
| [00:19:37.558585] Epoch: [1] [ 300/2877] eta: 0:05:29 lr: 0.000014 loss: 0.6855 (0.6861) time: 0.1246 data: 0.0001 max mem: 5308 | |
| [00:19:50.161205] Epoch: [1] [ 400/2877] eta: 0:05:15 lr: 0.000014 loss: 0.6858 (0.6863) time: 0.1269 data: 0.0002 max mem: 5308 | |
| [00:20:02.818130] Epoch: [1] [ 500/2877] eta: 0:05:02 lr: 0.000015 loss: 0.6854 (0.6862) time: 0.1271 data: 0.0002 max mem: 5308 | |
| [00:20:15.492771] Epoch: [1] [ 600/2877] eta: 0:04:49 lr: 0.000015 loss: 0.6831 (0.6861) time: 0.1262 data: 0.0001 max mem: 5308 | |
| [00:20:28.192790] Epoch: [1] [ 700/2877] eta: 0:04:36 lr: 0.000016 loss: 0.6785 (0.6856) time: 0.1267 data: 0.0001 max mem: 5308 | |
| [00:20:40.875019] Epoch: [1] [ 800/2877] eta: 0:04:23 lr: 0.000016 loss: 0.6807 (0.6855) time: 0.1265 data: 0.0001 max mem: 5308 | |
| [00:20:53.579096] Epoch: [1] [ 900/2877] eta: 0:04:11 lr: 0.000016 loss: 0.6776 (0.6851) time: 0.1273 data: 0.0001 max mem: 5308 | |
| [00:21:06.256113] Epoch: [1] [1000/2877] eta: 0:03:58 lr: 0.000017 loss: 0.6803 (0.6848) time: 0.1269 data: 0.0001 max mem: 5308 | |
| [00:21:18.917416] Epoch: [1] [1100/2877] eta: 0:03:45 lr: 0.000017 loss: 0.6775 (0.6846) time: 0.1269 data: 0.0001 max mem: 5308 | |
| [00:21:31.569934] Epoch: [1] [1200/2877] eta: 0:03:32 lr: 0.000018 loss: 0.6848 (0.6844) time: 0.1266 data: 0.0001 max mem: 5308 | |
| [00:21:44.052574] Epoch: [1] [1300/2877] eta: 0:03:19 lr: 0.000018 loss: 0.6839 (0.6843) time: 0.1252 data: 0.0001 max mem: 5308 | |
| [00:21:56.564054] Epoch: [1] [1400/2877] eta: 0:03:07 lr: 0.000019 loss: 0.6692 (0.6839) time: 0.1251 data: 0.0001 max mem: 5308 | |
| [00:22:09.219003] Epoch: [1] [1500/2877] eta: 0:02:54 lr: 0.000019 loss: 0.6901 (0.6836) time: 0.1267 data: 0.0001 max mem: 5308 | |
| [00:22:21.836430] Epoch: [1] [1600/2877] eta: 0:02:41 lr: 0.000019 loss: 0.6787 (0.6834) time: 0.1260 data: 0.0001 max mem: 5308 | |
| [00:22:34.428331] Epoch: [1] [1700/2877] eta: 0:02:28 lr: 0.000020 loss: 0.6732 (0.6832) time: 0.1262 data: 0.0001 max mem: 5308 | |
| [00:22:47.038348] Epoch: [1] [1800/2877] eta: 0:02:16 lr: 0.000020 loss: 0.6744 (0.6829) time: 0.1261 data: 0.0002 max mem: 5308 | |
| [00:22:59.640724] Epoch: [1] [1900/2877] eta: 0:02:03 lr: 0.000021 loss: 0.6760 (0.6827) time: 0.1265 data: 0.0001 max mem: 5308 | |
| [00:23:12.261777] Epoch: [1] [2000/2877] eta: 0:01:50 lr: 0.000021 loss: 0.6745 (0.6822) time: 0.1271 data: 0.0001 max mem: 5308 | |
| [00:23:24.919129] Epoch: [1] [2100/2877] eta: 0:01:38 lr: 0.000022 loss: 0.6872 (0.6821) time: 0.1266 data: 0.0001 max mem: 5308 | |
| [00:23:37.527688] Epoch: [1] [2200/2877] eta: 0:01:25 lr: 0.000022 loss: 0.6902 (0.6817) time: 0.1267 data: 0.0001 max mem: 5308 | |
| [00:23:50.171278] Epoch: [1] [2300/2877] eta: 0:01:12 lr: 0.000022 loss: 0.6730 (0.6813) time: 0.1259 data: 0.0001 max mem: 5308 | |
| [00:24:02.792101] Epoch: [1] [2400/2877] eta: 0:01:00 lr: 0.000023 loss: 0.6745 (0.6809) time: 0.1263 data: 0.0001 max mem: 5308 | |
| [00:24:15.307510] Epoch: [1] [2500/2877] eta: 0:00:47 lr: 0.000023 loss: 0.6791 (0.6807) time: 0.1246 data: 0.0001 max mem: 5308 | |
| [00:24:27.859600] Epoch: [1] [2600/2877] eta: 0:00:35 lr: 0.000024 loss: 0.6788 (0.6803) time: 0.1253 data: 0.0001 max mem: 5308 | |
| [00:24:40.378644] Epoch: [1] [2700/2877] eta: 0:00:22 lr: 0.000024 loss: 0.6632 (0.6798) time: 0.1250 data: 0.0001 max mem: 5308 | |
| [00:24:52.885982] Epoch: [1] [2800/2877] eta: 0:00:09 lr: 0.000025 loss: 0.6751 (0.6796) time: 0.1253 data: 0.0001 max mem: 5308 | |
| [00:25:02.388689] Epoch: [1] [2876/2877] eta: 0:00:00 lr: 0.000025 loss: 0.6872 (0.6794) time: 0.1247 data: 0.0002 max mem: 5308 | |
| [00:25:02.638960] Epoch: [1] Total time: 0:06:03 (0.1264 s / it) | |
| [00:25:02.640425] Averaged stats: lr: 0.000025 loss: 0.6872 (0.6792) | |
| [00:25:03.926542] Test: [ 0/560] eta: 0:11:57 loss: 0.6199 (0.6199) auc: 77.2549 (77.2549) time: 1.2808 data: 1.2474 max mem: 5308 | |
| [00:25:04.224669] Test: [ 10/560] eta: 0:01:18 loss: 0.5863 (0.5673) auc: 79.7571 (81.0819) time: 0.1434 data: 0.1135 max mem: 5308 | |
| [00:25:04.524959] Test: [ 20/560] eta: 0:00:48 loss: 0.5400 (0.5459) auc: 85.4167 (85.0816) time: 0.0298 data: 0.0002 max mem: 5308 | |
| [00:25:04.824908] Test: [ 30/560] eta: 0:00:37 loss: 0.5356 (0.5336) auc: 89.6104 (86.6654) time: 0.0299 data: 0.0002 max mem: 5308 | |
| [00:25:05.126139] Test: [ 40/560] eta: 0:00:31 loss: 0.5345 (0.5379) auc: 87.0588 (85.5144) time: 0.0300 data: 0.0002 max mem: 5308 | |
| [00:25:05.427360] Test: [ 50/560] eta: 0:00:27 loss: 0.5345 (0.5371) auc: 86.2500 (86.0696) time: 0.0300 data: 0.0002 max mem: 5308 | |
| [00:25:05.727130] Test: [ 60/560] eta: 0:00:25 loss: 0.5321 (0.5359) auc: 84.5703 (86.0086) time: 0.0299 data: 0.0002 max mem: 5308 | |
| [00:25:06.027724] Test: [ 70/560] eta: 0:00:23 loss: 0.5244 (0.5342) auc: 86.4372 (86.3364) time: 0.0299 data: 0.0002 max mem: 5308 | |
| [00:25:06.326076] Test: [ 80/560] eta: 0:00:21 loss: 0.5179 (0.5336) auc: 87.4510 (86.3074) time: 0.0299 data: 0.0002 max mem: 5308 | |
| [00:25:06.622826] Test: [ 90/560] eta: 0:00:20 loss: 0.5179 (0.5330) auc: 86.8182 (86.3460) time: 0.0297 data: 0.0001 max mem: 5308 | |
| [00:25:06.919051] Test: [100/560] eta: 0:00:19 loss: 0.5150 (0.5330) auc: 86.6667 (86.3862) time: 0.0296 data: 0.0002 max mem: 5308 | |
| [00:25:07.214541] Test: [110/560] eta: 0:00:18 loss: 0.5150 (0.5317) auc: 86.6667 (86.4107) time: 0.0295 data: 0.0001 max mem: 5308 | |
| [00:25:07.510425] Test: [120/560] eta: 0:00:17 loss: 0.5234 (0.5346) auc: 85.7488 (86.1502) time: 0.0295 data: 0.0001 max mem: 5308 | |
| [00:25:07.805902] Test: [130/560] eta: 0:00:16 loss: 0.5421 (0.5351) auc: 85.8824 (86.1065) time: 0.0295 data: 0.0001 max mem: 5308 | |
| [00:25:08.101976] Test: [140/560] eta: 0:00:16 loss: 0.5461 (0.5368) auc: 85.9375 (85.9855) time: 0.0295 data: 0.0001 max mem: 5308 | |
| [00:25:08.397848] Test: [150/560] eta: 0:00:15 loss: 0.5461 (0.5370) auc: 85.7143 (85.9060) time: 0.0295 data: 0.0001 max mem: 5308 | |
| [00:25:08.693736] Test: [160/560] eta: 0:00:14 loss: 0.5056 (0.5359) auc: 87.8431 (86.0864) time: 0.0295 data: 0.0002 max mem: 5308 | |
| [00:25:08.988819] Test: [170/560] eta: 0:00:14 loss: 0.5083 (0.5358) auc: 89.0625 (86.1361) time: 0.0295 data: 0.0001 max mem: 5308 | |
| [00:25:09.283833] Test: [180/560] eta: 0:00:13 loss: 0.5421 (0.5372) auc: 85.0202 (86.0175) time: 0.0294 data: 0.0001 max mem: 5308 | |
| [00:25:09.579600] Test: [190/560] eta: 0:00:13 loss: 0.5465 (0.5377) auc: 85.4902 (85.9749) time: 0.0295 data: 0.0001 max mem: 5308 | |
| [00:25:09.875145] Test: [200/560] eta: 0:00:12 loss: 0.5329 (0.5368) auc: 86.9048 (86.0595) time: 0.0295 data: 0.0001 max mem: 5308 | |
| [00:25:10.171149] Test: [210/560] eta: 0:00:12 loss: 0.5329 (0.5370) auc: 85.9375 (86.0292) time: 0.0295 data: 0.0001 max mem: 5308 | |
| [00:25:10.464874] Test: [220/560] eta: 0:00:12 loss: 0.5339 (0.5369) auc: 86.2745 (86.0260) time: 0.0294 data: 0.0001 max mem: 5308 | |
| [00:25:10.760141] Test: [230/560] eta: 0:00:11 loss: 0.5339 (0.5367) auc: 87.4494 (86.0625) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [00:25:11.053293] Test: [240/560] eta: 0:00:11 loss: 0.5358 (0.5365) auc: 87.6984 (86.1192) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [00:25:11.345776] Test: [250/560] eta: 0:00:10 loss: 0.5170 (0.5357) auc: 87.4510 (86.2013) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:25:11.638910] Test: [260/560] eta: 0:00:10 loss: 0.5075 (0.5342) auc: 88.4921 (86.3637) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:25:11.932536] Test: [270/560] eta: 0:00:09 loss: 0.5216 (0.5353) auc: 85.8824 (86.2147) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [00:25:12.225864] Test: [280/560] eta: 0:00:09 loss: 0.5390 (0.5350) auc: 87.4494 (86.3088) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [00:25:12.518589] Test: [290/560] eta: 0:00:09 loss: 0.5389 (0.5358) auc: 87.3016 (86.2926) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:25:12.811552] Test: [300/560] eta: 0:00:08 loss: 0.5389 (0.5347) auc: 87.1094 (86.4196) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:25:13.103998] Test: [310/560] eta: 0:00:08 loss: 0.5244 (0.5353) auc: 87.4510 (86.3325) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:25:13.397238] Test: [320/560] eta: 0:00:08 loss: 0.5555 (0.5357) auc: 82.7451 (86.3011) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:25:13.690036] Test: [330/560] eta: 0:00:07 loss: 0.5249 (0.5353) auc: 86.1111 (86.3852) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:25:13.982909] Test: [340/560] eta: 0:00:07 loss: 0.5117 (0.5347) auc: 89.0688 (86.4461) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:25:14.275736] Test: [350/560] eta: 0:00:06 loss: 0.5117 (0.5342) auc: 90.0794 (86.5140) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:25:14.568707] Test: [360/560] eta: 0:00:06 loss: 0.5313 (0.5349) auc: 87.0588 (86.4405) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:25:14.862176] Test: [370/560] eta: 0:00:06 loss: 0.5425 (0.5353) auc: 85.7143 (86.4359) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [00:25:15.155285] Test: [380/560] eta: 0:00:05 loss: 0.5366 (0.5350) auc: 87.5000 (86.5020) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [00:25:15.448506] Test: [390/560] eta: 0:00:05 loss: 0.5415 (0.5357) auc: 85.5469 (86.3943) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [00:25:15.742508] Test: [400/560] eta: 0:00:05 loss: 0.5587 (0.5363) auc: 81.3765 (86.3117) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [00:25:16.036011] Test: [410/560] eta: 0:00:04 loss: 0.5478 (0.5367) auc: 80.4688 (86.2255) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [00:25:16.328854] Test: [420/560] eta: 0:00:04 loss: 0.5345 (0.5367) auc: 85.7143 (86.2285) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [00:25:16.621195] Test: [430/560] eta: 0:00:04 loss: 0.5312 (0.5366) auc: 87.1094 (86.2376) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:25:16.914716] Test: [440/560] eta: 0:00:03 loss: 0.5059 (0.5365) auc: 87.3016 (86.2549) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:25:17.207782] Test: [450/560] eta: 0:00:03 loss: 0.5143 (0.5365) auc: 86.7188 (86.2521) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [00:25:17.501158] Test: [460/560] eta: 0:00:03 loss: 0.5445 (0.5369) auc: 86.6397 (86.2055) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:25:17.796264] Test: [470/560] eta: 0:00:02 loss: 0.5445 (0.5371) auc: 86.6397 (86.1717) time: 0.0294 data: 0.0001 max mem: 5308 | |
| [00:25:18.091291] Test: [480/560] eta: 0:00:02 loss: 0.5358 (0.5372) auc: 87.0588 (86.1623) time: 0.0294 data: 0.0001 max mem: 5308 | |
| [00:25:18.386693] Test: [490/560] eta: 0:00:02 loss: 0.5359 (0.5370) auc: 86.2500 (86.1819) time: 0.0295 data: 0.0001 max mem: 5308 | |
| [00:25:18.681731] Test: [500/560] eta: 0:00:01 loss: 0.5387 (0.5368) auc: 86.2500 (86.2110) time: 0.0295 data: 0.0001 max mem: 5308 | |
| [00:25:18.977551] Test: [510/560] eta: 0:00:01 loss: 0.5268 (0.5370) auc: 86.8627 (86.1983) time: 0.0295 data: 0.0001 max mem: 5308 | |
| [00:25:19.272466] Test: [520/560] eta: 0:00:01 loss: 0.5404 (0.5372) auc: 85.3175 (86.1499) time: 0.0295 data: 0.0001 max mem: 5308 | |
| [00:25:19.567738] Test: [530/560] eta: 0:00:00 loss: 0.5661 (0.5379) auc: 83.3333 (86.0771) time: 0.0294 data: 0.0001 max mem: 5308 | |
| [00:25:19.863840] Test: [540/560] eta: 0:00:00 loss: 0.5661 (0.5380) auc: 84.3137 (86.0803) time: 0.0295 data: 0.0001 max mem: 5308 | |
| [00:25:20.156326] Test: [550/560] eta: 0:00:00 loss: 0.5323 (0.5378) auc: 90.6883 (86.1869) time: 0.0294 data: 0.0001 max mem: 5308 | |
| [00:25:20.401921] Test: [559/560] eta: 0:00:00 loss: 0.5252 (0.5374) auc: 90.4545 (86.1971) time: 0.0283 data: 0.0001 max mem: 5308 | |
| [00:25:20.578828] Test: Total time: 0:00:17 (0.0320 s / it) | |
| [00:25:20.580980] * Auc 86.264 loss 0.538 | |
| [00:25:20.581359] AUC of the network on the 35796 val images: 86.26% | |
| [00:25:20.581399] Max auc: 86.26% | |
| [00:25:20.581438] Save model with min_val_loss at epoch: 1 | |
| [00:25:26.449187] log_dir: ./checkpoint/finetuned_models/FF++_c23_32frames | |
| [00:25:27.448892] Epoch: [2] [ 0/2877] eta: 0:47:50 lr: 0.000025 loss: 0.7165 (0.7165) time: 0.9978 data: 0.8690 max mem: 5308 | |
| [00:25:40.035421] Epoch: [2] [ 100/2877] eta: 0:06:13 lr: 0.000025 loss: 0.6604 (0.6752) time: 0.1252 data: 0.0001 max mem: 5308 | |
| [00:25:52.554871] Epoch: [2] [ 200/2877] eta: 0:05:47 lr: 0.000026 loss: 0.6904 (0.6746) time: 0.1254 data: 0.0001 max mem: 5308 | |
| [00:26:05.078046] Epoch: [2] [ 300/2877] eta: 0:05:30 lr: 0.000026 loss: 0.6594 (0.6719) time: 0.1256 data: 0.0001 max mem: 5308 | |
| [00:26:17.653242] Epoch: [2] [ 400/2877] eta: 0:05:16 lr: 0.000027 loss: 0.6765 (0.6732) time: 0.1250 data: 0.0001 max mem: 5308 | |
| [00:26:30.297838] Epoch: [2] [ 500/2877] eta: 0:05:02 lr: 0.000027 loss: 0.6765 (0.6721) time: 0.1255 data: 0.0001 max mem: 5308 | |
| [00:26:42.923532] Epoch: [2] [ 600/2877] eta: 0:04:49 lr: 0.000028 loss: 0.6658 (0.6726) time: 0.1260 data: 0.0001 max mem: 5308 | |
| [00:26:55.518100] Epoch: [2] [ 700/2877] eta: 0:04:36 lr: 0.000028 loss: 0.6737 (0.6726) time: 0.1258 data: 0.0001 max mem: 5308 | |
| [00:27:08.084048] Epoch: [2] [ 800/2877] eta: 0:04:23 lr: 0.000028 loss: 0.6704 (0.6724) time: 0.1254 data: 0.0001 max mem: 5308 | |
| [00:27:20.604327] Epoch: [2] [ 900/2877] eta: 0:04:10 lr: 0.000029 loss: 0.6576 (0.6718) time: 0.1246 data: 0.0001 max mem: 5308 | |
| [00:27:33.088180] Epoch: [2] [1000/2877] eta: 0:03:57 lr: 0.000029 loss: 0.6528 (0.6716) time: 0.1248 data: 0.0001 max mem: 5308 | |
| [00:27:45.575711] Epoch: [2] [1100/2877] eta: 0:03:44 lr: 0.000030 loss: 0.6631 (0.6712) time: 0.1245 data: 0.0001 max mem: 5308 | |
| [00:27:58.088422] Epoch: [2] [1200/2877] eta: 0:03:31 lr: 0.000030 loss: 0.6615 (0.6704) time: 0.1250 data: 0.0001 max mem: 5308 | |
| [00:28:10.648903] Epoch: [2] [1300/2877] eta: 0:03:19 lr: 0.000031 loss: 0.6593 (0.6699) time: 0.1255 data: 0.0001 max mem: 5308 | |
| [00:28:23.216740] Epoch: [2] [1400/2877] eta: 0:03:06 lr: 0.000031 loss: 0.6724 (0.6699) time: 0.1252 data: 0.0001 max mem: 5308 | |
| [00:28:35.787925] Epoch: [2] [1500/2877] eta: 0:02:53 lr: 0.000032 loss: 0.6577 (0.6696) time: 0.1260 data: 0.0001 max mem: 5308 | |
| [00:28:48.282810] Epoch: [2] [1600/2877] eta: 0:02:40 lr: 0.000032 loss: 0.6637 (0.6692) time: 0.1250 data: 0.0001 max mem: 5308 | |
| [00:29:00.789699] Epoch: [2] [1700/2877] eta: 0:02:28 lr: 0.000032 loss: 0.6608 (0.6687) time: 0.1251 data: 0.0001 max mem: 5308 | |
| [00:29:13.333240] Epoch: [2] [1800/2877] eta: 0:02:15 lr: 0.000033 loss: 0.6728 (0.6684) time: 0.1253 data: 0.0001 max mem: 5308 | |
| [00:29:25.887531] Epoch: [2] [1900/2877] eta: 0:02:03 lr: 0.000033 loss: 0.6593 (0.6683) time: 0.1262 data: 0.0001 max mem: 5308 | |
| [00:29:38.444183] Epoch: [2] [2000/2877] eta: 0:01:50 lr: 0.000034 loss: 0.6531 (0.6680) time: 0.1255 data: 0.0001 max mem: 5308 | |
| [00:29:50.980335] Epoch: [2] [2100/2877] eta: 0:01:37 lr: 0.000034 loss: 0.6766 (0.6679) time: 0.1249 data: 0.0001 max mem: 5308 | |
| [00:30:03.520585] Epoch: [2] [2200/2877] eta: 0:01:25 lr: 0.000035 loss: 0.6689 (0.6678) time: 0.1248 data: 0.0001 max mem: 5308 | |
| [00:30:16.030366] Epoch: [2] [2300/2877] eta: 0:01:12 lr: 0.000035 loss: 0.6509 (0.6675) time: 0.1251 data: 0.0001 max mem: 5308 | |
| [00:30:28.524183] Epoch: [2] [2400/2877] eta: 0:01:00 lr: 0.000035 loss: 0.6481 (0.6673) time: 0.1250 data: 0.0001 max mem: 5308 | |
| [00:30:41.039094] Epoch: [2] [2500/2877] eta: 0:00:47 lr: 0.000036 loss: 0.6668 (0.6670) time: 0.1251 data: 0.0001 max mem: 5308 | |
| [00:30:53.560242] Epoch: [2] [2600/2877] eta: 0:00:34 lr: 0.000036 loss: 0.6613 (0.6667) time: 0.1252 data: 0.0001 max mem: 5308 | |
| [00:31:06.040554] Epoch: [2] [2700/2877] eta: 0:00:22 lr: 0.000037 loss: 0.6785 (0.6669) time: 0.1245 data: 0.0001 max mem: 5308 | |
| [00:31:18.723489] Epoch: [2] [2800/2877] eta: 0:00:09 lr: 0.000037 loss: 0.6531 (0.6667) time: 0.1280 data: 0.0001 max mem: 5308 | |
| [00:31:28.285828] Epoch: [2] [2876/2877] eta: 0:00:00 lr: 0.000037 loss: 0.6555 (0.6665) time: 0.1242 data: 0.0002 max mem: 5308 | |
| [00:31:28.470542] Epoch: [2] Total time: 0:06:02 (0.1258 s / it) | |
| [00:31:28.496365] Averaged stats: lr: 0.000037 loss: 0.6555 (0.6670) | |
| [00:31:29.803145] Test: [ 0/560] eta: 0:12:09 loss: 0.5531 (0.5531) auc: 84.3137 (84.3137) time: 1.3035 data: 1.2707 max mem: 5308 | |
| [00:31:30.102323] Test: [ 10/560] eta: 0:01:20 loss: 0.5306 (0.5092) auc: 84.6154 (85.2935) time: 0.1456 data: 0.1157 max mem: 5308 | |
| [00:31:30.396819] Test: [ 20/560] eta: 0:00:48 loss: 0.4709 (0.4845) auc: 88.4921 (88.1831) time: 0.0296 data: 0.0001 max mem: 5308 | |
| [00:31:30.691770] Test: [ 30/560] eta: 0:00:37 loss: 0.4340 (0.4645) auc: 92.1569 (89.8603) time: 0.0294 data: 0.0001 max mem: 5308 | |
| [00:31:30.987408] Test: [ 40/560] eta: 0:00:31 loss: 0.4404 (0.4692) auc: 92.1569 (89.1604) time: 0.0295 data: 0.0001 max mem: 5308 | |
| [00:31:31.282142] Test: [ 50/560] eta: 0:00:27 loss: 0.4447 (0.4692) auc: 89.0196 (89.4849) time: 0.0295 data: 0.0001 max mem: 5308 | |
| [00:31:31.577102] Test: [ 60/560] eta: 0:00:25 loss: 0.4571 (0.4679) auc: 89.2857 (89.4706) time: 0.0294 data: 0.0001 max mem: 5308 | |
| [00:31:31.872930] Test: [ 70/560] eta: 0:00:23 loss: 0.4538 (0.4656) auc: 90.4545 (89.7818) time: 0.0295 data: 0.0001 max mem: 5308 | |
| [00:31:32.167975] Test: [ 80/560] eta: 0:00:21 loss: 0.4326 (0.4630) auc: 90.6250 (90.0154) time: 0.0295 data: 0.0001 max mem: 5308 | |
| [00:31:32.463464] Test: [ 90/560] eta: 0:00:20 loss: 0.4345 (0.4628) auc: 90.9091 (90.1639) time: 0.0295 data: 0.0001 max mem: 5308 | |
| [00:31:32.757848] Test: [100/560] eta: 0:00:19 loss: 0.4393 (0.4615) auc: 92.1569 (90.3776) time: 0.0294 data: 0.0001 max mem: 5308 | |
| [00:31:33.053549] Test: [110/560] eta: 0:00:18 loss: 0.4393 (0.4594) auc: 91.6667 (90.4877) time: 0.0294 data: 0.0001 max mem: 5308 | |
| [00:31:33.348999] Test: [120/560] eta: 0:00:17 loss: 0.4542 (0.4640) auc: 89.3720 (90.1903) time: 0.0295 data: 0.0001 max mem: 5308 | |
| [00:31:33.644358] Test: [130/560] eta: 0:00:16 loss: 0.4643 (0.4642) auc: 89.0873 (90.2240) time: 0.0295 data: 0.0001 max mem: 5308 | |
| [00:31:33.939380] Test: [140/560] eta: 0:00:16 loss: 0.4643 (0.4665) auc: 89.4531 (90.1040) time: 0.0295 data: 0.0001 max mem: 5308 | |
| [00:31:34.231676] Test: [150/560] eta: 0:00:15 loss: 0.4592 (0.4659) auc: 89.4531 (90.0881) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [00:31:34.524205] Test: [160/560] eta: 0:00:14 loss: 0.4181 (0.4635) auc: 92.9167 (90.3370) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:31:34.816383] Test: [170/560] eta: 0:00:14 loss: 0.4133 (0.4626) auc: 92.9688 (90.4135) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:31:35.109258] Test: [180/560] eta: 0:00:13 loss: 0.4463 (0.4639) auc: 91.9028 (90.4030) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:31:35.401890] Test: [190/560] eta: 0:00:13 loss: 0.4716 (0.4649) auc: 89.4118 (90.3332) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:31:35.693768] Test: [200/560] eta: 0:00:12 loss: 0.4675 (0.4636) auc: 89.7917 (90.4004) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:31:35.986795] Test: [210/560] eta: 0:00:12 loss: 0.4640 (0.4640) auc: 90.5882 (90.3557) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:31:36.279320] Test: [220/560] eta: 0:00:11 loss: 0.4640 (0.4636) auc: 89.8039 (90.3635) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:31:36.571708] Test: [230/560] eta: 0:00:11 loss: 0.4636 (0.4636) auc: 90.8730 (90.3789) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:31:36.864387] Test: [240/560] eta: 0:00:11 loss: 0.4665 (0.4633) auc: 90.8730 (90.4067) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:31:37.156512] Test: [250/560] eta: 0:00:10 loss: 0.4435 (0.4629) auc: 90.0794 (90.3966) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:31:37.450344] Test: [260/560] eta: 0:00:10 loss: 0.4049 (0.4607) auc: 94.7917 (90.5515) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:31:37.744442] Test: [270/560] eta: 0:00:09 loss: 0.4417 (0.4617) auc: 91.7647 (90.4599) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [00:31:38.040672] Test: [280/560] eta: 0:00:09 loss: 0.4523 (0.4609) auc: 90.8730 (90.5731) time: 0.0294 data: 0.0001 max mem: 5308 | |
| [00:31:38.335743] Test: [290/560] eta: 0:00:09 loss: 0.4692 (0.4622) auc: 90.8333 (90.5487) time: 0.0295 data: 0.0001 max mem: 5308 | |
| [00:31:38.630906] Test: [300/560] eta: 0:00:08 loss: 0.4692 (0.4610) auc: 90.8333 (90.6489) time: 0.0294 data: 0.0001 max mem: 5308 | |
| [00:31:38.926568] Test: [310/560] eta: 0:00:08 loss: 0.4482 (0.4614) auc: 92.9688 (90.5925) time: 0.0295 data: 0.0001 max mem: 5308 | |
| [00:31:39.223376] Test: [320/560] eta: 0:00:08 loss: 0.4773 (0.4619) auc: 89.0196 (90.5660) time: 0.0295 data: 0.0001 max mem: 5308 | |
| [00:31:39.519709] Test: [330/560] eta: 0:00:07 loss: 0.4512 (0.4615) auc: 90.4762 (90.6389) time: 0.0296 data: 0.0002 max mem: 5308 | |
| [00:31:39.818975] Test: [340/560] eta: 0:00:07 loss: 0.4437 (0.4608) auc: 93.3333 (90.6772) time: 0.0297 data: 0.0002 max mem: 5308 | |
| [00:31:40.114867] Test: [350/560] eta: 0:00:06 loss: 0.4366 (0.4602) auc: 92.5490 (90.7098) time: 0.0297 data: 0.0001 max mem: 5308 | |
| [00:31:40.407664] Test: [360/560] eta: 0:00:06 loss: 0.4639 (0.4611) auc: 90.1961 (90.6646) time: 0.0294 data: 0.0001 max mem: 5308 | |
| [00:31:40.700373] Test: [370/560] eta: 0:00:06 loss: 0.4716 (0.4615) auc: 90.1961 (90.6945) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:31:40.993323] Test: [380/560] eta: 0:00:05 loss: 0.4561 (0.4614) auc: 90.0810 (90.7008) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:31:41.286752] Test: [390/560] eta: 0:00:05 loss: 0.4660 (0.4622) auc: 89.2157 (90.6406) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [00:31:41.579830] Test: [400/560] eta: 0:00:05 loss: 0.4820 (0.4631) auc: 88.3333 (90.5659) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [00:31:41.872147] Test: [410/560] eta: 0:00:04 loss: 0.4800 (0.4635) auc: 85.4902 (90.4969) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:31:42.165055] Test: [420/560] eta: 0:00:04 loss: 0.4650 (0.4634) auc: 88.0952 (90.5133) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:31:42.457554] Test: [430/560] eta: 0:00:04 loss: 0.4337 (0.4628) auc: 90.2344 (90.5516) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:31:42.750847] Test: [440/560] eta: 0:00:03 loss: 0.4285 (0.4625) auc: 92.9412 (90.5902) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:31:43.043691] Test: [450/560] eta: 0:00:03 loss: 0.4416 (0.4626) auc: 92.5781 (90.5940) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:31:43.336653] Test: [460/560] eta: 0:00:03 loss: 0.4745 (0.4634) auc: 91.4062 (90.5519) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:31:43.629045] Test: [470/560] eta: 0:00:02 loss: 0.4949 (0.4636) auc: 91.2500 (90.5349) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:31:43.921138] Test: [480/560] eta: 0:00:02 loss: 0.4733 (0.4636) auc: 90.9804 (90.5347) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:31:44.213613] Test: [490/560] eta: 0:00:02 loss: 0.4581 (0.4632) auc: 90.8333 (90.5474) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:31:44.506931] Test: [500/560] eta: 0:00:01 loss: 0.4391 (0.4628) auc: 91.2698 (90.5832) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:31:44.800145] Test: [510/560] eta: 0:00:01 loss: 0.4369 (0.4629) auc: 91.0931 (90.5886) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [00:31:45.092936] Test: [520/560] eta: 0:00:01 loss: 0.4962 (0.4635) auc: 88.4921 (90.5208) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:31:45.385507] Test: [530/560] eta: 0:00:00 loss: 0.4980 (0.4645) auc: 86.7188 (90.4640) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:31:45.678289] Test: [540/560] eta: 0:00:00 loss: 0.4834 (0.4649) auc: 90.8730 (90.4612) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:31:45.968560] Test: [550/560] eta: 0:00:00 loss: 0.4619 (0.4647) auc: 93.7500 (90.5577) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:31:46.211312] Test: [559/560] eta: 0:00:00 loss: 0.4506 (0.4643) auc: 93.1174 (90.5584) time: 0.0281 data: 0.0001 max mem: 5308 | |
| [00:31:46.355755] Test: Total time: 0:00:17 (0.0319 s / it) | |
| [00:31:46.695343] * Auc 90.655 loss 0.465 | |
| [00:31:46.695480] AUC of the network on the 35796 val images: 90.66% | |
| [00:31:46.695490] Max auc: 90.66% | |
| [00:31:46.695500] Save model with min_val_loss at epoch: 2 | |
| [00:31:52.250080] log_dir: ./checkpoint/finetuned_models/FF++_c23_32frames | |
| [00:31:53.168522] Epoch: [3] [ 0/2877] eta: 0:43:58 lr: 0.000038 loss: 0.6943 (0.6943) time: 0.9172 data: 0.7895 max mem: 5308 | |
| [00:32:05.815382] Epoch: [3] [ 100/2877] eta: 0:06:12 lr: 0.000038 loss: 0.6621 (0.6595) time: 0.1264 data: 0.0001 max mem: 5308 | |
| [00:32:18.386325] Epoch: [3] [ 200/2877] eta: 0:05:48 lr: 0.000038 loss: 0.6638 (0.6616) time: 0.1257 data: 0.0001 max mem: 5308 | |
| [00:32:30.942019] Epoch: [3] [ 300/2877] eta: 0:05:31 lr: 0.000039 loss: 0.6623 (0.6614) time: 0.1255 data: 0.0001 max mem: 5308 | |
| [00:32:43.457030] Epoch: [3] [ 400/2877] eta: 0:05:16 lr: 0.000039 loss: 0.6630 (0.6607) time: 0.1250 data: 0.0001 max mem: 5308 | |
| [00:32:56.052847] Epoch: [3] [ 500/2877] eta: 0:05:02 lr: 0.000040 loss: 0.6543 (0.6599) time: 0.1256 data: 0.0001 max mem: 5308 | |
| [00:33:08.618776] Epoch: [3] [ 600/2877] eta: 0:04:49 lr: 0.000040 loss: 0.6741 (0.6592) time: 0.1251 data: 0.0001 max mem: 5308 | |
| [00:33:21.159612] Epoch: [3] [ 700/2877] eta: 0:04:36 lr: 0.000041 loss: 0.6428 (0.6581) time: 0.1254 data: 0.0001 max mem: 5308 | |
| [00:33:33.696632] Epoch: [3] [ 800/2877] eta: 0:04:23 lr: 0.000041 loss: 0.6669 (0.6593) time: 0.1249 data: 0.0001 max mem: 5308 | |
| [00:33:46.234910] Epoch: [3] [ 900/2877] eta: 0:04:10 lr: 0.000041 loss: 0.6650 (0.6594) time: 0.1248 data: 0.0001 max mem: 5308 | |
| [00:33:58.744093] Epoch: [3] [1000/2877] eta: 0:03:57 lr: 0.000042 loss: 0.6498 (0.6593) time: 0.1256 data: 0.0001 max mem: 5308 | |
| [00:34:11.262006] Epoch: [3] [1100/2877] eta: 0:03:44 lr: 0.000042 loss: 0.6678 (0.6601) time: 0.1259 data: 0.0001 max mem: 5308 | |
| [00:34:23.860538] Epoch: [3] [1200/2877] eta: 0:03:31 lr: 0.000043 loss: 0.6594 (0.6600) time: 0.1259 data: 0.0001 max mem: 5308 | |
| [00:34:36.413083] Epoch: [3] [1300/2877] eta: 0:03:18 lr: 0.000043 loss: 0.6500 (0.6595) time: 0.1249 data: 0.0001 max mem: 5308 | |
| [00:34:48.914768] Epoch: [3] [1400/2877] eta: 0:03:06 lr: 0.000044 loss: 0.6544 (0.6592) time: 0.1251 data: 0.0001 max mem: 5308 | |
| [00:35:01.494580] Epoch: [3] [1500/2877] eta: 0:02:53 lr: 0.000044 loss: 0.6399 (0.6588) time: 0.1265 data: 0.0001 max mem: 5308 | |
| [00:35:14.073074] Epoch: [3] [1600/2877] eta: 0:02:40 lr: 0.000044 loss: 0.6411 (0.6585) time: 0.1259 data: 0.0001 max mem: 5308 | |
| [00:35:26.644698] Epoch: [3] [1700/2877] eta: 0:02:28 lr: 0.000045 loss: 0.6385 (0.6583) time: 0.1256 data: 0.0001 max mem: 5308 | |
| [00:35:39.212622] Epoch: [3] [1800/2877] eta: 0:02:15 lr: 0.000045 loss: 0.6388 (0.6580) time: 0.1253 data: 0.0001 max mem: 5308 | |
| [00:35:51.756785] Epoch: [3] [1900/2877] eta: 0:02:03 lr: 0.000046 loss: 0.6377 (0.6577) time: 0.1253 data: 0.0001 max mem: 5308 | |
| [00:36:04.244501] Epoch: [3] [2000/2877] eta: 0:01:50 lr: 0.000046 loss: 0.6447 (0.6573) time: 0.1248 data: 0.0001 max mem: 5308 | |
| [00:36:16.731907] Epoch: [3] [2100/2877] eta: 0:01:37 lr: 0.000047 loss: 0.6652 (0.6572) time: 0.1248 data: 0.0001 max mem: 5308 | |
| [00:36:29.312774] Epoch: [3] [2200/2877] eta: 0:01:25 lr: 0.000047 loss: 0.6440 (0.6570) time: 0.1264 data: 0.0001 max mem: 5308 | |
| [00:36:41.881269] Epoch: [3] [2300/2877] eta: 0:01:12 lr: 0.000047 loss: 0.6484 (0.6567) time: 0.1253 data: 0.0001 max mem: 5308 | |
| [00:36:54.399780] Epoch: [3] [2400/2877] eta: 0:01:00 lr: 0.000048 loss: 0.6542 (0.6568) time: 0.1252 data: 0.0001 max mem: 5308 | |
| [00:37:06.942431] Epoch: [3] [2500/2877] eta: 0:00:47 lr: 0.000048 loss: 0.6394 (0.6564) time: 0.1257 data: 0.0001 max mem: 5308 | |
| [00:37:19.464025] Epoch: [3] [2600/2877] eta: 0:00:34 lr: 0.000049 loss: 0.6660 (0.6563) time: 0.1251 data: 0.0001 max mem: 5308 | |
| [00:37:32.022485] Epoch: [3] [2700/2877] eta: 0:00:22 lr: 0.000049 loss: 0.6547 (0.6562) time: 0.1266 data: 0.0001 max mem: 5308 | |
| [00:37:44.582786] Epoch: [3] [2800/2877] eta: 0:00:09 lr: 0.000050 loss: 0.6472 (0.6562) time: 0.1264 data: 0.0001 max mem: 5308 | |
| [00:37:54.107408] Epoch: [3] [2876/2877] eta: 0:00:00 lr: 0.000050 loss: 0.6689 (0.6563) time: 0.1244 data: 0.0002 max mem: 5308 | |
| [00:37:54.331850] Epoch: [3] Total time: 0:06:02 (0.1259 s / it) | |
| [00:37:54.358459] Averaged stats: lr: 0.000050 loss: 0.6689 (0.6556) | |
| [00:37:55.702486] Test: [ 0/560] eta: 0:12:30 loss: 0.4656 (0.4656) auc: 89.8039 (89.8039) time: 1.3408 data: 1.3065 max mem: 5308 | |
| [00:37:55.997421] Test: [ 10/560] eta: 0:01:21 loss: 0.4633 (0.4501) auc: 89.4737 (89.1802) time: 0.1486 data: 0.1189 max mem: 5308 | |
| [00:37:56.292199] Test: [ 20/560] eta: 0:00:49 loss: 0.4258 (0.4267) auc: 90.8730 (91.0200) time: 0.0294 data: 0.0001 max mem: 5308 | |
| [00:37:56.586936] Test: [ 30/560] eta: 0:00:38 loss: 0.3888 (0.4056) auc: 93.5065 (92.1459) time: 0.0294 data: 0.0001 max mem: 5308 | |
| [00:37:56.881619] Test: [ 40/560] eta: 0:00:31 loss: 0.3762 (0.4098) auc: 92.9412 (91.6432) time: 0.0294 data: 0.0001 max mem: 5308 | |
| [00:37:57.176002] Test: [ 50/560] eta: 0:00:28 loss: 0.3905 (0.4110) auc: 90.5882 (91.7305) time: 0.0294 data: 0.0001 max mem: 5308 | |
| [00:37:57.470092] Test: [ 60/560] eta: 0:00:25 loss: 0.4165 (0.4106) auc: 90.8730 (91.6471) time: 0.0294 data: 0.0001 max mem: 5308 | |
| [00:37:57.764574] Test: [ 70/560] eta: 0:00:23 loss: 0.4148 (0.4081) auc: 91.2500 (91.7978) time: 0.0294 data: 0.0001 max mem: 5308 | |
| [00:37:58.057866] Test: [ 80/560] eta: 0:00:21 loss: 0.3648 (0.4042) auc: 92.7083 (92.1027) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [00:37:58.349930] Test: [ 90/560] eta: 0:00:20 loss: 0.3535 (0.4040) auc: 93.9394 (92.3078) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:37:58.642623] Test: [100/560] eta: 0:00:19 loss: 0.3846 (0.4021) auc: 94.1176 (92.4474) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:37:58.935050] Test: [110/560] eta: 0:00:18 loss: 0.3763 (0.4003) auc: 94.1176 (92.4311) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:37:59.227338] Test: [120/560] eta: 0:00:17 loss: 0.3895 (0.4054) auc: 90.9804 (92.2035) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:37:59.520240] Test: [130/560] eta: 0:00:16 loss: 0.4027 (0.4055) auc: 92.7451 (92.2564) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:37:59.812982] Test: [140/560] eta: 0:00:16 loss: 0.4011 (0.4073) auc: 92.8571 (92.2185) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:38:00.104924] Test: [150/560] eta: 0:00:15 loss: 0.4011 (0.4064) auc: 91.4062 (92.2495) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:38:00.397617] Test: [160/560] eta: 0:00:14 loss: 0.3548 (0.4028) auc: 94.7917 (92.4752) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:38:00.689656] Test: [170/560] eta: 0:00:14 loss: 0.3475 (0.4015) auc: 94.5312 (92.5682) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:38:00.982111] Test: [180/560] eta: 0:00:13 loss: 0.3812 (0.4022) auc: 94.3320 (92.5973) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:38:01.274218] Test: [190/560] eta: 0:00:13 loss: 0.4044 (0.4032) auc: 92.0635 (92.5135) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:38:01.566416] Test: [200/560] eta: 0:00:12 loss: 0.4016 (0.4014) auc: 92.9412 (92.6151) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:38:01.858323] Test: [210/560] eta: 0:00:12 loss: 0.4014 (0.4020) auc: 92.9167 (92.5588) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:38:02.151288] Test: [220/560] eta: 0:00:11 loss: 0.4014 (0.4015) auc: 92.7126 (92.5921) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:38:02.443335] Test: [230/560] eta: 0:00:11 loss: 0.3993 (0.4017) auc: 93.3333 (92.6013) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:38:02.735607] Test: [240/560] eta: 0:00:11 loss: 0.3970 (0.4012) auc: 92.8571 (92.6410) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:38:03.027554] Test: [250/560] eta: 0:00:10 loss: 0.3769 (0.4007) auc: 92.4603 (92.6418) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:38:03.319758] Test: [260/560] eta: 0:00:10 loss: 0.3375 (0.3984) auc: 95.6863 (92.7598) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:38:03.614864] Test: [270/560] eta: 0:00:09 loss: 0.3788 (0.3992) auc: 94.0476 (92.7416) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [00:38:03.909147] Test: [280/560] eta: 0:00:09 loss: 0.3893 (0.3977) auc: 95.2381 (92.8583) time: 0.0294 data: 0.0001 max mem: 5308 | |
| [00:38:04.204377] Test: [290/560] eta: 0:00:09 loss: 0.3932 (0.3986) auc: 94.5833 (92.8585) time: 0.0294 data: 0.0001 max mem: 5308 | |
| [00:38:04.498378] Test: [300/560] eta: 0:00:08 loss: 0.3932 (0.3975) auc: 94.4444 (92.9134) time: 0.0294 data: 0.0001 max mem: 5308 | |
| [00:38:04.793269] Test: [310/560] eta: 0:00:08 loss: 0.3861 (0.3980) auc: 94.3359 (92.8552) time: 0.0294 data: 0.0001 max mem: 5308 | |
| [00:38:05.087719] Test: [320/560] eta: 0:00:08 loss: 0.4158 (0.3987) auc: 90.5882 (92.8233) time: 0.0294 data: 0.0001 max mem: 5308 | |
| [00:38:05.382341] Test: [330/560] eta: 0:00:07 loss: 0.3940 (0.3981) auc: 93.6508 (92.8770) time: 0.0294 data: 0.0001 max mem: 5308 | |
| [00:38:05.678584] Test: [340/560] eta: 0:00:07 loss: 0.3766 (0.3975) auc: 94.9219 (92.9039) time: 0.0295 data: 0.0001 max mem: 5308 | |
| [00:38:05.971373] Test: [350/560] eta: 0:00:06 loss: 0.3710 (0.3971) auc: 94.4444 (92.9181) time: 0.0294 data: 0.0001 max mem: 5308 | |
| [00:38:06.263297] Test: [360/560] eta: 0:00:06 loss: 0.3913 (0.3981) auc: 94.3320 (92.8836) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:38:06.555809] Test: [370/560] eta: 0:00:06 loss: 0.4071 (0.3984) auc: 91.3725 (92.9106) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:38:06.848038] Test: [380/560] eta: 0:00:05 loss: 0.3977 (0.3984) auc: 91.7749 (92.8958) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:38:07.140172] Test: [390/560] eta: 0:00:05 loss: 0.3983 (0.3990) auc: 92.8571 (92.8694) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:38:07.433549] Test: [400/560] eta: 0:00:05 loss: 0.4159 (0.4001) auc: 92.4603 (92.8002) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:38:07.725248] Test: [410/560] eta: 0:00:04 loss: 0.4183 (0.4004) auc: 90.2778 (92.7698) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:38:08.017756] Test: [420/560] eta: 0:00:04 loss: 0.3997 (0.4005) auc: 90.2778 (92.7564) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:38:08.310522] Test: [430/560] eta: 0:00:04 loss: 0.3553 (0.3997) auc: 92.0635 (92.7807) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:38:08.602569] Test: [440/560] eta: 0:00:03 loss: 0.3489 (0.3991) auc: 96.0317 (92.8355) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:38:08.895148] Test: [450/560] eta: 0:00:03 loss: 0.3726 (0.3992) auc: 94.5312 (92.8254) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:38:09.188367] Test: [460/560] eta: 0:00:03 loss: 0.4115 (0.4002) auc: 92.7126 (92.7846) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:38:09.480570] Test: [470/560] eta: 0:00:02 loss: 0.4388 (0.4004) auc: 92.9688 (92.7787) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:38:09.773063] Test: [480/560] eta: 0:00:02 loss: 0.4187 (0.4003) auc: 92.9688 (92.7861) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:38:10.065174] Test: [490/560] eta: 0:00:02 loss: 0.3820 (0.3999) auc: 92.9688 (92.8000) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:38:10.357809] Test: [500/560] eta: 0:00:01 loss: 0.3784 (0.3994) auc: 93.6508 (92.8325) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:38:10.649834] Test: [510/560] eta: 0:00:01 loss: 0.3516 (0.3993) auc: 94.9020 (92.8449) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:38:10.941705] Test: [520/560] eta: 0:00:01 loss: 0.4263 (0.4003) auc: 90.2834 (92.7724) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:38:11.234307] Test: [530/560] eta: 0:00:00 loss: 0.4544 (0.4013) auc: 89.0688 (92.7377) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:38:11.526456] Test: [540/560] eta: 0:00:00 loss: 0.4042 (0.4018) auc: 92.1569 (92.7251) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:38:11.816705] Test: [550/560] eta: 0:00:00 loss: 0.3983 (0.4016) auc: 95.1417 (92.7946) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:38:12.058707] Test: [559/560] eta: 0:00:00 loss: 0.3917 (0.4013) auc: 94.7368 (92.7929) time: 0.0280 data: 0.0001 max mem: 5308 | |
| [00:38:12.226384] Test: Total time: 0:00:17 (0.0319 s / it) | |
| [00:38:12.377858] * Auc 92.783 loss 0.402 | |
| [00:38:12.378014] AUC of the network on the 35796 val images: 92.78% | |
| [00:38:12.378026] Max auc: 92.78% | |
| [00:38:12.378036] Save model with min_val_loss at epoch: 3 | |
| [00:38:17.928782] log_dir: ./checkpoint/finetuned_models/FF++_c23_32frames | |
| [00:38:18.877450] Epoch: [4] [ 0/2877] eta: 0:45:23 lr: 0.000050 loss: 0.6511 (0.6511) time: 0.9468 data: 0.8136 max mem: 5308 | |
| [00:38:31.305840] Epoch: [4] [ 100/2877] eta: 0:06:07 lr: 0.000050 loss: 0.6458 (0.6455) time: 0.1242 data: 0.0001 max mem: 5308 | |
| [00:38:43.790790] Epoch: [4] [ 200/2877] eta: 0:05:44 lr: 0.000051 loss: 0.6566 (0.6468) time: 0.1252 data: 0.0001 max mem: 5308 | |
| [00:38:56.337403] Epoch: [4] [ 300/2877] eta: 0:05:28 lr: 0.000051 loss: 0.6598 (0.6468) time: 0.1252 data: 0.0001 max mem: 5308 | |
| [00:39:08.859809] Epoch: [4] [ 400/2877] eta: 0:05:14 lr: 0.000052 loss: 0.6701 (0.6481) time: 0.1258 data: 0.0001 max mem: 5308 | |
| [00:39:21.415473] Epoch: [4] [ 500/2877] eta: 0:05:01 lr: 0.000052 loss: 0.6552 (0.6482) time: 0.1259 data: 0.0001 max mem: 5308 | |
| [00:39:33.936023] Epoch: [4] [ 600/2877] eta: 0:04:47 lr: 0.000053 loss: 0.6406 (0.6481) time: 0.1247 data: 0.0001 max mem: 5308 | |
| [00:39:46.441328] Epoch: [4] [ 700/2877] eta: 0:04:34 lr: 0.000053 loss: 0.6201 (0.6467) time: 0.1248 data: 0.0001 max mem: 5308 | |
| [00:39:58.997595] Epoch: [4] [ 800/2877] eta: 0:04:22 lr: 0.000053 loss: 0.6587 (0.6466) time: 0.1252 data: 0.0001 max mem: 5308 | |
| [00:40:11.525094] Epoch: [4] [ 900/2877] eta: 0:04:09 lr: 0.000054 loss: 0.6448 (0.6464) time: 0.1256 data: 0.0001 max mem: 5308 | |
| [00:40:24.019803] Epoch: [4] [1000/2877] eta: 0:03:56 lr: 0.000054 loss: 0.6556 (0.6460) time: 0.1245 data: 0.0001 max mem: 5308 | |
| [00:40:36.499126] Epoch: [4] [1100/2877] eta: 0:03:43 lr: 0.000055 loss: 0.6480 (0.6463) time: 0.1246 data: 0.0001 max mem: 5308 | |
| [00:40:48.985060] Epoch: [4] [1200/2877] eta: 0:03:30 lr: 0.000055 loss: 0.6571 (0.6471) time: 0.1248 data: 0.0001 max mem: 5308 | |
| [00:41:01.550402] Epoch: [4] [1300/2877] eta: 0:03:18 lr: 0.000056 loss: 0.6494 (0.6471) time: 0.1269 data: 0.0001 max mem: 5308 | |
| [00:41:14.199924] Epoch: [4] [1400/2877] eta: 0:03:05 lr: 0.000056 loss: 0.6425 (0.6474) time: 0.1262 data: 0.0002 max mem: 5308 | |
| [00:41:26.715522] Epoch: [4] [1500/2877] eta: 0:02:53 lr: 0.000057 loss: 0.6421 (0.6470) time: 0.1266 data: 0.0001 max mem: 5308 | |
| [00:41:39.358848] Epoch: [4] [1600/2877] eta: 0:02:40 lr: 0.000057 loss: 0.6546 (0.6476) time: 0.1256 data: 0.0001 max mem: 5308 | |
| [00:41:51.851005] Epoch: [4] [1700/2877] eta: 0:02:28 lr: 0.000057 loss: 0.6416 (0.6474) time: 0.1252 data: 0.0001 max mem: 5308 | |
| [00:42:04.348099] Epoch: [4] [1800/2877] eta: 0:02:15 lr: 0.000058 loss: 0.6335 (0.6471) time: 0.1252 data: 0.0001 max mem: 5308 | |
| [00:42:16.819752] Epoch: [4] [1900/2877] eta: 0:02:02 lr: 0.000058 loss: 0.5991 (0.6469) time: 0.1245 data: 0.0001 max mem: 5308 | |
| [00:42:29.275274] Epoch: [4] [2000/2877] eta: 0:01:50 lr: 0.000059 loss: 0.6542 (0.6466) time: 0.1241 data: 0.0001 max mem: 5308 | |
| [00:42:41.669066] Epoch: [4] [2100/2877] eta: 0:01:37 lr: 0.000059 loss: 0.6430 (0.6465) time: 0.1237 data: 0.0001 max mem: 5308 | |
| [00:42:54.040756] Epoch: [4] [2200/2877] eta: 0:01:24 lr: 0.000060 loss: 0.6393 (0.6461) time: 0.1236 data: 0.0001 max mem: 5308 | |
| [00:43:06.423581] Epoch: [4] [2300/2877] eta: 0:01:12 lr: 0.000060 loss: 0.6165 (0.6461) time: 0.1237 data: 0.0001 max mem: 5308 | |
| [00:43:18.796163] Epoch: [4] [2400/2877] eta: 0:00:59 lr: 0.000060 loss: 0.6441 (0.6461) time: 0.1237 data: 0.0001 max mem: 5308 | |
| [00:43:31.164647] Epoch: [4] [2500/2877] eta: 0:00:47 lr: 0.000061 loss: 0.6400 (0.6460) time: 0.1238 data: 0.0001 max mem: 5308 | |
| [00:43:43.541234] Epoch: [4] [2600/2877] eta: 0:00:34 lr: 0.000061 loss: 0.6409 (0.6460) time: 0.1237 data: 0.0001 max mem: 5308 | |
| [00:43:55.900515] Epoch: [4] [2700/2877] eta: 0:00:22 lr: 0.000062 loss: 0.6399 (0.6460) time: 0.1235 data: 0.0001 max mem: 5308 | |
| [00:44:08.300360] Epoch: [4] [2800/2877] eta: 0:00:09 lr: 0.000062 loss: 0.6222 (0.6457) time: 0.1243 data: 0.0001 max mem: 5308 | |
| [00:44:17.714982] Epoch: [4] [2876/2877] eta: 0:00:00 lr: 0.000062 loss: 0.6469 (0.6455) time: 0.1232 data: 0.0002 max mem: 5308 | |
| [00:44:17.935941] Epoch: [4] Total time: 0:06:00 (0.1251 s / it) | |
| [00:44:17.943953] Averaged stats: lr: 0.000062 loss: 0.6469 (0.6465) | |
| [00:44:19.267670] Test: [ 0/560] eta: 0:12:18 loss: 0.3983 (0.3983) auc: 91.9608 (91.9608) time: 1.3189 data: 1.2844 max mem: 5308 | |
| [00:44:19.561357] Test: [ 10/560] eta: 0:01:20 loss: 0.4247 (0.4286) auc: 90.3382 (90.6281) time: 0.1465 data: 0.1169 max mem: 5308 | |
| [00:44:19.855057] Test: [ 20/560] eta: 0:00:48 loss: 0.3992 (0.4044) auc: 92.0833 (92.5250) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [00:44:20.149196] Test: [ 30/560] eta: 0:00:37 loss: 0.3631 (0.3804) auc: 95.2941 (93.5242) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [00:44:20.443469] Test: [ 40/560] eta: 0:00:31 loss: 0.3395 (0.3816) auc: 95.4167 (93.3740) time: 0.0294 data: 0.0001 max mem: 5308 | |
| [00:44:20.737593] Test: [ 50/560] eta: 0:00:27 loss: 0.3426 (0.3834) auc: 94.4444 (93.4503) time: 0.0294 data: 0.0001 max mem: 5308 | |
| [00:44:21.032698] Test: [ 60/560] eta: 0:00:25 loss: 0.4023 (0.3844) auc: 92.2078 (93.2596) time: 0.0294 data: 0.0001 max mem: 5308 | |
| [00:44:21.325717] Test: [ 70/560] eta: 0:00:23 loss: 0.3979 (0.3820) auc: 91.6667 (93.2924) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [00:44:21.621461] Test: [ 80/560] eta: 0:00:21 loss: 0.3253 (0.3775) auc: 96.3563 (93.6133) time: 0.0294 data: 0.0001 max mem: 5308 | |
| [00:44:21.915037] Test: [ 90/560] eta: 0:00:20 loss: 0.3180 (0.3777) auc: 96.3563 (93.7296) time: 0.0294 data: 0.0001 max mem: 5308 | |
| [00:44:22.209463] Test: [100/560] eta: 0:00:19 loss: 0.3612 (0.3754) auc: 95.6710 (93.9056) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [00:44:22.502569] Test: [110/560] eta: 0:00:18 loss: 0.3424 (0.3735) auc: 96.0938 (93.9680) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [00:44:22.796380] Test: [120/560] eta: 0:00:17 loss: 0.3904 (0.3800) auc: 91.2698 (93.6949) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [00:44:23.090863] Test: [130/560] eta: 0:00:16 loss: 0.3918 (0.3804) auc: 92.0635 (93.7517) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [00:44:23.384587] Test: [140/560] eta: 0:00:16 loss: 0.3775 (0.3816) auc: 94.7368 (93.7320) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [00:44:23.679271] Test: [150/560] eta: 0:00:15 loss: 0.3741 (0.3801) auc: 95.1417 (93.8032) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [00:44:23.973074] Test: [160/560] eta: 0:00:14 loss: 0.3328 (0.3759) auc: 96.8627 (94.0329) time: 0.0294 data: 0.0001 max mem: 5308 | |
| [00:44:24.266697] Test: [170/560] eta: 0:00:14 loss: 0.3082 (0.3742) auc: 96.8750 (94.1022) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [00:44:24.561641] Test: [180/560] eta: 0:00:13 loss: 0.3351 (0.3746) auc: 95.5466 (94.1337) time: 0.0294 data: 0.0001 max mem: 5308 | |
| [00:44:24.855708] Test: [190/560] eta: 0:00:13 loss: 0.3845 (0.3755) auc: 92.5490 (94.0857) time: 0.0294 data: 0.0001 max mem: 5308 | |
| [00:44:25.149425] Test: [200/560] eta: 0:00:12 loss: 0.3530 (0.3737) auc: 93.7255 (94.1707) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [00:44:25.440641] Test: [210/560] eta: 0:00:12 loss: 0.3530 (0.3746) auc: 94.2460 (94.1157) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:44:25.732176] Test: [220/560] eta: 0:00:11 loss: 0.3847 (0.3744) auc: 94.5312 (94.1312) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:44:26.023128] Test: [230/560] eta: 0:00:11 loss: 0.3771 (0.3744) auc: 94.5098 (94.1270) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:44:26.315247] Test: [240/560] eta: 0:00:11 loss: 0.3674 (0.3741) auc: 93.7255 (94.1363) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:44:26.606880] Test: [250/560] eta: 0:00:10 loss: 0.3531 (0.3740) auc: 94.0476 (94.1160) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:44:26.898686] Test: [260/560] eta: 0:00:10 loss: 0.3019 (0.3719) auc: 95.2941 (94.2010) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:44:27.189905] Test: [270/560] eta: 0:00:09 loss: 0.3676 (0.3725) auc: 94.8413 (94.1887) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:44:27.481367] Test: [280/560] eta: 0:00:09 loss: 0.3593 (0.3707) auc: 95.6349 (94.2884) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:44:27.773101] Test: [290/560] eta: 0:00:09 loss: 0.3593 (0.3717) auc: 96.5368 (94.2940) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:44:28.065001] Test: [300/560] eta: 0:00:08 loss: 0.3833 (0.3708) auc: 95.2381 (94.3361) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:44:28.356452] Test: [310/560] eta: 0:00:08 loss: 0.3677 (0.3712) auc: 94.3723 (94.2765) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:44:28.648142] Test: [320/560] eta: 0:00:07 loss: 0.3743 (0.3717) auc: 92.9412 (94.2592) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:44:28.939554] Test: [330/560] eta: 0:00:07 loss: 0.3712 (0.3714) auc: 94.5833 (94.2804) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:44:29.231666] Test: [340/560] eta: 0:00:07 loss: 0.3503 (0.3708) auc: 95.5466 (94.3077) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:44:29.523577] Test: [350/560] eta: 0:00:06 loss: 0.3503 (0.3704) auc: 95.3125 (94.3168) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:44:29.816151] Test: [360/560] eta: 0:00:06 loss: 0.3745 (0.3716) auc: 94.0476 (94.2794) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:44:30.108348] Test: [370/560] eta: 0:00:06 loss: 0.3907 (0.3719) auc: 94.1176 (94.3029) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:44:30.401093] Test: [380/560] eta: 0:00:05 loss: 0.3695 (0.3719) auc: 93.3333 (94.2928) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:44:30.694633] Test: [390/560] eta: 0:00:05 loss: 0.3512 (0.3723) auc: 94.1406 (94.2889) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [00:44:30.987127] Test: [400/560] eta: 0:00:05 loss: 0.3946 (0.3734) auc: 94.4444 (94.2279) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:44:31.279831] Test: [410/560] eta: 0:00:04 loss: 0.3965 (0.3739) auc: 91.2698 (94.1800) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:44:31.571639] Test: [420/560] eta: 0:00:04 loss: 0.3843 (0.3738) auc: 92.5781 (94.1766) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:44:31.864631] Test: [430/560] eta: 0:00:04 loss: 0.3280 (0.3728) auc: 95.4365 (94.2180) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:44:32.156518] Test: [440/560] eta: 0:00:03 loss: 0.3213 (0.3721) auc: 97.2222 (94.2700) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:44:32.448085] Test: [450/560] eta: 0:00:03 loss: 0.3544 (0.3724) auc: 95.5466 (94.2590) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:44:32.739995] Test: [460/560] eta: 0:00:03 loss: 0.3787 (0.3733) auc: 94.1406 (94.2267) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:44:33.032195] Test: [470/560] eta: 0:00:02 loss: 0.3958 (0.3735) auc: 94.1406 (94.2243) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:44:33.324603] Test: [480/560] eta: 0:00:02 loss: 0.3958 (0.3735) auc: 94.1406 (94.2142) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:44:33.616329] Test: [490/560] eta: 0:00:02 loss: 0.3516 (0.3729) auc: 94.3320 (94.2351) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:44:33.908483] Test: [500/560] eta: 0:00:01 loss: 0.3226 (0.3724) auc: 94.9020 (94.2624) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:44:34.200520] Test: [510/560] eta: 0:00:01 loss: 0.3202 (0.3721) auc: 95.2381 (94.2822) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:44:34.493085] Test: [520/560] eta: 0:00:01 loss: 0.3958 (0.3731) auc: 91.9028 (94.2206) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:44:34.785122] Test: [530/560] eta: 0:00:00 loss: 0.4133 (0.3742) auc: 90.6883 (94.1936) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:44:35.077635] Test: [540/560] eta: 0:00:00 loss: 0.4007 (0.3748) auc: 94.6860 (94.1792) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:44:35.367456] Test: [550/560] eta: 0:00:00 loss: 0.3773 (0.3748) auc: 96.0317 (94.2335) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:44:35.609580] Test: [559/560] eta: 0:00:00 loss: 0.3609 (0.3746) auc: 95.9514 (94.2390) time: 0.0280 data: 0.0001 max mem: 5308 | |
| [00:44:35.724161] Test: Total time: 0:00:17 (0.0317 s / it) | |
| [00:44:35.863697] * Auc 94.252 loss 0.374 | |
| [00:44:35.863852] AUC of the network on the 35796 val images: 94.25% | |
| [00:44:35.863865] Max auc: 94.25% | |
| [00:44:35.863875] Save model with min_val_loss at epoch: 4 | |
| [00:44:41.418031] log_dir: ./checkpoint/finetuned_models/FF++_c23_32frames | |
| [00:44:42.429455] Epoch: [5] [ 0/2877] eta: 0:48:23 lr: 0.000063 loss: 0.5772 (0.5772) time: 1.0092 data: 0.8771 max mem: 5308 | |
| [00:44:54.956798] Epoch: [5] [ 100/2877] eta: 0:06:12 lr: 0.000062 loss: 0.6558 (0.6514) time: 0.1255 data: 0.0001 max mem: 5308 | |
| [00:45:07.535102] Epoch: [5] [ 200/2877] eta: 0:05:47 lr: 0.000062 loss: 0.6386 (0.6489) time: 0.1258 data: 0.0001 max mem: 5308 | |
| [00:45:20.097919] Epoch: [5] [ 300/2877] eta: 0:05:31 lr: 0.000062 loss: 0.6481 (0.6462) time: 0.1257 data: 0.0001 max mem: 5308 | |
| [00:45:32.656067] Epoch: [5] [ 400/2877] eta: 0:05:16 lr: 0.000062 loss: 0.6311 (0.6430) time: 0.1249 data: 0.0001 max mem: 5308 | |
| [00:45:45.220379] Epoch: [5] [ 500/2877] eta: 0:05:02 lr: 0.000062 loss: 0.6430 (0.6411) time: 0.1254 data: 0.0001 max mem: 5308 | |
| [00:45:57.779087] Epoch: [5] [ 600/2877] eta: 0:04:49 lr: 0.000062 loss: 0.6408 (0.6417) time: 0.1257 data: 0.0001 max mem: 5308 | |
| [00:46:10.339266] Epoch: [5] [ 700/2877] eta: 0:04:36 lr: 0.000062 loss: 0.6562 (0.6414) time: 0.1256 data: 0.0001 max mem: 5308 | |
| [00:46:22.834182] Epoch: [5] [ 800/2877] eta: 0:04:22 lr: 0.000062 loss: 0.6654 (0.6425) time: 0.1251 data: 0.0001 max mem: 5308 | |
| [00:46:35.326585] Epoch: [5] [ 900/2877] eta: 0:04:09 lr: 0.000062 loss: 0.6338 (0.6422) time: 0.1248 data: 0.0001 max mem: 5308 | |
| [00:46:47.924661] Epoch: [5] [1000/2877] eta: 0:03:57 lr: 0.000062 loss: 0.6396 (0.6424) time: 0.1263 data: 0.0001 max mem: 5308 | |
| [00:47:00.553016] Epoch: [5] [1100/2877] eta: 0:03:44 lr: 0.000062 loss: 0.6217 (0.6406) time: 0.1262 data: 0.0001 max mem: 5308 | |
| [00:47:13.184718] Epoch: [5] [1200/2877] eta: 0:03:31 lr: 0.000061 loss: 0.6310 (0.6404) time: 0.1264 data: 0.0001 max mem: 5308 | |
| [00:47:25.832961] Epoch: [5] [1300/2877] eta: 0:03:19 lr: 0.000061 loss: 0.6452 (0.6403) time: 0.1264 data: 0.0001 max mem: 5308 | |
| [00:47:38.466588] Epoch: [5] [1400/2877] eta: 0:03:06 lr: 0.000061 loss: 0.6413 (0.6405) time: 0.1264 data: 0.0001 max mem: 5308 | |
| [00:47:51.102458] Epoch: [5] [1500/2877] eta: 0:02:53 lr: 0.000061 loss: 0.6363 (0.6406) time: 0.1266 data: 0.0001 max mem: 5308 | |
| [00:48:03.742834] Epoch: [5] [1600/2877] eta: 0:02:41 lr: 0.000061 loss: 0.6070 (0.6396) time: 0.1264 data: 0.0001 max mem: 5308 | |
| [00:48:16.384534] Epoch: [5] [1700/2877] eta: 0:02:28 lr: 0.000060 loss: 0.6470 (0.6391) time: 0.1264 data: 0.0001 max mem: 5308 | |
| [00:48:29.010907] Epoch: [5] [1800/2877] eta: 0:02:16 lr: 0.000060 loss: 0.6069 (0.6387) time: 0.1266 data: 0.0001 max mem: 5308 | |
| [00:48:41.660756] Epoch: [5] [1900/2877] eta: 0:02:03 lr: 0.000060 loss: 0.6338 (0.6386) time: 0.1262 data: 0.0001 max mem: 5308 | |
| [00:48:54.298016] Epoch: [5] [2000/2877] eta: 0:01:50 lr: 0.000060 loss: 0.6231 (0.6382) time: 0.1264 data: 0.0001 max mem: 5308 | |
| [00:49:06.943561] Epoch: [5] [2100/2877] eta: 0:01:38 lr: 0.000059 loss: 0.6318 (0.6380) time: 0.1263 data: 0.0001 max mem: 5308 | |
| [00:49:19.486008] Epoch: [5] [2200/2877] eta: 0:01:25 lr: 0.000059 loss: 0.6595 (0.6381) time: 0.1253 data: 0.0001 max mem: 5308 | |
| [00:49:32.082932] Epoch: [5] [2300/2877] eta: 0:01:12 lr: 0.000059 loss: 0.6247 (0.6380) time: 0.1263 data: 0.0001 max mem: 5308 | |
| [00:49:44.695572] Epoch: [5] [2400/2877] eta: 0:01:00 lr: 0.000058 loss: 0.6491 (0.6379) time: 0.1258 data: 0.0002 max mem: 5308 | |
| [00:49:57.301357] Epoch: [5] [2500/2877] eta: 0:00:47 lr: 0.000058 loss: 0.6367 (0.6377) time: 0.1261 data: 0.0002 max mem: 5308 | |
| [00:50:09.930045] Epoch: [5] [2600/2877] eta: 0:00:34 lr: 0.000058 loss: 0.6260 (0.6374) time: 0.1261 data: 0.0001 max mem: 5308 | |
| [00:50:22.541205] Epoch: [5] [2700/2877] eta: 0:00:22 lr: 0.000057 loss: 0.6473 (0.6372) time: 0.1264 data: 0.0001 max mem: 5308 | |
| [00:50:35.155263] Epoch: [5] [2800/2877] eta: 0:00:09 lr: 0.000057 loss: 0.6279 (0.6370) time: 0.1260 data: 0.0001 max mem: 5308 | |
| [00:50:44.735326] Epoch: [5] [2876/2877] eta: 0:00:00 lr: 0.000057 loss: 0.6627 (0.6368) time: 0.1257 data: 0.0002 max mem: 5308 | |
| [00:50:44.919808] Epoch: [5] Total time: 0:06:03 (0.1263 s / it) | |
| [00:50:44.921342] Averaged stats: lr: 0.000057 loss: 0.6627 (0.6372) | |
| [00:50:46.161394] Test: [ 0/560] eta: 0:11:31 loss: 0.3417 (0.3417) auc: 93.3333 (93.3333) time: 1.2353 data: 1.1977 max mem: 5308 | |
| [00:50:46.530047] Test: [ 10/560] eta: 0:01:20 loss: 0.3665 (0.3749) auc: 92.5000 (91.9170) time: 0.1457 data: 0.1157 max mem: 5308 | |
| [00:50:46.824588] Test: [ 20/560] eta: 0:00:48 loss: 0.3368 (0.3430) auc: 94.3320 (93.9786) time: 0.0331 data: 0.0038 max mem: 5308 | |
| [00:50:47.119207] Test: [ 30/560] eta: 0:00:37 loss: 0.2718 (0.3218) auc: 97.2222 (94.7001) time: 0.0294 data: 0.0001 max mem: 5308 | |
| [00:50:47.414814] Test: [ 40/560] eta: 0:00:31 loss: 0.2909 (0.3223) auc: 97.2222 (94.6276) time: 0.0294 data: 0.0001 max mem: 5308 | |
| [00:50:47.709143] Test: [ 50/560] eta: 0:00:27 loss: 0.3022 (0.3232) auc: 96.0317 (94.8007) time: 0.0294 data: 0.0001 max mem: 5308 | |
| [00:50:48.005358] Test: [ 60/560] eta: 0:00:25 loss: 0.3461 (0.3246) auc: 95.3125 (94.6625) time: 0.0295 data: 0.0001 max mem: 5308 | |
| [00:50:48.300629] Test: [ 70/560] eta: 0:00:23 loss: 0.3461 (0.3220) auc: 93.5065 (94.6922) time: 0.0295 data: 0.0002 max mem: 5308 | |
| [00:50:48.593998] Test: [ 80/560] eta: 0:00:21 loss: 0.2888 (0.3173) auc: 95.8333 (94.8807) time: 0.0294 data: 0.0001 max mem: 5308 | |
| [00:50:48.888138] Test: [ 90/560] eta: 0:00:20 loss: 0.2574 (0.3169) auc: 96.4286 (94.9502) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [00:50:49.179188] Test: [100/560] eta: 0:00:19 loss: 0.2890 (0.3137) auc: 96.0938 (95.1163) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:50:49.471037] Test: [110/560] eta: 0:00:18 loss: 0.2890 (0.3131) auc: 96.0938 (95.0909) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:50:49.762650] Test: [120/560] eta: 0:00:17 loss: 0.3217 (0.3185) auc: 92.9412 (94.9032) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:50:50.054543] Test: [130/560] eta: 0:00:16 loss: 0.3217 (0.3193) auc: 95.2381 (94.9605) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:50:50.346070] Test: [140/560] eta: 0:00:16 loss: 0.3146 (0.3200) auc: 96.2500 (94.9655) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:50:50.636747] Test: [150/560] eta: 0:00:15 loss: 0.3055 (0.3189) auc: 96.2500 (95.0111) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:50:50.929352] Test: [160/560] eta: 0:00:14 loss: 0.2829 (0.3145) auc: 97.2222 (95.1870) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:50:51.221174] Test: [170/560] eta: 0:00:14 loss: 0.2550 (0.3128) auc: 96.8750 (95.2374) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:50:51.512406] Test: [180/560] eta: 0:00:13 loss: 0.2721 (0.3133) auc: 96.3563 (95.2388) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:50:51.804504] Test: [190/560] eta: 0:00:13 loss: 0.3207 (0.3139) auc: 94.5098 (95.2146) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:50:52.096247] Test: [200/560] eta: 0:00:12 loss: 0.2928 (0.3118) auc: 94.9219 (95.3148) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:50:52.387644] Test: [210/560] eta: 0:00:12 loss: 0.2958 (0.3127) auc: 95.6710 (95.2594) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:50:52.679175] Test: [220/560] eta: 0:00:11 loss: 0.3158 (0.3127) auc: 95.5466 (95.2623) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:50:52.971171] Test: [230/560] eta: 0:00:11 loss: 0.3048 (0.3126) auc: 95.5466 (95.2817) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:50:53.262192] Test: [240/560] eta: 0:00:11 loss: 0.2938 (0.3122) auc: 96.2302 (95.2830) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:50:53.553626] Test: [250/560] eta: 0:00:10 loss: 0.3221 (0.3123) auc: 95.4167 (95.2702) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:50:53.844977] Test: [260/560] eta: 0:00:10 loss: 0.2594 (0.3105) auc: 96.0784 (95.3243) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:50:54.135874] Test: [270/560] eta: 0:00:09 loss: 0.3123 (0.3113) auc: 94.6429 (95.3007) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:50:54.427304] Test: [280/560] eta: 0:00:09 loss: 0.2806 (0.3090) auc: 96.7611 (95.3904) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:50:54.718486] Test: [290/560] eta: 0:00:09 loss: 0.2896 (0.3099) auc: 96.7611 (95.3811) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:50:55.010076] Test: [300/560] eta: 0:00:08 loss: 0.3148 (0.3091) auc: 95.2941 (95.4169) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:50:55.301278] Test: [310/560] eta: 0:00:08 loss: 0.2955 (0.3095) auc: 95.2941 (95.3757) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:50:55.593273] Test: [320/560] eta: 0:00:07 loss: 0.3148 (0.3099) auc: 94.9020 (95.3609) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:50:55.884847] Test: [330/560] eta: 0:00:07 loss: 0.3085 (0.3096) auc: 95.6349 (95.3773) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:50:56.176081] Test: [340/560] eta: 0:00:07 loss: 0.2823 (0.3092) auc: 96.4286 (95.3975) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:50:56.468361] Test: [350/560] eta: 0:00:06 loss: 0.2823 (0.3088) auc: 96.0938 (95.4076) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:50:56.760214] Test: [360/560] eta: 0:00:06 loss: 0.3106 (0.3097) auc: 95.6863 (95.3927) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:50:57.051379] Test: [370/560] eta: 0:00:06 loss: 0.3244 (0.3101) auc: 95.0000 (95.3994) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:50:57.343918] Test: [380/560] eta: 0:00:05 loss: 0.3064 (0.3100) auc: 95.0000 (95.3999) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:50:57.634816] Test: [390/560] eta: 0:00:05 loss: 0.2989 (0.3105) auc: 95.6863 (95.3849) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:50:57.926509] Test: [400/560] eta: 0:00:05 loss: 0.3050 (0.3115) auc: 96.0317 (95.3316) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:50:58.218400] Test: [410/560] eta: 0:00:04 loss: 0.3357 (0.3119) auc: 94.4444 (95.3020) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:50:58.510219] Test: [420/560] eta: 0:00:04 loss: 0.3204 (0.3122) auc: 94.4444 (95.2765) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:50:58.802749] Test: [430/560] eta: 0:00:04 loss: 0.2655 (0.3112) auc: 96.4706 (95.3185) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:50:59.095677] Test: [440/560] eta: 0:00:03 loss: 0.2513 (0.3105) auc: 98.0159 (95.3584) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:50:59.388518] Test: [450/560] eta: 0:00:03 loss: 0.2803 (0.3107) auc: 96.7611 (95.3477) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:50:59.681272] Test: [460/560] eta: 0:00:03 loss: 0.3275 (0.3115) auc: 95.3125 (95.3159) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:50:59.973078] Test: [470/560] eta: 0:00:02 loss: 0.3496 (0.3118) auc: 95.2941 (95.3085) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:51:00.265583] Test: [480/560] eta: 0:00:02 loss: 0.3201 (0.3117) auc: 94.9219 (95.2830) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:51:00.561165] Test: [490/560] eta: 0:00:02 loss: 0.2932 (0.3112) auc: 94.9219 (95.3049) time: 0.0294 data: 0.0001 max mem: 5308 | |
| [00:51:00.854955] Test: [500/560] eta: 0:00:01 loss: 0.2709 (0.3107) auc: 97.6471 (95.3380) time: 0.0294 data: 0.0002 max mem: 5308 | |
| [00:51:01.149361] Test: [510/560] eta: 0:00:01 loss: 0.2687 (0.3102) auc: 97.2549 (95.3574) time: 0.0294 data: 0.0001 max mem: 5308 | |
| [00:51:01.441977] Test: [520/560] eta: 0:00:01 loss: 0.3191 (0.3113) auc: 94.5098 (95.3149) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [00:51:01.734298] Test: [530/560] eta: 0:00:00 loss: 0.3565 (0.3122) auc: 94.1176 (95.3098) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:51:02.027573] Test: [540/560] eta: 0:00:00 loss: 0.3362 (0.3127) auc: 95.2941 (95.2958) time: 0.0292 data: 0.0002 max mem: 5308 | |
| [00:51:02.318877] Test: [550/560] eta: 0:00:00 loss: 0.2815 (0.3122) auc: 97.9757 (95.3462) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:51:02.561096] Test: [559/560] eta: 0:00:00 loss: 0.2856 (0.3124) auc: 97.9757 (95.3413) time: 0.0281 data: 0.0001 max mem: 5308 | |
| [00:51:02.743830] Test: Total time: 0:00:17 (0.0318 s / it) | |
| [00:51:02.904319] * Auc 95.324 loss 0.312 | |
| [00:51:02.904476] AUC of the network on the 35796 val images: 95.32% | |
| [00:51:02.904488] Max auc: 95.32% | |
| [00:51:02.904499] Save model with min_val_loss at epoch: 5 | |
| [00:51:08.455291] log_dir: ./checkpoint/finetuned_models/FF++_c23_32frames | |
| [00:51:09.391404] Epoch: [6] [ 0/2877] eta: 0:44:48 lr: 0.000057 loss: 0.6522 (0.6522) time: 0.9343 data: 0.7988 max mem: 5308 | |
| [00:51:21.848110] Epoch: [6] [ 100/2877] eta: 0:06:08 lr: 0.000056 loss: 0.6370 (0.6369) time: 0.1244 data: 0.0002 max mem: 5308 | |
| [00:51:34.292190] Epoch: [6] [ 200/2877] eta: 0:05:44 lr: 0.000056 loss: 0.6109 (0.6326) time: 0.1249 data: 0.0001 max mem: 5308 | |
| [00:51:46.784271] Epoch: [6] [ 300/2877] eta: 0:05:28 lr: 0.000055 loss: 0.6318 (0.6329) time: 0.1245 data: 0.0002 max mem: 5308 | |
| [00:51:59.215823] Epoch: [6] [ 400/2877] eta: 0:05:13 lr: 0.000055 loss: 0.6355 (0.6350) time: 0.1245 data: 0.0002 max mem: 5308 | |
| [00:52:11.650445] Epoch: [6] [ 500/2877] eta: 0:04:59 lr: 0.000055 loss: 0.6371 (0.6341) time: 0.1245 data: 0.0002 max mem: 5308 | |
| [00:52:24.125893] Epoch: [6] [ 600/2877] eta: 0:04:46 lr: 0.000054 loss: 0.6495 (0.6320) time: 0.1247 data: 0.0001 max mem: 5308 | |
| [00:52:36.784395] Epoch: [6] [ 700/2877] eta: 0:04:34 lr: 0.000054 loss: 0.6092 (0.6322) time: 0.1253 data: 0.0001 max mem: 5308 | |
| [00:52:49.299469] Epoch: [6] [ 800/2877] eta: 0:04:21 lr: 0.000053 loss: 0.6313 (0.6323) time: 0.1249 data: 0.0001 max mem: 5308 | |
| [00:53:01.775841] Epoch: [6] [ 900/2877] eta: 0:04:08 lr: 0.000053 loss: 0.6463 (0.6330) time: 0.1244 data: 0.0001 max mem: 5308 | |
| [00:53:14.255686] Epoch: [6] [1000/2877] eta: 0:03:55 lr: 0.000052 loss: 0.6471 (0.6339) time: 0.1249 data: 0.0001 max mem: 5308 | |
| [00:53:26.728907] Epoch: [6] [1100/2877] eta: 0:03:43 lr: 0.000052 loss: 0.6291 (0.6335) time: 0.1250 data: 0.0001 max mem: 5308 | |
| [00:53:39.350019] Epoch: [6] [1200/2877] eta: 0:03:30 lr: 0.000051 loss: 0.6272 (0.6325) time: 0.1267 data: 0.0001 max mem: 5308 | |
| [00:53:51.986487] Epoch: [6] [1300/2877] eta: 0:03:18 lr: 0.000051 loss: 0.6416 (0.6324) time: 0.1260 data: 0.0001 max mem: 5308 | |
| [00:54:04.509934] Epoch: [6] [1400/2877] eta: 0:03:05 lr: 0.000050 loss: 0.6062 (0.6314) time: 0.1249 data: 0.0001 max mem: 5308 | |
| [00:54:17.042783] Epoch: [6] [1500/2877] eta: 0:02:52 lr: 0.000049 loss: 0.6197 (0.6308) time: 0.1247 data: 0.0001 max mem: 5308 | |
| [00:54:29.623359] Epoch: [6] [1600/2877] eta: 0:02:40 lr: 0.000049 loss: 0.6336 (0.6308) time: 0.1257 data: 0.0001 max mem: 5308 | |
| [00:54:42.166389] Epoch: [6] [1700/2877] eta: 0:02:27 lr: 0.000048 loss: 0.6170 (0.6309) time: 0.1250 data: 0.0001 max mem: 5308 | |
| [00:54:54.776203] Epoch: [6] [1800/2877] eta: 0:02:15 lr: 0.000048 loss: 0.6475 (0.6308) time: 0.1259 data: 0.0001 max mem: 5308 | |
| [00:55:07.374464] Epoch: [6] [1900/2877] eta: 0:02:02 lr: 0.000047 loss: 0.6297 (0.6306) time: 0.1256 data: 0.0001 max mem: 5308 | |
| [00:55:19.915123] Epoch: [6] [2000/2877] eta: 0:01:50 lr: 0.000047 loss: 0.6358 (0.6312) time: 0.1255 data: 0.0001 max mem: 5308 | |
| [00:55:32.458316] Epoch: [6] [2100/2877] eta: 0:01:37 lr: 0.000046 loss: 0.6350 (0.6306) time: 0.1254 data: 0.0001 max mem: 5308 | |
| [00:55:45.000756] Epoch: [6] [2200/2877] eta: 0:01:25 lr: 0.000045 loss: 0.6371 (0.6305) time: 0.1260 data: 0.0001 max mem: 5308 | |
| [00:55:57.527789] Epoch: [6] [2300/2877] eta: 0:01:12 lr: 0.000045 loss: 0.6300 (0.6302) time: 0.1253 data: 0.0001 max mem: 5308 | |
| [00:56:10.054477] Epoch: [6] [2400/2877] eta: 0:00:59 lr: 0.000044 loss: 0.6274 (0.6301) time: 0.1252 data: 0.0001 max mem: 5308 | |
| [00:56:22.579854] Epoch: [6] [2500/2877] eta: 0:00:47 lr: 0.000044 loss: 0.6433 (0.6301) time: 0.1252 data: 0.0001 max mem: 5308 | |
| [00:56:35.102836] Epoch: [6] [2600/2877] eta: 0:00:34 lr: 0.000043 loss: 0.6139 (0.6298) time: 0.1250 data: 0.0001 max mem: 5308 | |
| [00:56:47.627313] Epoch: [6] [2700/2877] eta: 0:00:22 lr: 0.000042 loss: 0.6323 (0.6299) time: 0.1256 data: 0.0001 max mem: 5308 | |
| [00:57:00.147236] Epoch: [6] [2800/2877] eta: 0:00:09 lr: 0.000042 loss: 0.6214 (0.6301) time: 0.1251 data: 0.0001 max mem: 5308 | |
| [00:57:09.632640] Epoch: [6] [2876/2877] eta: 0:00:00 lr: 0.000041 loss: 0.6351 (0.6300) time: 0.1243 data: 0.0002 max mem: 5308 | |
| [00:57:09.833727] Epoch: [6] Total time: 0:06:01 (0.1256 s / it) | |
| [00:57:09.855876] Averaged stats: lr: 0.000041 loss: 0.6351 (0.6286) | |
| [00:57:11.089727] Test: [ 0/560] eta: 0:11:29 loss: 0.3152 (0.3152) auc: 96.0784 (96.0784) time: 1.2306 data: 1.1965 max mem: 5308 | |
| [00:57:11.430544] Test: [ 10/560] eta: 0:01:18 loss: 0.3369 (0.3541) auc: 94.1667 (92.9577) time: 0.1428 data: 0.1129 max mem: 5308 | |
| [00:57:11.724266] Test: [ 20/560] eta: 0:00:47 loss: 0.3322 (0.3237) auc: 94.6429 (94.7081) time: 0.0317 data: 0.0023 max mem: 5308 | |
| [00:57:12.018871] Test: [ 30/560] eta: 0:00:36 loss: 0.2635 (0.3032) auc: 97.5709 (95.3566) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [00:57:12.312491] Test: [ 40/560] eta: 0:00:31 loss: 0.2684 (0.3025) auc: 96.6667 (95.3752) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [00:57:12.606026] Test: [ 50/560] eta: 0:00:27 loss: 0.2769 (0.3024) auc: 95.2381 (95.5836) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [00:57:12.899856] Test: [ 60/560] eta: 0:00:24 loss: 0.3153 (0.3037) auc: 95.4167 (95.4739) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [00:57:13.193225] Test: [ 70/560] eta: 0:00:22 loss: 0.2950 (0.3009) auc: 95.4167 (95.5010) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [00:57:13.486073] Test: [ 80/560] eta: 0:00:21 loss: 0.2617 (0.2951) auc: 96.0317 (95.6791) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:57:13.779479] Test: [ 90/560] eta: 0:00:20 loss: 0.2432 (0.2947) auc: 96.8254 (95.7502) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:57:14.070692] Test: [100/560] eta: 0:00:19 loss: 0.2532 (0.2906) auc: 96.8254 (95.9114) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:57:14.361966] Test: [110/560] eta: 0:00:18 loss: 0.2608 (0.2904) auc: 96.9697 (95.8937) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:57:14.653284] Test: [120/560] eta: 0:00:17 loss: 0.3035 (0.2965) auc: 93.7198 (95.6860) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:57:14.944186] Test: [130/560] eta: 0:00:16 loss: 0.3088 (0.2978) auc: 94.6429 (95.7137) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:57:15.235374] Test: [140/560] eta: 0:00:15 loss: 0.2994 (0.2984) auc: 96.4844 (95.7124) time: 0.0290 data: 0.0001 max mem: 5308 | |
| [00:57:15.526789] Test: [150/560] eta: 0:00:15 loss: 0.2794 (0.2969) auc: 95.9514 (95.7346) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:57:15.818055] Test: [160/560] eta: 0:00:14 loss: 0.2472 (0.2923) auc: 98.0159 (95.9040) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:57:16.109118] Test: [170/560] eta: 0:00:14 loss: 0.2297 (0.2909) auc: 97.9167 (95.9473) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:57:16.399694] Test: [180/560] eta: 0:00:13 loss: 0.2545 (0.2916) auc: 97.5709 (95.9468) time: 0.0290 data: 0.0001 max mem: 5308 | |
| [00:57:16.690584] Test: [190/560] eta: 0:00:13 loss: 0.2947 (0.2923) auc: 96.0317 (95.9191) time: 0.0290 data: 0.0001 max mem: 5308 | |
| [00:57:16.982252] Test: [200/560] eta: 0:00:12 loss: 0.2833 (0.2904) auc: 95.6349 (95.9764) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:57:17.273096] Test: [210/560] eta: 0:00:12 loss: 0.2893 (0.2913) auc: 96.7611 (95.9413) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:57:17.564871] Test: [220/560] eta: 0:00:11 loss: 0.2901 (0.2910) auc: 96.7611 (95.9664) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:57:17.855611] Test: [230/560] eta: 0:00:11 loss: 0.2897 (0.2911) auc: 96.0784 (95.9685) time: 0.0290 data: 0.0001 max mem: 5308 | |
| [00:57:18.146887] Test: [240/560] eta: 0:00:10 loss: 0.2982 (0.2909) auc: 96.0317 (95.9659) time: 0.0290 data: 0.0001 max mem: 5308 | |
| [00:57:18.437540] Test: [250/560] eta: 0:00:10 loss: 0.3017 (0.2913) auc: 95.6863 (95.9489) time: 0.0290 data: 0.0001 max mem: 5308 | |
| [00:57:18.728977] Test: [260/560] eta: 0:00:10 loss: 0.2435 (0.2895) auc: 96.2500 (96.0034) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:57:19.020558] Test: [270/560] eta: 0:00:09 loss: 0.2808 (0.2904) auc: 95.0000 (95.9762) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:57:19.311668] Test: [280/560] eta: 0:00:09 loss: 0.2720 (0.2880) auc: 96.8254 (96.0529) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:57:19.603480] Test: [290/560] eta: 0:00:09 loss: 0.2720 (0.2888) auc: 97.9167 (96.0496) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:57:19.894717] Test: [300/560] eta: 0:00:08 loss: 0.2885 (0.2879) auc: 96.3563 (96.0825) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:57:20.186341] Test: [310/560] eta: 0:00:08 loss: 0.2698 (0.2883) auc: 96.0784 (96.0512) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:57:20.477844] Test: [320/560] eta: 0:00:07 loss: 0.2904 (0.2886) auc: 95.6863 (96.0351) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:57:20.768538] Test: [330/560] eta: 0:00:07 loss: 0.2789 (0.2882) auc: 96.0417 (96.0405) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:57:21.060463] Test: [340/560] eta: 0:00:07 loss: 0.2700 (0.2878) auc: 96.0784 (96.0502) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:57:21.352135] Test: [350/560] eta: 0:00:06 loss: 0.2623 (0.2876) auc: 96.0784 (96.0496) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:57:21.642933] Test: [360/560] eta: 0:00:06 loss: 0.2944 (0.2887) auc: 96.0784 (96.0334) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:57:21.933338] Test: [370/560] eta: 0:00:06 loss: 0.3074 (0.2891) auc: 96.4286 (96.0230) time: 0.0290 data: 0.0001 max mem: 5308 | |
| [00:57:22.224891] Test: [380/560] eta: 0:00:05 loss: 0.2839 (0.2889) auc: 96.0784 (96.0354) time: 0.0290 data: 0.0001 max mem: 5308 | |
| [00:57:22.516879] Test: [390/560] eta: 0:00:05 loss: 0.2736 (0.2893) auc: 96.4286 (96.0216) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:57:22.808209] Test: [400/560] eta: 0:00:05 loss: 0.2790 (0.2904) auc: 96.4286 (95.9658) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:57:23.100367] Test: [410/560] eta: 0:00:04 loss: 0.3138 (0.2909) auc: 94.1176 (95.9280) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:57:23.392386] Test: [420/560] eta: 0:00:04 loss: 0.3102 (0.2910) auc: 94.5098 (95.9083) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:57:23.684001] Test: [430/560] eta: 0:00:04 loss: 0.2360 (0.2900) auc: 97.2549 (95.9382) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:57:23.975538] Test: [440/560] eta: 0:00:03 loss: 0.2216 (0.2891) auc: 98.4127 (95.9747) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:57:24.266611] Test: [450/560] eta: 0:00:03 loss: 0.2510 (0.2891) auc: 97.9757 (95.9819) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:57:24.558965] Test: [460/560] eta: 0:00:03 loss: 0.3055 (0.2899) auc: 97.1660 (95.9681) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:57:24.850513] Test: [470/560] eta: 0:00:02 loss: 0.3258 (0.2903) auc: 95.6349 (95.9551) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:57:25.143914] Test: [480/560] eta: 0:00:02 loss: 0.2915 (0.2901) auc: 94.5098 (95.9295) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:57:25.435696] Test: [490/560] eta: 0:00:02 loss: 0.2696 (0.2894) auc: 95.2381 (95.9511) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:57:25.727466] Test: [500/560] eta: 0:00:01 loss: 0.2494 (0.2890) auc: 96.6667 (95.9653) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:57:26.019498] Test: [510/560] eta: 0:00:01 loss: 0.2494 (0.2886) auc: 96.3636 (95.9733) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:57:26.311854] Test: [520/560] eta: 0:00:01 loss: 0.2964 (0.2897) auc: 94.9219 (95.9298) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:57:26.603653] Test: [530/560] eta: 0:00:00 loss: 0.3291 (0.2907) auc: 94.9219 (95.9282) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [00:57:26.895733] Test: [540/560] eta: 0:00:00 loss: 0.3206 (0.2912) auc: 96.1353 (95.9222) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [00:57:27.184803] Test: [550/560] eta: 0:00:00 loss: 0.2766 (0.2907) auc: 98.8095 (95.9683) time: 0.0290 data: 0.0001 max mem: 5308 | |
| [00:57:27.426875] Test: [559/560] eta: 0:00:00 loss: 0.2626 (0.2908) auc: 98.7854 (95.9745) time: 0.0280 data: 0.0001 max mem: 5308 | |
| [00:57:27.567442] Test: Total time: 0:00:17 (0.0316 s / it) | |
| [00:57:27.807525] * Auc 95.957 loss 0.291 | |
| [00:57:27.807856] AUC of the network on the 35796 val images: 95.96% | |
| [00:57:27.807867] Max auc: 95.96% | |
| [00:57:27.807876] Save model with min_val_loss at epoch: 6 | |
| [00:57:33.274146] log_dir: ./checkpoint/finetuned_models/FF++_c23_32frames | |
| [00:57:34.116864] Epoch: [7] [ 0/2877] eta: 0:40:17 lr: 0.000041 loss: 0.6367 (0.6367) time: 0.8404 data: 0.7034 max mem: 5308 | |
| [00:57:46.695106] Epoch: [7] [ 100/2877] eta: 0:06:08 lr: 0.000041 loss: 0.6129 (0.6186) time: 0.1257 data: 0.0001 max mem: 5308 | |
| [00:57:59.269698] Epoch: [7] [ 200/2877] eta: 0:05:46 lr: 0.000040 loss: 0.6395 (0.6242) time: 0.1262 data: 0.0001 max mem: 5308 | |
| [00:58:11.854887] Epoch: [7] [ 300/2877] eta: 0:05:30 lr: 0.000039 loss: 0.6434 (0.6244) time: 0.1262 data: 0.0001 max mem: 5308 | |
| [00:58:24.433363] Epoch: [7] [ 400/2877] eta: 0:05:15 lr: 0.000039 loss: 0.6400 (0.6259) time: 0.1255 data: 0.0001 max mem: 5308 | |
| [00:58:36.950589] Epoch: [7] [ 500/2877] eta: 0:05:02 lr: 0.000038 loss: 0.6243 (0.6278) time: 0.1257 data: 0.0001 max mem: 5308 | |
| [00:58:49.400245] Epoch: [7] [ 600/2877] eta: 0:04:48 lr: 0.000037 loss: 0.6134 (0.6268) time: 0.1238 data: 0.0001 max mem: 5308 | |
| [00:59:01.795593] Epoch: [7] [ 700/2877] eta: 0:04:34 lr: 0.000037 loss: 0.6317 (0.6275) time: 0.1240 data: 0.0001 max mem: 5308 | |
| [00:59:14.193540] Epoch: [7] [ 800/2877] eta: 0:04:21 lr: 0.000036 loss: 0.6211 (0.6270) time: 0.1243 data: 0.0001 max mem: 5308 | |
| [00:59:26.589997] Epoch: [7] [ 900/2877] eta: 0:04:08 lr: 0.000035 loss: 0.6230 (0.6256) time: 0.1242 data: 0.0001 max mem: 5308 | |
| [00:59:38.993289] Epoch: [7] [1000/2877] eta: 0:03:55 lr: 0.000035 loss: 0.6272 (0.6249) time: 0.1240 data: 0.0001 max mem: 5308 | |
| [00:59:51.389384] Epoch: [7] [1100/2877] eta: 0:03:42 lr: 0.000034 loss: 0.6261 (0.6251) time: 0.1238 data: 0.0001 max mem: 5308 | |
| [01:00:03.783923] Epoch: [7] [1200/2877] eta: 0:03:30 lr: 0.000033 loss: 0.6326 (0.6244) time: 0.1241 data: 0.0001 max mem: 5308 | |
| [01:00:16.203556] Epoch: [7] [1300/2877] eta: 0:03:17 lr: 0.000033 loss: 0.6146 (0.6245) time: 0.1241 data: 0.0001 max mem: 5308 | |
| [01:00:28.609128] Epoch: [7] [1400/2877] eta: 0:03:04 lr: 0.000032 loss: 0.6135 (0.6237) time: 0.1242 data: 0.0001 max mem: 5308 | |
| [01:00:41.011156] Epoch: [7] [1500/2877] eta: 0:02:52 lr: 0.000031 loss: 0.6286 (0.6233) time: 0.1239 data: 0.0001 max mem: 5308 | |
| [01:00:53.414680] Epoch: [7] [1600/2877] eta: 0:02:39 lr: 0.000031 loss: 0.6257 (0.6235) time: 0.1240 data: 0.0001 max mem: 5308 | |
| [01:01:05.862996] Epoch: [7] [1700/2877] eta: 0:02:27 lr: 0.000030 loss: 0.6087 (0.6238) time: 0.1266 data: 0.0001 max mem: 5308 | |
| [01:01:18.562776] Epoch: [7] [1800/2877] eta: 0:02:14 lr: 0.000029 loss: 0.6253 (0.6239) time: 0.1275 data: 0.0001 max mem: 5308 | |
| [01:01:31.290130] Epoch: [7] [1900/2877] eta: 0:02:02 lr: 0.000029 loss: 0.5862 (0.6233) time: 0.1275 data: 0.0001 max mem: 5308 | |
| [01:01:43.967953] Epoch: [7] [2000/2877] eta: 0:01:49 lr: 0.000028 loss: 0.6206 (0.6230) time: 0.1262 data: 0.0001 max mem: 5308 | |
| [01:01:56.590858] Epoch: [7] [2100/2877] eta: 0:01:37 lr: 0.000027 loss: 0.5977 (0.6233) time: 0.1263 data: 0.0001 max mem: 5308 | |
| [01:02:09.202200] Epoch: [7] [2200/2877] eta: 0:01:24 lr: 0.000027 loss: 0.6593 (0.6237) time: 0.1254 data: 0.0001 max mem: 5308 | |
| [01:02:21.729047] Epoch: [7] [2300/2877] eta: 0:01:12 lr: 0.000026 loss: 0.6463 (0.6241) time: 0.1256 data: 0.0001 max mem: 5308 | |
| [01:02:34.260483] Epoch: [7] [2400/2877] eta: 0:00:59 lr: 0.000025 loss: 0.6176 (0.6242) time: 0.1253 data: 0.0001 max mem: 5308 | |
| [01:02:46.776117] Epoch: [7] [2500/2877] eta: 0:00:47 lr: 0.000025 loss: 0.6550 (0.6244) time: 0.1254 data: 0.0001 max mem: 5308 | |
| [01:02:59.310650] Epoch: [7] [2600/2877] eta: 0:00:34 lr: 0.000024 loss: 0.6365 (0.6236) time: 0.1257 data: 0.0001 max mem: 5308 | |
| [01:03:11.841252] Epoch: [7] [2700/2877] eta: 0:00:22 lr: 0.000023 loss: 0.6156 (0.6236) time: 0.1255 data: 0.0001 max mem: 5308 | |
| [01:03:24.377411] Epoch: [7] [2800/2877] eta: 0:00:09 lr: 0.000023 loss: 0.6397 (0.6238) time: 0.1254 data: 0.0001 max mem: 5308 | |
| [01:03:33.896735] Epoch: [7] [2876/2877] eta: 0:00:00 lr: 0.000022 loss: 0.6353 (0.6237) time: 0.1245 data: 0.0002 max mem: 5308 | |
| [01:03:34.127837] Epoch: [7] Total time: 0:06:00 (0.1254 s / it) | |
| [01:03:34.129357] Averaged stats: lr: 0.000022 loss: 0.6353 (0.6236) | |
| [01:03:35.500795] Test: [ 0/560] eta: 0:12:45 loss: 0.2804 (0.2804) auc: 97.2549 (97.2549) time: 1.3669 data: 1.3310 max mem: 5308 | |
| [01:03:35.794019] Test: [ 10/560] eta: 0:01:22 loss: 0.3160 (0.3266) auc: 94.5833 (93.2085) time: 0.1508 data: 0.1211 max mem: 5308 | |
| [01:03:36.088288] Test: [ 20/560] eta: 0:00:50 loss: 0.2878 (0.2932) auc: 95.6349 (95.1651) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [01:03:36.382229] Test: [ 30/560] eta: 0:00:38 loss: 0.2503 (0.2785) auc: 97.6471 (95.6137) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [01:03:36.676924] Test: [ 40/560] eta: 0:00:32 loss: 0.2523 (0.2792) auc: 97.2222 (95.5100) time: 0.0294 data: 0.0001 max mem: 5308 | |
| [01:03:36.970211] Test: [ 50/560] eta: 0:00:28 loss: 0.2580 (0.2761) auc: 96.2302 (95.8369) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [01:03:37.263089] Test: [ 60/560] eta: 0:00:25 loss: 0.2975 (0.2777) auc: 96.3636 (95.7366) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [01:03:37.557469] Test: [ 70/560] eta: 0:00:23 loss: 0.2685 (0.2763) auc: 96.3636 (95.7890) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [01:03:37.850336] Test: [ 80/560] eta: 0:00:21 loss: 0.2400 (0.2708) auc: 97.5709 (95.9685) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [01:03:38.140756] Test: [ 90/560] eta: 0:00:20 loss: 0.2479 (0.2700) auc: 96.8750 (96.0333) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:03:38.431825] Test: [100/560] eta: 0:00:19 loss: 0.2479 (0.2661) auc: 97.2222 (96.2029) time: 0.0290 data: 0.0001 max mem: 5308 | |
| [01:03:38.722760] Test: [110/560] eta: 0:00:18 loss: 0.2361 (0.2658) auc: 97.5709 (96.1847) time: 0.0290 data: 0.0001 max mem: 5308 | |
| [01:03:39.013666] Test: [120/560] eta: 0:00:17 loss: 0.2871 (0.2712) auc: 94.2029 (95.9796) time: 0.0290 data: 0.0001 max mem: 5308 | |
| [01:03:39.304184] Test: [130/560] eta: 0:00:16 loss: 0.2743 (0.2716) auc: 94.9393 (96.0059) time: 0.0290 data: 0.0001 max mem: 5308 | |
| [01:03:39.595637] Test: [140/560] eta: 0:00:16 loss: 0.2607 (0.2716) auc: 96.7611 (96.0276) time: 0.0290 data: 0.0001 max mem: 5308 | |
| [01:03:39.887532] Test: [150/560] eta: 0:00:15 loss: 0.2475 (0.2706) auc: 97.1660 (96.0421) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:03:40.178121] Test: [160/560] eta: 0:00:14 loss: 0.2266 (0.2666) auc: 98.4314 (96.1835) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:03:40.469189] Test: [170/560] eta: 0:00:14 loss: 0.2215 (0.2655) auc: 98.0392 (96.2151) time: 0.0290 data: 0.0001 max mem: 5308 | |
| [01:03:40.759662] Test: [180/560] eta: 0:00:13 loss: 0.2457 (0.2659) auc: 97.1660 (96.2138) time: 0.0290 data: 0.0001 max mem: 5308 | |
| [01:03:41.050525] Test: [190/560] eta: 0:00:13 loss: 0.2640 (0.2663) auc: 96.0784 (96.1995) time: 0.0290 data: 0.0001 max mem: 5308 | |
| [01:03:41.341142] Test: [200/560] eta: 0:00:12 loss: 0.2582 (0.2647) auc: 96.0938 (96.2578) time: 0.0290 data: 0.0001 max mem: 5308 | |
| [01:03:41.632434] Test: [210/560] eta: 0:00:12 loss: 0.2517 (0.2657) auc: 97.0833 (96.2255) time: 0.0290 data: 0.0001 max mem: 5308 | |
| [01:03:41.924001] Test: [220/560] eta: 0:00:11 loss: 0.2545 (0.2659) auc: 97.0833 (96.2420) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:03:42.215238] Test: [230/560] eta: 0:00:11 loss: 0.2545 (0.2660) auc: 96.8627 (96.2288) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:03:42.506525] Test: [240/560] eta: 0:00:11 loss: 0.2486 (0.2659) auc: 96.4706 (96.2313) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:03:42.797981] Test: [250/560] eta: 0:00:10 loss: 0.2745 (0.2664) auc: 96.4706 (96.2166) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:03:43.088888] Test: [260/560] eta: 0:00:10 loss: 0.2434 (0.2651) auc: 96.6667 (96.2575) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:03:43.380368] Test: [270/560] eta: 0:00:09 loss: 0.2765 (0.2659) auc: 96.4286 (96.2298) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:03:43.672006] Test: [280/560] eta: 0:00:09 loss: 0.2304 (0.2635) auc: 97.2222 (96.3077) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:03:43.963257] Test: [290/560] eta: 0:00:09 loss: 0.2411 (0.2641) auc: 97.9167 (96.2991) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:03:44.253946] Test: [300/560] eta: 0:00:08 loss: 0.2502 (0.2634) auc: 96.5587 (96.3297) time: 0.0290 data: 0.0001 max mem: 5308 | |
| [01:03:44.545426] Test: [310/560] eta: 0:00:08 loss: 0.2377 (0.2639) auc: 96.0784 (96.2910) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:03:44.836873] Test: [320/560] eta: 0:00:07 loss: 0.2653 (0.2643) auc: 96.0784 (96.2756) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:03:45.128318] Test: [330/560] eta: 0:00:07 loss: 0.2607 (0.2642) auc: 96.7611 (96.2714) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:03:45.419214] Test: [340/560] eta: 0:00:07 loss: 0.2518 (0.2637) auc: 97.2222 (96.2886) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:03:45.710522] Test: [350/560] eta: 0:00:06 loss: 0.2518 (0.2638) auc: 96.6667 (96.2853) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:03:46.001570] Test: [360/560] eta: 0:00:06 loss: 0.2743 (0.2646) auc: 96.0317 (96.2739) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:03:46.293083] Test: [370/560] eta: 0:00:06 loss: 0.2743 (0.2649) auc: 96.2500 (96.2597) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:03:46.583727] Test: [380/560] eta: 0:00:05 loss: 0.2427 (0.2647) auc: 97.1660 (96.2774) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:03:46.875074] Test: [390/560] eta: 0:00:05 loss: 0.2339 (0.2648) auc: 97.6190 (96.2774) time: 0.0290 data: 0.0001 max mem: 5308 | |
| [01:03:47.166270] Test: [400/560] eta: 0:00:05 loss: 0.2628 (0.2657) auc: 96.4844 (96.2288) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:03:47.457823] Test: [410/560] eta: 0:00:04 loss: 0.2862 (0.2662) auc: 95.2381 (96.1946) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:03:47.749759] Test: [420/560] eta: 0:00:04 loss: 0.2902 (0.2665) auc: 95.2381 (96.1666) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:03:48.041195] Test: [430/560] eta: 0:00:04 loss: 0.2337 (0.2656) auc: 97.2222 (96.2036) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:03:48.333068] Test: [440/560] eta: 0:00:03 loss: 0.2272 (0.2650) auc: 97.9757 (96.2268) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:03:48.624644] Test: [450/560] eta: 0:00:03 loss: 0.2293 (0.2651) auc: 97.7733 (96.2265) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:03:48.916137] Test: [460/560] eta: 0:00:03 loss: 0.2627 (0.2655) auc: 96.8254 (96.2162) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:03:49.208057] Test: [470/560] eta: 0:00:02 loss: 0.2862 (0.2660) auc: 96.0784 (96.2105) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:03:49.499393] Test: [480/560] eta: 0:00:02 loss: 0.2681 (0.2660) auc: 96.0784 (96.1890) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:03:49.791344] Test: [490/560] eta: 0:00:02 loss: 0.2418 (0.2655) auc: 96.0938 (96.2103) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:03:50.082870] Test: [500/560] eta: 0:00:01 loss: 0.2452 (0.2651) auc: 97.1660 (96.2262) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:03:50.374732] Test: [510/560] eta: 0:00:01 loss: 0.2594 (0.2645) auc: 96.4706 (96.2345) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:03:50.666248] Test: [520/560] eta: 0:00:01 loss: 0.2717 (0.2655) auc: 95.9514 (96.1982) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:03:50.957802] Test: [530/560] eta: 0:00:00 loss: 0.2986 (0.2660) auc: 95.6863 (96.1974) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:03:51.249633] Test: [540/560] eta: 0:00:00 loss: 0.2673 (0.2663) auc: 96.4286 (96.1839) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:03:51.539109] Test: [550/560] eta: 0:00:00 loss: 0.2385 (0.2656) auc: 98.4127 (96.2246) time: 0.0290 data: 0.0001 max mem: 5308 | |
| [01:03:51.781151] Test: [559/560] eta: 0:00:00 loss: 0.2417 (0.2658) auc: 98.0159 (96.2217) time: 0.0280 data: 0.0001 max mem: 5308 | |
| [01:03:51.950791] Test: Total time: 0:00:17 (0.0318 s / it) | |
| [01:03:52.060250] * Auc 96.194 loss 0.265 | |
| [01:03:52.060404] AUC of the network on the 35796 val images: 96.19% | |
| [01:03:52.060416] Max auc: 96.19% | |
| [01:03:52.060426] Save model with min_val_loss at epoch: 7 | |
| [01:03:57.597585] log_dir: ./checkpoint/finetuned_models/FF++_c23_32frames | |
| [01:03:58.616622] Epoch: [8] [ 0/2877] eta: 0:48:43 lr: 0.000022 loss: 0.6962 (0.6962) time: 1.0162 data: 0.8766 max mem: 5308 | |
| [01:04:11.118234] Epoch: [8] [ 100/2877] eta: 0:06:11 lr: 0.000022 loss: 0.6390 (0.6219) time: 0.1247 data: 0.0001 max mem: 5308 | |
| [01:04:23.621199] Epoch: [8] [ 200/2877] eta: 0:05:46 lr: 0.000021 loss: 0.6071 (0.6208) time: 0.1254 data: 0.0001 max mem: 5308 | |
| [01:04:36.120321] Epoch: [8] [ 300/2877] eta: 0:05:29 lr: 0.000020 loss: 0.6445 (0.6232) time: 0.1250 data: 0.0001 max mem: 5308 | |
| [01:04:48.642848] Epoch: [8] [ 400/2877] eta: 0:05:15 lr: 0.000020 loss: 0.5857 (0.6225) time: 0.1247 data: 0.0001 max mem: 5308 | |
| [01:05:01.139258] Epoch: [8] [ 500/2877] eta: 0:05:01 lr: 0.000019 loss: 0.6168 (0.6209) time: 0.1246 data: 0.0001 max mem: 5308 | |
| [01:05:13.763883] Epoch: [8] [ 600/2877] eta: 0:04:48 lr: 0.000019 loss: 0.6445 (0.6232) time: 0.1249 data: 0.0001 max mem: 5308 | |
| [01:05:26.363935] Epoch: [8] [ 700/2877] eta: 0:04:35 lr: 0.000018 loss: 0.6126 (0.6242) time: 0.1264 data: 0.0001 max mem: 5308 | |
| [01:05:39.017757] Epoch: [8] [ 800/2877] eta: 0:04:22 lr: 0.000017 loss: 0.6363 (0.6234) time: 0.1262 data: 0.0001 max mem: 5308 | |
| [01:05:51.644199] Epoch: [8] [ 900/2877] eta: 0:04:10 lr: 0.000017 loss: 0.6205 (0.6228) time: 0.1253 data: 0.0001 max mem: 5308 | |
| [01:06:04.330699] Epoch: [8] [1000/2877] eta: 0:03:57 lr: 0.000016 loss: 0.6318 (0.6231) time: 0.1268 data: 0.0001 max mem: 5308 | |
| [01:06:16.986229] Epoch: [8] [1100/2877] eta: 0:03:44 lr: 0.000016 loss: 0.6062 (0.6225) time: 0.1265 data: 0.0001 max mem: 5308 | |
| [01:06:29.655257] Epoch: [8] [1200/2877] eta: 0:03:32 lr: 0.000015 loss: 0.6201 (0.6223) time: 0.1260 data: 0.0001 max mem: 5308 | |
| [01:06:42.250083] Epoch: [8] [1300/2877] eta: 0:03:19 lr: 0.000014 loss: 0.6430 (0.6222) time: 0.1252 data: 0.0001 max mem: 5308 | |
| [01:06:54.828924] Epoch: [8] [1400/2877] eta: 0:03:06 lr: 0.000014 loss: 0.6127 (0.6219) time: 0.1261 data: 0.0001 max mem: 5308 | |
| [01:07:07.444810] Epoch: [8] [1500/2877] eta: 0:02:54 lr: 0.000013 loss: 0.6248 (0.6217) time: 0.1260 data: 0.0001 max mem: 5308 | |
| [01:07:20.119861] Epoch: [8] [1600/2877] eta: 0:02:41 lr: 0.000013 loss: 0.6255 (0.6222) time: 0.1245 data: 0.0001 max mem: 5308 | |
| [01:07:32.586340] Epoch: [8] [1700/2877] eta: 0:02:28 lr: 0.000012 loss: 0.6227 (0.6221) time: 0.1259 data: 0.0001 max mem: 5308 | |
| [01:07:45.209583] Epoch: [8] [1800/2877] eta: 0:02:16 lr: 0.000012 loss: 0.6198 (0.6222) time: 0.1260 data: 0.0001 max mem: 5308 | |
| [01:07:57.768397] Epoch: [8] [1900/2877] eta: 0:02:03 lr: 0.000011 loss: 0.5774 (0.6218) time: 0.1256 data: 0.0001 max mem: 5308 | |
| [01:08:10.364510] Epoch: [8] [2000/2877] eta: 0:01:50 lr: 0.000011 loss: 0.6533 (0.6222) time: 0.1257 data: 0.0001 max mem: 5308 | |
| [01:08:22.937602] Epoch: [8] [2100/2877] eta: 0:01:38 lr: 0.000010 loss: 0.5891 (0.6218) time: 0.1258 data: 0.0002 max mem: 5308 | |
| [01:08:35.481542] Epoch: [8] [2200/2877] eta: 0:01:25 lr: 0.000010 loss: 0.6360 (0.6213) time: 0.1254 data: 0.0002 max mem: 5308 | |
| [01:08:48.033869] Epoch: [8] [2300/2877] eta: 0:01:12 lr: 0.000009 loss: 0.6396 (0.6213) time: 0.1256 data: 0.0001 max mem: 5308 | |
| [01:09:00.615105] Epoch: [8] [2400/2877] eta: 0:01:00 lr: 0.000009 loss: 0.6095 (0.6208) time: 0.1256 data: 0.0001 max mem: 5308 | |
| [01:09:13.180280] Epoch: [8] [2500/2877] eta: 0:00:47 lr: 0.000008 loss: 0.6199 (0.6206) time: 0.1258 data: 0.0001 max mem: 5308 | |
| [01:09:25.741711] Epoch: [8] [2600/2877] eta: 0:00:34 lr: 0.000008 loss: 0.6360 (0.6204) time: 0.1253 data: 0.0001 max mem: 5308 | |
| [01:09:38.311900] Epoch: [8] [2700/2877] eta: 0:00:22 lr: 0.000008 loss: 0.6387 (0.6202) time: 0.1262 data: 0.0001 max mem: 5308 | |
| [01:09:50.879523] Epoch: [8] [2800/2877] eta: 0:00:09 lr: 0.000007 loss: 0.6330 (0.6200) time: 0.1252 data: 0.0001 max mem: 5308 | |
| [01:10:00.371399] Epoch: [8] [2876/2877] eta: 0:00:00 lr: 0.000007 loss: 0.5983 (0.6200) time: 0.1239 data: 0.0003 max mem: 5308 | |
| [01:10:00.586492] Epoch: [8] Total time: 0:06:02 (0.1262 s / it) | |
| [01:10:00.602296] Averaged stats: lr: 0.000007 loss: 0.5983 (0.6206) | |
| [01:10:02.136995] Test: [ 0/560] eta: 0:14:16 loss: 0.2754 (0.2754) auc: 97.2549 (97.2549) time: 1.5290 data: 1.4938 max mem: 5308 | |
| [01:10:02.438105] Test: [ 10/560] eta: 0:01:31 loss: 0.3183 (0.3282) auc: 94.3750 (93.2773) time: 0.1663 data: 0.1360 max mem: 5308 | |
| [01:10:02.738848] Test: [ 20/560] eta: 0:00:54 loss: 0.2958 (0.2975) auc: 95.6349 (95.3388) time: 0.0300 data: 0.0002 max mem: 5308 | |
| [01:10:03.037895] Test: [ 30/560] eta: 0:00:41 loss: 0.2550 (0.2818) auc: 98.3806 (95.7652) time: 0.0299 data: 0.0002 max mem: 5308 | |
| [01:10:03.339161] Test: [ 40/560] eta: 0:00:34 loss: 0.2596 (0.2808) auc: 97.0833 (95.7116) time: 0.0299 data: 0.0002 max mem: 5308 | |
| [01:10:03.639676] Test: [ 50/560] eta: 0:00:30 loss: 0.2617 (0.2794) auc: 96.4286 (96.0119) time: 0.0300 data: 0.0002 max mem: 5308 | |
| [01:10:03.942354] Test: [ 60/560] eta: 0:00:27 loss: 0.2936 (0.2812) auc: 96.4286 (95.9175) time: 0.0301 data: 0.0002 max mem: 5308 | |
| [01:10:04.243264] Test: [ 70/560] eta: 0:00:25 loss: 0.2743 (0.2793) auc: 96.8627 (95.9671) time: 0.0301 data: 0.0002 max mem: 5308 | |
| [01:10:04.546704] Test: [ 80/560] eta: 0:00:23 loss: 0.2441 (0.2734) auc: 97.5709 (96.1631) time: 0.0301 data: 0.0002 max mem: 5308 | |
| [01:10:04.842942] Test: [ 90/560] eta: 0:00:21 loss: 0.2341 (0.2726) auc: 97.2222 (96.2109) time: 0.0299 data: 0.0002 max mem: 5308 | |
| [01:10:05.140069] Test: [100/560] eta: 0:00:20 loss: 0.2347 (0.2688) auc: 96.9697 (96.3751) time: 0.0296 data: 0.0002 max mem: 5308 | |
| [01:10:05.435727] Test: [110/560] eta: 0:00:19 loss: 0.2347 (0.2684) auc: 97.1660 (96.3786) time: 0.0296 data: 0.0002 max mem: 5308 | |
| [01:10:05.729658] Test: [120/560] eta: 0:00:18 loss: 0.2726 (0.2741) auc: 94.5312 (96.1964) time: 0.0294 data: 0.0002 max mem: 5308 | |
| [01:10:06.027358] Test: [130/560] eta: 0:00:17 loss: 0.2768 (0.2750) auc: 95.3441 (96.2212) time: 0.0295 data: 0.0002 max mem: 5308 | |
| [01:10:06.329352] Test: [140/560] eta: 0:00:17 loss: 0.2729 (0.2753) auc: 96.8627 (96.2604) time: 0.0299 data: 0.0002 max mem: 5308 | |
| [01:10:06.630612] Test: [150/560] eta: 0:00:16 loss: 0.2596 (0.2740) auc: 97.5709 (96.2675) time: 0.0301 data: 0.0002 max mem: 5308 | |
| [01:10:06.931903] Test: [160/560] eta: 0:00:15 loss: 0.2313 (0.2694) auc: 98.4314 (96.4105) time: 0.0300 data: 0.0002 max mem: 5308 | |
| [01:10:07.233926] Test: [170/560] eta: 0:00:15 loss: 0.2190 (0.2682) auc: 98.3806 (96.4370) time: 0.0301 data: 0.0002 max mem: 5308 | |
| [01:10:07.535288] Test: [180/560] eta: 0:00:14 loss: 0.2434 (0.2686) auc: 97.5709 (96.4521) time: 0.0301 data: 0.0002 max mem: 5308 | |
| [01:10:07.829823] Test: [190/560] eta: 0:00:13 loss: 0.2718 (0.2691) auc: 96.4706 (96.4486) time: 0.0297 data: 0.0002 max mem: 5308 | |
| [01:10:08.125182] Test: [200/560] eta: 0:00:13 loss: 0.2584 (0.2673) auc: 96.3563 (96.5113) time: 0.0294 data: 0.0002 max mem: 5308 | |
| [01:10:08.421369] Test: [210/560] eta: 0:00:12 loss: 0.2584 (0.2684) auc: 97.5000 (96.4752) time: 0.0295 data: 0.0002 max mem: 5308 | |
| [01:10:08.716651] Test: [220/560] eta: 0:00:12 loss: 0.2662 (0.2684) auc: 97.5000 (96.4914) time: 0.0295 data: 0.0001 max mem: 5308 | |
| [01:10:09.013390] Test: [230/560] eta: 0:00:11 loss: 0.2622 (0.2685) auc: 96.8627 (96.4778) time: 0.0295 data: 0.0001 max mem: 5308 | |
| [01:10:09.308984] Test: [240/560] eta: 0:00:11 loss: 0.2622 (0.2684) auc: 96.8627 (96.4781) time: 0.0295 data: 0.0002 max mem: 5308 | |
| [01:10:09.603617] Test: [250/560] eta: 0:00:11 loss: 0.2870 (0.2691) auc: 96.6667 (96.4596) time: 0.0294 data: 0.0002 max mem: 5308 | |
| [01:10:09.899882] Test: [260/560] eta: 0:00:10 loss: 0.2332 (0.2676) auc: 97.2549 (96.4988) time: 0.0295 data: 0.0002 max mem: 5308 | |
| [01:10:10.192757] Test: [270/560] eta: 0:00:10 loss: 0.2728 (0.2684) auc: 96.4706 (96.4777) time: 0.0294 data: 0.0002 max mem: 5308 | |
| [01:10:10.487449] Test: [280/560] eta: 0:00:09 loss: 0.2354 (0.2659) auc: 97.6190 (96.5461) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [01:10:10.780532] Test: [290/560] eta: 0:00:09 loss: 0.2471 (0.2667) auc: 97.9167 (96.5354) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [01:10:11.073230] Test: [300/560] eta: 0:00:09 loss: 0.2603 (0.2659) auc: 97.1660 (96.5654) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [01:10:11.369712] Test: [310/560] eta: 0:00:08 loss: 0.2490 (0.2663) auc: 96.4706 (96.5318) time: 0.0294 data: 0.0002 max mem: 5308 | |
| [01:10:11.671027] Test: [320/560] eta: 0:00:08 loss: 0.2725 (0.2666) auc: 96.4706 (96.5182) time: 0.0298 data: 0.0002 max mem: 5308 | |
| [01:10:11.965639] Test: [330/560] eta: 0:00:07 loss: 0.2695 (0.2665) auc: 97.1660 (96.5135) time: 0.0297 data: 0.0002 max mem: 5308 | |
| [01:10:12.260077] Test: [340/560] eta: 0:00:07 loss: 0.2572 (0.2661) auc: 97.1660 (96.5231) time: 0.0294 data: 0.0002 max mem: 5308 | |
| [01:10:12.555143] Test: [350/560] eta: 0:00:07 loss: 0.2522 (0.2661) auc: 96.2500 (96.5183) time: 0.0294 data: 0.0002 max mem: 5308 | |
| [01:10:12.849546] Test: [360/560] eta: 0:00:06 loss: 0.2813 (0.2670) auc: 96.2500 (96.5038) time: 0.0294 data: 0.0002 max mem: 5308 | |
| [01:10:13.147945] Test: [370/560] eta: 0:00:06 loss: 0.2865 (0.2674) auc: 96.4286 (96.4792) time: 0.0296 data: 0.0002 max mem: 5308 | |
| [01:10:13.442243] Test: [380/560] eta: 0:00:06 loss: 0.2473 (0.2672) auc: 97.1660 (96.4978) time: 0.0296 data: 0.0002 max mem: 5308 | |
| [01:10:13.737050] Test: [390/560] eta: 0:00:05 loss: 0.2379 (0.2674) auc: 97.6190 (96.4987) time: 0.0294 data: 0.0001 max mem: 5308 | |
| [01:10:14.031616] Test: [400/560] eta: 0:00:05 loss: 0.2594 (0.2683) auc: 96.8254 (96.4631) time: 0.0294 data: 0.0001 max mem: 5308 | |
| [01:10:14.325437] Test: [410/560] eta: 0:00:04 loss: 0.2852 (0.2688) auc: 95.2381 (96.4242) time: 0.0294 data: 0.0001 max mem: 5308 | |
| [01:10:14.621015] Test: [420/560] eta: 0:00:04 loss: 0.2852 (0.2690) auc: 96.0784 (96.4048) time: 0.0294 data: 0.0001 max mem: 5308 | |
| [01:10:14.915630] Test: [430/560] eta: 0:00:04 loss: 0.2291 (0.2680) auc: 97.9757 (96.4436) time: 0.0294 data: 0.0001 max mem: 5308 | |
| [01:10:15.207293] Test: [440/560] eta: 0:00:03 loss: 0.2124 (0.2673) auc: 98.4127 (96.4713) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [01:10:15.499454] Test: [450/560] eta: 0:00:03 loss: 0.2318 (0.2674) auc: 98.0159 (96.4662) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:10:15.791883] Test: [460/560] eta: 0:00:03 loss: 0.2797 (0.2680) auc: 96.9697 (96.4568) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [01:10:16.087897] Test: [470/560] eta: 0:00:02 loss: 0.2826 (0.2683) auc: 96.4706 (96.4567) time: 0.0294 data: 0.0001 max mem: 5308 | |
| [01:10:16.378443] Test: [480/560] eta: 0:00:02 loss: 0.2681 (0.2683) auc: 96.2500 (96.4371) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [01:10:16.669902] Test: [490/560] eta: 0:00:02 loss: 0.2446 (0.2676) auc: 96.2891 (96.4565) time: 0.0290 data: 0.0001 max mem: 5308 | |
| [01:10:16.962120] Test: [500/560] eta: 0:00:01 loss: 0.2446 (0.2672) auc: 97.9757 (96.4736) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:10:17.253615] Test: [510/560] eta: 0:00:01 loss: 0.2504 (0.2667) auc: 96.4706 (96.4854) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:10:17.545925] Test: [520/560] eta: 0:00:01 loss: 0.2821 (0.2677) auc: 96.0784 (96.4505) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:10:17.837626] Test: [530/560] eta: 0:00:00 loss: 0.2905 (0.2684) auc: 94.9219 (96.4542) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:10:18.129759] Test: [540/560] eta: 0:00:00 loss: 0.2814 (0.2689) auc: 96.6667 (96.4368) time: 0.0291 data: 0.0002 max mem: 5308 | |
| [01:10:18.418843] Test: [550/560] eta: 0:00:00 loss: 0.2483 (0.2684) auc: 98.4375 (96.4758) time: 0.0290 data: 0.0001 max mem: 5308 | |
| [01:10:18.663405] Test: [559/560] eta: 0:00:00 loss: 0.2540 (0.2686) auc: 98.4127 (96.4731) time: 0.0281 data: 0.0001 max mem: 5308 | |
| [01:10:18.822296] Test: Total time: 0:00:18 (0.0325 s / it) | |
| [01:10:18.823921] * Auc 96.445 loss 0.268 | |
| [01:10:18.824582] AUC of the network on the 35796 val images: 96.44% | |
| [01:10:18.824623] Max auc: 96.44% | |
| [01:10:18.831374] log_dir: ./checkpoint/finetuned_models/FF++_c23_32frames | |
| [01:10:19.987049] Epoch: [9] [ 0/2877] eta: 0:55:17 lr: 0.000007 loss: 0.6032 (0.6032) time: 1.1532 data: 0.9549 max mem: 5308 | |
| [01:10:32.517135] Epoch: [9] [ 100/2877] eta: 0:06:16 lr: 0.000006 loss: 0.6102 (0.6082) time: 0.1251 data: 0.0001 max mem: 5308 | |
| [01:10:45.016705] Epoch: [9] [ 200/2877] eta: 0:05:48 lr: 0.000006 loss: 0.6095 (0.6100) time: 0.1250 data: 0.0001 max mem: 5308 | |
| [01:10:57.566999] Epoch: [9] [ 300/2877] eta: 0:05:31 lr: 0.000006 loss: 0.6123 (0.6104) time: 0.1254 data: 0.0001 max mem: 5308 | |
| [01:11:10.104553] Epoch: [9] [ 400/2877] eta: 0:05:16 lr: 0.000005 loss: 0.6216 (0.6157) time: 0.1254 data: 0.0001 max mem: 5308 | |
| [01:11:22.616465] Epoch: [9] [ 500/2877] eta: 0:05:02 lr: 0.000005 loss: 0.6054 (0.6169) time: 0.1249 data: 0.0001 max mem: 5308 | |
| [01:11:35.154926] Epoch: [9] [ 600/2877] eta: 0:04:49 lr: 0.000005 loss: 0.6183 (0.6168) time: 0.1252 data: 0.0001 max mem: 5308 | |
| [01:11:47.678403] Epoch: [9] [ 700/2877] eta: 0:04:35 lr: 0.000004 loss: 0.6176 (0.6184) time: 0.1247 data: 0.0001 max mem: 5308 | |
| [01:12:00.196012] Epoch: [9] [ 800/2877] eta: 0:04:22 lr: 0.000004 loss: 0.6251 (0.6185) time: 0.1252 data: 0.0001 max mem: 5308 | |
| [01:12:12.725046] Epoch: [9] [ 900/2877] eta: 0:04:09 lr: 0.000004 loss: 0.6224 (0.6186) time: 0.1251 data: 0.0001 max mem: 5308 | |
| [01:12:25.235129] Epoch: [9] [1000/2877] eta: 0:03:56 lr: 0.000004 loss: 0.6258 (0.6193) time: 0.1253 data: 0.0001 max mem: 5308 | |
| [01:12:37.767788] Epoch: [9] [1100/2877] eta: 0:03:44 lr: 0.000003 loss: 0.5994 (0.6199) time: 0.1254 data: 0.0001 max mem: 5308 | |
| [01:12:50.299492] Epoch: [9] [1200/2877] eta: 0:03:31 lr: 0.000003 loss: 0.6504 (0.6205) time: 0.1254 data: 0.0001 max mem: 5308 | |
| [01:13:02.828986] Epoch: [9] [1300/2877] eta: 0:03:18 lr: 0.000003 loss: 0.6244 (0.6211) time: 0.1251 data: 0.0001 max mem: 5308 | |
| [01:13:15.349215] Epoch: [9] [1400/2877] eta: 0:03:06 lr: 0.000003 loss: 0.6132 (0.6206) time: 0.1252 data: 0.0001 max mem: 5308 | |
| [01:13:27.901900] Epoch: [9] [1500/2877] eta: 0:02:53 lr: 0.000002 loss: 0.6366 (0.6204) time: 0.1260 data: 0.0001 max mem: 5308 | |
| [01:13:40.454839] Epoch: [9] [1600/2877] eta: 0:02:40 lr: 0.000002 loss: 0.6490 (0.6199) time: 0.1258 data: 0.0001 max mem: 5308 | |
| [01:13:52.989293] Epoch: [9] [1700/2877] eta: 0:02:28 lr: 0.000002 loss: 0.5923 (0.6198) time: 0.1253 data: 0.0001 max mem: 5308 | |
| [01:14:05.520317] Epoch: [9] [1800/2877] eta: 0:02:15 lr: 0.000002 loss: 0.6158 (0.6198) time: 0.1250 data: 0.0001 max mem: 5308 | |
| [01:14:18.029331] Epoch: [9] [1900/2877] eta: 0:02:02 lr: 0.000002 loss: 0.6244 (0.6196) time: 0.1251 data: 0.0001 max mem: 5308 | |
| [01:14:30.562023] Epoch: [9] [2000/2877] eta: 0:01:50 lr: 0.000002 loss: 0.6158 (0.6202) time: 0.1254 data: 0.0001 max mem: 5308 | |
| [01:14:43.076502] Epoch: [9] [2100/2877] eta: 0:01:37 lr: 0.000001 loss: 0.6236 (0.6196) time: 0.1250 data: 0.0001 max mem: 5308 | |
| [01:14:55.605703] Epoch: [9] [2200/2877] eta: 0:01:25 lr: 0.000001 loss: 0.6389 (0.6194) time: 0.1253 data: 0.0001 max mem: 5308 | |
| [01:15:08.125929] Epoch: [9] [2300/2877] eta: 0:01:12 lr: 0.000001 loss: 0.6032 (0.6192) time: 0.1251 data: 0.0001 max mem: 5308 | |
| [01:15:20.646091] Epoch: [9] [2400/2877] eta: 0:00:59 lr: 0.000001 loss: 0.6254 (0.6190) time: 0.1253 data: 0.0001 max mem: 5308 | |
| [01:15:33.161260] Epoch: [9] [2500/2877] eta: 0:00:47 lr: 0.000001 loss: 0.6126 (0.6189) time: 0.1251 data: 0.0001 max mem: 5308 | |
| [01:15:45.681507] Epoch: [9] [2600/2877] eta: 0:00:34 lr: 0.000001 loss: 0.5802 (0.6186) time: 0.1253 data: 0.0001 max mem: 5308 | |
| [01:15:58.215074] Epoch: [9] [2700/2877] eta: 0:00:22 lr: 0.000001 loss: 0.6309 (0.6190) time: 0.1252 data: 0.0001 max mem: 5308 | |
| [01:16:10.735795] Epoch: [9] [2800/2877] eta: 0:00:09 lr: 0.000001 loss: 0.6427 (0.6193) time: 0.1251 data: 0.0001 max mem: 5308 | |
| [01:16:20.226203] Epoch: [9] [2876/2877] eta: 0:00:00 lr: 0.000001 loss: 0.6170 (0.6194) time: 0.1242 data: 0.0002 max mem: 5308 | |
| [01:16:20.402915] Epoch: [9] Total time: 0:06:01 (0.1257 s / it) | |
| [01:16:20.454787] Averaged stats: lr: 0.000001 loss: 0.6170 (0.6199) | |
| [01:16:21.708234] Test: [ 0/560] eta: 0:11:40 loss: 0.2722 (0.2722) auc: 97.2549 (97.2549) time: 1.2504 data: 1.2138 max mem: 5308 | |
| [01:16:22.004596] Test: [ 10/560] eta: 0:01:17 loss: 0.3206 (0.3271) auc: 95.0000 (93.4459) time: 0.1405 data: 0.1106 max mem: 5308 | |
| [01:16:22.298368] Test: [ 20/560] eta: 0:00:47 loss: 0.2955 (0.2959) auc: 95.6349 (95.4811) time: 0.0294 data: 0.0002 max mem: 5308 | |
| [01:16:22.592826] Test: [ 30/560] eta: 0:00:36 loss: 0.2514 (0.2803) auc: 98.3806 (95.8764) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [01:16:22.886076] Test: [ 40/560] eta: 0:00:30 loss: 0.2580 (0.2793) auc: 97.0833 (95.8053) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [01:16:23.180335] Test: [ 50/560] eta: 0:00:27 loss: 0.2610 (0.2777) auc: 96.8254 (96.0871) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [01:16:23.474660] Test: [ 60/560] eta: 0:00:24 loss: 0.2934 (0.2794) auc: 96.8254 (95.9960) time: 0.0294 data: 0.0001 max mem: 5308 | |
| [01:16:23.768348] Test: [ 70/560] eta: 0:00:22 loss: 0.2725 (0.2776) auc: 96.8627 (96.0572) time: 0.0293 data: 0.0001 max mem: 5308 | |
| [01:16:24.059256] Test: [ 80/560] eta: 0:00:21 loss: 0.2436 (0.2715) auc: 97.9757 (96.2697) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [01:16:24.350458] Test: [ 90/560] eta: 0:00:20 loss: 0.2328 (0.2708) auc: 97.2656 (96.3182) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:16:24.641348] Test: [100/560] eta: 0:00:19 loss: 0.2356 (0.2671) auc: 97.1660 (96.4760) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:16:24.932870] Test: [110/560] eta: 0:00:18 loss: 0.2380 (0.2668) auc: 97.1660 (96.4704) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:16:25.224038] Test: [120/560] eta: 0:00:17 loss: 0.2711 (0.2727) auc: 94.5833 (96.2833) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:16:25.515932] Test: [130/560] eta: 0:00:16 loss: 0.2764 (0.2735) auc: 95.8333 (96.3002) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:16:25.808029] Test: [140/560] eta: 0:00:15 loss: 0.2708 (0.2739) auc: 96.8627 (96.3366) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:16:26.099476] Test: [150/560] eta: 0:00:15 loss: 0.2604 (0.2726) auc: 97.5709 (96.3466) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:16:26.390534] Test: [160/560] eta: 0:00:14 loss: 0.2305 (0.2680) auc: 98.3333 (96.4796) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:16:26.681532] Test: [170/560] eta: 0:00:14 loss: 0.2185 (0.2668) auc: 98.3333 (96.5032) time: 0.0290 data: 0.0001 max mem: 5308 | |
| [01:16:26.972755] Test: [180/560] eta: 0:00:13 loss: 0.2432 (0.2672) auc: 97.6562 (96.5190) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:16:27.264079] Test: [190/560] eta: 0:00:13 loss: 0.2711 (0.2677) auc: 96.8627 (96.5226) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:16:27.557501] Test: [200/560] eta: 0:00:12 loss: 0.2630 (0.2660) auc: 96.3563 (96.5816) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [01:16:27.849878] Test: [210/560] eta: 0:00:12 loss: 0.2630 (0.2671) auc: 97.1660 (96.5477) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [01:16:28.142789] Test: [220/560] eta: 0:00:11 loss: 0.2652 (0.2671) auc: 97.1660 (96.5606) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [01:16:28.434510] Test: [230/560] eta: 0:00:11 loss: 0.2627 (0.2673) auc: 97.5709 (96.5524) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [01:16:28.727363] Test: [240/560] eta: 0:00:10 loss: 0.2627 (0.2671) auc: 97.6471 (96.5505) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [01:16:29.020999] Test: [250/560] eta: 0:00:10 loss: 0.2834 (0.2678) auc: 96.6667 (96.5275) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [01:16:29.313728] Test: [260/560] eta: 0:00:10 loss: 0.2338 (0.2664) auc: 97.2549 (96.5704) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [01:16:29.605908] Test: [270/560] eta: 0:00:09 loss: 0.2719 (0.2671) auc: 96.4706 (96.5534) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [01:16:29.897973] Test: [280/560] eta: 0:00:09 loss: 0.2327 (0.2646) auc: 97.6190 (96.6211) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [01:16:30.189998] Test: [290/560] eta: 0:00:09 loss: 0.2470 (0.2654) auc: 97.9167 (96.6098) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:16:30.481622] Test: [300/560] eta: 0:00:08 loss: 0.2571 (0.2647) auc: 97.1660 (96.6394) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:16:30.773965] Test: [310/560] eta: 0:00:08 loss: 0.2501 (0.2650) auc: 96.8627 (96.6078) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:16:31.066974] Test: [320/560] eta: 0:00:07 loss: 0.2691 (0.2654) auc: 96.8627 (96.5913) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [01:16:31.359413] Test: [330/560] eta: 0:00:07 loss: 0.2688 (0.2654) auc: 97.0833 (96.5869) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [01:16:31.652098] Test: [340/560] eta: 0:00:07 loss: 0.2554 (0.2650) auc: 96.8254 (96.5908) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [01:16:31.944557] Test: [350/560] eta: 0:00:06 loss: 0.2463 (0.2650) auc: 96.8254 (96.5862) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [01:16:32.236934] Test: [360/560] eta: 0:00:06 loss: 0.2791 (0.2659) auc: 96.8254 (96.5709) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [01:16:32.528580] Test: [370/560] eta: 0:00:06 loss: 0.2842 (0.2663) auc: 96.8254 (96.5491) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:16:32.820704] Test: [380/560] eta: 0:00:05 loss: 0.2488 (0.2661) auc: 97.5709 (96.5647) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:16:33.113117] Test: [390/560] eta: 0:00:05 loss: 0.2341 (0.2663) auc: 97.6190 (96.5660) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [01:16:33.404999] Test: [400/560] eta: 0:00:05 loss: 0.2575 (0.2673) auc: 97.2222 (96.5297) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [01:16:33.696626] Test: [410/560] eta: 0:00:04 loss: 0.2843 (0.2677) auc: 95.6349 (96.4911) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:16:33.988500] Test: [420/560] eta: 0:00:04 loss: 0.2843 (0.2679) auc: 96.0784 (96.4682) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:16:34.280262] Test: [430/560] eta: 0:00:04 loss: 0.2232 (0.2669) auc: 97.9757 (96.5079) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:16:34.571775] Test: [440/560] eta: 0:00:03 loss: 0.2123 (0.2662) auc: 98.4127 (96.5350) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:16:34.864627] Test: [450/560] eta: 0:00:03 loss: 0.2309 (0.2663) auc: 98.0469 (96.5324) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:16:35.156390] Test: [460/560] eta: 0:00:03 loss: 0.2795 (0.2669) auc: 96.9697 (96.5207) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:16:35.448807] Test: [470/560] eta: 0:00:02 loss: 0.2803 (0.2672) auc: 96.4706 (96.5188) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:16:35.741410] Test: [480/560] eta: 0:00:02 loss: 0.2657 (0.2672) auc: 96.2500 (96.4992) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [01:16:36.033626] Test: [490/560] eta: 0:00:02 loss: 0.2419 (0.2665) auc: 96.2891 (96.5190) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [01:16:36.325583] Test: [500/560] eta: 0:00:01 loss: 0.2419 (0.2661) auc: 97.5709 (96.5363) time: 0.0292 data: 0.0001 max mem: 5308 | |
| [01:16:36.617329] Test: [510/560] eta: 0:00:01 loss: 0.2526 (0.2656) auc: 96.8627 (96.5470) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:16:36.908946] Test: [520/560] eta: 0:00:01 loss: 0.2786 (0.2666) auc: 96.0784 (96.5132) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:16:37.200657] Test: [530/560] eta: 0:00:00 loss: 0.2901 (0.2673) auc: 95.3125 (96.5152) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:16:37.492868] Test: [540/560] eta: 0:00:00 loss: 0.2839 (0.2678) auc: 97.0833 (96.4989) time: 0.0291 data: 0.0001 max mem: 5308 | |
| [01:16:37.782418] Test: [550/560] eta: 0:00:00 loss: 0.2509 (0.2673) auc: 98.4375 (96.5387) time: 0.0290 data: 0.0001 max mem: 5308 | |
| [01:16:38.024509] Test: [559/560] eta: 0:00:00 loss: 0.2576 (0.2676) auc: 98.4127 (96.5354) time: 0.0280 data: 0.0001 max mem: 5308 | |
| [01:16:38.208761] Test: Total time: 0:00:17 (0.0317 s / it) | |
| [01:16:38.377206] * Auc 96.507 loss 0.267 | |
| [01:16:38.377348] AUC of the network on the 35796 val images: 96.51% | |
| [01:16:38.377359] Max auc: 96.51% | |
| [01:16:39.258283] Training time 1:04:49 | |