train.log

2024-09-09 12:49:35.169739: I tensorflow/core/util/port.cc:153] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-09-09 12:49:35.188435: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:485] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2024-09-09 12:49:35.210470: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:8454] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2024-09-09 12:49:35.217107: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1452] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-09-09 12:49:35.233109: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-09-09 12:49:36.523400: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
/usr/local/lib/python3.10/dist-packages/transformers/training_args.py:1525: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
  warnings.warn(
09/09/2024 12:49:38 - WARNING - __main__ -   Process rank: 0, device: cuda:0, n_gpu: 1distributed training: True, 16-bits training: False
09/09/2024 12:49:38 - INFO - __main__ -   Training/evaluation parameters TrainingArguments(
_n_gpu=1,
accelerator_config={'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None, 'use_configured_state': False},
adafactor=False,
adam_beta1=0.9,
adam_beta2=0.999,
adam_epsilon=1e-08,
auto_find_batch_size=False,
batch_eval_metrics=False,
bf16=False,
bf16_full_eval=False,
data_seed=None,
dataloader_drop_last=False,
dataloader_num_workers=0,
dataloader_persistent_workers=False,
dataloader_pin_memory=True,
dataloader_prefetch_factor=None,
ddp_backend=None,
ddp_broadcast_buffers=None,
ddp_bucket_cap_mb=None,
ddp_find_unused_parameters=None,
ddp_timeout=1800,
debug=[],
deepspeed=None,
disable_tqdm=False,
dispatch_batches=None,
do_eval=True,
do_predict=True,
do_train=True,
eval_accumulation_steps=None,
eval_delay=0,
eval_do_concat_batches=True,
eval_on_start=False,
eval_steps=None,
eval_strategy=epoch,
eval_use_gather_object=False,
evaluation_strategy=epoch,
fp16=False,
fp16_backend=auto,
fp16_full_eval=False,
fp16_opt_level=O1,
fsdp=[],
fsdp_config={'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False},
fsdp_min_num_params=0,
fsdp_transformer_layer_cls_to_wrap=None,
full_determinism=False,
gradient_accumulation_steps=2,
gradient_checkpointing=False,
gradient_checkpointing_kwargs=None,
greater_is_better=True,
group_by_length=False,
half_precision_backend=auto,
hub_always_push=False,
hub_model_id=None,
hub_private_repo=False,
hub_strategy=every_save,
hub_token=<HUB_TOKEN>,
ignore_data_skip=False,
include_inputs_for_metrics=False,
include_num_input_tokens_seen=False,
include_tokens_per_second=False,
jit_mode_eval=False,
label_names=None,
label_smoothing_factor=0.0,
learning_rate=5e-05,
length_column_name=length,
load_best_model_at_end=True,
local_rank=0,
log_level=passive,
log_level_replica=warning,
log_on_each_node=True,
logging_dir=/content/dissertation/scripts/ner/output/tb,
logging_first_step=False,
logging_nan_inf_filter=True,
logging_steps=500,
logging_strategy=steps,
lr_scheduler_kwargs={},
lr_scheduler_type=linear,
max_grad_norm=1.0,
max_steps=-1,
metric_for_best_model=f1,
mp_parameters=,
neftune_noise_alpha=None,
no_cuda=False,
num_train_epochs=10.0,
optim=adamw_torch,
optim_args=None,
optim_target_modules=None,
output_dir=/content/dissertation/scripts/ner/output,
overwrite_output_dir=True,
past_index=-1,
per_device_eval_batch_size=8,
per_device_train_batch_size=32,
prediction_loss_only=False,
push_to_hub=True,
push_to_hub_model_id=None,
push_to_hub_organization=None,
push_to_hub_token=<PUSH_TO_HUB_TOKEN>,
ray_scope=last,
remove_unused_columns=True,
report_to=['tensorboard'],
restore_callback_states_from_checkpoint=False,
resume_from_checkpoint=None,
run_name=/content/dissertation/scripts/ner/output,
save_on_each_node=False,
save_only_model=False,
save_safetensors=True,
save_steps=500,
save_strategy=epoch,
save_total_limit=None,
seed=42,
skip_memory_metrics=True,
split_batches=None,
tf32=None,
torch_compile=False,
torch_compile_backend=None,
torch_compile_mode=None,
torch_empty_cache_steps=None,
torchdynamo=None,
tpu_metrics_debug=False,
tpu_num_cores=None,
use_cpu=False,
use_ipex=False,
use_legacy_prediction_loop=False,
use_mps_device=False,
warmup_ratio=0.0,
warmup_steps=0,
weight_decay=0.0,
)

Downloading builder script:   0%|          | 0.00/3.62k [00:00<?, ?B/s]
Downloading builder script: 100%|██████████| 3.62k/3.62k [00:00<00:00, 15.3kB/s]
Downloading builder script: 100%|██████████| 3.62k/3.62k [00:00<00:00, 15.3kB/s]

Downloading data:   0%|          | 0.00/33.9M [00:00<?, ?B/s]
Downloading data:  31%|███       | 10.5M/33.9M [00:00<00:02, 10.5MB/s]
Downloading data:  62%|██████▏   | 21.0M/33.9M [00:01<00:00, 15.9MB/s]
Downloading data:  93%|█████████▎| 31.5M/33.9M [00:01<00:00, 18.7MB/s]
Downloading data: 100%|██████████| 33.9M/33.9M [00:01<00:00, 17.8MB/s]

Downloading data:   0%|          | 0.00/6.66M [00:00<?, ?B/s]
Downloading data: 100%|██████████| 6.66M/6.66M [00:00<00:00, 7.41MB/s]
Downloading data: 100%|██████████| 6.66M/6.66M [00:00<00:00, 7.36MB/s]

Downloading data:   0%|          | 0.00/12.0M [00:00<?, ?B/s]
Downloading data:  87%|████████▋ | 10.5M/12.0M [00:01<00:00, 8.44MB/s]
Downloading data: 100%|██████████| 12.0M/12.0M [00:01<00:00, 9.56MB/s]

Generating train split: 0 examples [00:00, ? examples/s]
Generating train split: 645 examples [00:00, 6422.99 examples/s]
Generating train split: 1585 examples [00:00, 6310.50 examples/s]
Generating train split: 2239 examples [00:00, 6396.54 examples/s]
Generating train split: 2942 examples [00:00, 6620.97 examples/s]
Generating train split: 3958 examples [00:00, 6683.92 examples/s]
Generating train split: 4976 examples [00:00, 6719.22 examples/s]
Generating train split: 5941 examples [00:00, 6616.84 examples/s]
Generating train split: 6894 examples [00:01, 6523.22 examples/s]
Generating train split: 7910 examples [00:01, 6602.52 examples/s]
Generating train split: 8903 examples [00:01, 6604.68 examples/s]
Generating train split: 9878 examples [00:01, 6568.43 examples/s]
Generating train split: 10839 examples [00:01, 6515.53 examples/s]
Generating train split: 11854 examples [00:01, 6589.56 examples/s]
Generating train split: 12845 examples [00:01, 6591.65 examples/s]
Generating train split: 13855 examples [00:02, 6626.64 examples/s]
Generating train split: 14832 examples [00:02, 6589.42 examples/s]
Generating train split: 15831 examples [00:02, 6606.16 examples/s]
Generating train split: 16856 examples [00:02, 6669.54 examples/s]
Generating train split: 17866 examples [00:02, 6684.58 examples/s]
Generating train split: 18853 examples [00:02, 6648.95 examples/s]
Generating train split: 19863 examples [00:03, 6672.36 examples/s]
Generating train split: 20537 examples [00:03, 6685.56 examples/s]
Generating train split: 21553 examples [00:03, 6712.14 examples/s]
Generating train split: 22557 examples [00:03, 6703.84 examples/s]
Generating train split: 23543 examples [00:03, 6658.95 examples/s]
Generating train split: 24494 examples [00:03, 6558.35 examples/s]
Generating train split: 25486 examples [00:03, 6571.56 examples/s]
Generating train split: 26448 examples [00:04, 6522.66 examples/s]
Generating train split: 27423 examples [00:04, 6510.51 examples/s]
Generating train split: 28421 examples [00:04, 6550.53 examples/s]
Generating train split: 29376 examples [00:04, 6485.54 examples/s]
Generating train split: 30240 examples [00:04, 4496.54 examples/s]
Generating train split: 30931 examples [00:04, 4904.64 examples/s]
Generating train split: 31559 examples [00:05, 5172.47 examples/s]
Generating train split: 32216 examples [00:05, 5474.88 examples/s]
Generating train split: 32447 examples [00:05, 6307.49 examples/s]

Generating validation split: 0 examples [00:00, ? examples/s]
Generating validation split: 753 examples [00:00, 7503.26 examples/s]
Generating validation split: 1823 examples [00:00, 7241.51 examples/s]
Generating validation split: 2828 examples [00:00, 6970.53 examples/s]
Generating validation split: 3879 examples [00:00, 6983.17 examples/s]
Generating validation split: 4923 examples [00:00, 6965.65 examples/s]
Generating validation split: 5963 examples [00:00, 6947.29 examples/s]
Generating validation split: 6946 examples [00:00, 6916.82 examples/s]
Generating validation split: 6946 examples [00:01, 6918.93 examples/s]

Generating test split: 0 examples [00:00, ? examples/s]
Generating test split: 837 examples [00:00, 8333.65 examples/s]
Generating test split: 1992 examples [00:00, 7893.19 examples/s]
Generating test split: 3118 examples [00:00, 7697.73 examples/s]
Generating test split: 3905 examples [00:00, 7750.41 examples/s]
Generating test split: 5029 examples [00:00, 7643.58 examples/s]
Generating test split: 5922 examples [00:00, 8001.63 examples/s]
Generating test split: 6745 examples [00:00, 8065.26 examples/s]
Generating test split: 7957 examples [00:01, 8066.19 examples/s]
Generating test split: 9088 examples [00:01, 7878.75 examples/s]
Generating test split: 10230 examples [00:01, 7788.78 examples/s]
Generating test split: 11023 examples [00:01, 7817.04 examples/s]
Generating test split: 11883 examples [00:01, 8015.39 examples/s]
Generating test split: 12954 examples [00:01, 7696.61 examples/s]
Generating test split: 14049 examples [00:01, 7561.13 examples/s]
Generating test split: 14715 examples [00:01, 7754.44 examples/s]
[INFO|configuration_utils.py:733] 2024-09-09 12:49:56,508 >> loading configuration file config.json from cache at /root/.cache/huggingface/hub/models--michiyasunaga--BioLinkBERT-base/snapshots/b71f5d70f063d1c8f1124070ce86f1ee463ca1fe/config.json
[INFO|configuration_utils.py:800] 2024-09-09 12:49:56,512 >> Model config BertConfig {
  "_name_or_path": "michiyasunaga/BioLinkBERT-base",
  "architectures": [
    "BertModel"
  ],
  "attention_probs_dropout_prob": 0.1,
  "classifier_dropout": null,
  "finetuning_task": "ner",
  "gradient_checkpointing": false,
  "hidden_act": "gelu",
  "hidden_dropout_prob": 0.1,
  "hidden_size": 768,
  "id2label": {
    "0": "O",
    "1": "B-FARMACO",
    "2": "I-FARMACO"
  },
  "initializer_range": 0.02,
  "intermediate_size": 3072,
  "label2id": {
    "B-FARMACO": 1,
    "I-FARMACO": 2,
    "O": 0
  },
  "layer_norm_eps": 1e-12,
  "max_position_embeddings": 512,
  "model_type": "bert",
  "num_attention_heads": 12,
  "num_hidden_layers": 12,
  "pad_token_id": 0,
  "position_embedding_type": "absolute",
  "transformers_version": "4.44.2",
  "type_vocab_size": 2,
  "use_cache": true,
  "vocab_size": 28895
}

[INFO|tokenization_utils_base.py:2269] 2024-09-09 12:49:58,923 >> loading file vocab.txt from cache at /root/.cache/huggingface/hub/models--michiyasunaga--BioLinkBERT-base/snapshots/b71f5d70f063d1c8f1124070ce86f1ee463ca1fe/vocab.txt
[INFO|tokenization_utils_base.py:2269] 2024-09-09 12:49:58,923 >> loading file tokenizer.json from cache at /root/.cache/huggingface/hub/models--michiyasunaga--BioLinkBERT-base/snapshots/b71f5d70f063d1c8f1124070ce86f1ee463ca1fe/tokenizer.json
[INFO|tokenization_utils_base.py:2269] 2024-09-09 12:49:58,923 >> loading file added_tokens.json from cache at None
[INFO|tokenization_utils_base.py:2269] 2024-09-09 12:49:58,923 >> loading file special_tokens_map.json from cache at /root/.cache/huggingface/hub/models--michiyasunaga--BioLinkBERT-base/snapshots/b71f5d70f063d1c8f1124070ce86f1ee463ca1fe/special_tokens_map.json
[INFO|tokenization_utils_base.py:2269] 2024-09-09 12:49:58,923 >> loading file tokenizer_config.json from cache at /root/.cache/huggingface/hub/models--michiyasunaga--BioLinkBERT-base/snapshots/b71f5d70f063d1c8f1124070ce86f1ee463ca1fe/tokenizer_config.json
/usr/local/lib/python3.10/dist-packages/transformers/tokenization_utils_base.py:1601: FutureWarning: `clean_up_tokenization_spaces` was not set. It will be set to `True` by default. This behavior will be depracted in transformers v4.45, and will be then set to `False` by default. For more details check this issue: https://github.com/huggingface/transformers/issues/31884
  warnings.warn(
[INFO|modeling_utils.py:3678] 2024-09-09 12:50:02,660 >> loading weights file pytorch_model.bin from cache at /root/.cache/huggingface/hub/models--michiyasunaga--BioLinkBERT-base/snapshots/b71f5d70f063d1c8f1124070ce86f1ee463ca1fe/pytorch_model.bin
[INFO|modeling_utils.py:4497] 2024-09-09 12:50:02,740 >> Some weights of the model checkpoint at michiyasunaga/BioLinkBERT-base were not used when initializing BertForTokenClassification: ['bert.pooler.dense.bias', 'bert.pooler.dense.weight']
- This IS expected if you are initializing BertForTokenClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing BertForTokenClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
[WARNING|modeling_utils.py:4509] 2024-09-09 12:50:02,740 >> Some weights of BertForTokenClassification were not initialized from the model checkpoint at michiyasunaga/BioLinkBERT-base and are newly initialized: ['classifier.bias', 'classifier.weight']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.

Map:   0%|          | 0/32447 [00:00<?, ? examples/s]
Map:   3%|▎         | 1000/32447 [00:00<00:03, 9362.33 examples/s]
Map:   9%|▉         | 3000/32447 [00:00<00:02, 10280.00 examples/s]
Map:  15%|█▌        | 5000/32447 [00:00<00:02, 10674.68 examples/s]
Map:  22%|██▏       | 7000/32447 [00:00<00:02, 10414.12 examples/s]
Map:  28%|██▊       | 9000/32447 [00:00<00:02, 10509.84 examples/s]
Map:  34%|███▍      | 11000/32447 [00:01<00:02, 10515.16 examples/s]
Map:  40%|████      | 13000/32447 [00:01<00:01, 10620.05 examples/s]
Map:  46%|████▌     | 15000/32447 [00:01<00:01, 10528.03 examples/s]
Map:  52%|█████▏    | 17000/32447 [00:01<00:01, 10624.94 examples/s]
Map:  59%|█████▊    | 19000/32447 [00:01<00:01, 10748.91 examples/s]
Map:  65%|██████▍   | 21000/32447 [00:01<00:01, 10823.70 examples/s]
Map:  71%|███████   | 23000/32447 [00:02<00:00, 10784.11 examples/s]
Map:  77%|███████▋  | 25000/32447 [00:02<00:00, 7869.26 examples/s] 
Map:  83%|████████▎ | 27000/32447 [00:02<00:00, 8538.25 examples/s]
Map:  89%|████████▉ | 29000/32447 [00:02<00:00, 9022.56 examples/s]
Map:  96%|█████████▌| 31000/32447 [00:03<00:00, 9440.41 examples/s]
Map: 100%|██████████| 32447/32447 [00:03<00:00, 9474.30 examples/s]
Map: 100%|██████████| 32447/32447 [00:03<00:00, 9831.82 examples/s]

Map:   0%|          | 0/6946 [00:00<?, ? examples/s]
Map:  29%|██▉       | 2000/6946 [00:00<00:00, 11608.63 examples/s]
Map:  58%|█████▊    | 4000/6946 [00:00<00:00, 11380.90 examples/s]
Map:  86%|████████▋ | 6000/6946 [00:00<00:00, 11392.48 examples/s]
Map: 100%|██████████| 6946/6946 [00:00<00:00, 11297.05 examples/s]

Map:   0%|          | 0/14715 [00:00<?, ? examples/s]
Map:  14%|█▎        | 2000/14715 [00:00<00:01, 12264.98 examples/s]
Map:  27%|██▋       | 4000/14715 [00:00<00:00, 12320.92 examples/s]
Map:  41%|████      | 6000/14715 [00:00<00:00, 12772.71 examples/s]
Map:  54%|█████▍    | 8000/14715 [00:00<00:00, 12979.51 examples/s]
Map:  68%|██████▊   | 10000/14715 [00:00<00:00, 12993.32 examples/s]
Map:  82%|████████▏ | 12000/14715 [00:00<00:00, 13101.07 examples/s]
Map:  95%|█████████▌| 14000/14715 [00:01<00:00, 12505.75 examples/s]
Map: 100%|██████████| 14715/14715 [00:01<00:00, 12533.17 examples/s]
/content/dissertation/scripts/ner/run_ner_train.py:397: FutureWarning: load_metric is deprecated and will be removed in the next major version of datasets. Use 'evaluate.load' instead, from the new library 🤗 Evaluate: https://huggingface.co/docs/evaluate
  metric = load_metric("seqeval", trust_remote_code=True)
[INFO|trainer.py:811] 2024-09-09 12:50:10,148 >> The following columns in the training set don't have a corresponding argument in `BertForTokenClassification.forward` and have been ignored: id, tokens, ner_tags. If id, tokens, ner_tags are not expected by `BertForTokenClassification.forward`,  you can safely ignore this message.
[INFO|trainer.py:2134] 2024-09-09 12:50:10,704 >> ***** Running training *****
[INFO|trainer.py:2135] 2024-09-09 12:50:10,704 >>   Num examples = 32,447
[INFO|trainer.py:2136] 2024-09-09 12:50:10,704 >>   Num Epochs = 10
[INFO|trainer.py:2137] 2024-09-09 12:50:10,704 >>   Instantaneous batch size per device = 32
[INFO|trainer.py:2140] 2024-09-09 12:50:10,704 >>   Total train batch size (w. parallel, distributed & accumulation) = 64
[INFO|trainer.py:2141] 2024-09-09 12:50:10,704 >>   Gradient Accumulation steps = 2
[INFO|trainer.py:2142] 2024-09-09 12:50:10,704 >>   Total optimization steps = 5,070
[INFO|trainer.py:2143] 2024-09-09 12:50:10,705 >>   Number of trainable parameters = 107,644,419

  0%|          | 0/5070 [00:00<?, ?it/s]
  0%|          | 1/5070 [00:01<1:55:56,  1.37s/it]
  0%|          | 2/5070 [00:01<1:01:02,  1.38it/s]
  0%|          | 3/5070 [00:01<43:59,  1.92it/s]  
  0%|          | 4/5070 [00:02<37:03,  2.28it/s]
  0%|          | 5/5070 [00:02<33:54,  2.49it/s]
  0%|          | 6/5070 [00:02<29:56,  2.82it/s]
  0%|          | 7/5070 [00:03<29:13,  2.89it/s]
  0%|          | 8/5070 [00:03<29:19,  2.88it/s]
  0%|          | 9/5070 [00:03<28:01,  3.01it/s]
  0%|          | 10/5070 [00:04<29:27,  2.86it/s]
  0%|          | 11/5070 [00:04<31:52,  2.65it/s]
  0%|          | 12/5070 [00:04<31:16,  2.70it/s]
  0%|          | 13/5070 [00:05<30:19,  2.78it/s]
  0%|          | 14/5070 [00:05<32:35,  2.59it/s]
  0%|          | 15/5070 [00:06<29:33,  2.85it/s]
  0%|          | 16/5070 [00:06<29:08,  2.89it/s]
  0%|          | 17/5070 [00:06<32:23,  2.60it/s]
  0%|          | 18/5070 [00:07<32:56,  2.56it/s]
  0%|          | 19/5070 [00:07<30:14,  2.78it/s]
  0%|          | 20/5070 [00:07<31:25,  2.68it/s]
  0%|          | 21/5070 [00:08<28:05,  3.00it/s]
  0%|          | 22/5070 [00:08<27:40,  3.04it/s]
  0%|          | 23/5070 [00:08<29:31,  2.85it/s]
  0%|          | 24/5070 [00:09<27:57,  3.01it/s]
  0%|          | 25/5070 [00:09<29:57,  2.81it/s]
  1%|          | 26/5070 [00:09<30:14,  2.78it/s]
  1%|          | 27/5070 [00:10<31:00,  2.71it/s]
  1%|          | 28/5070 [00:10<33:13,  2.53it/s]
  1%|          | 29/5070 [00:11<31:44,  2.65it/s]
  1%|          | 30/5070 [00:11<34:06,  2.46it/s]
  1%|          | 31/5070 [00:11<31:33,  2.66it/s]
  1%|          | 32/5070 [00:12<30:32,  2.75it/s]
  1%|          | 33/5070 [00:12<32:34,  2.58it/s]
  1%|          | 34/5070 [00:13<33:22,  2.51it/s]
  1%|          | 35/5070 [00:13<32:11,  2.61it/s]
  1%|          | 36/5070 [00:13<29:47,  2.82it/s]
  1%|          | 37/5070 [00:14<34:03,  2.46it/s]
  1%|          | 38/5070 [00:14<33:12,  2.53it/s]
  1%|          | 39/5070 [00:15<32:01,  2.62it/s]
  1%|          | 40/5070 [00:15<32:04,  2.61it/s]
  1%|          | 41/5070 [00:15<31:28,  2.66it/s]
  1%|          | 42/5070 [00:16<29:43,  2.82it/s]
  1%|          | 43/5070 [00:16<29:54,  2.80it/s]
  1%|          | 44/5070 [00:16<30:08,  2.78it/s]
  1%|          | 45/5070 [00:17<28:16,  2.96it/s]
  1%|          | 46/5070 [00:17<28:34,  2.93it/s]
  1%|          | 47/5070 [00:17<29:04,  2.88it/s]
  1%|          | 48/5070 [00:18<30:31,  2.74it/s]
  1%|          | 49/5070 [00:18<29:37,  2.82it/s]
  1%|          | 50/5070 [00:19<33:53,  2.47it/s]
  1%|          | 51/5070 [00:19<34:07,  2.45it/s]
  1%|          | 52/5070 [00:19<34:08,  2.45it/s]
  1%|          | 53/5070 [00:20<31:49,  2.63it/s]
  1%|          | 54/5070 [00:20<31:13,  2.68it/s]
  1%|          | 55/5070 [00:20<31:05,  2.69it/s]
  1%|          | 56/5070 [00:21<30:58,  2.70it/s]
  1%|          | 57/5070 [00:21<29:45,  2.81it/s]
  1%|          | 58/5070 [00:21<30:11,  2.77it/s]
  1%|          | 59/5070 [00:22<30:56,  2.70it/s]
  1%|          | 60/5070 [00:22<31:45,  2.63it/s]
  1%|          | 61/5070 [00:23<31:48,  2.63it/s]
  1%|          | 62/5070 [00:23<31:35,  2.64it/s]
  1%|          | 63/5070 [00:23<31:04,  2.69it/s]
  1%|▏         | 64/5070 [00:24<28:24,  2.94it/s]
  1%|▏         | 65/5070 [00:24<27:01,  3.09it/s]
  1%|▏         | 66/5070 [00:24<30:56,  2.70it/s]
  1%|▏         | 67/5070 [00:25<33:25,  2.50it/s]
  1%|▏         | 68/5070 [00:25<37:18,  2.23it/s]
  1%|▏         | 69/5070 [00:26<39:18,  2.12it/s]
  1%|▏         | 70/5070 [00:26<35:57,  2.32it/s]
  1%|▏         | 71/5070 [00:27<35:19,  2.36it/s]
  1%|▏         | 72/5070 [00:27<36:17,  2.30it/s]
  1%|▏         | 73/5070 [00:28<38:25,  2.17it/s]
  1%|▏         | 74/5070 [00:28<35:23,  2.35it/s]
  1%|▏         | 75/5070 [00:28<35:34,  2.34it/s]
  1%|▏         | 76/5070 [00:29<35:16,  2.36it/s]
  2%|▏         | 77/5070 [00:29<37:14,  2.23it/s]
  2%|▏         | 78/5070 [00:30<36:13,  2.30it/s]
  2%|▏         | 79/5070 [00:30<37:43,  2.20it/s]
  2%|▏         | 80/5070 [00:31<35:44,  2.33it/s]
  2%|▏         | 81/5070 [00:31<35:40,  2.33it/s]
  2%|▏         | 82/5070 [00:31<32:56,  2.52it/s]
  2%|▏         | 83/5070 [00:32<32:34,  2.55it/s]
  2%|▏         | 84/5070 [00:32<36:48,  2.26it/s]
  2%|▏         | 85/5070 [00:33<34:40,  2.40it/s]
  2%|▏         | 86/5070 [00:33<35:53,  2.31it/s]
  2%|▏         | 87/5070 [00:34<38:14,  2.17it/s]
  2%|▏         | 88/5070 [00:34<37:25,  2.22it/s]
  2%|▏         | 89/5070 [00:34<33:37,  2.47it/s]
  2%|▏         | 90/5070 [00:35<32:47,  2.53it/s]
  2%|▏         | 91/5070 [00:35<30:14,  2.74it/s]
  2%|▏         | 92/5070 [00:36<34:18,  2.42it/s]
  2%|▏         | 93/5070 [00:36<33:53,  2.45it/s]
  2%|▏         | 94/5070 [00:36<34:27,  2.41it/s]
  2%|▏         | 95/5070 [00:37<32:28,  2.55it/s]
  2%|▏         | 96/5070 [00:37<31:05,  2.67it/s]
  2%|▏         | 97/5070 [00:37<28:50,  2.87it/s]
  2%|▏         | 98/5070 [00:38<32:59,  2.51it/s]
  2%|▏         | 99/5070 [00:38<31:29,  2.63it/s]
  2%|▏         | 100/5070 [00:39<31:13,  2.65it/s]
  2%|▏         | 101/5070 [00:39<34:40,  2.39it/s]
  2%|▏         | 102/5070 [00:39<31:42,  2.61it/s]
  2%|▏         | 103/5070 [00:40<32:17,  2.56it/s]
  2%|▏         | 104/5070 [00:40<30:58,  2.67it/s]
  2%|▏         | 105/5070 [00:41<29:59,  2.76it/s]
  2%|▏         | 106/5070 [00:41<36:53,  2.24it/s]
  2%|▏         | 107/5070 [00:42<34:40,  2.39it/s]
  2%|▏         | 108/5070 [00:42<33:38,  2.46it/s]
  2%|▏         | 109/5070 [00:42<33:31,  2.47it/s]
  2%|▏         | 110/5070 [00:43<31:03,  2.66it/s]
  2%|▏         | 111/5070 [00:43<31:56,  2.59it/s]
  2%|▏         | 112/5070 [00:43<31:17,  2.64it/s]
  2%|▏         | 113/5070 [00:44<31:57,  2.58it/s]
  2%|▏         | 114/5070 [00:44<29:33,  2.80it/s]
  2%|▏         | 115/5070 [00:45<31:12,  2.65it/s]
  2%|▏         | 116/5070 [00:45<31:17,  2.64it/s]
  2%|▏         | 117/5070 [00:45<33:04,  2.50it/s]
  2%|▏         | 118/5070 [00:46<36:26,  2.26it/s]
  2%|▏         | 119/5070 [00:47<42:31,  1.94it/s]
  2%|▏         | 120/5070 [00:47<37:20,  2.21it/s]
  2%|▏         | 121/5070 [00:47<35:21,  2.33it/s]
  2%|▏         | 122/5070 [00:48<34:00,  2.43it/s]
  2%|▏         | 123/5070 [00:48<32:01,  2.57it/s]
  2%|▏         | 124/5070 [00:48<31:24,  2.62it/s]
  2%|▏         | 125/5070 [00:49<29:14,  2.82it/s]
  2%|▏         | 126/5070 [00:49<31:42,  2.60it/s]
  3%|▎         | 127/5070 [00:50<36:51,  2.24it/s]
  3%|▎         | 128/5070 [00:50<35:05,  2.35it/s]
  3%|▎         | 129/5070 [00:50<32:31,  2.53it/s]
  3%|▎         | 130/5070 [00:51<33:29,  2.46it/s]
  3%|▎         | 131/5070 [00:51<34:54,  2.36it/s]
  3%|▎         | 132/5070 [00:52<32:01,  2.57it/s]
  3%|▎         | 133/5070 [00:52<30:11,  2.73it/s]
  3%|▎         | 134/5070 [00:52<27:55,  2.95it/s]
  3%|▎         | 135/5070 [00:53<27:47,  2.96it/s]
  3%|▎         | 136/5070 [00:53<34:07,  2.41it/s]
  3%|▎         | 137/5070 [00:54<36:48,  2.23it/s]
  3%|▎         | 138/5070 [00:54<34:08,  2.41it/s]
  3%|▎         | 139/5070 [00:54<32:28,  2.53it/s]
  3%|▎         | 140/5070 [00:55<32:32,  2.53it/s]
  3%|▎         | 141/5070 [00:55<34:10,  2.40it/s]
  3%|▎         | 142/5070 [00:56<33:01,  2.49it/s]
  3%|▎         | 143/5070 [00:56<34:43,  2.36it/s]
  3%|▎         | 144/5070 [00:56<32:40,  2.51it/s]
  3%|▎         | 145/5070 [00:57<34:42,  2.37it/s]
  3%|▎         | 146/5070 [00:57<33:20,  2.46it/s]
  3%|▎         | 147/5070 [00:58<38:03,  2.16it/s]
  3%|▎         | 148/5070 [00:58<38:10,  2.15it/s]
  3%|▎         | 149/5070 [00:59<39:25,  2.08it/s]
  3%|▎         | 150/5070 [00:59<37:15,  2.20it/s]
  3%|▎         | 151/5070 [01:00<35:10,  2.33it/s]
  3%|▎         | 152/5070 [01:00<35:57,  2.28it/s]
  3%|▎         | 153/5070 [01:00<32:39,  2.51it/s]
  3%|▎         | 154/5070 [01:01<31:50,  2.57it/s]
  3%|▎         | 155/5070 [01:01<30:24,  2.69it/s]
  3%|▎         | 156/5070 [01:01<29:07,  2.81it/s]
  3%|▎         | 157/5070 [01:02<27:52,  2.94it/s]
  3%|▎         | 158/5070 [01:02<26:45,  3.06it/s]
  3%|▎         | 159/5070 [01:02<27:09,  3.01it/s]
  3%|▎         | 160/5070 [01:03<28:05,  2.91it/s]
  3%|▎         | 161/5070 [01:03<28:55,  2.83it/s]
  3%|▎         | 162/5070 [01:04<32:05,  2.55it/s]
  3%|▎         | 163/5070 [01:04<32:59,  2.48it/s]
  3%|▎         | 164/5070 [01:04<33:29,  2.44it/s]
  3%|▎         | 165/5070 [01:05<38:00,  2.15it/s]
  3%|▎         | 166/5070 [01:05<37:31,  2.18it/s]
  3%|▎         | 167/5070 [01:06<34:33,  2.36it/s]
  3%|▎         | 168/5070 [01:06<30:48,  2.65it/s]
  3%|▎         | 169/5070 [01:06<29:41,  2.75it/s]
  3%|▎         | 170/5070 [01:07<28:36,  2.85it/s]
  3%|▎         | 171/5070 [01:07<29:44,  2.74it/s]
  3%|▎         | 172/5070 [01:07<29:24,  2.78it/s]
  3%|▎         | 173/5070 [01:08<30:28,  2.68it/s]
  3%|▎         | 174/5070 [01:08<31:08,  2.62it/s]
  3%|▎         | 175/5070 [01:09<30:05,  2.71it/s]
  3%|▎         | 176/5070 [01:09<31:47,  2.57it/s]
  3%|▎         | 177/5070 [01:09<30:57,  2.63it/s]
  4%|▎         | 178/5070 [01:10<30:51,  2.64it/s]
  4%|▎         | 179/5070 [01:10<32:25,  2.51it/s]
  4%|▎         | 180/5070 [01:11<32:40,  2.49it/s]
  4%|▎         | 181/5070 [01:11<30:53,  2.64it/s]
  4%|▎         | 182/5070 [01:11<29:51,  2.73it/s]
  4%|▎         | 183/5070 [01:12<30:08,  2.70it/s]
  4%|▎         | 184/5070 [01:12<29:08,  2.79it/s]
  4%|▎         | 185/5070 [01:12<27:38,  2.95it/s]
  4%|▎         | 186/5070 [01:13<29:08,  2.79it/s]
  4%|▎         | 187/5070 [01:13<30:50,  2.64it/s]
  4%|▎         | 188/5070 [01:13<29:46,  2.73it/s]
  4%|▎         | 189/5070 [01:14<27:02,  3.01it/s]
  4%|▎         | 190/5070 [01:14<32:28,  2.50it/s]
  4%|▍         | 191/5070 [01:15<33:35,  2.42it/s]
  4%|▍         | 192/5070 [01:15<33:34,  2.42it/s]
  4%|▍         | 193/5070 [01:16<39:01,  2.08it/s]
  4%|▍         | 194/5070 [01:16<38:27,  2.11it/s]
  4%|▍         | 195/5070 [01:17<35:38,  2.28it/s]
  4%|▍         | 196/5070 [01:17<33:20,  2.44it/s]
  4%|▍         | 197/5070 [01:17<31:14,  2.60it/s]
  4%|▍         | 198/5070 [01:18<32:57,  2.46it/s]
  4%|▍         | 199/5070 [01:18<30:26,  2.67it/s]
  4%|▍         | 200/5070 [01:19<34:45,  2.34it/s]
  4%|▍         | 201/5070 [01:19<31:49,  2.55it/s]
  4%|▍         | 202/5070 [01:19<32:15,  2.52it/s]
  4%|▍         | 203/5070 [01:20<36:02,  2.25it/s]
  4%|▍         | 204/5070 [01:20<36:30,  2.22it/s]
  4%|▍         | 205/5070 [01:21<39:03,  2.08it/s]
  4%|▍         | 206/5070 [01:21<36:32,  2.22it/s]
  4%|▍         | 207/5070 [01:22<34:56,  2.32it/s]
  4%|▍         | 208/5070 [01:22<34:18,  2.36it/s]
  4%|▍         | 209/5070 [01:22<32:08,  2.52it/s]
  4%|▍         | 210/5070 [01:23<31:17,  2.59it/s]
  4%|▍         | 211/5070 [01:23<29:35,  2.74it/s]
  4%|▍         | 212/5070 [01:23<33:09,  2.44it/s]
  4%|▍         | 213/5070 [01:24<32:48,  2.47it/s]
  4%|▍         | 214/5070 [01:24<36:04,  2.24it/s]
  4%|▍         | 215/5070 [01:25<33:15,  2.43it/s]
  4%|▍         | 216/5070 [01:25<32:15,  2.51it/s]
  4%|▍         | 217/5070 [01:26<32:45,  2.47it/s]
  4%|▍         | 218/5070 [01:26<32:25,  2.49it/s]
  4%|▍         | 219/5070 [01:26<34:45,  2.33it/s]
  4%|▍         | 220/5070 [01:27<37:32,  2.15it/s]
  4%|▍         | 221/5070 [01:27<38:50,  2.08it/s]
  4%|▍         | 222/5070 [01:28<34:44,  2.33it/s]
  4%|▍         | 223/5070 [01:28<34:06,  2.37it/s]
  4%|▍         | 224/5070 [01:29<34:41,  2.33it/s]
  4%|▍         | 225/5070 [01:29<34:43,  2.33it/s]
  4%|▍         | 226/5070 [01:30<34:36,  2.33it/s]
  4%|▍         | 227/5070 [01:30<35:31,  2.27it/s]
  4%|▍         | 228/5070 [01:30<35:10,  2.29it/s]
  5%|▍         | 229/5070 [01:31<31:33,  2.56it/s]
  5%|▍         | 230/5070 [01:31<29:53,  2.70it/s]
  5%|▍         | 231/5070 [01:31<27:03,  2.98it/s]
  5%|▍         | 232/5070 [01:32<27:04,  2.98it/s]
  5%|▍         | 233/5070 [01:32<28:05,  2.87it/s]
  5%|▍         | 234/5070 [01:32<27:50,  2.90it/s]
  5%|▍         | 235/5070 [01:33<29:12,  2.76it/s]
  5%|▍         | 236/5070 [01:33<29:10,  2.76it/s]
  5%|▍         | 237/5070 [01:33<29:14,  2.75it/s]
  5%|▍         | 238/5070 [01:34<28:12,  2.85it/s]
  5%|▍         | 239/5070 [01:34<28:52,  2.79it/s]
  5%|▍         | 240/5070 [01:35<28:58,  2.78it/s]
  5%|▍         | 241/5070 [01:35<30:33,  2.63it/s]
  5%|▍         | 242/5070 [01:35<27:48,  2.89it/s]
  5%|▍         | 243/5070 [01:36<27:25,  2.93it/s]
  5%|▍         | 244/5070 [01:36<26:48,  3.00it/s]
  5%|▍         | 245/5070 [01:36<29:15,  2.75it/s]
  5%|▍         | 246/5070 [01:37<27:55,  2.88it/s]
  5%|▍         | 247/5070 [01:37<28:18,  2.84it/s]
  5%|▍         | 248/5070 [01:37<30:50,  2.61it/s]
  5%|▍         | 249/5070 [01:38<31:34,  2.54it/s]
  5%|▍         | 250/5070 [01:38<30:21,  2.65it/s]
  5%|▍         | 251/5070 [01:39<29:52,  2.69it/s]
  5%|▍         | 252/5070 [01:39<32:46,  2.45it/s]
  5%|▍         | 253/5070 [01:39<31:44,  2.53it/s]
  5%|▌         | 254/5070 [01:40<31:13,  2.57it/s]
  5%|▌         | 255/5070 [01:40<31:48,  2.52it/s]
  5%|▌         | 256/5070 [01:41<31:07,  2.58it/s]
  5%|▌         | 257/5070 [01:41<31:06,  2.58it/s]
  5%|▌         | 258/5070 [01:41<30:15,  2.65it/s]
  5%|▌         | 259/5070 [01:42<29:15,  2.74it/s]
  5%|▌         | 260/5070 [01:42<31:19,  2.56it/s]
  5%|▌         | 261/5070 [01:43<36:08,  2.22it/s]
  5%|▌         | 262/5070 [01:43<33:22,  2.40it/s]
  5%|▌         | 263/5070 [01:43<32:11,  2.49it/s]
  5%|▌         | 264/5070 [01:44<29:48,  2.69it/s]
  5%|▌         | 265/5070 [01:44<28:33,  2.80it/s]
  5%|▌         | 266/5070 [01:44<28:52,  2.77it/s]
  5%|▌         | 267/5070 [01:45<32:16,  2.48it/s]
  5%|▌         | 268/5070 [01:45<33:51,  2.36it/s]
  5%|▌         | 269/5070 [01:46<33:57,  2.36it/s]
  5%|▌         | 270/5070 [01:46<31:25,  2.55it/s]
  5%|▌         | 271/5070 [01:46<31:23,  2.55it/s]
  5%|▌         | 272/5070 [01:47<32:46,  2.44it/s]
  5%|▌         | 273/5070 [01:47<32:44,  2.44it/s]
  5%|▌         | 274/5070 [01:48<33:13,  2.41it/s]
  5%|▌         | 275/5070 [01:48<32:30,  2.46it/s]
  5%|▌         | 276/5070 [01:49<32:22,  2.47it/s]
  5%|▌         | 277/5070 [01:49<31:44,  2.52it/s]
  5%|▌         | 278/5070 [01:49<31:56,  2.50it/s]
  6%|▌         | 279/5070 [01:50<31:59,  2.50it/s]
  6%|▌         | 280/5070 [01:50<30:52,  2.59it/s]
  6%|▌         | 281/5070 [01:50<29:44,  2.68it/s]
  6%|▌         | 282/5070 [01:51<31:30,  2.53it/s]
  6%|▌         | 283/5070 [01:51<33:37,  2.37it/s]
  6%|▌         | 284/5070 [01:52<32:58,  2.42it/s]
  6%|▌         | 285/5070 [01:52<32:31,  2.45it/s]
  6%|▌         | 286/5070 [01:53<31:45,  2.51it/s]
  6%|▌         | 287/5070 [01:53<30:22,  2.63it/s]
  6%|▌         | 288/5070 [01:53<28:53,  2.76it/s]
  6%|▌         | 289/5070 [01:54<29:05,  2.74it/s]
  6%|▌         | 290/5070 [01:54<29:51,  2.67it/s]
  6%|▌         | 291/5070 [01:54<29:46,  2.67it/s]
  6%|▌         | 292/5070 [01:55<28:56,  2.75it/s]
  6%|▌         | 293/5070 [01:55<27:36,  2.88it/s]
  6%|▌         | 294/5070 [01:55<27:28,  2.90it/s]
  6%|▌         | 295/5070 [01:56<29:02,  2.74it/s]
  6%|▌         | 296/5070 [01:56<34:38,  2.30it/s]
  6%|▌         | 297/5070 [01:57<33:12,  2.40it/s]
  6%|▌         | 298/5070 [01:57<30:01,  2.65it/s]
  6%|▌         | 299/5070 [01:58<35:52,  2.22it/s]
  6%|▌         | 300/5070 [01:58<32:11,  2.47it/s]
  6%|▌         | 301/5070 [01:58<30:00,  2.65it/s]
  6%|▌         | 302/5070 [01:59<32:15,  2.46it/s]
  6%|▌         | 303/5070 [01:59<36:24,  2.18it/s]
  6%|▌         | 304/5070 [02:00<32:07,  2.47it/s]
  6%|▌         | 305/5070 [02:00<33:30,  2.37it/s]
  6%|▌         | 306/5070 [02:00<33:06,  2.40it/s]
  6%|▌         | 307/5070 [02:01<29:43,  2.67it/s]
  6%|▌         | 308/5070 [02:01<34:19,  2.31it/s]
  6%|▌         | 309/5070 [02:02<31:41,  2.50it/s]
  6%|▌         | 310/5070 [02:02<32:56,  2.41it/s]
  6%|▌         | 311/5070 [02:02<29:29,  2.69it/s]
  6%|▌         | 312/5070 [02:03<33:51,  2.34it/s]
  6%|▌         | 313/5070 [02:03<31:25,  2.52it/s]
  6%|▌         | 314/5070 [02:04<33:19,  2.38it/s]
  6%|▌         | 315/5070 [02:04<32:28,  2.44it/s]
  6%|▌         | 316/5070 [02:04<31:52,  2.49it/s]
  6%|▋         | 317/5070 [02:05<30:42,  2.58it/s]
  6%|▋         | 318/5070 [02:05<29:19,  2.70it/s]
  6%|▋         | 319/5070 [02:05<28:40,  2.76it/s]
  6%|▋         | 320/5070 [02:06<27:29,  2.88it/s]
  6%|▋         | 321/5070 [02:06<29:14,  2.71it/s]
  6%|▋         | 322/5070 [02:07<28:11,  2.81it/s]
  6%|▋         | 323/5070 [02:07<27:18,  2.90it/s]
  6%|▋         | 324/5070 [02:07<29:24,  2.69it/s]
  6%|▋         | 325/5070 [02:08<28:57,  2.73it/s]
  6%|▋         | 326/5070 [02:08<28:15,  2.80it/s]
  6%|▋         | 327/5070 [02:08<27:16,  2.90it/s]
  6%|▋         | 328/5070 [02:09<28:10,  2.81it/s]
  6%|▋         | 329/5070 [02:09<25:59,  3.04it/s]
  7%|▋         | 330/5070 [02:09<26:56,  2.93it/s]
  7%|▋         | 331/5070 [02:10<41:18,  1.91it/s]
  7%|▋         | 332/5070 [02:11<37:36,  2.10it/s]
  7%|▋         | 333/5070 [02:11<35:17,  2.24it/s]
  7%|▋         | 334/5070 [02:11<30:48,  2.56it/s]
  7%|▋         | 335/5070 [02:12<32:01,  2.46it/s]
  7%|▋         | 336/5070 [02:12<30:39,  2.57it/s]
  7%|▋         | 337/5070 [02:12<31:04,  2.54it/s]
  7%|▋         | 338/5070 [02:13<31:22,  2.51it/s]
  7%|▋         | 339/5070 [02:13<29:31,  2.67it/s]
  7%|▋         | 340/5070 [02:14<32:09,  2.45it/s]
  7%|▋         | 341/5070 [02:14<32:00,  2.46it/s]
  7%|▋         | 342/5070 [02:14<31:12,  2.53it/s]
  7%|▋         | 343/5070 [02:15<31:17,  2.52it/s]
  7%|▋         | 344/5070 [02:15<31:32,  2.50it/s]
  7%|▋         | 345/5070 [02:16<34:21,  2.29it/s]
  7%|▋         | 346/5070 [02:16<31:52,  2.47it/s]
  7%|▋         | 347/5070 [02:16<30:24,  2.59it/s]
  7%|▋         | 348/5070 [02:17<29:56,  2.63it/s]
  7%|▋         | 349/5070 [02:17<32:38,  2.41it/s]
  7%|▋         | 350/5070 [02:18<31:33,  2.49it/s]
  7%|▋         | 351/5070 [02:18<29:10,  2.70it/s]
  7%|▋         | 352/5070 [02:18<29:07,  2.70it/s]
  7%|▋         | 353/5070 [02:19<29:14,  2.69it/s]
  7%|▋         | 354/5070 [02:19<31:25,  2.50it/s]
  7%|▋         | 355/5070 [02:19<29:31,  2.66it/s]
  7%|▋         | 356/5070 [02:20<27:26,  2.86it/s]
  7%|▋         | 357/5070 [02:20<25:35,  3.07it/s]
  7%|▋         | 358/5070 [02:20<27:36,  2.84it/s]
  7%|▋         | 359/5070 [02:21<28:06,  2.79it/s]
  7%|▋         | 360/5070 [02:21<28:07,  2.79it/s]
  7%|▋         | 361/5070 [02:21<26:24,  2.97it/s]
  7%|▋         | 362/5070 [02:22<28:17,  2.77it/s]
  7%|▋         | 363/5070 [02:22<29:25,  2.67it/s]
  7%|▋         | 364/5070 [02:23<26:55,  2.91it/s]
  7%|▋         | 365/5070 [02:23<32:24,  2.42it/s]
  7%|▋         | 366/5070 [02:24<31:13,  2.51it/s]
  7%|▋         | 367/5070 [02:24<29:52,  2.62it/s]
  7%|▋         | 368/5070 [02:24<28:41,  2.73it/s]
  7%|▋         | 369/5070 [02:25<27:59,  2.80it/s]
  7%|▋         | 370/5070 [02:25<28:45,  2.72it/s]
  7%|▋         | 371/5070 [02:25<31:32,  2.48it/s]
  7%|▋         | 372/5070 [02:26<28:45,  2.72it/s]
  7%|▋         | 373/5070 [02:26<28:16,  2.77it/s]
  7%|▋         | 374/5070 [02:26<29:23,  2.66it/s]
  7%|▋         | 375/5070 [02:27<32:17,  2.42it/s]
  7%|▋         | 376/5070 [02:27<30:05,  2.60it/s]
  7%|▋         | 377/5070 [02:28<32:03,  2.44it/s]
  7%|▋         | 378/5070 [02:28<31:05,  2.52it/s]
  7%|▋         | 379/5070 [02:28<30:05,  2.60it/s]
  7%|▋         | 380/5070 [02:29<32:30,  2.40it/s]
  8%|▊         | 381/5070 [02:29<31:32,  2.48it/s]
  8%|▊         | 382/5070 [02:30<34:09,  2.29it/s]
  8%|▊         | 383/5070 [02:30<33:53,  2.30it/s]
  8%|▊         | 384/5070 [02:31<32:36,  2.40it/s]
  8%|▊         | 385/5070 [02:31<32:58,  2.37it/s]
  8%|▊         | 386/5070 [02:32<33:49,  2.31it/s]
  8%|▊         | 387/5070 [02:32<34:36,  2.26it/s]
  8%|▊         | 388/5070 [02:32<30:40,  2.54it/s]
  8%|▊         | 389/5070 [02:33<33:50,  2.30it/s]
  8%|▊         | 390/5070 [02:33<34:57,  2.23it/s]
  8%|▊         | 391/5070 [02:34<31:32,  2.47it/s]
  8%|▊         | 392/5070 [02:34<29:57,  2.60it/s]
  8%|▊         | 393/5070 [02:34<28:23,  2.75it/s]
  8%|▊         | 394/5070 [02:35<28:33,  2.73it/s]
  8%|▊         | 395/5070 [02:35<29:05,  2.68it/s]
  8%|▊         | 396/5070 [02:35<29:46,  2.62it/s]
  8%|▊         | 397/5070 [02:36<27:19,  2.85it/s]
  8%|▊         | 398/5070 [02:36<28:19,  2.75it/s]
  8%|▊         | 399/5070 [02:36<28:51,  2.70it/s]
  8%|▊         | 400/5070 [02:37<28:11,  2.76it/s]
  8%|▊         | 401/5070 [02:37<29:33,  2.63it/s]
  8%|▊         | 402/5070 [02:38<31:49,  2.44it/s]
  8%|▊         | 403/5070 [02:38<31:43,  2.45it/s]
  8%|▊         | 404/5070 [02:39<31:30,  2.47it/s]
  8%|▊         | 405/5070 [02:39<29:03,  2.68it/s]
  8%|▊         | 406/5070 [02:39<29:03,  2.67it/s]
  8%|▊         | 407/5070 [02:40<30:11,  2.57it/s]
  8%|▊         | 408/5070 [02:40<30:14,  2.57it/s]
  8%|▊         | 409/5070 [02:41<33:16,  2.33it/s]
  8%|▊         | 410/5070 [02:41<30:50,  2.52it/s]
  8%|▊         | 411/5070 [02:41<30:26,  2.55it/s]
  8%|▊         | 412/5070 [02:42<30:50,  2.52it/s]
  8%|▊         | 413/5070 [02:42<33:27,  2.32it/s]
  8%|▊         | 414/5070 [02:43<32:54,  2.36it/s]
  8%|▊         | 415/5070 [02:43<30:43,  2.53it/s]
  8%|▊         | 416/5070 [02:43<27:35,  2.81it/s]
  8%|▊         | 417/5070 [02:43<27:18,  2.84it/s]
  8%|▊         | 418/5070 [02:44<28:26,  2.73it/s]
  8%|▊         | 419/5070 [02:44<28:17,  2.74it/s]
  8%|▊         | 420/5070 [02:45<27:58,  2.77it/s]
  8%|▊         | 421/5070 [02:45<26:19,  2.94it/s]
  8%|▊         | 422/5070 [02:45<24:50,  3.12it/s]
  8%|▊         | 423/5070 [02:46<26:30,  2.92it/s]
  8%|▊         | 424/5070 [02:46<26:57,  2.87it/s]
  8%|▊         | 425/5070 [02:46<29:17,  2.64it/s]
  8%|▊         | 426/5070 [02:47<29:36,  2.61it/s]
  8%|▊         | 427/5070 [02:47<30:52,  2.51it/s]
  8%|▊         | 428/5070 [02:48<29:51,  2.59it/s]
  8%|▊         | 429/5070 [02:48<28:39,  2.70it/s]
  8%|▊         | 430/5070 [02:48<28:36,  2.70it/s]
  9%|▊         | 431/5070 [02:49<28:34,  2.71it/s]
  9%|▊         | 432/5070 [02:49<28:04,  2.75it/s]
  9%|▊         | 433/5070 [02:49<27:33,  2.80it/s]
  9%|▊         | 434/5070 [02:50<26:46,  2.89it/s]
  9%|▊         | 435/5070 [02:50<30:11,  2.56it/s]
  9%|▊         | 436/5070 [02:50<27:57,  2.76it/s]
  9%|▊         | 437/5070 [02:51<27:25,  2.82it/s]
  9%|▊         | 438/5070 [02:51<28:27,  2.71it/s]
  9%|▊         | 439/5070 [02:52<29:18,  2.63it/s]
  9%|▊         | 440/5070 [02:52<29:07,  2.65it/s]
  9%|▊         | 441/5070 [02:52<29:23,  2.63it/s]
  9%|▊         | 442/5070 [02:53<28:21,  2.72it/s]
  9%|▊         | 443/5070 [02:53<26:46,  2.88it/s]
  9%|▉         | 444/5070 [02:53<27:21,  2.82it/s]
  9%|▉         | 445/5070 [02:54<27:35,  2.79it/s]
  9%|▉         | 446/5070 [02:54<31:53,  2.42it/s]
  9%|▉         | 447/5070 [02:55<29:38,  2.60it/s]
  9%|▉         | 448/5070 [02:55<32:06,  2.40it/s]
  9%|▉         | 449/5070 [02:55<30:18,  2.54it/s]
  9%|▉         | 450/5070 [02:56<30:30,  2.52it/s]
  9%|▉         | 451/5070 [02:56<35:58,  2.14it/s]
  9%|▉         | 452/5070 [02:57<31:53,  2.41it/s]
  9%|▉         | 453/5070 [02:57<30:16,  2.54it/s]
  9%|▉         | 454/5070 [02:58<32:03,  2.40it/s]
  9%|▉         | 455/5070 [02:58<32:14,  2.39it/s]
  9%|▉         | 456/5070 [02:58<30:19,  2.54it/s]
  9%|▉         | 457/5070 [02:59<29:53,  2.57it/s]
  9%|▉         | 458/5070 [02:59<29:59,  2.56it/s]
  9%|▉         | 459/5070 [02:59<29:36,  2.60it/s]
  9%|▉         | 460/5070 [03:00<32:30,  2.36it/s]
  9%|▉         | 461/5070 [03:00<31:48,  2.42it/s]
  9%|▉         | 462/5070 [03:01<30:53,  2.49it/s]
  9%|▉         | 463/5070 [03:01<29:15,  2.62it/s]
  9%|▉         | 464/5070 [03:01<26:44,  2.87it/s]
  9%|▉         | 465/5070 [03:02<28:36,  2.68it/s]
  9%|▉         | 466/5070 [03:02<28:37,  2.68it/s]
  9%|▉         | 467/5070 [03:02<27:06,  2.83it/s]
  9%|▉         | 468/5070 [03:03<26:37,  2.88it/s]
  9%|▉         | 469/5070 [03:03<26:46,  2.86it/s]
  9%|▉         | 470/5070 [03:03<27:13,  2.82it/s]
  9%|▉         | 471/5070 [03:04<27:24,  2.80it/s]
  9%|▉         | 472/5070 [03:04<27:36,  2.78it/s]
  9%|▉         | 473/5070 [03:05<26:50,  2.86it/s]
  9%|▉         | 474/5070 [03:05<28:13,  2.71it/s]
  9%|▉         | 475/5070 [03:05<28:49,  2.66it/s]
  9%|▉         | 476/5070 [03:06<29:09,  2.63it/s]
  9%|▉         | 477/5070 [03:06<27:36,  2.77it/s]
  9%|▉         | 478/5070 [03:07<29:31,  2.59it/s]
  9%|▉         | 479/5070 [03:07<27:13,  2.81it/s]
  9%|▉         | 480/5070 [03:07<27:09,  2.82it/s]
  9%|▉         | 481/5070 [03:07<24:54,  3.07it/s]
 10%|▉         | 482/5070 [03:08<28:23,  2.69it/s]
 10%|▉         | 483/5070 [03:08<27:57,  2.73it/s]
 10%|▉         | 484/5070 [03:08<25:38,  2.98it/s]
 10%|▉         | 485/5070 [03:09<25:07,  3.04it/s]
 10%|▉         | 486/5070 [03:09<26:52,  2.84it/s]
 10%|▉         | 487/5070 [03:10<29:34,  2.58it/s]
 10%|▉         | 488/5070 [03:10<30:52,  2.47it/s]
 10%|▉         | 489/5070 [03:10<28:52,  2.64it/s]
 10%|▉         | 490/5070 [03:11<28:11,  2.71it/s]
 10%|▉         | 491/5070 [03:11<28:24,  2.69it/s]
 10%|▉         | 492/5070 [03:12<30:32,  2.50it/s]
 10%|▉         | 493/5070 [03:12<30:43,  2.48it/s]
 10%|▉         | 494/5070 [03:12<29:25,  2.59it/s]
 10%|▉         | 495/5070 [03:13<26:43,  2.85it/s]
 10%|▉         | 496/5070 [03:13<27:07,  2.81it/s]
 10%|▉         | 497/5070 [03:13<29:10,  2.61it/s]
 10%|▉         | 498/5070 [03:14<30:41,  2.48it/s]
 10%|▉         | 499/5070 [03:14<29:46,  2.56it/s]
 10%|▉         | 500/5070 [03:15<29:14,  2.60it/s]
                                                  

 10%|▉         | 500/5070 [03:15<29:14,  2.60it/s]
 10%|▉         | 501/5070 [03:15<31:12,  2.44it/s]
 10%|▉         | 502/5070 [03:16<31:45,  2.40it/s]
 10%|▉         | 503/5070 [03:16<34:48,  2.19it/s]
 10%|▉         | 504/5070 [03:17<35:06,  2.17it/s]
 10%|▉         | 505/5070 [03:17<34:39,  2.20it/s]
 10%|▉         | 506/5070 [03:17<31:02,  2.45it/s]
 10%|█         | 507/5070 [03:18<27:23,  2.78it/s][INFO|trainer.py:811] 2024-09-09 12:53:28,785 >> The following columns in the evaluation set don't have a corresponding argument in `BertForTokenClassification.forward` and have been ignored: id, tokens, ner_tags. If id, tokens, ner_tags are not expected by `BertForTokenClassification.forward`,  you can safely ignore this message.
[INFO|trainer.py:3819] 2024-09-09 12:53:28,787 >> 
***** Running Evaluation *****
[INFO|trainer.py:3821] 2024-09-09 12:53:28,787 >>   Num examples = 6946
[INFO|trainer.py:3824] 2024-09-09 12:53:28,787 >>   Batch size = 8
{'loss': 0.0183, 'grad_norm': 0.16175590455532074, 'learning_rate': 4.5069033530571994e-05, 'epoch': 0.99}


  0%|          | 0/869 [00:00<?, ?it/s]

  1%|          | 10/869 [00:00<00:09, 91.31it/s]

  2%|▏         | 20/869 [00:00<00:10, 79.28it/s]

  3%|▎         | 29/869 [00:00<00:10, 77.60it/s]

  4%|▍         | 37/869 [00:00<00:11, 74.43it/s]

  5%|▌         | 46/869 [00:00<00:10, 78.18it/s]

  6%|▋         | 55/869 [00:00<00:10, 80.86it/s]

  7%|▋         | 64/869 [00:00<00:10, 77.08it/s]

  8%|▊         | 72/869 [00:00<00:10, 76.17it/s]

  9%|▉         | 82/869 [00:01<00:09, 80.57it/s]

 11%|█         | 92/869 [00:01<00:09, 83.71it/s]

 12%|█▏        | 102/869 [00:01<00:09, 84.98it/s]

 13%|█▎        | 111/869 [00:01<00:09, 82.02it/s]

 14%|█▍        | 120/869 [00:01<00:09, 80.94it/s]

 15%|█▍        | 129/869 [00:01<00:09, 78.76it/s]

 16%|█▌        | 138/869 [00:01<00:08, 81.75it/s]

 17%|█▋        | 147/869 [00:01<00:09, 77.19it/s]

 18%|█▊        | 156/869 [00:01<00:09, 78.39it/s]

 19%|█▉        | 164/869 [00:02<00:08, 78.36it/s]

 20%|█▉        | 172/869 [00:02<00:09, 76.45it/s]

 21%|██        | 180/869 [00:02<00:09, 75.56it/s]

 22%|██▏       | 189/869 [00:02<00:08, 77.85it/s]

 23%|██▎       | 197/869 [00:02<00:08, 74.92it/s]

 24%|██▎       | 206/869 [00:02<00:08, 77.53it/s]

 25%|██▍       | 215/869 [00:02<00:08, 79.20it/s]

 26%|██▌       | 224/869 [00:02<00:08, 80.25it/s]

 27%|██▋       | 233/869 [00:02<00:08, 76.55it/s]

 28%|██▊       | 241/869 [00:03<00:08, 71.54it/s]

 29%|██▉       | 250/869 [00:03<00:08, 74.36it/s]

 30%|██▉       | 259/869 [00:03<00:07, 76.53it/s]

 31%|███       | 267/869 [00:03<00:07, 75.31it/s]

 32%|███▏      | 275/869 [00:03<00:07, 74.63it/s]

 33%|███▎      | 283/869 [00:03<00:07, 73.75it/s]

 33%|███▎      | 291/869 [00:03<00:07, 74.15it/s]

 34%|███▍      | 299/869 [00:03<00:07, 71.68it/s]

 35%|███▌      | 307/869 [00:03<00:07, 72.22it/s]

 36%|███▌      | 315/869 [00:04<00:07, 72.79it/s]

 37%|███▋      | 323/869 [00:04<00:07, 72.47it/s]

 38%|███▊      | 333/869 [00:04<00:06, 78.26it/s]

 39%|███▉      | 341/869 [00:04<00:06, 77.41it/s]

 40%|████      | 349/869 [00:04<00:07, 71.93it/s]

 41%|████      | 358/869 [00:04<00:06, 74.83it/s]

 42%|████▏     | 367/869 [00:04<00:06, 78.08it/s]

 43%|████▎     | 375/869 [00:04<00:06, 74.02it/s]

 44%|████▍     | 383/869 [00:04<00:06, 74.40it/s]

 45%|████▍     | 391/869 [00:05<00:06, 71.69it/s]

 46%|████▌     | 399/869 [00:05<00:06, 73.08it/s]

 47%|████▋     | 408/869 [00:05<00:06, 76.09it/s]

 48%|████▊     | 417/869 [00:05<00:05, 78.91it/s]

 49%|████▉     | 425/869 [00:05<00:05, 79.05it/s]

 50%|████▉     | 434/869 [00:05<00:05, 79.59it/s]

 51%|█████     | 442/869 [00:05<00:05, 76.14it/s]

 52%|█████▏    | 450/869 [00:05<00:05, 77.00it/s]

 53%|█████▎    | 459/869 [00:05<00:05, 77.04it/s]

 54%|█████▎    | 467/869 [00:06<00:05, 76.09it/s]

 55%|█████▍    | 477/869 [00:06<00:04, 81.01it/s]

 56%|█████▌    | 486/869 [00:06<00:04, 82.35it/s]

 57%|█████▋    | 495/869 [00:06<00:04, 76.40it/s]

 58%|█████▊    | 503/869 [00:06<00:04, 74.27it/s]

 59%|█████▉    | 511/869 [00:06<00:04, 75.21it/s]

 60%|█████▉    | 520/869 [00:06<00:04, 78.71it/s]

 61%|██████    | 528/869 [00:06<00:04, 72.74it/s]

 62%|██████▏   | 536/869 [00:06<00:04, 73.74it/s]

 63%|██████▎   | 544/869 [00:07<00:04, 70.86it/s]

 64%|██████▎   | 553/869 [00:07<00:04, 73.98it/s]

 65%|██████▍   | 561/869 [00:07<00:04, 75.50it/s]

 65%|██████▌   | 569/869 [00:07<00:04, 73.18it/s]

 67%|██████▋   | 578/869 [00:07<00:03, 75.98it/s]

 67%|██████▋   | 586/869 [00:07<00:03, 73.37it/s]

 68%|██████▊   | 594/869 [00:07<00:03, 74.99it/s]

 69%|██████▉   | 603/869 [00:07<00:03, 78.26it/s]

 70%|███████   | 611/869 [00:07<00:03, 77.04it/s]

 71%|███████   | 619/869 [00:08<00:03, 77.37it/s]

 72%|███████▏  | 627/869 [00:08<00:03, 75.86it/s]

 73%|███████▎  | 635/869 [00:08<00:03, 73.79it/s]

 74%|███████▍  | 644/869 [00:08<00:02, 77.27it/s]

 75%|███████▌  | 653/869 [00:08<00:02, 79.43it/s]

 76%|███████▌  | 661/869 [00:08<00:02, 76.75it/s]

 77%|███████▋  | 670/869 [00:08<00:02, 79.36it/s]

 78%|███████▊  | 679/869 [00:08<00:02, 82.14it/s]

 79%|███████▉  | 688/869 [00:09<00:02, 70.58it/s]

 80%|████████  | 696/869 [00:09<00:02, 72.95it/s]

 81%|████████  | 704/869 [00:09<00:02, 72.45it/s]

 82%|████████▏ | 713/869 [00:09<00:02, 76.22it/s]

 83%|████████▎ | 721/869 [00:09<00:01, 76.48it/s]

 84%|████████▍ | 730/869 [00:09<00:01, 78.48it/s]

 85%|████████▍ | 738/869 [00:09<00:01, 77.62it/s]

 86%|████████▌ | 746/869 [00:09<00:01, 78.12it/s]

 87%|████████▋ | 754/869 [00:09<00:01, 75.65it/s]

 88%|████████▊ | 763/869 [00:09<00:01, 78.80it/s]

 89%|████████▊ | 771/869 [00:10<00:01, 73.95it/s]

 90%|████████▉ | 779/869 [00:10<00:01, 65.06it/s]

 91%|█████████ | 788/869 [00:10<00:01, 69.56it/s]

 92%|█████████▏| 797/869 [00:10<00:00, 73.01it/s]

 93%|█████████▎| 805/869 [00:10<00:00, 73.73it/s]

 94%|█████████▎| 814/869 [00:10<00:00, 75.79it/s]

 95%|█████████▍| 822/869 [00:10<00:00, 75.50it/s]

 96%|█████████▌| 831/869 [00:10<00:00, 75.78it/s]

 97%|█████████▋| 840/869 [00:11<00:00, 77.68it/s]

 98%|█████████▊| 849/869 [00:11<00:00, 79.00it/s]

 99%|█████████▊| 857/869 [00:11<00:00, 79.09it/s]

100%|█████████▉| 865/869 [00:11<00:00, 73.04it/s]
                                                  

                                                 

 10%|█         | 507/5070 [03:33<27:23,  2.78it/s]

100%|██████████| 869/869 [00:15<00:00, 73.04it/s]

                                                 [INFO|trainer.py:3503] 2024-09-09 12:53:44,013 >> Saving model checkpoint to /content/dissertation/scripts/ner/output/checkpoint-507
[INFO|configuration_utils.py:472] 2024-09-09 12:53:44,015 >> Configuration saved in /content/dissertation/scripts/ner/output/checkpoint-507/config.json
[INFO|modeling_utils.py:2799] 2024-09-09 12:53:44,902 >> Model weights saved in /content/dissertation/scripts/ner/output/checkpoint-507/model.safetensors
[INFO|tokenization_utils_base.py:2684] 2024-09-09 12:53:44,903 >> tokenizer config file saved in /content/dissertation/scripts/ner/output/checkpoint-507/tokenizer_config.json
[INFO|tokenization_utils_base.py:2693] 2024-09-09 12:53:44,903 >> Special tokens file saved in /content/dissertation/scripts/ner/output/checkpoint-507/special_tokens_map.json
[INFO|tokenization_utils_base.py:2684] 2024-09-09 12:53:48,529 >> tokenizer config file saved in /content/dissertation/scripts/ner/output/tokenizer_config.json
[INFO|tokenization_utils_base.py:2693] 2024-09-09 12:53:48,529 >> Special tokens file saved in /content/dissertation/scripts/ner/output/special_tokens_map.json

 10%|█         | 508/5070 [03:38<7:59:31,  6.31s/it]
 10%|█         | 509/5070 [03:38<5:43:00,  4.51s/it]
 10%|█         | 510/5070 [03:39<4:09:49,  3.29s/it]
 10%|█         | 511/5070 [03:39<3:02:23,  2.40s/it]
 10%|█         | 512/5070 [03:39<2:14:43,  1.77s/it]
 10%|█         | 513/5070 [03:40<1:43:58,  1.37s/it]
 10%|█         | 514/5070 [03:40<1:22:41,  1.09s/it]
 10%|█         | 515/5070 [03:40<1:04:22,  1.18it/s]
 10%|█         | 516/5070 [03:41<52:24,  1.45it/s]  
 10%|█         | 517/5070 [03:41<48:49,  1.55it/s]
 10%|█         | 518/5070 [03:42<46:44,  1.62it/s]
 10%|█         | 519/5070 [03:42<39:54,  1.90it/s]
 10%|█         | 520/5070 [03:42<34:40,  2.19it/s]
 10%|█         | 521/5070 [03:43<32:48,  2.31it/s]
 10%|█         | 522/5070 [03:43<31:00,  2.44it/s]
 10%|█         | 523/5070 [03:43<27:25,  2.76it/s]
 10%|█         | 524/5070 [03:44<26:38,  2.84it/s]
 10%|█         | 525/5070 [03:44<27:04,  2.80it/s]
 10%|█         | 526/5070 [03:44<27:18,  2.77it/s]
 10%|█         | 527/5070 [03:45<27:06,  2.79it/s]
 10%|█         | 528/5070 [03:45<27:34,  2.75it/s]
 10%|█         | 529/5070 [03:45<25:33,  2.96it/s]
 10%|█         | 530/5070 [03:46<27:32,  2.75it/s]
 10%|█         | 531/5070 [03:46<28:24,  2.66it/s]
 10%|█         | 532/5070 [03:47<26:56,  2.81it/s]
 11%|█         | 533/5070 [03:47<26:50,  2.82it/s]
 11%|█         | 534/5070 [03:47<25:43,  2.94it/s]
 11%|█         | 535/5070 [03:48<27:07,  2.79it/s]
 11%|█         | 536/5070 [03:48<27:33,  2.74it/s]
 11%|█         | 537/5070 [03:48<29:37,  2.55it/s]
 11%|█         | 538/5070 [03:49<32:00,  2.36it/s]
 11%|█         | 539/5070 [03:49<29:10,  2.59it/s]
 11%|█         | 540/5070 [03:50<28:23,  2.66it/s]
 11%|█         | 541/5070 [03:50<28:24,  2.66it/s]
 11%|█         | 542/5070 [03:50<29:45,  2.54it/s]
 11%|█         | 543/5070 [03:51<32:20,  2.33it/s]
 11%|█         | 544/5070 [03:51<32:47,  2.30it/s]
 11%|█         | 545/5070 [03:52<32:17,  2.34it/s]
 11%|█         | 546/5070 [03:52<29:43,  2.54it/s]
 11%|█         | 547/5070 [03:52<29:53,  2.52it/s]
 11%|█         | 548/5070 [03:53<28:05,  2.68it/s]
 11%|█         | 549/5070 [03:53<27:40,  2.72it/s]
 11%|█         | 550/5070 [03:53<25:17,  2.98it/s]
 11%|█         | 551/5070 [03:54<26:11,  2.87it/s]
 11%|█         | 552/5070 [03:54<26:47,  2.81it/s]
 11%|█         | 553/5070 [03:55<27:54,  2.70it/s]
 11%|█         | 554/5070 [03:55<31:30,  2.39it/s]
 11%|█         | 555/5070 [03:55<31:08,  2.42it/s]
 11%|█         | 556/5070 [03:56<28:54,  2.60it/s]
 11%|█         | 557/5070 [03:56<28:31,  2.64it/s]
 11%|█         | 558/5070 [03:56<27:29,  2.74it/s]
 11%|█         | 559/5070 [03:57<30:01,  2.50it/s]
 11%|█         | 560/5070 [03:57<28:51,  2.60it/s]
 11%|█         | 561/5070 [03:58<27:48,  2.70it/s]
 11%|█         | 562/5070 [03:58<27:04,  2.78it/s]
 11%|█         | 563/5070 [03:58<28:37,  2.62it/s]
 11%|█         | 564/5070 [03:59<32:25,  2.32it/s]
 11%|█         | 565/5070 [03:59<32:00,  2.35it/s]
 11%|█         | 566/5070 [04:00<32:54,  2.28it/s]
 11%|█         | 567/5070 [04:00<32:14,  2.33it/s]
 11%|█         | 568/5070 [04:01<31:07,  2.41it/s]
 11%|█         | 569/5070 [04:01<30:28,  2.46it/s]
 11%|█         | 570/5070 [04:01<28:51,  2.60it/s]
 11%|█▏        | 571/5070 [04:02<27:02,  2.77it/s]
 11%|█▏        | 572/5070 [04:02<27:49,  2.69it/s]
 11%|█▏        | 573/5070 [04:03<29:30,  2.54it/s]
 11%|█▏        | 574/5070 [04:03<29:06,  2.57it/s]
 11%|█▏        | 575/5070 [04:03<27:59,  2.68it/s]
 11%|█▏        | 576/5070 [04:04<27:40,  2.71it/s]
 11%|█▏        | 577/5070 [04:04<26:59,  2.78it/s]
 11%|█▏        | 578/5070 [04:04<27:13,  2.75it/s]
 11%|█▏        | 579/5070 [04:05<27:11,  2.75it/s]
 11%|█▏        | 580/5070 [04:05<27:55,  2.68it/s]
 11%|█▏        | 581/5070 [04:05<27:12,  2.75it/s]
 11%|█▏        | 582/5070 [04:06<25:46,  2.90it/s]
 11%|█▏        | 583/5070 [04:06<26:23,  2.83it/s]
 12%|█▏        | 584/5070 [04:06<26:58,  2.77it/s]
 12%|█▏        | 585/5070 [04:07<28:33,  2.62it/s]
 12%|█▏        | 586/5070 [04:07<28:31,  2.62it/s]
 12%|█▏        | 587/5070 [04:08<26:37,  2.81it/s]
 12%|█▏        | 588/5070 [04:08<26:51,  2.78it/s]
 12%|█▏        | 589/5070 [04:08<28:01,  2.66it/s]
 12%|█▏        | 590/5070 [04:09<31:48,  2.35it/s]
 12%|█▏        | 591/5070 [04:09<28:30,  2.62it/s]
 12%|█▏        | 592/5070 [04:10<29:37,  2.52it/s]
 12%|█▏        | 593/5070 [04:10<28:26,  2.62it/s]
 12%|█▏        | 594/5070 [04:10<28:30,  2.62it/s]
 12%|█▏        | 595/5070 [04:11<26:28,  2.82it/s]
 12%|█▏        | 596/5070 [04:11<24:56,  2.99it/s]
 12%|█▏        | 597/5070 [04:11<26:43,  2.79it/s]
 12%|█▏        | 598/5070 [04:12<26:47,  2.78it/s]
 12%|█▏        | 599/5070 [04:12<28:46,  2.59it/s]
 12%|█▏        | 600/5070 [04:12<28:04,  2.65it/s]
 12%|█▏        | 601/5070 [04:13<27:50,  2.68it/s]
 12%|█▏        | 602/5070 [04:13<28:36,  2.60it/s]
 12%|█▏        | 603/5070 [04:14<31:30,  2.36it/s]
 12%|█▏        | 604/5070 [04:14<30:04,  2.48it/s]
 12%|█▏        | 605/5070 [04:15<29:27,  2.53it/s]
 12%|█▏        | 606/5070 [04:15<27:32,  2.70it/s]
 12%|█▏        | 607/5070 [04:15<26:02,  2.86it/s]
 12%|█▏        | 608/5070 [04:15<24:33,  3.03it/s]
 12%|█▏        | 609/5070 [04:16<27:46,  2.68it/s]
 12%|█▏        | 610/5070 [04:16<27:02,  2.75it/s]
 12%|█▏        | 611/5070 [04:17<27:23,  2.71it/s]
 12%|█▏        | 612/5070 [04:17<26:46,  2.78it/s]
 12%|█▏        | 613/5070 [04:17<27:05,  2.74it/s]
 12%|█▏        | 614/5070 [04:18<27:24,  2.71it/s]
 12%|█▏        | 615/5070 [04:18<27:54,  2.66it/s]
 12%|█▏        | 616/5070 [04:18<28:24,  2.61it/s]
 12%|█▏        | 617/5070 [04:19<27:37,  2.69it/s]
 12%|█▏        | 618/5070 [04:19<27:37,  2.69it/s]
 12%|█▏        | 619/5070 [04:20<27:19,  2.71it/s]
 12%|█▏        | 620/5070 [04:20<32:19,  2.29it/s]
 12%|█▏        | 621/5070 [04:21<33:26,  2.22it/s]