[2025-02-09 11:31:41,136][00884] Saving configuration to /content/train_dir/default_experiment/config.json... [2025-02-09 11:31:41,138][00884] Rollout worker 0 uses device cpu [2025-02-09 11:31:41,139][00884] Rollout worker 1 uses device cpu [2025-02-09 11:31:41,141][00884] Rollout worker 2 uses device cpu [2025-02-09 11:31:41,142][00884] Rollout worker 3 uses device cpu [2025-02-09 11:31:41,144][00884] Rollout worker 4 uses device cpu [2025-02-09 11:31:41,145][00884] Rollout worker 5 uses device cpu [2025-02-09 11:31:41,147][00884] Rollout worker 6 uses device cpu [2025-02-09 11:31:41,148][00884] Rollout worker 7 uses device cpu [2025-02-09 11:31:41,305][00884] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2025-02-09 11:31:41,307][00884] InferenceWorker_p0-w0: min num requests: 2 [2025-02-09 11:31:41,341][00884] Starting all processes... [2025-02-09 11:31:41,342][00884] Starting process learner_proc0 [2025-02-09 11:31:41,402][00884] Starting all processes... [2025-02-09 11:31:41,412][00884] Starting process inference_proc0-0 [2025-02-09 11:31:41,412][00884] Starting process rollout_proc0 [2025-02-09 11:31:41,412][00884] Starting process rollout_proc1 [2025-02-09 11:31:41,412][00884] Starting process rollout_proc2 [2025-02-09 11:31:41,412][00884] Starting process rollout_proc3 [2025-02-09 11:31:41,413][00884] Starting process rollout_proc4 [2025-02-09 11:31:41,413][00884] Starting process rollout_proc5 [2025-02-09 11:31:41,413][00884] Starting process rollout_proc6 [2025-02-09 11:31:41,413][00884] Starting process rollout_proc7 [2025-02-09 11:31:58,922][04159] Worker 4 uses CPU cores [0] [2025-02-09 11:31:58,939][04141] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2025-02-09 11:31:58,939][04141] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 [2025-02-09 11:31:59,023][04141] Num visible devices: 1 [2025-02-09 11:31:59,072][04141] Starting seed is not provided [2025-02-09 11:31:59,073][04141] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2025-02-09 11:31:59,073][04141] Initializing actor-critic model on device cuda:0 [2025-02-09 11:31:59,074][04141] RunningMeanStd input shape: (3, 72, 128) [2025-02-09 11:31:59,076][04141] RunningMeanStd input shape: (1,) [2025-02-09 11:31:59,127][04155] Worker 0 uses CPU cores [0] [2025-02-09 11:31:59,146][04162] Worker 7 uses CPU cores [1] [2025-02-09 11:31:59,146][04141] ConvEncoder: input_channels=3 [2025-02-09 11:31:59,190][04158] Worker 3 uses CPU cores [1] [2025-02-09 11:31:59,275][04154] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2025-02-09 11:31:59,276][04154] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 [2025-02-09 11:31:59,294][04157] Worker 2 uses CPU cores [0] [2025-02-09 11:31:59,325][04160] Worker 5 uses CPU cores [1] [2025-02-09 11:31:59,332][04154] Num visible devices: 1 [2025-02-09 11:31:59,360][04156] Worker 1 uses CPU cores [1] [2025-02-09 11:31:59,394][04161] Worker 6 uses CPU cores [0] [2025-02-09 11:31:59,471][04141] Conv encoder output size: 512 [2025-02-09 11:31:59,471][04141] Policy head output size: 512 [2025-02-09 11:31:59,526][04141] Created Actor Critic model with architecture: [2025-02-09 11:31:59,526][04141] ActorCriticSharedWeights( (obs_normalizer): ObservationNormalizer( (running_mean_std): RunningMeanStdDictInPlace( (running_mean_std): ModuleDict( (obs): RunningMeanStdInPlace() ) ) ) (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) (encoder): VizdoomEncoder( (basic_encoder): ConvEncoder( (enc): RecursiveScriptModule( original_name=ConvEncoderImpl (conv_head): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Conv2d) (1): RecursiveScriptModule(original_name=ELU) (2): RecursiveScriptModule(original_name=Conv2d) (3): RecursiveScriptModule(original_name=ELU) (4): RecursiveScriptModule(original_name=Conv2d) (5): RecursiveScriptModule(original_name=ELU) ) (mlp_layers): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Linear) (1): RecursiveScriptModule(original_name=ELU) ) ) ) ) (core): ModelCoreRNN( (core): GRU(512, 512) ) (decoder): MlpDecoder( (mlp): Identity() ) (critic_linear): Linear(in_features=512, out_features=1, bias=True) (action_parameterization): ActionParameterizationDefault( (distribution_linear): Linear(in_features=512, out_features=5, bias=True) ) ) [2025-02-09 11:31:59,864][04141] Using optimizer [2025-02-09 11:32:01,300][00884] Heartbeat connected on Batcher_0 [2025-02-09 11:32:01,306][00884] Heartbeat connected on InferenceWorker_p0-w0 [2025-02-09 11:32:01,318][00884] Heartbeat connected on RolloutWorker_w1 [2025-02-09 11:32:01,320][00884] Heartbeat connected on RolloutWorker_w0 [2025-02-09 11:32:01,323][00884] Heartbeat connected on RolloutWorker_w2 [2025-02-09 11:32:01,325][00884] Heartbeat connected on RolloutWorker_w3 [2025-02-09 11:32:01,330][00884] Heartbeat connected on RolloutWorker_w4 [2025-02-09 11:32:01,333][00884] Heartbeat connected on RolloutWorker_w5 [2025-02-09 11:32:01,336][00884] Heartbeat connected on RolloutWorker_w6 [2025-02-09 11:32:01,346][00884] Heartbeat connected on RolloutWorker_w7 [2025-02-09 11:32:04,045][04141] No checkpoints found [2025-02-09 11:32:04,046][04141] Did not load from checkpoint, starting from scratch! [2025-02-09 11:32:04,046][04141] Initialized policy 0 weights for model version 0 [2025-02-09 11:32:04,049][04141] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2025-02-09 11:32:04,063][04141] LearnerWorker_p0 finished initialization! [2025-02-09 11:32:04,065][00884] Heartbeat connected on LearnerWorker_p0 [2025-02-09 11:32:04,275][04154] RunningMeanStd input shape: (3, 72, 128) [2025-02-09 11:32:04,276][04154] RunningMeanStd input shape: (1,) [2025-02-09 11:32:04,288][04154] ConvEncoder: input_channels=3 [2025-02-09 11:32:04,388][04154] Conv encoder output size: 512 [2025-02-09 11:32:04,388][04154] Policy head output size: 512 [2025-02-09 11:32:04,423][00884] Inference worker 0-0 is ready! [2025-02-09 11:32:04,424][00884] All inference workers are ready! Signal rollout workers to start! [2025-02-09 11:32:04,756][04161] Doom resolution: 160x120, resize resolution: (128, 72) [2025-02-09 11:32:04,754][04155] Doom resolution: 160x120, resize resolution: (128, 72) [2025-02-09 11:32:04,769][04160] Doom resolution: 160x120, resize resolution: (128, 72) [2025-02-09 11:32:04,782][04159] Doom resolution: 160x120, resize resolution: (128, 72) [2025-02-09 11:32:04,802][04157] Doom resolution: 160x120, resize resolution: (128, 72) [2025-02-09 11:32:04,828][04162] Doom resolution: 160x120, resize resolution: (128, 72) [2025-02-09 11:32:04,871][04158] Doom resolution: 160x120, resize resolution: (128, 72) [2025-02-09 11:32:04,879][04156] Doom resolution: 160x120, resize resolution: (128, 72) [2025-02-09 11:32:05,406][00884] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2025-02-09 11:32:06,696][04158] Decorrelating experience for 0 frames... [2025-02-09 11:32:06,697][04156] Decorrelating experience for 0 frames... [2025-02-09 11:32:06,699][04161] Decorrelating experience for 0 frames... [2025-02-09 11:32:06,700][04162] Decorrelating experience for 0 frames... [2025-02-09 11:32:06,701][04155] Decorrelating experience for 0 frames... [2025-02-09 11:32:08,084][04162] Decorrelating experience for 32 frames... [2025-02-09 11:32:08,093][04156] Decorrelating experience for 32 frames... [2025-02-09 11:32:08,100][04158] Decorrelating experience for 32 frames... [2025-02-09 11:32:08,116][04161] Decorrelating experience for 32 frames... [2025-02-09 11:32:08,143][04155] Decorrelating experience for 32 frames... [2025-02-09 11:32:09,766][04159] Decorrelating experience for 0 frames... [2025-02-09 11:32:09,793][04157] Decorrelating experience for 0 frames... [2025-02-09 11:32:10,090][04161] Decorrelating experience for 64 frames... [2025-02-09 11:32:10,297][04162] Decorrelating experience for 64 frames... [2025-02-09 11:32:10,319][04158] Decorrelating experience for 64 frames... [2025-02-09 11:32:10,323][04156] Decorrelating experience for 64 frames... [2025-02-09 11:32:10,406][00884] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2025-02-09 11:32:10,855][04158] Decorrelating experience for 96 frames... [2025-02-09 11:32:11,113][04155] Decorrelating experience for 64 frames... [2025-02-09 11:32:11,507][04159] Decorrelating experience for 32 frames... [2025-02-09 11:32:11,558][04161] Decorrelating experience for 96 frames... [2025-02-09 11:32:11,837][04157] Decorrelating experience for 32 frames... [2025-02-09 11:32:12,682][04160] Decorrelating experience for 0 frames... [2025-02-09 11:32:13,144][04162] Decorrelating experience for 96 frames... [2025-02-09 11:32:13,270][04155] Decorrelating experience for 96 frames... [2025-02-09 11:32:13,867][04156] Decorrelating experience for 96 frames... [2025-02-09 11:32:13,921][04159] Decorrelating experience for 64 frames... [2025-02-09 11:32:14,443][04160] Decorrelating experience for 32 frames... [2025-02-09 11:32:15,406][00884] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 57.2. Samples: 572. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2025-02-09 11:32:15,411][00884] Avg episode reward: [(0, '2.600')] [2025-02-09 11:32:15,977][04157] Decorrelating experience for 64 frames... [2025-02-09 11:32:16,583][04141] Signal inference workers to stop experience collection... [2025-02-09 11:32:16,599][04154] InferenceWorker_p0-w0: stopping experience collection [2025-02-09 11:32:16,850][04159] Decorrelating experience for 96 frames... [2025-02-09 11:32:17,100][04160] Decorrelating experience for 64 frames... [2025-02-09 11:32:17,496][04160] Decorrelating experience for 96 frames... [2025-02-09 11:32:17,699][04157] Decorrelating experience for 96 frames... [2025-02-09 11:32:18,379][04141] Signal inference workers to resume experience collection... [2025-02-09 11:32:18,381][04154] InferenceWorker_p0-w0: resuming experience collection [2025-02-09 11:32:20,406][00884] Fps is (10 sec: 1228.8, 60 sec: 819.2, 300 sec: 819.2). Total num frames: 12288. Throughput: 0: 226.5. Samples: 3398. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) [2025-02-09 11:32:20,407][00884] Avg episode reward: [(0, '3.294')] [2025-02-09 11:32:25,408][00884] Fps is (10 sec: 2457.0, 60 sec: 1228.7, 300 sec: 1228.7). Total num frames: 24576. Throughput: 0: 265.9. Samples: 5318. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2025-02-09 11:32:25,411][00884] Avg episode reward: [(0, '3.527')] [2025-02-09 11:32:29,269][04154] Updated weights for policy 0, policy_version 10 (0.0105) [2025-02-09 11:32:30,406][00884] Fps is (10 sec: 3276.8, 60 sec: 1802.3, 300 sec: 1802.3). Total num frames: 45056. Throughput: 0: 388.7. Samples: 9718. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2025-02-09 11:32:30,411][00884] Avg episode reward: [(0, '4.078')] [2025-02-09 11:32:35,407][00884] Fps is (10 sec: 4096.3, 60 sec: 2184.4, 300 sec: 2184.4). Total num frames: 65536. Throughput: 0: 561.4. Samples: 16842. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:32:35,414][00884] Avg episode reward: [(0, '4.422')] [2025-02-09 11:32:38,302][04154] Updated weights for policy 0, policy_version 20 (0.0020) [2025-02-09 11:32:40,406][00884] Fps is (10 sec: 4096.0, 60 sec: 2457.6, 300 sec: 2457.6). Total num frames: 86016. Throughput: 0: 579.3. Samples: 20274. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:32:40,407][00884] Avg episode reward: [(0, '4.408')] [2025-02-09 11:32:45,406][00884] Fps is (10 sec: 3687.0, 60 sec: 2560.0, 300 sec: 2560.0). Total num frames: 102400. Throughput: 0: 616.2. Samples: 24646. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2025-02-09 11:32:45,408][00884] Avg episode reward: [(0, '4.503')] [2025-02-09 11:32:45,417][04141] Saving new best policy, reward=4.503! [2025-02-09 11:32:49,862][04154] Updated weights for policy 0, policy_version 30 (0.0017) [2025-02-09 11:32:50,406][00884] Fps is (10 sec: 3686.4, 60 sec: 2730.7, 300 sec: 2730.7). Total num frames: 122880. Throughput: 0: 683.3. Samples: 30750. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2025-02-09 11:32:50,411][00884] Avg episode reward: [(0, '4.457')] [2025-02-09 11:32:55,406][00884] Fps is (10 sec: 4096.0, 60 sec: 2867.2, 300 sec: 2867.2). Total num frames: 143360. Throughput: 0: 758.4. Samples: 34126. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2025-02-09 11:32:55,409][00884] Avg episode reward: [(0, '4.352')] [2025-02-09 11:33:00,406][00884] Fps is (10 sec: 3686.4, 60 sec: 2904.4, 300 sec: 2904.4). Total num frames: 159744. Throughput: 0: 862.8. Samples: 39400. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2025-02-09 11:33:00,410][00884] Avg episode reward: [(0, '4.320')] [2025-02-09 11:33:01,536][04154] Updated weights for policy 0, policy_version 40 (0.0018) [2025-02-09 11:33:05,406][00884] Fps is (10 sec: 3686.4, 60 sec: 3003.7, 300 sec: 3003.7). Total num frames: 180224. Throughput: 0: 914.4. Samples: 44544. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2025-02-09 11:33:05,407][00884] Avg episode reward: [(0, '4.384')] [2025-02-09 11:33:10,406][00884] Fps is (10 sec: 4096.0, 60 sec: 3345.1, 300 sec: 3087.8). Total num frames: 200704. Throughput: 0: 950.6. Samples: 48092. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2025-02-09 11:33:10,411][00884] Avg episode reward: [(0, '4.612')] [2025-02-09 11:33:10,414][04141] Saving new best policy, reward=4.612! [2025-02-09 11:33:10,693][04154] Updated weights for policy 0, policy_version 50 (0.0014) [2025-02-09 11:33:15,406][00884] Fps is (10 sec: 4095.9, 60 sec: 3686.4, 300 sec: 3159.8). Total num frames: 221184. Throughput: 0: 995.7. Samples: 54524. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2025-02-09 11:33:15,410][00884] Avg episode reward: [(0, '4.528')] [2025-02-09 11:33:20,406][00884] Fps is (10 sec: 3686.4, 60 sec: 3754.7, 300 sec: 3167.6). Total num frames: 237568. Throughput: 0: 931.5. Samples: 58760. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2025-02-09 11:33:20,410][00884] Avg episode reward: [(0, '4.375')] [2025-02-09 11:33:22,076][04154] Updated weights for policy 0, policy_version 60 (0.0022) [2025-02-09 11:33:25,406][00884] Fps is (10 sec: 3686.4, 60 sec: 3891.4, 300 sec: 3225.6). Total num frames: 258048. Throughput: 0: 932.8. Samples: 62248. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2025-02-09 11:33:25,410][00884] Avg episode reward: [(0, '4.595')] [2025-02-09 11:33:30,406][00884] Fps is (10 sec: 4505.6, 60 sec: 3959.5, 300 sec: 3325.0). Total num frames: 282624. Throughput: 0: 996.4. Samples: 69482. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:33:30,408][00884] Avg episode reward: [(0, '4.634')] [2025-02-09 11:33:30,414][04141] Saving new best policy, reward=4.634! [2025-02-09 11:33:31,789][04154] Updated weights for policy 0, policy_version 70 (0.0014) [2025-02-09 11:33:35,406][00884] Fps is (10 sec: 3686.4, 60 sec: 3823.0, 300 sec: 3276.8). Total num frames: 294912. Throughput: 0: 964.5. Samples: 74154. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:33:35,412][00884] Avg episode reward: [(0, '4.661')] [2025-02-09 11:33:35,419][04141] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000072_294912.pth... [2025-02-09 11:33:35,604][04141] Saving new best policy, reward=4.661! [2025-02-09 11:33:40,406][00884] Fps is (10 sec: 3276.8, 60 sec: 3822.9, 300 sec: 3319.9). Total num frames: 315392. Throughput: 0: 944.0. Samples: 76608. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2025-02-09 11:33:40,408][00884] Avg episode reward: [(0, '4.544')] [2025-02-09 11:33:42,474][04154] Updated weights for policy 0, policy_version 80 (0.0013) [2025-02-09 11:33:45,406][00884] Fps is (10 sec: 4505.6, 60 sec: 3959.5, 300 sec: 3399.7). Total num frames: 339968. Throughput: 0: 982.7. Samples: 83620. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2025-02-09 11:33:45,409][00884] Avg episode reward: [(0, '4.565')] [2025-02-09 11:33:50,406][00884] Fps is (10 sec: 4096.0, 60 sec: 3891.2, 300 sec: 3393.8). Total num frames: 356352. Throughput: 0: 996.5. Samples: 89386. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2025-02-09 11:33:50,410][00884] Avg episode reward: [(0, '4.469')] [2025-02-09 11:33:53,921][04154] Updated weights for policy 0, policy_version 90 (0.0018) [2025-02-09 11:33:55,406][00884] Fps is (10 sec: 3276.8, 60 sec: 3822.9, 300 sec: 3388.5). Total num frames: 372736. Throughput: 0: 964.4. Samples: 91492. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2025-02-09 11:33:55,408][00884] Avg episode reward: [(0, '4.477')] [2025-02-09 11:34:00,406][00884] Fps is (10 sec: 4096.0, 60 sec: 3959.5, 300 sec: 3454.9). Total num frames: 397312. Throughput: 0: 964.0. Samples: 97904. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:34:00,408][00884] Avg episode reward: [(0, '4.634')] [2025-02-09 11:34:02,687][04154] Updated weights for policy 0, policy_version 100 (0.0018) [2025-02-09 11:34:05,406][00884] Fps is (10 sec: 4505.6, 60 sec: 3959.5, 300 sec: 3481.6). Total num frames: 417792. Throughput: 0: 1021.2. Samples: 104716. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:34:05,408][00884] Avg episode reward: [(0, '4.491')] [2025-02-09 11:34:10,406][00884] Fps is (10 sec: 3276.8, 60 sec: 3822.9, 300 sec: 3440.6). Total num frames: 430080. Throughput: 0: 990.9. Samples: 106838. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2025-02-09 11:34:10,409][00884] Avg episode reward: [(0, '4.450')] [2025-02-09 11:34:14,077][04154] Updated weights for policy 0, policy_version 110 (0.0020) [2025-02-09 11:34:15,406][00884] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3497.4). Total num frames: 454656. Throughput: 0: 954.2. Samples: 112420. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2025-02-09 11:34:15,412][00884] Avg episode reward: [(0, '4.637')] [2025-02-09 11:34:20,406][00884] Fps is (10 sec: 4915.2, 60 sec: 4027.7, 300 sec: 3549.9). Total num frames: 479232. Throughput: 0: 1007.2. Samples: 119478. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2025-02-09 11:34:20,408][00884] Avg episode reward: [(0, '4.717')] [2025-02-09 11:34:20,411][04141] Saving new best policy, reward=4.717! [2025-02-09 11:34:24,297][04154] Updated weights for policy 0, policy_version 120 (0.0030) [2025-02-09 11:34:25,412][00884] Fps is (10 sec: 3684.0, 60 sec: 3890.8, 300 sec: 3510.7). Total num frames: 491520. Throughput: 0: 1012.0. Samples: 122154. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2025-02-09 11:34:25,415][00884] Avg episode reward: [(0, '4.721')] [2025-02-09 11:34:25,432][04141] Saving new best policy, reward=4.721! [2025-02-09 11:34:30,406][00884] Fps is (10 sec: 3276.8, 60 sec: 3822.9, 300 sec: 3531.0). Total num frames: 512000. Throughput: 0: 954.8. Samples: 126586. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2025-02-09 11:34:30,408][00884] Avg episode reward: [(0, '4.896')] [2025-02-09 11:34:30,411][04141] Saving new best policy, reward=4.896! [2025-02-09 11:34:34,806][04154] Updated weights for policy 0, policy_version 130 (0.0028) [2025-02-09 11:34:35,406][00884] Fps is (10 sec: 4098.6, 60 sec: 3959.5, 300 sec: 3549.9). Total num frames: 532480. Throughput: 0: 979.6. Samples: 133466. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:34:35,408][00884] Avg episode reward: [(0, '5.086')] [2025-02-09 11:34:35,413][04141] Saving new best policy, reward=5.086! [2025-02-09 11:34:40,408][00884] Fps is (10 sec: 4095.1, 60 sec: 3959.3, 300 sec: 3567.4). Total num frames: 552960. Throughput: 0: 1007.6. Samples: 136836. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2025-02-09 11:34:40,410][00884] Avg episode reward: [(0, '5.159')] [2025-02-09 11:34:40,414][04141] Saving new best policy, reward=5.159! [2025-02-09 11:34:45,406][00884] Fps is (10 sec: 3276.8, 60 sec: 3754.7, 300 sec: 3532.8). Total num frames: 565248. Throughput: 0: 956.7. Samples: 140956. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:34:45,408][00884] Avg episode reward: [(0, '4.992')] [2025-02-09 11:34:48,151][04154] Updated weights for policy 0, policy_version 140 (0.0021) [2025-02-09 11:34:50,413][00884] Fps is (10 sec: 2456.4, 60 sec: 3686.0, 300 sec: 3500.1). Total num frames: 577536. Throughput: 0: 891.2. Samples: 144826. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2025-02-09 11:34:50,418][00884] Avg episode reward: [(0, '4.707')] [2025-02-09 11:34:55,406][00884] Fps is (10 sec: 3276.7, 60 sec: 3754.7, 300 sec: 3517.7). Total num frames: 598016. Throughput: 0: 906.3. Samples: 147620. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2025-02-09 11:34:55,408][00884] Avg episode reward: [(0, '4.656')] [2025-02-09 11:34:59,500][04154] Updated weights for policy 0, policy_version 150 (0.0029) [2025-02-09 11:35:00,406][00884] Fps is (10 sec: 3689.0, 60 sec: 3618.1, 300 sec: 3510.9). Total num frames: 614400. Throughput: 0: 910.0. Samples: 153370. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2025-02-09 11:35:00,409][00884] Avg episode reward: [(0, '4.749')] [2025-02-09 11:35:05,406][00884] Fps is (10 sec: 3686.5, 60 sec: 3618.1, 300 sec: 3527.1). Total num frames: 634880. Throughput: 0: 862.5. Samples: 158290. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:35:05,411][00884] Avg episode reward: [(0, '4.812')] [2025-02-09 11:35:09,542][04154] Updated weights for policy 0, policy_version 160 (0.0021) [2025-02-09 11:35:10,406][00884] Fps is (10 sec: 4505.5, 60 sec: 3822.9, 300 sec: 3564.6). Total num frames: 659456. Throughput: 0: 883.2. Samples: 161892. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2025-02-09 11:35:10,411][00884] Avg episode reward: [(0, '5.005')] [2025-02-09 11:35:15,406][00884] Fps is (10 sec: 4095.7, 60 sec: 3686.4, 300 sec: 3557.0). Total num frames: 675840. Throughput: 0: 942.5. Samples: 169000. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:35:15,411][00884] Avg episode reward: [(0, '4.983')] [2025-02-09 11:35:20,406][00884] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3549.9). Total num frames: 692224. Throughput: 0: 886.2. Samples: 173346. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2025-02-09 11:35:20,411][00884] Avg episode reward: [(0, '4.932')] [2025-02-09 11:35:20,944][04154] Updated weights for policy 0, policy_version 170 (0.0021) [2025-02-09 11:35:25,406][00884] Fps is (10 sec: 4096.3, 60 sec: 3755.1, 300 sec: 3584.0). Total num frames: 716800. Throughput: 0: 880.9. Samples: 176476. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2025-02-09 11:35:25,411][00884] Avg episode reward: [(0, '5.143')] [2025-02-09 11:35:29,574][04154] Updated weights for policy 0, policy_version 180 (0.0020) [2025-02-09 11:35:30,406][00884] Fps is (10 sec: 4915.3, 60 sec: 3822.9, 300 sec: 3616.5). Total num frames: 741376. Throughput: 0: 946.6. Samples: 183554. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2025-02-09 11:35:30,413][00884] Avg episode reward: [(0, '5.056')] [2025-02-09 11:35:35,406][00884] Fps is (10 sec: 3686.4, 60 sec: 3686.4, 300 sec: 3588.9). Total num frames: 753664. Throughput: 0: 978.5. Samples: 188852. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2025-02-09 11:35:35,408][00884] Avg episode reward: [(0, '4.939')] [2025-02-09 11:35:35,428][04141] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000184_753664.pth... [2025-02-09 11:35:40,406][00884] Fps is (10 sec: 3276.8, 60 sec: 3686.5, 300 sec: 3600.7). Total num frames: 774144. Throughput: 0: 963.5. Samples: 190978. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2025-02-09 11:35:40,408][00884] Avg episode reward: [(0, '4.982')] [2025-02-09 11:35:40,998][04154] Updated weights for policy 0, policy_version 190 (0.0019) [2025-02-09 11:35:45,406][00884] Fps is (10 sec: 4505.7, 60 sec: 3891.2, 300 sec: 3630.5). Total num frames: 798720. Throughput: 0: 991.2. Samples: 197974. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2025-02-09 11:35:45,408][00884] Avg episode reward: [(0, '5.167')] [2025-02-09 11:35:45,415][04141] Saving new best policy, reward=5.167! [2025-02-09 11:35:50,408][00884] Fps is (10 sec: 4095.2, 60 sec: 3959.8, 300 sec: 3622.7). Total num frames: 815104. Throughput: 0: 1020.3. Samples: 204204. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2025-02-09 11:35:50,413][00884] Avg episode reward: [(0, '5.063')] [2025-02-09 11:35:51,102][04154] Updated weights for policy 0, policy_version 200 (0.0031) [2025-02-09 11:35:55,406][00884] Fps is (10 sec: 3276.8, 60 sec: 3891.2, 300 sec: 3615.2). Total num frames: 831488. Throughput: 0: 986.2. Samples: 206272. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2025-02-09 11:35:55,408][00884] Avg episode reward: [(0, '4.802')] [2025-02-09 11:36:00,406][00884] Fps is (10 sec: 4096.8, 60 sec: 4027.7, 300 sec: 3642.8). Total num frames: 856064. Throughput: 0: 963.5. Samples: 212356. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:36:00,409][00884] Avg episode reward: [(0, '4.553')] [2025-02-09 11:36:01,204][04154] Updated weights for policy 0, policy_version 210 (0.0018) [2025-02-09 11:36:05,406][00884] Fps is (10 sec: 4505.6, 60 sec: 4027.7, 300 sec: 3652.3). Total num frames: 876544. Throughput: 0: 1023.2. Samples: 219390. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2025-02-09 11:36:05,410][00884] Avg episode reward: [(0, '4.520')] [2025-02-09 11:36:10,406][00884] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3644.6). Total num frames: 892928. Throughput: 0: 1007.5. Samples: 221814. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2025-02-09 11:36:10,408][00884] Avg episode reward: [(0, '4.422')] [2025-02-09 11:36:12,503][04154] Updated weights for policy 0, policy_version 220 (0.0014) [2025-02-09 11:36:15,406][00884] Fps is (10 sec: 3686.4, 60 sec: 3959.5, 300 sec: 3653.6). Total num frames: 913408. Throughput: 0: 962.3. Samples: 226856. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2025-02-09 11:36:15,408][00884] Avg episode reward: [(0, '4.652')] [2025-02-09 11:36:20,406][00884] Fps is (10 sec: 4505.6, 60 sec: 4096.0, 300 sec: 3678.4). Total num frames: 937984. Throughput: 0: 1003.8. Samples: 234024. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2025-02-09 11:36:20,409][00884] Avg episode reward: [(0, '4.893')] [2025-02-09 11:36:21,283][04154] Updated weights for policy 0, policy_version 230 (0.0013) [2025-02-09 11:36:25,406][00884] Fps is (10 sec: 4096.0, 60 sec: 3959.5, 300 sec: 3670.6). Total num frames: 954368. Throughput: 0: 1032.5. Samples: 237440. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:36:25,413][00884] Avg episode reward: [(0, '4.919')] [2025-02-09 11:36:30,406][00884] Fps is (10 sec: 3276.8, 60 sec: 3822.9, 300 sec: 3663.2). Total num frames: 970752. Throughput: 0: 973.1. Samples: 241764. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2025-02-09 11:36:30,411][00884] Avg episode reward: [(0, '4.853')] [2025-02-09 11:36:32,464][04154] Updated weights for policy 0, policy_version 240 (0.0031) [2025-02-09 11:36:35,406][00884] Fps is (10 sec: 4096.0, 60 sec: 4027.7, 300 sec: 3686.4). Total num frames: 995328. Throughput: 0: 984.0. Samples: 248482. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2025-02-09 11:36:35,411][00884] Avg episode reward: [(0, '5.272')] [2025-02-09 11:36:35,419][04141] Saving new best policy, reward=5.272! [2025-02-09 11:36:40,406][00884] Fps is (10 sec: 4505.6, 60 sec: 4027.7, 300 sec: 3693.8). Total num frames: 1015808. Throughput: 0: 1017.5. Samples: 252058. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2025-02-09 11:36:40,410][00884] Avg episode reward: [(0, '5.288')] [2025-02-09 11:36:40,412][04141] Saving new best policy, reward=5.288! [2025-02-09 11:36:42,489][04154] Updated weights for policy 0, policy_version 250 (0.0022) [2025-02-09 11:36:45,406][00884] Fps is (10 sec: 3686.2, 60 sec: 3891.2, 300 sec: 3686.4). Total num frames: 1032192. Throughput: 0: 994.7. Samples: 257118. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2025-02-09 11:36:45,411][00884] Avg episode reward: [(0, '5.217')] [2025-02-09 11:36:50,406][00884] Fps is (10 sec: 3686.4, 60 sec: 3959.6, 300 sec: 3693.6). Total num frames: 1052672. Throughput: 0: 967.9. Samples: 262944. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2025-02-09 11:36:50,408][00884] Avg episode reward: [(0, '5.356')] [2025-02-09 11:36:50,413][04141] Saving new best policy, reward=5.356! [2025-02-09 11:36:52,754][04154] Updated weights for policy 0, policy_version 260 (0.0021) [2025-02-09 11:36:55,406][00884] Fps is (10 sec: 4505.8, 60 sec: 4096.0, 300 sec: 3714.7). Total num frames: 1077248. Throughput: 0: 992.2. Samples: 266462. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2025-02-09 11:36:55,408][00884] Avg episode reward: [(0, '5.439')] [2025-02-09 11:36:55,415][04141] Saving new best policy, reward=5.439! [2025-02-09 11:37:00,406][00884] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3693.3). Total num frames: 1089536. Throughput: 0: 1011.6. Samples: 272380. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:37:00,407][00884] Avg episode reward: [(0, '5.432')] [2025-02-09 11:37:04,426][04154] Updated weights for policy 0, policy_version 270 (0.0044) [2025-02-09 11:37:05,408][00884] Fps is (10 sec: 3276.2, 60 sec: 3891.1, 300 sec: 3762.7). Total num frames: 1110016. Throughput: 0: 959.2. Samples: 277188. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:37:05,412][00884] Avg episode reward: [(0, '5.441')] [2025-02-09 11:37:05,419][04141] Saving new best policy, reward=5.441! [2025-02-09 11:37:10,406][00884] Fps is (10 sec: 4505.6, 60 sec: 4027.7, 300 sec: 3846.1). Total num frames: 1134592. Throughput: 0: 962.9. Samples: 280772. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2025-02-09 11:37:10,408][00884] Avg episode reward: [(0, '5.462')] [2025-02-09 11:37:10,413][04141] Saving new best policy, reward=5.462! [2025-02-09 11:37:13,045][04154] Updated weights for policy 0, policy_version 280 (0.0021) [2025-02-09 11:37:15,406][00884] Fps is (10 sec: 4506.5, 60 sec: 4027.7, 300 sec: 3873.8). Total num frames: 1155072. Throughput: 0: 1027.1. Samples: 287982. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2025-02-09 11:37:15,413][00884] Avg episode reward: [(0, '5.599')] [2025-02-09 11:37:15,422][04141] Saving new best policy, reward=5.599! [2025-02-09 11:37:20,406][00884] Fps is (10 sec: 3276.7, 60 sec: 3822.9, 300 sec: 3873.9). Total num frames: 1167360. Throughput: 0: 976.9. Samples: 292444. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2025-02-09 11:37:20,408][00884] Avg episode reward: [(0, '5.683')] [2025-02-09 11:37:20,410][04141] Saving new best policy, reward=5.683! [2025-02-09 11:37:24,331][04154] Updated weights for policy 0, policy_version 290 (0.0014) [2025-02-09 11:37:25,406][00884] Fps is (10 sec: 3686.4, 60 sec: 3959.5, 300 sec: 3887.7). Total num frames: 1191936. Throughput: 0: 961.2. Samples: 295310. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2025-02-09 11:37:25,411][00884] Avg episode reward: [(0, '5.607')] [2025-02-09 11:37:30,406][00884] Fps is (10 sec: 4915.3, 60 sec: 4096.0, 300 sec: 3901.6). Total num frames: 1216512. Throughput: 0: 1007.2. Samples: 302442. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2025-02-09 11:37:30,407][00884] Avg episode reward: [(0, '5.450')] [2025-02-09 11:37:34,207][04154] Updated weights for policy 0, policy_version 300 (0.0032) [2025-02-09 11:37:35,406][00884] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3873.8). Total num frames: 1228800. Throughput: 0: 997.0. Samples: 307810. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2025-02-09 11:37:35,411][00884] Avg episode reward: [(0, '5.517')] [2025-02-09 11:37:35,421][04141] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000300_1228800.pth... [2025-02-09 11:37:35,607][04141] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000072_294912.pth [2025-02-09 11:37:40,406][00884] Fps is (10 sec: 3276.8, 60 sec: 3891.2, 300 sec: 3887.7). Total num frames: 1249280. Throughput: 0: 966.8. Samples: 309968. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2025-02-09 11:37:40,411][00884] Avg episode reward: [(0, '6.047')] [2025-02-09 11:37:40,415][04141] Saving new best policy, reward=6.047! [2025-02-09 11:37:45,407][00884] Fps is (10 sec: 3276.3, 60 sec: 3822.9, 300 sec: 3859.9). Total num frames: 1261568. Throughput: 0: 953.6. Samples: 315292. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2025-02-09 11:37:45,410][00884] Avg episode reward: [(0, '6.053')] [2025-02-09 11:37:45,424][04141] Saving new best policy, reward=6.053! [2025-02-09 11:37:46,988][04154] Updated weights for policy 0, policy_version 310 (0.0013) [2025-02-09 11:37:50,406][00884] Fps is (10 sec: 2867.1, 60 sec: 3754.6, 300 sec: 3846.1). Total num frames: 1277952. Throughput: 0: 953.1. Samples: 320078. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2025-02-09 11:37:50,408][00884] Avg episode reward: [(0, '6.452')] [2025-02-09 11:37:50,415][04141] Saving new best policy, reward=6.452! [2025-02-09 11:37:55,406][00884] Fps is (10 sec: 3277.1, 60 sec: 3618.1, 300 sec: 3846.1). Total num frames: 1294336. Throughput: 0: 917.9. Samples: 322078. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2025-02-09 11:37:55,414][00884] Avg episode reward: [(0, '6.484')] [2025-02-09 11:37:55,419][04141] Saving new best policy, reward=6.484! [2025-02-09 11:37:58,686][04154] Updated weights for policy 0, policy_version 320 (0.0017) [2025-02-09 11:38:00,406][00884] Fps is (10 sec: 3686.5, 60 sec: 3754.7, 300 sec: 3846.1). Total num frames: 1314816. Throughput: 0: 885.4. Samples: 327824. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2025-02-09 11:38:00,407][00884] Avg episode reward: [(0, '6.589')] [2025-02-09 11:38:00,494][04141] Saving new best policy, reward=6.589! [2025-02-09 11:38:05,406][00884] Fps is (10 sec: 4505.8, 60 sec: 3823.1, 300 sec: 3860.0). Total num frames: 1339392. Throughput: 0: 945.9. Samples: 335008. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2025-02-09 11:38:05,411][00884] Avg episode reward: [(0, '6.653')] [2025-02-09 11:38:05,421][04141] Saving new best policy, reward=6.653! [2025-02-09 11:38:08,963][04154] Updated weights for policy 0, policy_version 330 (0.0016) [2025-02-09 11:38:10,406][00884] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3846.1). Total num frames: 1355776. Throughput: 0: 933.5. Samples: 337316. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2025-02-09 11:38:10,408][00884] Avg episode reward: [(0, '6.918')] [2025-02-09 11:38:10,410][04141] Saving new best policy, reward=6.918! [2025-02-09 11:38:15,406][00884] Fps is (10 sec: 3686.4, 60 sec: 3686.4, 300 sec: 3860.0). Total num frames: 1376256. Throughput: 0: 888.9. Samples: 342444. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2025-02-09 11:38:15,407][00884] Avg episode reward: [(0, '7.531')] [2025-02-09 11:38:15,419][04141] Saving new best policy, reward=7.531! [2025-02-09 11:38:18,724][04154] Updated weights for policy 0, policy_version 340 (0.0019) [2025-02-09 11:38:20,406][00884] Fps is (10 sec: 4505.6, 60 sec: 3891.2, 300 sec: 3873.8). Total num frames: 1400832. Throughput: 0: 930.7. Samples: 349692. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:38:20,408][00884] Avg episode reward: [(0, '8.283')] [2025-02-09 11:38:20,412][04141] Saving new best policy, reward=8.283! [2025-02-09 11:38:25,406][00884] Fps is (10 sec: 4096.0, 60 sec: 3754.7, 300 sec: 3846.1). Total num frames: 1417216. Throughput: 0: 957.7. Samples: 353064. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2025-02-09 11:38:25,413][00884] Avg episode reward: [(0, '8.415')] [2025-02-09 11:38:25,422][04141] Saving new best policy, reward=8.415! [2025-02-09 11:38:30,309][04154] Updated weights for policy 0, policy_version 350 (0.0012) [2025-02-09 11:38:30,406][00884] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3860.0). Total num frames: 1433600. Throughput: 0: 929.0. Samples: 357096. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:38:30,409][00884] Avg episode reward: [(0, '8.751')] [2025-02-09 11:38:30,412][04141] Saving new best policy, reward=8.751! [2025-02-09 11:38:35,406][00884] Fps is (10 sec: 3686.4, 60 sec: 3754.7, 300 sec: 3860.0). Total num frames: 1454080. Throughput: 0: 973.4. Samples: 363880. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:38:35,408][00884] Avg episode reward: [(0, '9.374')] [2025-02-09 11:38:35,433][04141] Saving new best policy, reward=9.374! [2025-02-09 11:38:39,005][04154] Updated weights for policy 0, policy_version 360 (0.0023) [2025-02-09 11:38:40,406][00884] Fps is (10 sec: 4505.5, 60 sec: 3822.9, 300 sec: 3860.0). Total num frames: 1478656. Throughput: 0: 1007.7. Samples: 367424. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:38:40,408][00884] Avg episode reward: [(0, '9.367')] [2025-02-09 11:38:45,406][00884] Fps is (10 sec: 3686.4, 60 sec: 3823.0, 300 sec: 3846.1). Total num frames: 1490944. Throughput: 0: 997.6. Samples: 372714. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2025-02-09 11:38:45,411][00884] Avg episode reward: [(0, '9.179')] [2025-02-09 11:38:49,987][04154] Updated weights for policy 0, policy_version 370 (0.0034) [2025-02-09 11:38:50,406][00884] Fps is (10 sec: 3686.5, 60 sec: 3959.5, 300 sec: 3873.8). Total num frames: 1515520. Throughput: 0: 972.5. Samples: 378770. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2025-02-09 11:38:50,411][00884] Avg episode reward: [(0, '8.766')] [2025-02-09 11:38:55,406][00884] Fps is (10 sec: 4915.2, 60 sec: 4096.0, 300 sec: 3873.8). Total num frames: 1540096. Throughput: 0: 1002.8. Samples: 382444. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:38:55,412][00884] Avg episode reward: [(0, '9.433')] [2025-02-09 11:38:55,421][04141] Saving new best policy, reward=9.433! [2025-02-09 11:39:00,049][04154] Updated weights for policy 0, policy_version 380 (0.0018) [2025-02-09 11:39:00,406][00884] Fps is (10 sec: 4096.0, 60 sec: 4027.7, 300 sec: 3860.0). Total num frames: 1556480. Throughput: 0: 1024.7. Samples: 388554. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:39:00,408][00884] Avg episode reward: [(0, '9.942')] [2025-02-09 11:39:00,411][04141] Saving new best policy, reward=9.942! [2025-02-09 11:39:05,406][00884] Fps is (10 sec: 3276.8, 60 sec: 3891.2, 300 sec: 3873.8). Total num frames: 1572864. Throughput: 0: 969.8. Samples: 393334. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:39:05,412][00884] Avg episode reward: [(0, '10.291')] [2025-02-09 11:39:05,420][04141] Saving new best policy, reward=10.291! [2025-02-09 11:39:09,764][04154] Updated weights for policy 0, policy_version 390 (0.0018) [2025-02-09 11:39:10,406][00884] Fps is (10 sec: 4096.0, 60 sec: 4027.7, 300 sec: 3873.8). Total num frames: 1597440. Throughput: 0: 974.2. Samples: 396904. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2025-02-09 11:39:10,408][00884] Avg episode reward: [(0, '11.147')] [2025-02-09 11:39:10,417][04141] Saving new best policy, reward=11.147! [2025-02-09 11:39:15,406][00884] Fps is (10 sec: 4505.6, 60 sec: 4027.7, 300 sec: 3860.0). Total num frames: 1617920. Throughput: 0: 1046.2. Samples: 404176. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2025-02-09 11:39:15,412][00884] Avg episode reward: [(0, '11.170')] [2025-02-09 11:39:15,421][04141] Saving new best policy, reward=11.170! [2025-02-09 11:39:20,406][00884] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3873.9). Total num frames: 1634304. Throughput: 0: 994.6. Samples: 408636. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2025-02-09 11:39:20,411][00884] Avg episode reward: [(0, '11.555')] [2025-02-09 11:39:20,413][04141] Saving new best policy, reward=11.555! [2025-02-09 11:39:21,181][04154] Updated weights for policy 0, policy_version 400 (0.0024) [2025-02-09 11:39:25,406][00884] Fps is (10 sec: 3686.4, 60 sec: 3959.5, 300 sec: 3873.8). Total num frames: 1654784. Throughput: 0: 984.6. Samples: 411732. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:39:25,411][00884] Avg episode reward: [(0, '12.164')] [2025-02-09 11:39:25,420][04141] Saving new best policy, reward=12.164! [2025-02-09 11:39:29,910][04154] Updated weights for policy 0, policy_version 410 (0.0022) [2025-02-09 11:39:30,406][00884] Fps is (10 sec: 4505.6, 60 sec: 4096.0, 300 sec: 3887.7). Total num frames: 1679360. Throughput: 0: 1020.0. Samples: 418614. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:39:30,408][00884] Avg episode reward: [(0, '11.670')] [2025-02-09 11:39:35,406][00884] Fps is (10 sec: 4096.0, 60 sec: 4027.7, 300 sec: 3873.9). Total num frames: 1695744. Throughput: 0: 1003.1. Samples: 423910. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2025-02-09 11:39:35,408][00884] Avg episode reward: [(0, '12.379')] [2025-02-09 11:39:35,417][04141] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000414_1695744.pth... [2025-02-09 11:39:35,592][04141] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000184_753664.pth [2025-02-09 11:39:35,608][04141] Saving new best policy, reward=12.379! [2025-02-09 11:39:40,406][00884] Fps is (10 sec: 3276.8, 60 sec: 3891.2, 300 sec: 3887.7). Total num frames: 1712128. Throughput: 0: 965.2. Samples: 425878. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2025-02-09 11:39:40,410][00884] Avg episode reward: [(0, '11.808')] [2025-02-09 11:39:41,717][04154] Updated weights for policy 0, policy_version 420 (0.0025) [2025-02-09 11:39:45,406][00884] Fps is (10 sec: 4096.0, 60 sec: 4096.0, 300 sec: 3929.5). Total num frames: 1736704. Throughput: 0: 980.6. Samples: 432682. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2025-02-09 11:39:45,411][00884] Avg episode reward: [(0, '13.465')] [2025-02-09 11:39:45,420][04141] Saving new best policy, reward=13.465! [2025-02-09 11:39:50,406][00884] Fps is (10 sec: 4096.0, 60 sec: 3959.5, 300 sec: 3915.5). Total num frames: 1753088. Throughput: 0: 1011.9. Samples: 438870. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2025-02-09 11:39:50,412][00884] Avg episode reward: [(0, '13.201')] [2025-02-09 11:39:52,370][04154] Updated weights for policy 0, policy_version 430 (0.0013) [2025-02-09 11:39:55,406][00884] Fps is (10 sec: 3276.8, 60 sec: 3822.9, 300 sec: 3915.5). Total num frames: 1769472. Throughput: 0: 977.8. Samples: 440904. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2025-02-09 11:39:55,410][00884] Avg episode reward: [(0, '13.878')] [2025-02-09 11:39:55,416][04141] Saving new best policy, reward=13.878! [2025-02-09 11:40:00,406][00884] Fps is (10 sec: 4095.9, 60 sec: 3959.5, 300 sec: 3929.4). Total num frames: 1794048. Throughput: 0: 949.7. Samples: 446912. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2025-02-09 11:40:00,409][00884] Avg episode reward: [(0, '14.480')] [2025-02-09 11:40:00,410][04141] Saving new best policy, reward=14.480! [2025-02-09 11:40:02,174][04154] Updated weights for policy 0, policy_version 440 (0.0016) [2025-02-09 11:40:05,406][00884] Fps is (10 sec: 4505.6, 60 sec: 4027.7, 300 sec: 3915.5). Total num frames: 1814528. Throughput: 0: 1006.0. Samples: 453904. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:40:05,410][00884] Avg episode reward: [(0, '13.610')] [2025-02-09 11:40:10,407][00884] Fps is (10 sec: 3686.0, 60 sec: 3891.1, 300 sec: 3915.5). Total num frames: 1830912. Throughput: 0: 987.8. Samples: 456186. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2025-02-09 11:40:10,410][00884] Avg episode reward: [(0, '13.350')] [2025-02-09 11:40:13,341][04154] Updated weights for policy 0, policy_version 450 (0.0024) [2025-02-09 11:40:15,406][00884] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3929.4). Total num frames: 1851392. Throughput: 0: 953.7. Samples: 461532. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:40:15,412][00884] Avg episode reward: [(0, '12.926')] [2025-02-09 11:40:20,406][00884] Fps is (10 sec: 4506.2, 60 sec: 4027.7, 300 sec: 3929.4). Total num frames: 1875968. Throughput: 0: 998.0. Samples: 468822. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2025-02-09 11:40:20,411][00884] Avg episode reward: [(0, '12.584')] [2025-02-09 11:40:21,581][04154] Updated weights for policy 0, policy_version 460 (0.0024) [2025-02-09 11:40:25,406][00884] Fps is (10 sec: 4095.9, 60 sec: 3959.5, 300 sec: 3901.6). Total num frames: 1892352. Throughput: 0: 1028.9. Samples: 472180. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2025-02-09 11:40:25,410][00884] Avg episode reward: [(0, '13.401')] [2025-02-09 11:40:30,406][00884] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3929.4). Total num frames: 1912832. Throughput: 0: 976.9. Samples: 476644. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2025-02-09 11:40:30,408][00884] Avg episode reward: [(0, '14.406')] [2025-02-09 11:40:32,870][04154] Updated weights for policy 0, policy_version 470 (0.0034) [2025-02-09 11:40:35,406][00884] Fps is (10 sec: 4505.7, 60 sec: 4027.7, 300 sec: 3943.3). Total num frames: 1937408. Throughput: 0: 999.1. Samples: 483830. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2025-02-09 11:40:35,408][00884] Avg episode reward: [(0, '15.627')] [2025-02-09 11:40:35,414][04141] Saving new best policy, reward=15.627! [2025-02-09 11:40:40,407][00884] Fps is (10 sec: 4095.4, 60 sec: 4027.6, 300 sec: 3915.5). Total num frames: 1953792. Throughput: 0: 1034.6. Samples: 487462. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:40:40,415][00884] Avg episode reward: [(0, '15.338')] [2025-02-09 11:40:44,696][04154] Updated weights for policy 0, policy_version 480 (0.0023) [2025-02-09 11:40:45,406][00884] Fps is (10 sec: 2867.2, 60 sec: 3822.9, 300 sec: 3901.6). Total num frames: 1966080. Throughput: 0: 987.2. Samples: 491334. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2025-02-09 11:40:45,411][00884] Avg episode reward: [(0, '14.682')] [2025-02-09 11:40:50,406][00884] Fps is (10 sec: 2867.6, 60 sec: 3822.9, 300 sec: 3901.6). Total num frames: 1982464. Throughput: 0: 924.2. Samples: 495492. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2025-02-09 11:40:50,407][00884] Avg episode reward: [(0, '13.662')] [2025-02-09 11:40:55,073][04154] Updated weights for policy 0, policy_version 490 (0.0029) [2025-02-09 11:40:55,406][00884] Fps is (10 sec: 4096.0, 60 sec: 3959.5, 300 sec: 3901.6). Total num frames: 2007040. Throughput: 0: 954.3. Samples: 499130. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2025-02-09 11:40:55,408][00884] Avg episode reward: [(0, '14.055')] [2025-02-09 11:41:00,408][00884] Fps is (10 sec: 4504.5, 60 sec: 3891.0, 300 sec: 3901.6). Total num frames: 2027520. Throughput: 0: 996.7. Samples: 506386. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2025-02-09 11:41:00,413][00884] Avg episode reward: [(0, '14.614')] [2025-02-09 11:41:05,409][00884] Fps is (10 sec: 3685.1, 60 sec: 3822.7, 300 sec: 3901.6). Total num frames: 2043904. Throughput: 0: 936.8. Samples: 510982. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:41:05,412][00884] Avg episode reward: [(0, '15.720')] [2025-02-09 11:41:05,420][04141] Saving new best policy, reward=15.720! [2025-02-09 11:41:06,356][04154] Updated weights for policy 0, policy_version 500 (0.0019) [2025-02-09 11:41:10,406][00884] Fps is (10 sec: 3687.4, 60 sec: 3891.3, 300 sec: 3901.6). Total num frames: 2064384. Throughput: 0: 928.1. Samples: 513946. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:41:10,414][00884] Avg episode reward: [(0, '16.922')] [2025-02-09 11:41:10,417][04141] Saving new best policy, reward=16.922! [2025-02-09 11:41:14,728][04154] Updated weights for policy 0, policy_version 510 (0.0018) [2025-02-09 11:41:15,406][00884] Fps is (10 sec: 4507.1, 60 sec: 3959.5, 300 sec: 3901.6). Total num frames: 2088960. Throughput: 0: 989.9. Samples: 521190. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:41:15,408][00884] Avg episode reward: [(0, '16.748')] [2025-02-09 11:41:20,406][00884] Fps is (10 sec: 4096.0, 60 sec: 3822.9, 300 sec: 3901.6). Total num frames: 2105344. Throughput: 0: 955.0. Samples: 526806. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2025-02-09 11:41:20,412][00884] Avg episode reward: [(0, '17.085')] [2025-02-09 11:41:20,416][04141] Saving new best policy, reward=17.085! [2025-02-09 11:41:25,406][00884] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3915.5). Total num frames: 2125824. Throughput: 0: 922.7. Samples: 528982. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2025-02-09 11:41:25,411][00884] Avg episode reward: [(0, '17.344')] [2025-02-09 11:41:25,420][04141] Saving new best policy, reward=17.344! [2025-02-09 11:41:26,077][04154] Updated weights for policy 0, policy_version 520 (0.0024) [2025-02-09 11:41:30,406][00884] Fps is (10 sec: 4096.0, 60 sec: 3891.2, 300 sec: 3901.6). Total num frames: 2146304. Throughput: 0: 984.9. Samples: 535656. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2025-02-09 11:41:30,408][00884] Avg episode reward: [(0, '16.296')] [2025-02-09 11:41:35,409][00884] Fps is (10 sec: 4094.7, 60 sec: 3822.7, 300 sec: 3901.6). Total num frames: 2166784. Throughput: 0: 1033.3. Samples: 541994. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2025-02-09 11:41:35,413][00884] Avg episode reward: [(0, '17.341')] [2025-02-09 11:41:35,421][04141] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000529_2166784.pth... [2025-02-09 11:41:35,603][04141] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000300_1228800.pth [2025-02-09 11:41:36,540][04154] Updated weights for policy 0, policy_version 530 (0.0021) [2025-02-09 11:41:40,406][00884] Fps is (10 sec: 3276.8, 60 sec: 3754.8, 300 sec: 3887.7). Total num frames: 2179072. Throughput: 0: 994.8. Samples: 543894. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:41:40,408][00884] Avg episode reward: [(0, '16.382')] [2025-02-09 11:41:45,406][00884] Fps is (10 sec: 3687.6, 60 sec: 3959.5, 300 sec: 3901.6). Total num frames: 2203648. Throughput: 0: 960.1. Samples: 549588. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2025-02-09 11:41:45,408][00884] Avg episode reward: [(0, '15.424')] [2025-02-09 11:41:46,609][04154] Updated weights for policy 0, policy_version 540 (0.0026) [2025-02-09 11:41:50,409][00884] Fps is (10 sec: 4913.5, 60 sec: 4095.8, 300 sec: 3901.6). Total num frames: 2228224. Throughput: 0: 1017.8. Samples: 556784. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2025-02-09 11:41:50,413][00884] Avg episode reward: [(0, '16.027')] [2025-02-09 11:41:55,407][00884] Fps is (10 sec: 3685.8, 60 sec: 3891.1, 300 sec: 3901.6). Total num frames: 2240512. Throughput: 0: 1007.6. Samples: 559290. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2025-02-09 11:41:55,417][00884] Avg episode reward: [(0, '15.870')] [2025-02-09 11:41:58,284][04154] Updated weights for policy 0, policy_version 550 (0.0027) [2025-02-09 11:42:00,406][00884] Fps is (10 sec: 3277.9, 60 sec: 3891.4, 300 sec: 3901.6). Total num frames: 2260992. Throughput: 0: 952.9. Samples: 564072. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2025-02-09 11:42:00,409][00884] Avg episode reward: [(0, '16.414')] [2025-02-09 11:42:05,406][00884] Fps is (10 sec: 4506.3, 60 sec: 4028.0, 300 sec: 3901.6). Total num frames: 2285568. Throughput: 0: 987.1. Samples: 571224. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:42:05,407][00884] Avg episode reward: [(0, '17.095')] [2025-02-09 11:42:06,886][04154] Updated weights for policy 0, policy_version 560 (0.0012) [2025-02-09 11:42:10,406][00884] Fps is (10 sec: 4096.0, 60 sec: 3959.5, 300 sec: 3887.7). Total num frames: 2301952. Throughput: 0: 1019.0. Samples: 574838. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:42:10,410][00884] Avg episode reward: [(0, '17.638')] [2025-02-09 11:42:10,413][04141] Saving new best policy, reward=17.638! [2025-02-09 11:42:15,406][00884] Fps is (10 sec: 3276.8, 60 sec: 3822.9, 300 sec: 3901.6). Total num frames: 2318336. Throughput: 0: 963.7. Samples: 579022. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2025-02-09 11:42:15,411][00884] Avg episode reward: [(0, '16.969')] [2025-02-09 11:42:18,469][04154] Updated weights for policy 0, policy_version 570 (0.0019) [2025-02-09 11:42:20,406][00884] Fps is (10 sec: 4096.0, 60 sec: 3959.5, 300 sec: 3901.6). Total num frames: 2342912. Throughput: 0: 962.1. Samples: 585286. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2025-02-09 11:42:20,408][00884] Avg episode reward: [(0, '16.429')] [2025-02-09 11:42:25,406][00884] Fps is (10 sec: 4505.6, 60 sec: 3959.5, 300 sec: 3887.7). Total num frames: 2363392. Throughput: 0: 997.1. Samples: 588764. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:42:25,411][00884] Avg episode reward: [(0, '16.320')] [2025-02-09 11:42:29,105][04154] Updated weights for policy 0, policy_version 580 (0.0026) [2025-02-09 11:42:30,406][00884] Fps is (10 sec: 3276.8, 60 sec: 3822.9, 300 sec: 3887.7). Total num frames: 2375680. Throughput: 0: 985.2. Samples: 593924. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2025-02-09 11:42:30,410][00884] Avg episode reward: [(0, '15.581')] [2025-02-09 11:42:35,406][00884] Fps is (10 sec: 3276.8, 60 sec: 3823.1, 300 sec: 3887.7). Total num frames: 2396160. Throughput: 0: 942.5. Samples: 599192. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2025-02-09 11:42:35,408][00884] Avg episode reward: [(0, '16.744')] [2025-02-09 11:42:39,253][04154] Updated weights for policy 0, policy_version 590 (0.0017) [2025-02-09 11:42:40,406][00884] Fps is (10 sec: 4505.6, 60 sec: 4027.7, 300 sec: 3929.4). Total num frames: 2420736. Throughput: 0: 965.1. Samples: 602716. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2025-02-09 11:42:40,411][00884] Avg episode reward: [(0, '17.809')] [2025-02-09 11:42:40,413][04141] Saving new best policy, reward=17.809! [2025-02-09 11:42:45,406][00884] Fps is (10 sec: 4095.9, 60 sec: 3891.2, 300 sec: 3929.4). Total num frames: 2437120. Throughput: 0: 1003.0. Samples: 609206. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:42:45,408][00884] Avg episode reward: [(0, '18.886')] [2025-02-09 11:42:45,417][04141] Saving new best policy, reward=18.886! [2025-02-09 11:42:50,409][00884] Fps is (10 sec: 3275.7, 60 sec: 3754.7, 300 sec: 3929.3). Total num frames: 2453504. Throughput: 0: 946.2. Samples: 613804. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2025-02-09 11:42:50,411][00884] Avg episode reward: [(0, '18.007')] [2025-02-09 11:42:50,447][04154] Updated weights for policy 0, policy_version 600 (0.0016) [2025-02-09 11:42:55,406][00884] Fps is (10 sec: 4096.1, 60 sec: 3959.6, 300 sec: 3943.3). Total num frames: 2478080. Throughput: 0: 946.8. Samples: 617444. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2025-02-09 11:42:55,411][00884] Avg episode reward: [(0, '17.578')] [2025-02-09 11:42:58,989][04154] Updated weights for policy 0, policy_version 610 (0.0026) [2025-02-09 11:43:00,406][00884] Fps is (10 sec: 4916.9, 60 sec: 4027.7, 300 sec: 3943.3). Total num frames: 2502656. Throughput: 0: 1012.9. Samples: 624602. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2025-02-09 11:43:00,408][00884] Avg episode reward: [(0, '16.571')] [2025-02-09 11:43:05,406][00884] Fps is (10 sec: 3686.4, 60 sec: 3822.9, 300 sec: 3929.4). Total num frames: 2514944. Throughput: 0: 976.7. Samples: 629236. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:43:05,410][00884] Avg episode reward: [(0, '16.142')] [2025-02-09 11:43:10,120][04154] Updated weights for policy 0, policy_version 620 (0.0032) [2025-02-09 11:43:10,406][00884] Fps is (10 sec: 3686.4, 60 sec: 3959.5, 300 sec: 3943.3). Total num frames: 2539520. Throughput: 0: 966.2. Samples: 632242. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2025-02-09 11:43:10,412][00884] Avg episode reward: [(0, '15.632')] [2025-02-09 11:43:15,406][00884] Fps is (10 sec: 4915.2, 60 sec: 4096.0, 300 sec: 3943.3). Total num frames: 2564096. Throughput: 0: 1013.1. Samples: 639512. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:43:15,408][00884] Avg episode reward: [(0, '15.487')] [2025-02-09 11:43:20,031][04154] Updated weights for policy 0, policy_version 630 (0.0023) [2025-02-09 11:43:20,406][00884] Fps is (10 sec: 4096.0, 60 sec: 3959.5, 300 sec: 3943.3). Total num frames: 2580480. Throughput: 0: 1021.0. Samples: 645136. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2025-02-09 11:43:20,409][00884] Avg episode reward: [(0, '16.136')] [2025-02-09 11:43:25,406][00884] Fps is (10 sec: 3276.8, 60 sec: 3891.2, 300 sec: 3943.3). Total num frames: 2596864. Throughput: 0: 993.1. Samples: 647404. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2025-02-09 11:43:25,411][00884] Avg episode reward: [(0, '15.809')] [2025-02-09 11:43:29,872][04154] Updated weights for policy 0, policy_version 640 (0.0019) [2025-02-09 11:43:30,406][00884] Fps is (10 sec: 4096.0, 60 sec: 4096.0, 300 sec: 3957.2). Total num frames: 2621440. Throughput: 0: 1003.7. Samples: 654370. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2025-02-09 11:43:30,411][00884] Avg episode reward: [(0, '17.006')] [2025-02-09 11:43:35,408][00884] Fps is (10 sec: 4505.6, 60 sec: 4096.0, 300 sec: 3943.3). Total num frames: 2641920. Throughput: 0: 1050.6. Samples: 661076. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2025-02-09 11:43:35,410][00884] Avg episode reward: [(0, '16.559')] [2025-02-09 11:43:35,422][04141] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000645_2641920.pth... [2025-02-09 11:43:35,657][04141] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000414_1695744.pth [2025-02-09 11:43:40,409][00884] Fps is (10 sec: 3275.7, 60 sec: 3891.0, 300 sec: 3943.2). Total num frames: 2654208. Throughput: 0: 1008.6. Samples: 662836. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:43:40,415][00884] Avg episode reward: [(0, '17.074')] [2025-02-09 11:43:43,099][04154] Updated weights for policy 0, policy_version 650 (0.0023) [2025-02-09 11:43:45,406][00884] Fps is (10 sec: 2867.2, 60 sec: 3891.2, 300 sec: 3915.5). Total num frames: 2670592. Throughput: 0: 930.0. Samples: 666454. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:43:45,408][00884] Avg episode reward: [(0, '16.566')] [2025-02-09 11:43:50,406][00884] Fps is (10 sec: 4097.4, 60 sec: 4028.0, 300 sec: 3915.5). Total num frames: 2695168. Throughput: 0: 977.1. Samples: 673204. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2025-02-09 11:43:50,408][00884] Avg episode reward: [(0, '18.083')] [2025-02-09 11:43:52,045][04154] Updated weights for policy 0, policy_version 660 (0.0016) [2025-02-09 11:43:55,406][00884] Fps is (10 sec: 4096.1, 60 sec: 3891.2, 300 sec: 3915.5). Total num frames: 2711552. Throughput: 0: 988.7. Samples: 676734. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2025-02-09 11:43:55,407][00884] Avg episode reward: [(0, '17.200')] [2025-02-09 11:44:00,406][00884] Fps is (10 sec: 3276.8, 60 sec: 3754.7, 300 sec: 3915.5). Total num frames: 2727936. Throughput: 0: 927.4. Samples: 681246. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2025-02-09 11:44:00,408][00884] Avg episode reward: [(0, '18.433')] [2025-02-09 11:44:03,625][04154] Updated weights for policy 0, policy_version 670 (0.0020) [2025-02-09 11:44:05,406][00884] Fps is (10 sec: 4096.0, 60 sec: 3959.5, 300 sec: 3915.5). Total num frames: 2752512. Throughput: 0: 946.9. Samples: 687748. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:44:05,408][00884] Avg episode reward: [(0, '18.723')] [2025-02-09 11:44:10,406][00884] Fps is (10 sec: 4505.6, 60 sec: 3891.2, 300 sec: 3915.5). Total num frames: 2772992. Throughput: 0: 979.1. Samples: 691464. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2025-02-09 11:44:10,408][00884] Avg episode reward: [(0, '18.463')] [2025-02-09 11:44:13,241][04154] Updated weights for policy 0, policy_version 680 (0.0021) [2025-02-09 11:44:15,406][00884] Fps is (10 sec: 3686.4, 60 sec: 3754.7, 300 sec: 3915.5). Total num frames: 2789376. Throughput: 0: 947.9. Samples: 697026. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:44:15,408][00884] Avg episode reward: [(0, '19.617')] [2025-02-09 11:44:15,418][04141] Saving new best policy, reward=19.617! [2025-02-09 11:44:20,406][00884] Fps is (10 sec: 3686.4, 60 sec: 3822.9, 300 sec: 3915.5). Total num frames: 2809856. Throughput: 0: 924.0. Samples: 702656. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2025-02-09 11:44:20,410][00884] Avg episode reward: [(0, '19.049')] [2025-02-09 11:44:23,063][04154] Updated weights for policy 0, policy_version 690 (0.0018) [2025-02-09 11:44:25,406][00884] Fps is (10 sec: 4505.6, 60 sec: 3959.5, 300 sec: 3915.5). Total num frames: 2834432. Throughput: 0: 967.2. Samples: 706356. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2025-02-09 11:44:25,411][00884] Avg episode reward: [(0, '20.779')] [2025-02-09 11:44:25,431][04141] Saving new best policy, reward=20.779! [2025-02-09 11:44:30,406][00884] Fps is (10 sec: 4096.0, 60 sec: 3822.9, 300 sec: 3915.5). Total num frames: 2850816. Throughput: 0: 1028.5. Samples: 712738. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:44:30,409][00884] Avg episode reward: [(0, '20.959')] [2025-02-09 11:44:30,411][04141] Saving new best policy, reward=20.959! [2025-02-09 11:44:34,611][04154] Updated weights for policy 0, policy_version 700 (0.0016) [2025-02-09 11:44:35,409][00884] Fps is (10 sec: 3275.7, 60 sec: 3754.5, 300 sec: 3915.5). Total num frames: 2867200. Throughput: 0: 978.1. Samples: 717224. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:44:35,412][00884] Avg episode reward: [(0, '21.422')] [2025-02-09 11:44:35,443][04141] Saving new best policy, reward=21.422! [2025-02-09 11:44:40,406][00884] Fps is (10 sec: 4096.0, 60 sec: 3959.7, 300 sec: 3915.5). Total num frames: 2891776. Throughput: 0: 978.4. Samples: 720762. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2025-02-09 11:44:40,407][00884] Avg episode reward: [(0, '21.063')] [2025-02-09 11:44:43,148][04154] Updated weights for policy 0, policy_version 710 (0.0018) [2025-02-09 11:44:45,406][00884] Fps is (10 sec: 4916.8, 60 sec: 4096.0, 300 sec: 3943.3). Total num frames: 2916352. Throughput: 0: 1039.8. Samples: 728038. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:44:45,414][00884] Avg episode reward: [(0, '20.753')] [2025-02-09 11:44:50,406][00884] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3929.4). Total num frames: 2928640. Throughput: 0: 1000.9. Samples: 732790. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:44:50,409][00884] Avg episode reward: [(0, '19.070')] [2025-02-09 11:44:54,235][04154] Updated weights for policy 0, policy_version 720 (0.0024) [2025-02-09 11:44:55,407][00884] Fps is (10 sec: 3685.8, 60 sec: 4027.6, 300 sec: 3929.4). Total num frames: 2953216. Throughput: 0: 982.9. Samples: 735696. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:44:55,412][00884] Avg episode reward: [(0, '19.648')] [2025-02-09 11:45:00,406][00884] Fps is (10 sec: 4915.2, 60 sec: 4164.3, 300 sec: 3943.3). Total num frames: 2977792. Throughput: 0: 1022.1. Samples: 743022. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2025-02-09 11:45:00,407][00884] Avg episode reward: [(0, '19.143')] [2025-02-09 11:45:03,660][04154] Updated weights for policy 0, policy_version 730 (0.0028) [2025-02-09 11:45:05,406][00884] Fps is (10 sec: 4096.7, 60 sec: 4027.7, 300 sec: 3943.3). Total num frames: 2994176. Throughput: 0: 1022.6. Samples: 748674. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:45:05,408][00884] Avg episode reward: [(0, '19.208')] [2025-02-09 11:45:10,406][00884] Fps is (10 sec: 3276.8, 60 sec: 3959.5, 300 sec: 3929.4). Total num frames: 3010560. Throughput: 0: 990.6. Samples: 750934. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2025-02-09 11:45:10,408][00884] Avg episode reward: [(0, '21.984')] [2025-02-09 11:45:10,413][04141] Saving new best policy, reward=21.984! [2025-02-09 11:45:14,037][04154] Updated weights for policy 0, policy_version 740 (0.0018) [2025-02-09 11:45:15,406][00884] Fps is (10 sec: 4096.0, 60 sec: 4096.0, 300 sec: 3929.4). Total num frames: 3035136. Throughput: 0: 1000.9. Samples: 757778. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:45:15,411][00884] Avg episode reward: [(0, '20.243')] [2025-02-09 11:45:20,408][00884] Fps is (10 sec: 4504.5, 60 sec: 4095.8, 300 sec: 3943.2). Total num frames: 3055616. Throughput: 0: 1052.4. Samples: 764582. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2025-02-09 11:45:20,413][00884] Avg episode reward: [(0, '20.877')] [2025-02-09 11:45:24,634][04154] Updated weights for policy 0, policy_version 750 (0.0015) [2025-02-09 11:45:25,406][00884] Fps is (10 sec: 3686.4, 60 sec: 3959.5, 300 sec: 3929.4). Total num frames: 3072000. Throughput: 0: 1022.0. Samples: 766754. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2025-02-09 11:45:25,411][00884] Avg episode reward: [(0, '21.507')] [2025-02-09 11:45:30,406][00884] Fps is (10 sec: 4097.0, 60 sec: 4096.0, 300 sec: 3929.4). Total num frames: 3096576. Throughput: 0: 996.3. Samples: 772872. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2025-02-09 11:45:30,411][00884] Avg episode reward: [(0, '22.998')] [2025-02-09 11:45:30,414][04141] Saving new best policy, reward=22.998! [2025-02-09 11:45:33,418][04154] Updated weights for policy 0, policy_version 760 (0.0014) [2025-02-09 11:45:35,406][00884] Fps is (10 sec: 4915.2, 60 sec: 4232.8, 300 sec: 3957.2). Total num frames: 3121152. Throughput: 0: 1049.0. Samples: 779994. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2025-02-09 11:45:35,408][00884] Avg episode reward: [(0, '21.397')] [2025-02-09 11:45:35,415][04141] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000762_3121152.pth... [2025-02-09 11:45:35,573][04141] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000529_2166784.pth [2025-02-09 11:45:40,406][00884] Fps is (10 sec: 3686.4, 60 sec: 4027.7, 300 sec: 3957.2). Total num frames: 3133440. Throughput: 0: 1041.8. Samples: 782576. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2025-02-09 11:45:40,412][00884] Avg episode reward: [(0, '22.386')] [2025-02-09 11:45:44,430][04154] Updated weights for policy 0, policy_version 770 (0.0012) [2025-02-09 11:45:45,406][00884] Fps is (10 sec: 3686.4, 60 sec: 4027.7, 300 sec: 3984.9). Total num frames: 3158016. Throughput: 0: 995.1. Samples: 787800. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2025-02-09 11:45:45,411][00884] Avg episode reward: [(0, '23.073')] [2025-02-09 11:45:45,416][04141] Saving new best policy, reward=23.073! [2025-02-09 11:45:50,406][00884] Fps is (10 sec: 4915.2, 60 sec: 4232.5, 300 sec: 3984.9). Total num frames: 3182592. Throughput: 0: 1030.1. Samples: 795028. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2025-02-09 11:45:50,412][00884] Avg episode reward: [(0, '21.458')] [2025-02-09 11:45:53,470][04154] Updated weights for policy 0, policy_version 780 (0.0018) [2025-02-09 11:45:55,406][00884] Fps is (10 sec: 4096.0, 60 sec: 4096.1, 300 sec: 3971.1). Total num frames: 3198976. Throughput: 0: 1058.2. Samples: 798554. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:45:55,408][00884] Avg episode reward: [(0, '20.110')] [2025-02-09 11:46:00,406][00884] Fps is (10 sec: 3276.8, 60 sec: 3959.5, 300 sec: 3971.1). Total num frames: 3215360. Throughput: 0: 1007.6. Samples: 803118. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:46:00,408][00884] Avg episode reward: [(0, '20.181')] [2025-02-09 11:46:04,105][04154] Updated weights for policy 0, policy_version 790 (0.0033) [2025-02-09 11:46:05,406][00884] Fps is (10 sec: 4096.0, 60 sec: 4096.0, 300 sec: 3984.9). Total num frames: 3239936. Throughput: 0: 1009.7. Samples: 810018. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:46:05,413][00884] Avg episode reward: [(0, '20.372')] [2025-02-09 11:46:10,406][00884] Fps is (10 sec: 4505.6, 60 sec: 4164.3, 300 sec: 3971.0). Total num frames: 3260416. Throughput: 0: 1042.7. Samples: 813674. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:46:10,411][00884] Avg episode reward: [(0, '19.383')] [2025-02-09 11:46:14,557][04154] Updated weights for policy 0, policy_version 800 (0.0015) [2025-02-09 11:46:15,407][00884] Fps is (10 sec: 3685.9, 60 sec: 4027.6, 300 sec: 3971.0). Total num frames: 3276800. Throughput: 0: 1026.0. Samples: 819044. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:46:15,413][00884] Avg episode reward: [(0, '20.147')] [2025-02-09 11:46:20,406][00884] Fps is (10 sec: 4096.0, 60 sec: 4096.2, 300 sec: 3984.9). Total num frames: 3301376. Throughput: 0: 1002.3. Samples: 825096. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:46:20,412][00884] Avg episode reward: [(0, '21.575')] [2025-02-09 11:46:23,536][04154] Updated weights for policy 0, policy_version 810 (0.0016) [2025-02-09 11:46:25,406][00884] Fps is (10 sec: 4915.9, 60 sec: 4232.5, 300 sec: 3998.8). Total num frames: 3325952. Throughput: 0: 1027.4. Samples: 828810. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2025-02-09 11:46:25,412][00884] Avg episode reward: [(0, '23.413')] [2025-02-09 11:46:25,420][04141] Saving new best policy, reward=23.413! [2025-02-09 11:46:30,406][00884] Fps is (10 sec: 4096.0, 60 sec: 4096.0, 300 sec: 3985.0). Total num frames: 3342336. Throughput: 0: 1049.2. Samples: 835014. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:46:30,408][00884] Avg episode reward: [(0, '22.498')] [2025-02-09 11:46:35,407][00884] Fps is (10 sec: 2866.9, 60 sec: 3891.1, 300 sec: 3984.9). Total num frames: 3354624. Throughput: 0: 987.0. Samples: 839446. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:46:35,412][00884] Avg episode reward: [(0, '22.988')] [2025-02-09 11:46:35,694][04154] Updated weights for policy 0, policy_version 820 (0.0019) [2025-02-09 11:46:40,406][00884] Fps is (10 sec: 2867.2, 60 sec: 3959.5, 300 sec: 3957.2). Total num frames: 3371008. Throughput: 0: 957.3. Samples: 841634. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:46:40,409][00884] Avg episode reward: [(0, '22.109')] [2025-02-09 11:46:45,406][00884] Fps is (10 sec: 3686.8, 60 sec: 3891.2, 300 sec: 3943.3). Total num frames: 3391488. Throughput: 0: 984.0. Samples: 847400. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:46:45,408][00884] Avg episode reward: [(0, '21.118')] [2025-02-09 11:46:47,387][04154] Updated weights for policy 0, policy_version 830 (0.0023) [2025-02-09 11:46:50,406][00884] Fps is (10 sec: 3686.3, 60 sec: 3754.7, 300 sec: 3957.2). Total num frames: 3407872. Throughput: 0: 931.4. Samples: 851930. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2025-02-09 11:46:50,413][00884] Avg episode reward: [(0, '19.751')] [2025-02-09 11:46:55,406][00884] Fps is (10 sec: 3686.4, 60 sec: 3822.9, 300 sec: 3957.2). Total num frames: 3428352. Throughput: 0: 921.5. Samples: 855140. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:46:55,416][00884] Avg episode reward: [(0, '19.712')] [2025-02-09 11:46:57,105][04154] Updated weights for policy 0, policy_version 840 (0.0037) [2025-02-09 11:47:00,409][00884] Fps is (10 sec: 4913.8, 60 sec: 4027.5, 300 sec: 3971.0). Total num frames: 3457024. Throughput: 0: 965.5. Samples: 862492. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2025-02-09 11:47:00,410][00884] Avg episode reward: [(0, '20.692')] [2025-02-09 11:47:05,406][00884] Fps is (10 sec: 4096.0, 60 sec: 3822.9, 300 sec: 3957.2). Total num frames: 3469312. Throughput: 0: 947.4. Samples: 867730. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2025-02-09 11:47:05,410][00884] Avg episode reward: [(0, '21.404')] [2025-02-09 11:47:08,283][04154] Updated weights for policy 0, policy_version 850 (0.0036) [2025-02-09 11:47:10,406][00884] Fps is (10 sec: 3277.8, 60 sec: 3822.9, 300 sec: 3971.0). Total num frames: 3489792. Throughput: 0: 913.6. Samples: 869924. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2025-02-09 11:47:10,407][00884] Avg episode reward: [(0, '21.922')] [2025-02-09 11:47:15,406][00884] Fps is (10 sec: 4505.6, 60 sec: 3959.6, 300 sec: 3971.0). Total num frames: 3514368. Throughput: 0: 937.2. Samples: 877190. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:47:15,409][00884] Avg episode reward: [(0, '21.796')] [2025-02-09 11:47:16,888][04154] Updated weights for policy 0, policy_version 860 (0.0012) [2025-02-09 11:47:20,406][00884] Fps is (10 sec: 4096.0, 60 sec: 3822.9, 300 sec: 3957.2). Total num frames: 3530752. Throughput: 0: 976.2. Samples: 883372. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2025-02-09 11:47:20,411][00884] Avg episode reward: [(0, '21.955')] [2025-02-09 11:47:25,406][00884] Fps is (10 sec: 3276.8, 60 sec: 3686.4, 300 sec: 3971.0). Total num frames: 3547136. Throughput: 0: 973.8. Samples: 885456. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:47:25,412][00884] Avg episode reward: [(0, '22.653')] [2025-02-09 11:47:28,525][04154] Updated weights for policy 0, policy_version 870 (0.0018) [2025-02-09 11:47:30,406][00884] Fps is (10 sec: 4096.0, 60 sec: 3822.9, 300 sec: 3984.9). Total num frames: 3571712. Throughput: 0: 980.5. Samples: 891524. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2025-02-09 11:47:30,412][00884] Avg episode reward: [(0, '21.672')] [2025-02-09 11:47:35,406][00884] Fps is (10 sec: 4505.6, 60 sec: 3959.5, 300 sec: 3971.0). Total num frames: 3592192. Throughput: 0: 1037.2. Samples: 898602. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2025-02-09 11:47:35,407][00884] Avg episode reward: [(0, '21.975')] [2025-02-09 11:47:35,422][04141] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000877_3592192.pth... [2025-02-09 11:47:35,600][04141] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000645_2641920.pth [2025-02-09 11:47:39,141][04154] Updated weights for policy 0, policy_version 880 (0.0013) [2025-02-09 11:47:40,406][00884] Fps is (10 sec: 3276.8, 60 sec: 3891.2, 300 sec: 3957.2). Total num frames: 3604480. Throughput: 0: 1009.6. Samples: 900570. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2025-02-09 11:47:40,416][00884] Avg episode reward: [(0, '22.784')] [2025-02-09 11:47:45,406][00884] Fps is (10 sec: 3276.8, 60 sec: 3891.2, 300 sec: 3971.1). Total num frames: 3624960. Throughput: 0: 953.4. Samples: 905390. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2025-02-09 11:47:45,408][00884] Avg episode reward: [(0, '21.533')] [2025-02-09 11:47:49,241][04154] Updated weights for policy 0, policy_version 890 (0.0019) [2025-02-09 11:47:50,406][00884] Fps is (10 sec: 4505.6, 60 sec: 4027.7, 300 sec: 3971.0). Total num frames: 3649536. Throughput: 0: 986.7. Samples: 912130. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2025-02-09 11:47:50,412][00884] Avg episode reward: [(0, '20.391')] [2025-02-09 11:47:55,406][00884] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3929.4). Total num frames: 3661824. Throughput: 0: 1006.5. Samples: 915218. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2025-02-09 11:47:55,412][00884] Avg episode reward: [(0, '21.388')] [2025-02-09 11:48:00,406][00884] Fps is (10 sec: 3276.8, 60 sec: 3754.9, 300 sec: 3957.2). Total num frames: 3682304. Throughput: 0: 934.7. Samples: 919250. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2025-02-09 11:48:00,408][00884] Avg episode reward: [(0, '21.567')] [2025-02-09 11:48:01,315][04154] Updated weights for policy 0, policy_version 900 (0.0033) [2025-02-09 11:48:05,406][00884] Fps is (10 sec: 4096.0, 60 sec: 3891.2, 300 sec: 3943.3). Total num frames: 3702784. Throughput: 0: 944.2. Samples: 925862. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2025-02-09 11:48:05,408][00884] Avg episode reward: [(0, '20.216')] [2025-02-09 11:48:10,406][00884] Fps is (10 sec: 4096.0, 60 sec: 3891.2, 300 sec: 3929.4). Total num frames: 3723264. Throughput: 0: 976.4. Samples: 929392. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:48:10,408][00884] Avg episode reward: [(0, '21.774')] [2025-02-09 11:48:11,001][04154] Updated weights for policy 0, policy_version 910 (0.0017) [2025-02-09 11:48:15,406][00884] Fps is (10 sec: 3686.4, 60 sec: 3754.7, 300 sec: 3929.4). Total num frames: 3739648. Throughput: 0: 949.5. Samples: 934252. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) [2025-02-09 11:48:15,412][00884] Avg episode reward: [(0, '22.362')] [2025-02-09 11:48:20,406][00884] Fps is (10 sec: 3686.4, 60 sec: 3822.9, 300 sec: 3943.3). Total num frames: 3760128. Throughput: 0: 922.4. Samples: 940110. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:48:20,412][00884] Avg episode reward: [(0, '21.510')] [2025-02-09 11:48:21,798][04154] Updated weights for policy 0, policy_version 920 (0.0020) [2025-02-09 11:48:25,406][00884] Fps is (10 sec: 4505.6, 60 sec: 3959.5, 300 sec: 3943.3). Total num frames: 3784704. Throughput: 0: 957.2. Samples: 943644. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:48:25,412][00884] Avg episode reward: [(0, '21.521')] [2025-02-09 11:48:30,406][00884] Fps is (10 sec: 4096.0, 60 sec: 3822.9, 300 sec: 3929.4). Total num frames: 3801088. Throughput: 0: 985.1. Samples: 949718. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2025-02-09 11:48:30,408][00884] Avg episode reward: [(0, '22.220')] [2025-02-09 11:48:32,644][04154] Updated weights for policy 0, policy_version 930 (0.0029) [2025-02-09 11:48:35,406][00884] Fps is (10 sec: 3276.8, 60 sec: 3754.7, 300 sec: 3943.3). Total num frames: 3817472. Throughput: 0: 947.7. Samples: 954778. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2025-02-09 11:48:35,408][00884] Avg episode reward: [(0, '20.618')] [2025-02-09 11:48:40,406][00884] Fps is (10 sec: 4096.0, 60 sec: 3959.5, 300 sec: 3971.0). Total num frames: 3842048. Throughput: 0: 959.5. Samples: 958396. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2025-02-09 11:48:40,412][00884] Avg episode reward: [(0, '20.623')] [2025-02-09 11:48:41,342][04154] Updated weights for policy 0, policy_version 940 (0.0016) [2025-02-09 11:48:45,406][00884] Fps is (10 sec: 4505.6, 60 sec: 3959.5, 300 sec: 3957.2). Total num frames: 3862528. Throughput: 0: 1030.4. Samples: 965616. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2025-02-09 11:48:45,410][00884] Avg episode reward: [(0, '22.160')] [2025-02-09 11:48:50,406][00884] Fps is (10 sec: 3686.4, 60 sec: 3822.9, 300 sec: 3957.2). Total num frames: 3878912. Throughput: 0: 984.0. Samples: 970142. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:48:50,408][00884] Avg episode reward: [(0, '22.205')] [2025-02-09 11:48:52,449][04154] Updated weights for policy 0, policy_version 950 (0.0028) [2025-02-09 11:48:55,406][00884] Fps is (10 sec: 4096.0, 60 sec: 4027.7, 300 sec: 3984.9). Total num frames: 3903488. Throughput: 0: 978.2. Samples: 973410. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2025-02-09 11:48:55,412][00884] Avg episode reward: [(0, '21.836')] [2025-02-09 11:49:00,406][00884] Fps is (10 sec: 4915.2, 60 sec: 4096.0, 300 sec: 3984.9). Total num frames: 3928064. Throughput: 0: 1033.5. Samples: 980758. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:49:00,412][00884] Avg episode reward: [(0, '21.941')] [2025-02-09 11:49:00,901][04154] Updated weights for policy 0, policy_version 960 (0.0024) [2025-02-09 11:49:05,411][00884] Fps is (10 sec: 4093.7, 60 sec: 4027.4, 300 sec: 3971.0). Total num frames: 3944448. Throughput: 0: 1020.8. Samples: 986050. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2025-02-09 11:49:05,414][00884] Avg episode reward: [(0, '21.670')] [2025-02-09 11:49:10,406][00884] Fps is (10 sec: 3276.8, 60 sec: 3959.5, 300 sec: 3971.0). Total num frames: 3960832. Throughput: 0: 992.0. Samples: 988284. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2025-02-09 11:49:10,408][00884] Avg episode reward: [(0, '22.341')] [2025-02-09 11:49:12,285][04154] Updated weights for policy 0, policy_version 970 (0.0021) [2025-02-09 11:49:15,406][00884] Fps is (10 sec: 4098.2, 60 sec: 4096.0, 300 sec: 3984.9). Total num frames: 3985408. Throughput: 0: 1012.8. Samples: 995294. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2025-02-09 11:49:15,411][00884] Avg episode reward: [(0, '22.047')] [2025-02-09 11:49:20,349][04141] Stopping Batcher_0... [2025-02-09 11:49:20,351][04141] Loop batcher_evt_loop terminating... [2025-02-09 11:49:20,352][04141] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... [2025-02-09 11:49:20,361][00884] Component Batcher_0 stopped! [2025-02-09 11:49:20,479][04154] Weights refcount: 2 0 [2025-02-09 11:49:20,486][00884] Component InferenceWorker_p0-w0 stopped! [2025-02-09 11:49:20,484][04154] Stopping InferenceWorker_p0-w0... [2025-02-09 11:49:20,493][04154] Loop inference_proc0-0_evt_loop terminating... [2025-02-09 11:49:20,529][04141] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000762_3121152.pth [2025-02-09 11:49:20,549][04141] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... [2025-02-09 11:49:20,790][04141] Stopping LearnerWorker_p0... [2025-02-09 11:49:20,791][04141] Loop learner_proc0_evt_loop terminating... [2025-02-09 11:49:20,792][00884] Component LearnerWorker_p0 stopped! [2025-02-09 11:49:20,908][00884] Component RolloutWorker_w1 stopped! [2025-02-09 11:49:20,910][04156] Stopping RolloutWorker_w1... [2025-02-09 11:49:20,914][04156] Loop rollout_proc1_evt_loop terminating... [2025-02-09 11:49:20,955][00884] Component RolloutWorker_w3 stopped! [2025-02-09 11:49:20,961][04158] Stopping RolloutWorker_w3... [2025-02-09 11:49:20,968][00884] Component RolloutWorker_w5 stopped! [2025-02-09 11:49:20,973][04160] Stopping RolloutWorker_w5... [2025-02-09 11:49:20,961][04158] Loop rollout_proc3_evt_loop terminating... [2025-02-09 11:49:20,978][04160] Loop rollout_proc5_evt_loop terminating... [2025-02-09 11:49:21,005][00884] Component RolloutWorker_w7 stopped! [2025-02-09 11:49:21,014][04162] Stopping RolloutWorker_w7... [2025-02-09 11:49:21,016][04162] Loop rollout_proc7_evt_loop terminating... [2025-02-09 11:49:21,056][04161] Stopping RolloutWorker_w6... [2025-02-09 11:49:21,056][04161] Loop rollout_proc6_evt_loop terminating... [2025-02-09 11:49:21,056][00884] Component RolloutWorker_w6 stopped! [2025-02-09 11:49:21,106][04155] Stopping RolloutWorker_w0... [2025-02-09 11:49:21,106][04155] Loop rollout_proc0_evt_loop terminating... [2025-02-09 11:49:21,106][00884] Component RolloutWorker_w0 stopped! [2025-02-09 11:49:21,145][00884] Component RolloutWorker_w2 stopped! [2025-02-09 11:49:21,147][04157] Stopping RolloutWorker_w2... [2025-02-09 11:49:21,148][04157] Loop rollout_proc2_evt_loop terminating... [2025-02-09 11:49:21,169][00884] Component RolloutWorker_w4 stopped! [2025-02-09 11:49:21,171][00884] Waiting for process learner_proc0 to stop... [2025-02-09 11:49:21,176][04159] Stopping RolloutWorker_w4... [2025-02-09 11:49:21,177][04159] Loop rollout_proc4_evt_loop terminating... [2025-02-09 11:49:23,352][00884] Waiting for process inference_proc0-0 to join... [2025-02-09 11:49:23,357][00884] Waiting for process rollout_proc0 to join... [2025-02-09 11:49:25,753][00884] Waiting for process rollout_proc1 to join... [2025-02-09 11:49:25,763][00884] Waiting for process rollout_proc2 to join... [2025-02-09 11:49:25,771][00884] Waiting for process rollout_proc3 to join... [2025-02-09 11:49:25,779][00884] Waiting for process rollout_proc4 to join... [2025-02-09 11:49:25,784][00884] Waiting for process rollout_proc5 to join... [2025-02-09 11:49:25,789][00884] Waiting for process rollout_proc6 to join... [2025-02-09 11:49:25,797][00884] Waiting for process rollout_proc7 to join... [2025-02-09 11:49:25,801][00884] Batcher 0 profile tree view: batching: 26.3360, releasing_batches: 0.0279 [2025-02-09 11:49:25,805][00884] InferenceWorker_p0-w0 profile tree view: wait_policy: 0.0074 wait_policy_total: 418.9512 update_model: 8.3162 weight_update: 0.0013 one_step: 0.0120 handle_policy_step: 570.1043 deserialize: 13.8535, stack: 2.9655, obs_to_device_normalize: 120.6554, forward: 293.7605, send_messages: 27.9761 prepare_outputs: 86.2119 to_cpu: 53.0826 [2025-02-09 11:49:25,807][00884] Learner 0 profile tree view: misc: 0.0040, prepare_batch: 12.8060 train: 72.9067 epoch_init: 0.0067, minibatch_init: 0.0055, losses_postprocess: 0.5639, kl_divergence: 0.6418, after_optimizer: 33.7543 calculate_losses: 25.8151 losses_init: 0.0033, forward_head: 1.2417, bptt_initial: 17.0366, tail: 1.0230, advantages_returns: 0.2836, losses: 3.9080 bptt: 2.0284 bptt_forward_core: 1.9423 update: 11.6065 clip: 0.8669 [2025-02-09 11:49:25,810][00884] RolloutWorker_w0 profile tree view: wait_for_trajectories: 0.2660, enqueue_policy_requests: 107.1974, env_step: 809.5226, overhead: 11.2900, complete_rollouts: 6.8076 save_policy_outputs: 17.6194 split_output_tensors: 6.8398 [2025-02-09 11:49:25,812][00884] RolloutWorker_w7 profile tree view: wait_for_trajectories: 0.3730, enqueue_policy_requests: 105.9898, env_step: 808.5389, overhead: 11.3889, complete_rollouts: 6.7120 save_policy_outputs: 18.0621 split_output_tensors: 7.0097 [2025-02-09 11:49:25,815][00884] Loop Runner_EvtLoop terminating... [2025-02-09 11:49:25,817][00884] Runner profile tree view: main_loop: 1064.4768 [2025-02-09 11:49:25,819][00884] Collected {0: 4005888}, FPS: 3763.2 [2025-02-09 11:51:07,085][00884] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json [2025-02-09 11:51:07,087][00884] Overriding arg 'num_workers' with value 1 passed from command line [2025-02-09 11:51:07,089][00884] Adding new argument 'no_render'=True that is not in the saved config file! [2025-02-09 11:51:07,092][00884] Adding new argument 'save_video'=True that is not in the saved config file! [2025-02-09 11:51:07,093][00884] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! [2025-02-09 11:51:07,095][00884] Adding new argument 'video_name'=None that is not in the saved config file! [2025-02-09 11:51:07,096][00884] Adding new argument 'max_num_frames'=1000000000.0 that is not in the saved config file! [2025-02-09 11:51:07,098][00884] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! [2025-02-09 11:51:07,100][00884] Adding new argument 'push_to_hub'=False that is not in the saved config file! [2025-02-09 11:51:07,105][00884] Adding new argument 'hf_repository'=None that is not in the saved config file! [2025-02-09 11:51:07,105][00884] Adding new argument 'policy_index'=0 that is not in the saved config file! [2025-02-09 11:51:07,107][00884] Adding new argument 'eval_deterministic'=False that is not in the saved config file! [2025-02-09 11:51:07,109][00884] Adding new argument 'train_script'=None that is not in the saved config file! [2025-02-09 11:51:07,112][00884] Adding new argument 'enjoy_script'=None that is not in the saved config file! [2025-02-09 11:51:07,114][00884] Using frameskip 1 and render_action_repeat=4 for evaluation [2025-02-09 11:51:07,143][00884] Doom resolution: 160x120, resize resolution: (128, 72) [2025-02-09 11:51:07,147][00884] RunningMeanStd input shape: (3, 72, 128) [2025-02-09 11:51:07,149][00884] RunningMeanStd input shape: (1,) [2025-02-09 11:51:07,164][00884] ConvEncoder: input_channels=3 [2025-02-09 11:51:07,259][00884] Conv encoder output size: 512 [2025-02-09 11:51:07,261][00884] Policy head output size: 512 [2025-02-09 11:51:07,536][00884] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... [2025-02-09 11:51:08,279][00884] Num frames 100... [2025-02-09 11:51:08,412][00884] Num frames 200... [2025-02-09 11:51:08,540][00884] Num frames 300... [2025-02-09 11:51:08,669][00884] Num frames 400... [2025-02-09 11:51:08,803][00884] Num frames 500... [2025-02-09 11:51:08,934][00884] Num frames 600... [2025-02-09 11:51:09,063][00884] Num frames 700... [2025-02-09 11:51:09,189][00884] Num frames 800... [2025-02-09 11:51:09,318][00884] Num frames 900... [2025-02-09 11:51:09,448][00884] Num frames 1000... [2025-02-09 11:51:09,580][00884] Num frames 1100... [2025-02-09 11:51:09,715][00884] Num frames 1200... [2025-02-09 11:51:09,847][00884] Num frames 1300... [2025-02-09 11:51:09,977][00884] Num frames 1400... [2025-02-09 11:51:10,107][00884] Num frames 1500... [2025-02-09 11:51:10,235][00884] Num frames 1600... [2025-02-09 11:51:10,364][00884] Num frames 1700... [2025-02-09 11:51:10,460][00884] Avg episode rewards: #0: 42.320, true rewards: #0: 17.320 [2025-02-09 11:51:10,461][00884] Avg episode reward: 42.320, avg true_objective: 17.320 [2025-02-09 11:51:10,550][00884] Num frames 1800... [2025-02-09 11:51:10,676][00884] Num frames 1900... [2025-02-09 11:51:10,812][00884] Num frames 2000... [2025-02-09 11:51:10,947][00884] Num frames 2100... [2025-02-09 11:51:11,075][00884] Num frames 2200... [2025-02-09 11:51:11,203][00884] Num frames 2300... [2025-02-09 11:51:11,333][00884] Num frames 2400... [2025-02-09 11:51:11,462][00884] Num frames 2500... [2025-02-09 11:51:11,590][00884] Num frames 2600... [2025-02-09 11:51:11,723][00884] Avg episode rewards: #0: 29.300, true rewards: #0: 13.300 [2025-02-09 11:51:11,725][00884] Avg episode reward: 29.300, avg true_objective: 13.300 [2025-02-09 11:51:11,786][00884] Num frames 2700... [2025-02-09 11:51:11,918][00884] Num frames 2800... [2025-02-09 11:51:12,049][00884] Num frames 2900... [2025-02-09 11:51:12,175][00884] Num frames 3000... [2025-02-09 11:51:12,303][00884] Num frames 3100... [2025-02-09 11:51:12,433][00884] Num frames 3200... [2025-02-09 11:51:12,563][00884] Num frames 3300... [2025-02-09 11:51:12,662][00884] Avg episode rewards: #0: 24.107, true rewards: #0: 11.107 [2025-02-09 11:51:12,663][00884] Avg episode reward: 24.107, avg true_objective: 11.107 [2025-02-09 11:51:12,757][00884] Num frames 3400... [2025-02-09 11:51:12,900][00884] Num frames 3500... [2025-02-09 11:51:13,029][00884] Num frames 3600... [2025-02-09 11:51:13,159][00884] Num frames 3700... [2025-02-09 11:51:13,286][00884] Num frames 3800... [2025-02-09 11:51:13,414][00884] Num frames 3900... [2025-02-09 11:51:13,548][00884] Num frames 4000... [2025-02-09 11:51:13,679][00884] Num frames 4100... [2025-02-09 11:51:13,820][00884] Num frames 4200... [2025-02-09 11:51:13,955][00884] Num frames 4300... [2025-02-09 11:51:14,086][00884] Num frames 4400... [2025-02-09 11:51:14,214][00884] Num frames 4500... [2025-02-09 11:51:14,344][00884] Num frames 4600... [2025-02-09 11:51:14,472][00884] Num frames 4700... [2025-02-09 11:51:14,600][00884] Num frames 4800... [2025-02-09 11:51:14,736][00884] Num frames 4900... [2025-02-09 11:51:14,834][00884] Avg episode rewards: #0: 27.580, true rewards: #0: 12.330 [2025-02-09 11:51:14,837][00884] Avg episode reward: 27.580, avg true_objective: 12.330 [2025-02-09 11:51:14,928][00884] Num frames 5000... [2025-02-09 11:51:15,058][00884] Num frames 5100... [2025-02-09 11:51:15,183][00884] Num frames 5200... [2025-02-09 11:51:15,308][00884] Num frames 5300... [2025-02-09 11:51:15,466][00884] Avg episode rewards: #0: 23.160, true rewards: #0: 10.760 [2025-02-09 11:51:15,467][00884] Avg episode reward: 23.160, avg true_objective: 10.760 [2025-02-09 11:51:15,496][00884] Num frames 5400... [2025-02-09 11:51:15,625][00884] Num frames 5500... [2025-02-09 11:51:15,752][00884] Num frames 5600... [2025-02-09 11:51:15,863][00884] Avg episode rewards: #0: 19.740, true rewards: #0: 9.407 [2025-02-09 11:51:15,865][00884] Avg episode reward: 19.740, avg true_objective: 9.407 [2025-02-09 11:51:15,936][00884] Num frames 5700... [2025-02-09 11:51:16,066][00884] Num frames 5800... [2025-02-09 11:51:16,203][00884] Num frames 5900... [2025-02-09 11:51:16,256][00884] Avg episode rewards: #0: 17.286, true rewards: #0: 8.429 [2025-02-09 11:51:16,257][00884] Avg episode reward: 17.286, avg true_objective: 8.429 [2025-02-09 11:51:16,385][00884] Num frames 6000... [2025-02-09 11:51:16,556][00884] Num frames 6100... [2025-02-09 11:51:16,727][00884] Num frames 6200... [2025-02-09 11:51:16,909][00884] Num frames 6300... [2025-02-09 11:51:17,081][00884] Num frames 6400... [2025-02-09 11:51:17,248][00884] Num frames 6500... [2025-02-09 11:51:17,418][00884] Num frames 6600... [2025-02-09 11:51:17,588][00884] Num frames 6700... [2025-02-09 11:51:17,764][00884] Num frames 6800... [2025-02-09 11:51:17,947][00884] Num frames 6900... [2025-02-09 11:51:18,125][00884] Num frames 7000... [2025-02-09 11:51:18,303][00884] Num frames 7100... [2025-02-09 11:51:18,488][00884] Num frames 7200... [2025-02-09 11:51:18,680][00884] Num frames 7300... [2025-02-09 11:51:18,864][00884] Num frames 7400... [2025-02-09 11:51:19,020][00884] Num frames 7500... [2025-02-09 11:51:19,150][00884] Num frames 7600... [2025-02-09 11:51:19,280][00884] Num frames 7700... [2025-02-09 11:51:19,411][00884] Num frames 7800... [2025-02-09 11:51:19,493][00884] Avg episode rewards: #0: 21.400, true rewards: #0: 9.775 [2025-02-09 11:51:19,495][00884] Avg episode reward: 21.400, avg true_objective: 9.775 [2025-02-09 11:51:19,598][00884] Num frames 7900... [2025-02-09 11:51:19,727][00884] Num frames 8000... [2025-02-09 11:51:19,861][00884] Num frames 8100... [2025-02-09 11:51:19,994][00884] Num frames 8200... [2025-02-09 11:51:20,130][00884] Num frames 8300... [2025-02-09 11:51:20,259][00884] Num frames 8400... [2025-02-09 11:51:20,388][00884] Num frames 8500... [2025-02-09 11:51:20,519][00884] Num frames 8600... [2025-02-09 11:51:20,649][00884] Num frames 8700... [2025-02-09 11:51:20,779][00884] Num frames 8800... [2025-02-09 11:51:20,911][00884] Num frames 8900... [2025-02-09 11:51:21,046][00884] Num frames 9000... [2025-02-09 11:51:21,175][00884] Num frames 9100... [2025-02-09 11:51:21,306][00884] Num frames 9200... [2025-02-09 11:51:21,437][00884] Num frames 9300... [2025-02-09 11:51:21,567][00884] Num frames 9400... [2025-02-09 11:51:21,698][00884] Num frames 9500... [2025-02-09 11:51:21,826][00884] Num frames 9600... [2025-02-09 11:51:21,980][00884] Avg episode rewards: #0: 23.973, true rewards: #0: 10.751 [2025-02-09 11:51:21,981][00884] Avg episode reward: 23.973, avg true_objective: 10.751 [2025-02-09 11:51:22,016][00884] Num frames 9700... [2025-02-09 11:51:22,150][00884] Num frames 9800... [2025-02-09 11:51:22,277][00884] Num frames 9900... [2025-02-09 11:51:22,405][00884] Num frames 10000... [2025-02-09 11:51:22,534][00884] Num frames 10100... [2025-02-09 11:51:22,662][00884] Num frames 10200... [2025-02-09 11:51:22,789][00884] Num frames 10300... [2025-02-09 11:51:22,924][00884] Num frames 10400... [2025-02-09 11:51:23,050][00884] Num frames 10500... [2025-02-09 11:51:23,124][00884] Avg episode rewards: #0: 23.311, true rewards: #0: 10.511 [2025-02-09 11:51:23,126][00884] Avg episode reward: 23.311, avg true_objective: 10.511 [2025-02-09 11:52:20,464][00884] Replay video saved to /content/train_dir/default_experiment/replay.mp4! [2025-02-09 11:54:58,947][00884] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json [2025-02-09 11:54:58,949][00884] Overriding arg 'num_workers' with value 1 passed from command line [2025-02-09 11:54:58,950][00884] Adding new argument 'no_render'=True that is not in the saved config file! [2025-02-09 11:54:58,952][00884] Adding new argument 'save_video'=True that is not in the saved config file! [2025-02-09 11:54:58,953][00884] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! [2025-02-09 11:54:58,956][00884] Adding new argument 'video_name'=None that is not in the saved config file! [2025-02-09 11:54:58,957][00884] Adding new argument 'max_num_frames'=100000 that is not in the saved config file! [2025-02-09 11:54:58,958][00884] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! [2025-02-09 11:54:58,959][00884] Adding new argument 'push_to_hub'=True that is not in the saved config file! [2025-02-09 11:54:58,959][00884] Adding new argument 'hf_repository'='ThomasSimonini/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file! [2025-02-09 11:54:58,960][00884] Adding new argument 'policy_index'=0 that is not in the saved config file! [2025-02-09 11:54:58,961][00884] Adding new argument 'eval_deterministic'=False that is not in the saved config file! [2025-02-09 11:54:58,962][00884] Adding new argument 'train_script'=None that is not in the saved config file! [2025-02-09 11:54:58,963][00884] Adding new argument 'enjoy_script'=None that is not in the saved config file! [2025-02-09 11:54:58,964][00884] Using frameskip 1 and render_action_repeat=4 for evaluation [2025-02-09 11:54:58,994][00884] RunningMeanStd input shape: (3, 72, 128) [2025-02-09 11:54:58,996][00884] RunningMeanStd input shape: (1,) [2025-02-09 11:54:59,008][00884] ConvEncoder: input_channels=3 [2025-02-09 11:54:59,042][00884] Conv encoder output size: 512 [2025-02-09 11:54:59,044][00884] Policy head output size: 512 [2025-02-09 11:54:59,063][00884] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... [2025-02-09 11:54:59,501][00884] Num frames 100... [2025-02-09 11:54:59,629][00884] Num frames 200... [2025-02-09 11:54:59,756][00884] Num frames 300... [2025-02-09 11:54:59,904][00884] Num frames 400... [2025-02-09 11:55:00,033][00884] Num frames 500... [2025-02-09 11:55:00,158][00884] Num frames 600... [2025-02-09 11:55:00,283][00884] Num frames 700... [2025-02-09 11:55:00,410][00884] Num frames 800... [2025-02-09 11:55:00,537][00884] Num frames 900... [2025-02-09 11:55:00,668][00884] Num frames 1000... [2025-02-09 11:55:00,834][00884] Avg episode rewards: #0: 21.880, true rewards: #0: 10.880 [2025-02-09 11:55:00,835][00884] Avg episode reward: 21.880, avg true_objective: 10.880 [2025-02-09 11:55:00,856][00884] Num frames 1100... [2025-02-09 11:55:00,989][00884] Num frames 1200... [2025-02-09 11:55:01,114][00884] Num frames 1300... [2025-02-09 11:55:01,271][00884] Num frames 1400... [2025-02-09 11:55:01,446][00884] Num frames 1500... [2025-02-09 11:55:01,566][00884] Avg episode rewards: #0: 13.680, true rewards: #0: 7.680 [2025-02-09 11:55:01,568][00884] Avg episode reward: 13.680, avg true_objective: 7.680 [2025-02-09 11:55:01,686][00884] Num frames 1600... [2025-02-09 11:55:01,868][00884] Num frames 1700... [2025-02-09 11:55:02,045][00884] Num frames 1800... [2025-02-09 11:55:02,213][00884] Num frames 1900... [2025-02-09 11:55:02,376][00884] Num frames 2000... [2025-02-09 11:55:02,548][00884] Num frames 2100... [2025-02-09 11:55:02,720][00884] Num frames 2200... [2025-02-09 11:55:02,919][00884] Num frames 2300... [2025-02-09 11:55:03,128][00884] Num frames 2400... [2025-02-09 11:55:03,319][00884] Num frames 2500... [2025-02-09 11:55:03,476][00884] Avg episode rewards: #0: 16.523, true rewards: #0: 8.523 [2025-02-09 11:55:03,478][00884] Avg episode reward: 16.523, avg true_objective: 8.523 [2025-02-09 11:55:03,555][00884] Num frames 2600... [2025-02-09 11:55:03,721][00884] Num frames 2700... [2025-02-09 11:55:03,854][00884] Num frames 2800... [2025-02-09 11:55:03,980][00884] Num frames 2900... [2025-02-09 11:55:04,116][00884] Num frames 3000... [2025-02-09 11:55:04,244][00884] Num frames 3100... [2025-02-09 11:55:04,371][00884] Num frames 3200... [2025-02-09 11:55:04,498][00884] Num frames 3300... [2025-02-09 11:55:04,628][00884] Num frames 3400... [2025-02-09 11:55:04,757][00884] Num frames 3500... [2025-02-09 11:55:04,886][00884] Num frames 3600... [2025-02-09 11:55:05,014][00884] Num frames 3700... [2025-02-09 11:55:05,145][00884] Num frames 3800... [2025-02-09 11:55:05,248][00884] Avg episode rewards: #0: 19.093, true rewards: #0: 9.592 [2025-02-09 11:55:05,249][00884] Avg episode reward: 19.093, avg true_objective: 9.592 [2025-02-09 11:55:05,329][00884] Num frames 3900... [2025-02-09 11:55:05,456][00884] Num frames 4000... [2025-02-09 11:55:05,583][00884] Num frames 4100... [2025-02-09 11:55:05,713][00884] Num frames 4200... [2025-02-09 11:55:05,847][00884] Num frames 4300... [2025-02-09 11:55:05,973][00884] Num frames 4400... [2025-02-09 11:55:06,102][00884] Num frames 4500... [2025-02-09 11:55:06,239][00884] Num frames 4600... [2025-02-09 11:55:06,365][00884] Num frames 4700... [2025-02-09 11:55:06,495][00884] Num frames 4800... [2025-02-09 11:55:06,623][00884] Num frames 4900... [2025-02-09 11:55:06,753][00884] Num frames 5000... [2025-02-09 11:55:06,886][00884] Num frames 5100... [2025-02-09 11:55:07,017][00884] Num frames 5200... [2025-02-09 11:55:07,151][00884] Num frames 5300... [2025-02-09 11:55:07,284][00884] Num frames 5400... [2025-02-09 11:55:07,416][00884] Num frames 5500... [2025-02-09 11:55:07,545][00884] Num frames 5600... [2025-02-09 11:55:07,676][00884] Num frames 5700... [2025-02-09 11:55:07,804][00884] Avg episode rewards: #0: 25.114, true rewards: #0: 11.514 [2025-02-09 11:55:07,805][00884] Avg episode reward: 25.114, avg true_objective: 11.514 [2025-02-09 11:55:07,863][00884] Num frames 5800... [2025-02-09 11:55:07,993][00884] Num frames 5900... [2025-02-09 11:55:08,124][00884] Num frames 6000... [2025-02-09 11:55:08,255][00884] Num frames 6100... [2025-02-09 11:55:08,379][00884] Num frames 6200... [2025-02-09 11:55:08,507][00884] Num frames 6300... [2025-02-09 11:55:08,636][00884] Num frames 6400... [2025-02-09 11:55:08,716][00884] Avg episode rewards: #0: 22.695, true rewards: #0: 10.695 [2025-02-09 11:55:08,717][00884] Avg episode reward: 22.695, avg true_objective: 10.695 [2025-02-09 11:55:08,823][00884] Num frames 6500... [2025-02-09 11:55:08,954][00884] Num frames 6600... [2025-02-09 11:55:09,082][00884] Num frames 6700... [2025-02-09 11:55:09,215][00884] Num frames 6800... [2025-02-09 11:55:09,343][00884] Num frames 6900... [2025-02-09 11:55:09,469][00884] Num frames 7000... [2025-02-09 11:55:09,558][00884] Avg episode rewards: #0: 20.750, true rewards: #0: 10.036 [2025-02-09 11:55:09,559][00884] Avg episode reward: 20.750, avg true_objective: 10.036 [2025-02-09 11:55:09,656][00884] Num frames 7100... [2025-02-09 11:55:09,782][00884] Num frames 7200... [2025-02-09 11:55:09,920][00884] Num frames 7300... [2025-02-09 11:55:10,057][00884] Num frames 7400... [2025-02-09 11:55:10,191][00884] Num frames 7500... [2025-02-09 11:55:10,330][00884] Num frames 7600... [2025-02-09 11:55:10,457][00884] Num frames 7700... [2025-02-09 11:55:10,588][00884] Num frames 7800... [2025-02-09 11:55:10,717][00884] Num frames 7900... [2025-02-09 11:55:10,849][00884] Num frames 8000... [2025-02-09 11:55:10,977][00884] Num frames 8100... [2025-02-09 11:55:11,106][00884] Num frames 8200... [2025-02-09 11:55:11,233][00884] Num frames 8300... [2025-02-09 11:55:11,369][00884] Num frames 8400... [2025-02-09 11:55:11,498][00884] Num frames 8500... [2025-02-09 11:55:11,627][00884] Num frames 8600... [2025-02-09 11:55:11,758][00884] Num frames 8700... [2025-02-09 11:55:11,891][00884] Num frames 8800... [2025-02-09 11:55:12,023][00884] Num frames 8900... [2025-02-09 11:55:12,150][00884] Num frames 9000... [2025-02-09 11:55:12,282][00884] Num frames 9100... [2025-02-09 11:55:12,376][00884] Avg episode rewards: #0: 25.531, true rewards: #0: 11.406 [2025-02-09 11:55:12,378][00884] Avg episode reward: 25.531, avg true_objective: 11.406 [2025-02-09 11:55:12,472][00884] Num frames 9200... [2025-02-09 11:55:12,602][00884] Num frames 9300... [2025-02-09 11:55:12,732][00884] Num frames 9400... [2025-02-09 11:55:12,865][00884] Num frames 9500... [2025-02-09 11:55:12,994][00884] Num frames 9600... [2025-02-09 11:55:13,131][00884] Num frames 9700... [2025-02-09 11:55:13,257][00884] Num frames 9800... [2025-02-09 11:55:13,390][00884] Num frames 9900... [2025-02-09 11:55:13,519][00884] Num frames 10000... [2025-02-09 11:55:13,602][00884] Avg episode rewards: #0: 24.468, true rewards: #0: 11.134 [2025-02-09 11:55:13,604][00884] Avg episode reward: 24.468, avg true_objective: 11.134 [2025-02-09 11:55:13,714][00884] Num frames 10100... [2025-02-09 11:55:13,901][00884] Num frames 10200... [2025-02-09 11:55:14,075][00884] Num frames 10300... [2025-02-09 11:55:14,241][00884] Num frames 10400... [2025-02-09 11:55:14,420][00884] Num frames 10500... [2025-02-09 11:55:14,587][00884] Num frames 10600... [2025-02-09 11:55:14,754][00884] Num frames 10700... [2025-02-09 11:55:14,926][00884] Num frames 10800... [2025-02-09 11:55:15,102][00884] Num frames 10900... [2025-02-09 11:55:15,301][00884] Avg episode rewards: #0: 24.077, true rewards: #0: 10.977 [2025-02-09 11:55:15,303][00884] Avg episode reward: 24.077, avg true_objective: 10.977 [2025-02-09 11:56:17,016][00884] Replay video saved to /content/train_dir/default_experiment/replay.mp4! [2025-02-09 11:56:44,045][00884] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json [2025-02-09 11:56:44,047][00884] Overriding arg 'num_workers' with value 1 passed from command line [2025-02-09 11:56:44,049][00884] Adding new argument 'no_render'=True that is not in the saved config file! [2025-02-09 11:56:44,051][00884] Adding new argument 'save_video'=True that is not in the saved config file! [2025-02-09 11:56:44,052][00884] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! [2025-02-09 11:56:44,053][00884] Adding new argument 'video_name'=None that is not in the saved config file! [2025-02-09 11:56:44,055][00884] Adding new argument 'max_num_frames'=100000 that is not in the saved config file! [2025-02-09 11:56:44,056][00884] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! [2025-02-09 11:56:44,058][00884] Adding new argument 'push_to_hub'=True that is not in the saved config file! [2025-02-09 11:56:44,059][00884] Adding new argument 'hf_repository'='TaoZewen/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file! [2025-02-09 11:56:44,060][00884] Adding new argument 'policy_index'=0 that is not in the saved config file! [2025-02-09 11:56:44,062][00884] Adding new argument 'eval_deterministic'=False that is not in the saved config file! [2025-02-09 11:56:44,063][00884] Adding new argument 'train_script'=None that is not in the saved config file! [2025-02-09 11:56:44,064][00884] Adding new argument 'enjoy_script'=None that is not in the saved config file! [2025-02-09 11:56:44,066][00884] Using frameskip 1 and render_action_repeat=4 for evaluation [2025-02-09 11:56:44,098][00884] RunningMeanStd input shape: (3, 72, 128) [2025-02-09 11:56:44,101][00884] RunningMeanStd input shape: (1,) [2025-02-09 11:56:44,112][00884] ConvEncoder: input_channels=3 [2025-02-09 11:56:44,145][00884] Conv encoder output size: 512 [2025-02-09 11:56:44,146][00884] Policy head output size: 512 [2025-02-09 11:56:44,165][00884] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... [2025-02-09 11:56:44,604][00884] Num frames 100... [2025-02-09 11:56:44,737][00884] Num frames 200... [2025-02-09 11:56:44,874][00884] Num frames 300... [2025-02-09 11:56:45,003][00884] Num frames 400... [2025-02-09 11:56:45,130][00884] Num frames 500... [2025-02-09 11:56:45,268][00884] Num frames 600... [2025-02-09 11:56:45,398][00884] Num frames 700... [2025-02-09 11:56:45,527][00884] Num frames 800... [2025-02-09 11:56:45,658][00884] Num frames 900... [2025-02-09 11:56:45,788][00884] Num frames 1000... [2025-02-09 11:56:45,919][00884] Num frames 1100... [2025-02-09 11:56:46,046][00884] Num frames 1200... [2025-02-09 11:56:46,105][00884] Avg episode rewards: #0: 28.010, true rewards: #0: 12.010 [2025-02-09 11:56:46,107][00884] Avg episode reward: 28.010, avg true_objective: 12.010 [2025-02-09 11:56:46,236][00884] Num frames 1300... [2025-02-09 11:56:46,373][00884] Num frames 1400... [2025-02-09 11:56:46,502][00884] Num frames 1500... [2025-02-09 11:56:46,630][00884] Num frames 1600... [2025-02-09 11:56:46,757][00884] Num frames 1700... [2025-02-09 11:56:46,898][00884] Num frames 1800... [2025-02-09 11:56:47,024][00884] Num frames 1900... [2025-02-09 11:56:47,150][00884] Num frames 2000... [2025-02-09 11:56:47,275][00884] Num frames 2100... [2025-02-09 11:56:47,408][00884] Num frames 2200... [2025-02-09 11:56:47,536][00884] Num frames 2300... [2025-02-09 11:56:47,619][00884] Avg episode rewards: #0: 26.105, true rewards: #0: 11.605 [2025-02-09 11:56:47,620][00884] Avg episode reward: 26.105, avg true_objective: 11.605 [2025-02-09 11:56:47,735][00884] Num frames 2400... [2025-02-09 11:56:47,868][00884] Num frames 2500... [2025-02-09 11:56:47,994][00884] Num frames 2600... [2025-02-09 11:56:48,120][00884] Num frames 2700... [2025-02-09 11:56:48,245][00884] Num frames 2800... [2025-02-09 11:56:48,380][00884] Num frames 2900... [2025-02-09 11:56:48,444][00884] Avg episode rewards: #0: 21.020, true rewards: #0: 9.687 [2025-02-09 11:56:48,446][00884] Avg episode reward: 21.020, avg true_objective: 9.687 [2025-02-09 11:56:48,569][00884] Num frames 3000... [2025-02-09 11:56:48,700][00884] Num frames 3100... [2025-02-09 11:56:48,825][00884] Num frames 3200... [2025-02-09 11:56:48,961][00884] Num frames 3300... [2025-02-09 11:56:49,090][00884] Num frames 3400... [2025-02-09 11:56:49,215][00884] Num frames 3500... [2025-02-09 11:56:49,342][00884] Num frames 3600... [2025-02-09 11:56:49,476][00884] Num frames 3700... [2025-02-09 11:56:49,605][00884] Num frames 3800... [2025-02-09 11:56:49,737][00884] Num frames 3900... [2025-02-09 11:56:49,870][00884] Num frames 4000... [2025-02-09 11:56:50,004][00884] Num frames 4100... [2025-02-09 11:56:50,135][00884] Num frames 4200... [2025-02-09 11:56:50,265][00884] Num frames 4300... [2025-02-09 11:56:50,406][00884] Num frames 4400... [2025-02-09 11:56:50,540][00884] Num frames 4500... [2025-02-09 11:56:50,674][00884] Num frames 4600... [2025-02-09 11:56:50,807][00884] Num frames 4700... [2025-02-09 11:56:50,940][00884] Num frames 4800... [2025-02-09 11:56:51,072][00884] Num frames 4900... [2025-02-09 11:56:51,204][00884] Num frames 5000... [2025-02-09 11:56:51,268][00884] Avg episode rewards: #0: 31.015, true rewards: #0: 12.515 [2025-02-09 11:56:51,269][00884] Avg episode reward: 31.015, avg true_objective: 12.515 [2025-02-09 11:56:51,389][00884] Num frames 5100... [2025-02-09 11:56:51,524][00884] Num frames 5200... [2025-02-09 11:56:51,650][00884] Num frames 5300... [2025-02-09 11:56:51,821][00884] Avg episode rewards: #0: 25.980, true rewards: #0: 10.780 [2025-02-09 11:56:51,823][00884] Avg episode reward: 25.980, avg true_objective: 10.780 [2025-02-09 11:56:51,842][00884] Num frames 5400... [2025-02-09 11:56:51,970][00884] Num frames 5500... [2025-02-09 11:56:52,098][00884] Num frames 5600... [2025-02-09 11:56:52,227][00884] Num frames 5700... [2025-02-09 11:56:52,356][00884] Num frames 5800... [2025-02-09 11:56:52,491][00884] Num frames 5900... [2025-02-09 11:56:52,620][00884] Num frames 6000... [2025-02-09 11:56:52,749][00884] Num frames 6100... [2025-02-09 11:56:52,885][00884] Num frames 6200... [2025-02-09 11:56:53,036][00884] Avg episode rewards: #0: 25.458, true rewards: #0: 10.458 [2025-02-09 11:56:53,038][00884] Avg episode reward: 25.458, avg true_objective: 10.458 [2025-02-09 11:56:53,072][00884] Num frames 6300... [2025-02-09 11:56:53,198][00884] Num frames 6400... [2025-02-09 11:56:53,325][00884] Num frames 6500... [2025-02-09 11:56:53,493][00884] Num frames 6600... [2025-02-09 11:56:53,678][00884] Num frames 6700... [2025-02-09 11:56:53,786][00884] Avg episode rewards: #0: 22.604, true rewards: #0: 9.604 [2025-02-09 11:56:53,788][00884] Avg episode reward: 22.604, avg true_objective: 9.604 [2025-02-09 11:56:53,932][00884] Num frames 6800... [2025-02-09 11:56:54,099][00884] Num frames 6900... [2025-02-09 11:56:54,270][00884] Num frames 7000... [2025-02-09 11:56:54,439][00884] Num frames 7100... [2025-02-09 11:56:54,543][00884] Avg episode rewards: #0: 20.409, true rewards: #0: 8.909 [2025-02-09 11:56:54,545][00884] Avg episode reward: 20.409, avg true_objective: 8.909 [2025-02-09 11:56:54,670][00884] Num frames 7200... [2025-02-09 11:56:54,850][00884] Num frames 7300... [2025-02-09 11:56:55,026][00884] Num frames 7400... [2025-02-09 11:56:55,199][00884] Num frames 7500... [2025-02-09 11:56:55,379][00884] Num frames 7600... [2025-02-09 11:56:55,506][00884] Avg episode rewards: #0: 18.932, true rewards: #0: 8.488 [2025-02-09 11:56:55,507][00884] Avg episode reward: 18.932, avg true_objective: 8.488 [2025-02-09 11:56:55,629][00884] Num frames 7700... [2025-02-09 11:56:55,805][00884] Num frames 7800... [2025-02-09 11:56:55,939][00884] Num frames 7900... [2025-02-09 11:56:56,067][00884] Num frames 8000... [2025-02-09 11:56:56,192][00884] Num frames 8100... [2025-02-09 11:56:56,320][00884] Num frames 8200... [2025-02-09 11:56:56,395][00884] Avg episode rewards: #0: 17.915, true rewards: #0: 8.215 [2025-02-09 11:56:56,397][00884] Avg episode reward: 17.915, avg true_objective: 8.215 [2025-02-09 11:57:41,582][00884] Replay video saved to /content/train_dir/default_experiment/replay.mp4!