Hugging Face's logo

sb3
/
a2c-BipedalWalker-v3

Reinforcement Learning
stable-baselines3
BipedalWalker-v3
deep-reinforcement-learning
Eval Results
Model card Files Community
Antonin Raffin commited on
Commit
00ca0ad
·
1 Parent(s): 989d36c

Initial commit

Browse files
Files changed (16) hide show
  1. .gitattributes +2 -0
  2. README.md +69 -0
  3. a2c-BipedalWalker-v3.zip +3 -0
  4. a2c-BipedalWalker-v3/_stable_baselines3_version +1 -0
  5. a2c-BipedalWalker-v3/data +106 -0
  6. a2c-BipedalWalker-v3/policy.optimizer.pth +3 -0
  7. a2c-BipedalWalker-v3/policy.pth +3 -0
  8. a2c-BipedalWalker-v3/pytorch_variables.pth +3 -0
  9. a2c-BipedalWalker-v3/system_info.txt +7 -0
  10. args.yml +59 -0
  11. config.yml +31 -0
  12. env_kwargs.yml +1 -0
  13. replay.mp4 +3 -0
  14. results.json +1 -0
  15. train_eval_metrics.zip +3 -0
  16. vec_normalize.pkl +3 -0
.gitattributes CHANGED
@@ -25,3 +25,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
25
  *.zip filter=lfs diff=lfs merge=lfs -text
26
  *.zstandard filter=lfs diff=lfs merge=lfs -text
27
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
25
  *.zip filter=lfs diff=lfs merge=lfs -text
26
  *.zstandard filter=lfs diff=lfs merge=lfs -text
27
  *tfevents* filter=lfs diff=lfs merge=lfs -text
28
+ *.mp4 filter=lfs diff=lfs merge=lfs -text
29
+ vec_normalize.pkl filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,69 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: stable-baselines3
3
+ tags:
4
+ - BipedalWalker-v3
5
+ - deep-reinforcement-learning
6
+ - reinforcement-learning
7
+ - stable-baselines3
8
+ model-index:
9
+ - name: A2C
10
+ results:
11
+ - metrics:
12
+ - type: mean_reward
13
+ value: 300.85 +/- 1.44
14
+ name: mean_reward
15
+ task:
16
+ type: reinforcement-learning
17
+ name: reinforcement-learning
18
+ dataset:
19
+ name: BipedalWalker-v3
20
+ type: BipedalWalker-v3
21
+ ---
22
+
23
+ # **A2C** Agent playing **BipedalWalker-v3**
24
+ This is a trained model of a **A2C** agent playing **BipedalWalker-v3**
25
+ using the [stable-baselines3 library](https://github.com/DLR-RM/stable-baselines3)
26
+ and the [RL Zoo](https://github.com/DLR-RM/rl-baselines3-zoo).
27
+
28
+ The RL Zoo is a training framework for Stable Baselines3
29
+ reinforcement learning agents,
30
+ with hyperparameter optimization and pre-trained agents included.
31
+
32
+ ## Usage (with SB3 RL Zoo)
33
+
34
+ RL Zoo: https://github.com/DLR-RM/rl-baselines3-zoo<br/>
35
+ SB3: https://github.com/DLR-RM/stable-baselines3<br/>
36
+ SB3 Contrib: https://github.com/Stable-Baselines-Team/stable-baselines3-contrib
37
+
38
+ ```
39
+ # Download model and save it into the logs/ folder
40
+ python -m utils.load_from_hub --algo a2c --env BipedalWalker-v3 -orga sb3 -f logs/
41
+ python enjoy.py --algo a2c --env BipedalWalker-v3 -f logs/
42
+ ```
43
+
44
+ ## Training (with the RL Zoo)
45
+ ```
46
+ python train.py --algo a2c --env BipedalWalker-v3 -f logs/
47
+ # Upload the model and generate video (when possible)
48
+ python -m utils.push_to_hub --algo a2c --env BipedalWalker-v3 -f logs/ -orga sb3
49
+ ```
50
+
51
+ ## Hyperparameters
52
+ ```python
53
+ OrderedDict([('ent_coef', 0.0),
54
+ ('gae_lambda', 0.9),
55
+ ('gamma', 0.99),
56
+ ('learning_rate', 'lin_0.00096'),
57
+ ('max_grad_norm', 0.5),
58
+ ('n_envs', 16),
59
+ ('n_steps', 8),
60
+ ('n_timesteps', 5000000.0),
61
+ ('normalize', True),
62
+ ('normalize_advantage', False),
63
+ ('policy', 'MlpPolicy'),
64
+ ('policy_kwargs', 'dict(log_std_init=-2, ortho_init=False)'),
65
+ ('use_rms_prop', True),
66
+ ('use_sde', True),
67
+ ('vf_coef', 0.4),
68
+ ('normalize_kwargs', {'norm_obs': True, 'norm_reward': False})])
69
+ ```
a2c-BipedalWalker-v3.zip ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1347120e5f37521e08163027a83205ed929a1b9644a58280825d9f0f96901674
3
+ size 130095
a2c-BipedalWalker-v3/_stable_baselines3_version ADDED
@@ -0,0 +1 @@
 
 
1
+ 1.5.1a8
a2c-BipedalWalker-v3/data ADDED
@@ -0,0 +1,106 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "policy_class": {
3
+ ":type:": "<class 'abc.ABCMeta'>",
4
+ ":serialized:": "gASVOwAAAAAAAACMIXN0YWJsZV9iYXNlbGluZXMzLmNvbW1vbi5wb2xpY2llc5SMEUFjdG9yQ3JpdGljUG9saWN5lJOULg==",
5
+ "__module__": "stable_baselines3.common.policies",
6
+ "__doc__": "\n Policy class for actor-critic algorithms (has both policy and value prediction).\n Used by A2C, PPO and the likes.\n\n :param observation_space: Observation space\n :param action_space: Action space\n :param lr_schedule: Learning rate schedule (could be constant)\n :param net_arch: The specification of the policy and value networks.\n :param activation_fn: Activation function\n :param ortho_init: Whether to use or not orthogonal initialization\n :param use_sde: Whether to use State Dependent Exploration or not\n :param log_std_init: Initial value for the log standard deviation\n :param full_std: Whether to use (n_features x n_actions) parameters\n for the std instead of only (n_features,) when using gSDE\n :param sde_net_arch: Network architecture for extracting features\n when using gSDE. If None, the latent features from the policy will be used.\n Pass an empty list to use the states as features.\n :param use_expln: Use ``expln()`` function instead of ``exp()`` to ensure\n a positive standard deviation (cf paper). It allows to keep variance\n above zero and prevent it from growing too fast. In practice, ``exp()`` is usually enough.\n :param squash_output: Whether to squash the output using a tanh function,\n this allows to ensure boundaries when using gSDE.\n :param features_extractor_class: Features extractor to use.\n :param features_extractor_kwargs: Keyword arguments\n to pass to the features extractor.\n :param normalize_images: Whether to normalize images or not,\n dividing by 255.0 (True by default)\n :param optimizer_class: The optimizer to use,\n ``th.optim.Adam`` by default\n :param optimizer_kwargs: Additional keyword arguments,\n excluding the learning rate, to pass to the optimizer\n ",
7
+ "__init__": "<function ActorCriticPolicy.__init__ at 0x7f4193176950>",
8
+ "_get_constructor_parameters": "<function ActorCriticPolicy._get_constructor_parameters at 0x7f41931769e0>",
9
+ "reset_noise": "<function ActorCriticPolicy.reset_noise at 0x7f4193176a70>",
10
+ "_build_mlp_extractor": "<function ActorCriticPolicy._build_mlp_extractor at 0x7f4193176b00>",
11
+ "_build": "<function ActorCriticPolicy._build at 0x7f4193176b90>",
12
+ "forward": "<function ActorCriticPolicy.forward at 0x7f4193176c20>",
13
+ "_get_action_dist_from_latent": "<function ActorCriticPolicy._get_action_dist_from_latent at 0x7f4193176cb0>",
14
+ "_predict": "<function ActorCriticPolicy._predict at 0x7f4193176d40>",
15
+ "evaluate_actions": "<function ActorCriticPolicy.evaluate_actions at 0x7f4193176dd0>",
16
+ "get_distribution": "<function ActorCriticPolicy.get_distribution at 0x7f4193176e60>",
17
+ "predict_values": "<function ActorCriticPolicy.predict_values at 0x7f4193176ef0>",
18
+ "__abstractmethods__": "frozenset()",
19
+ "_abc_impl": "<_abc_data object at 0x7f41931c7840>"
20
+ },
21
+ "verbose": 1,
22
+ "policy_kwargs": {
23
+ ":type:": "<class 'dict'>",
24
+ ":serialized:": "gASVowAAAAAAAAB9lCiMDGxvZ19zdGRfaW5pdJRK/v///4wKb3J0aG9faW5pdJSJjA9vcHRpbWl6ZXJfY2xhc3OUjBN0b3JjaC5vcHRpbS5ybXNwcm9wlIwHUk1TcHJvcJSTlIwQb3B0aW1pemVyX2t3YXJnc5R9lCiMBWFscGhhlEc/764UeuFHrowDZXBzlEc+5Pi1iONo8YwMd2VpZ2h0X2RlY2F5lEsAdXUu",
25
+ "log_std_init": -2,
26
+ "ortho_init": false,
27
+ "optimizer_class": "<class 'torch.optim.rmsprop.RMSprop'>",
28
+ "optimizer_kwargs": {
29
+ "alpha": 0.99,
30
+ "eps": 1e-05,
31
+ "weight_decay": 0
32
+ }
33
+ },
34
+ "observation_space": {
35
+ ":type:": "<class 'gym.spaces.box.Box'>",
36
+ ":serialized:": "gASVYwIAAAAAAACMDmd5bS5zcGFjZXMuYm94lIwDQm94lJOUKYGUfZQojAVkdHlwZZSMBW51bXB5lGgFk5SMAmY0lImIh5RSlChLA4wBPJROTk5K/////0r/////SwB0lGKMA2xvd5SMFW51bXB5LmNvcmUubXVsdGlhcnJheZSMDF9yZWNvbnN0cnVjdJSTlGgGjAduZGFycmF5lJOUSwCFlEMBYpSHlFKUKEsBSxiFlGgKiUNgAACA/wAAgP8AAID/AACA/wAAgP8AAID/AACA/wAAgP8AAID/AACA/wAAgP8AAID/AACA/wAAgP8AAID/AACA/wAAgP8AAID/AACA/wAAgP8AAID/AACA/wAAgP8AAID/lHSUYowEaGlnaJRoEGgSSwCFlGgUh5RSlChLAUsYhZRoColDYAAAgH8AAIB/AACAfwAAgH8AAIB/AACAfwAAgH8AAIB/AACAfwAAgH8AAIB/AACAfwAAgH8AAIB/AACAfwAAgH8AAIB/AACAfwAAgH8AAIB/AACAfwAAgH8AAIB/AACAf5R0lGKMDWJvdW5kZWRfYmVsb3eUaBBoEksAhZRoFIeUUpQoSwFLGIWUaAeMAmIxlImIh5RSlChLA4wBfJROTk5K/////0r/////SwB0lGKJQxgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACUdJRijA1ib3VuZGVkX2Fib3ZllGgQaBJLAIWUaBSHlFKUKEsBSxiFlGgoiUMYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAlHSUYowKX25wX3JhbmRvbZROjAZfc2hhcGWUSxiFlHViLg==",
37
+ "dtype": "float32",
38
+ "low": "[-inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf\n -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf]",
39
+ "high": "[inf inf inf inf inf inf inf inf inf inf inf inf inf inf inf inf inf inf\n inf inf inf inf inf inf]",
40
+ "bounded_below": "[False False False False False False False False False False False False\n False False False False False False False False False False False False]",
41
+ "bounded_above": "[False False False False False False False False False False False False\n False False False False False False False False False False False False]",
42
+ "_np_random": null,
43
+ "_shape": [
44
+ 24
45
+ ]
46
+ },
47
+ "action_space": {
48
+ ":type:": "<class 'gym.spaces.box.Box'>",
49
+ ":serialized:": "gASVKwwAAAAAAACMDmd5bS5zcGFjZXMuYm94lIwDQm94lJOUKYGUfZQojAVkdHlwZZSMBW51bXB5lGgFk5SMAmY0lImIh5RSlChLA4wBPJROTk5K/////0r/////SwB0lGKMA2xvd5SMFW51bXB5LmNvcmUubXVsdGlhcnJheZSMDF9yZWNvbnN0cnVjdJSTlGgGjAduZGFycmF5lJOUSwCFlEMBYpSHlFKUKEsBSwSFlGgKiUMQAACAvwAAgL8AAIC/AACAv5R0lGKMBGhpZ2iUaBBoEksAhZRoFIeUUpQoSwFLBIWUaAqJQxAAAIA/AACAPwAAgD8AAIA/lHSUYowNYm91bmRlZF9iZWxvd5RoEGgSSwCFlGgUh5RSlChLAUsEhZRoB4wCYjGUiYiHlFKUKEsDjAF8lE5OTkr/////Sv////9LAHSUYolDBAEBAQGUdJRijA1ib3VuZGVkX2Fib3ZllGgQaBJLAIWUaBSHlFKUKEsBSwSFlGgoiUMEAQEBAZR0lGKMCl9ucF9yYW5kb22UjBRudW1weS5yYW5kb20uX3BpY2tsZZSMEl9fcmFuZG9tc3RhdGVfY3RvcpSTlIwHTVQxOTkzN5SFlFKUfZQojA1iaXRfZ2VuZXJhdG9ylGg4jAVzdGF0ZZR9lCiMA2tleZRoEGgSSwCFlGgUh5RSlChLAU1wAoWUaAeMAnU0lImIh5RSlChLA2gLTk5OSv////9K/////0sAdJRiiULACQAAAAAAgFPCs5yHA7WTcuyrW+jmsvLPtoHa1QbvYaExTaBrtczZE+YYn3SceS/IpRuAxHkBSZ4PQ+Rd4BiTkhNVzNRKKWTSCJW+NNCeRHJ6A/Ctvwpzm3s+6WCxBtp1A0ULbn3WFmrrBDRmg7fz9aUvae9CY0O1XPfCd1LMRkQ3LQiJbtCOrnf7GLaAT3ue+U8y7BLYuY5ehakZyq49di1nK0KAnsXuxx/1IgFdS88kD8wZUmREV5hwnQr1Ehe16VreO8T/Qc86sV+2h90z2FiJqqLNf3s/yZS3bA9DHzGZfRtgkKu3Bx0ZTN7I+466APXPqRreJf4gBqUW/NB248FO9cpD0wFaed9QV63NqpiFOs/RYeEwD8e32dZFRfi9SK5aLvuApJMu2LOfZweypHIkyPjeY5W+tsm2bdPmVoCAOiyi73cf5k0LQeJNWqZU/wuy/f8myghZ9qrjf+2JSJMaB9VNMXWmxuq4Dq0fkhzJr1ML7SgftfSG29O5koFUMozQL58gAzyX96ZMcpWbZ+3/zlaVhGln5egXC8MtIK6xIFCvh/vD/F1jLgYIp14MM597MuPmTpa+OaKek7bql9Cp8/0skhg5QSCvCaijm8wenxrfqLyRxDPCpS+L3isJC5LrjFgWnsdxQXVrJK8uaobJcTIJ5NrYYfA2l5gH27iPI9EqhzFtZJXiE4vXpH18f3kouYV9RowPzOtmYsbmstR/Mx/VY7E0XBmnMidL4dYTUXgxaDbFxWy3y6miL2yw0I2O09vPWV7LqbwMbthlU26lrLfnJDz88B+7y4pFCwvsHgCsMWq6pvroAF4Ms+++JnhzrL0GLrtfJ7667p42Vg78GirmKIRMFH0p6aLRPV4V/fclE3PLBj0InU315M5v7fDFj/IreJeFUhIAy5/BlvLdgwIfBMBWvyjhqGRBo2KmdiDAO9BPhdi6oGmZUCxTDjUyMd8rjeRdntTE+L9sHQUSvUfpRjKcSIjZXcjsMMSwwU0QzIXewD4nZg8EAZ72iHnChWveW7cB5EQRFE+YPvvyNAw4OvORF+DjDtmLUz6jNz0JbkuKbhIQEEVbpDRkoeIGA4HATymZeE4TX/hvhOxJfX5liXD9/Uon8OjJ/hhIrDNUYR2NOV2R1L+VuK/JA9o0izGPiCC9vi7UK1u4re8tvPrBInC6BH2DpMvWSRepdH0frFxGsH/kRB/S77USeMs38a+yorDaa9Wsb9WnuQg10vl087jhvwSZx77fGq52NV1h/UnZAjnqJOjCPBgQrY0wsSeSW6SOSdItAOupiWkVvSzHz+gopd/3FUUPbjdik7Vd6O6ycGOFwp4+wyZqq1MKHww5exJND6De8NB4fGBzsvKoT8O1fAC3Y2Z+3mLYwkCJXULx2zNSO90bQi0L0BhTF1AidNtqRkak72V+V1kos6m4F+kyZiOiwxfcHKWHmpQe3A8HpX6nacx0Zso0WyF/W/sOLo//2g0YD9koIjgKvw//f9Em4CvxNBUMnJYTVQJGysF4QMShnM6byaMNJhRVALf+X0+SgLJkEwLfKdBbKxjA035OEc2YKmUpvooVUUt+9U8d7cKRgNYKKPwCHyYfLJSQt4ZEdThgjeywDxgsGzPax5SklOLroSYn39feofatZDzJXevfPTHyi1ZLwpe6Hwkbqz1FuOnHiShPedbEA9b/HCtBytSZrgUwlwHpQlAiBTSxpN7TFzVZrrhRd6N8W+FeE9SAGCHwXchFR4SCGThPyRGO/XWkMPJ42BLUOmMGepDQgWH646tjoJSE3EXcA5iDS+Nq6Oh521oC2UPAnvxIj1QbVh8IbxlBytRTTjc0c14E9cyhIPlgIoHriuEFSMXSEzOGL1MmL6UCbiXfsRg9Z6OwWCCl3VeGg5bEZ1kjJkvs08k7wtPk4ATAjaTL3QoY2gf106zFbJtL4D5gmLMJ9OuzE2Fn5uaAqqpjXIqqEXxS9jtpsRU9VTHCg68RwXQVIUhuVJgHq8fOigBMrW7Am5+jjo/GNNlcFcp813dFiXy4qHhjGSEjNXp3ln03NZkOgqXQ8SalJlPOvyrAS9wW9EtjQKhcrBSWSsQ8C3o/Mc/sR/CMRB85ZIojR/tiCKtOutxQMIusIOnYHK8g6kPpTQ8J/PfJ8pa3GEoYoA67axQTXsysd2Y6ZDwpz1HkAeISVK2AlCcuQssrQv8dVLAcins/2kjRFp5Vp82HSX9j6Ci9GH5mkdyqV84vWsdwRz4JNXHZoHVZKnrxSdA1HUwRUI/5oWiqnGk9KyiS4Mv9dQIVluJ1+/pAHYEdG9YgLHiNE2zA7aIQbqqlGX6jH1CrHSPL9mnlHdPiKjozwRXu02UQuzlGJn+/PUkU6cPYLLeLc8e7S2qfCZxbdpHioand7wYKqb5bMb8dA3Dwvm6P6iJXSogJ+Q+0z43li8ydYJqZNZlSjsljr/2c5UU33vMhLDLEXIHT8WzrsS45TsKNbhYfTYx6Ds/8W5yOtiOPWc0+fRlAjbQC++FKo5UaMl2eRCxI4U5/heX2HxJGNk50rICjuFsG/8Q+NUuJgS+y3FMhe+sm1e2MdC+ldkBqEn8oRxajECbP4Wizz1tfJliW/1A5fdGTfUxM3HV72bJgnqswmoAumAx6d36KfuZwEEp0/wcrdo+8/unJ5f2mYeqCrOcaxDJrs6SxW1zVaH/YTZl+RNA0NjPLgaqnlveaes/MkpzsVEQDtvKGFrG1cnmGjZVi2azrSDGQ0Y423nEksDC5awYcOJmVYbeA4DkMSNfj+7Dx2SzH/PVPuXLX9aw9K9QF4Ml48zsSrwsVjIa8+gIdffs2pf2wCKcPtFez6vOT0UDuFHknJjMDg0fI3DnyC7jJqO8V4XpmPyarTp3JJRGhmqTHhpZInn70JMfS+RFry5+rLSOM0T+KWV8fYjs9eyCiZijlR4AiADooXm9G8JIzZCLZX2Dty83iyz7gQzSxYO7ULuTT1stvGuJwbBP4LMhLXkbxdhAmBSDiYNOnc3O+yFsO6Ps9UOQD8S4Pbr8hZ4mFjbicpO635SwpmHINYDeuewln3/GHz69LpCjmpnKPeF9ZxXcq6MR4kJUV2j/dQzqjLniNaQmrMkULdI7W1sMXRFcsz9xs1GVwVqmtMVws8HtvXMYNmosCrrgAFX2ghPz7dXCV6vML5YhfNbDAzzG6MHffrslrhMav3vtlt8Fnld4VaH6IhMkowayT1lSVvfvlKHCWwtKaTcOZrR5LZGalJOpFbVIFUOAo+LnY/25bmc3KloyLzgiTudjPsXEGPNPBIvE/5cMEvU4Lrs0N3tCke4abYDXF9f14QrwLlHSUYowDcG9zlE1wAnWMCWhhc19nYXVzc5RLAIwFZ2F1c3OURwAAAAAAAAAAdWKMBl9zaGFwZZRLBIWUdWIu",
50
+ "dtype": "float32",
51
+ "low": "[-1. -1. -1. -1.]",
52
+ "high": "[1. 1. 1. 1.]",
53
+ "bounded_below": "[ True True True True]",
54
+ "bounded_above": "[ True True True True]",
55
+ "_np_random": "RandomState(MT19937)",
56
+ "_shape": [
57
+ 4
58
+ ]
59
+ },
60
+ "n_envs": 16,
61
+ "num_timesteps": 2260000,
62
+ "_total_timesteps": 5000000,
63
+ "_num_timesteps_at_start": 0,
64
+ "seed": 0,
65
+ "action_noise": null,
66
+ "start_time": 1614852199.497987,
67
+ "learning_rate": {
68
+ ":type:": "<class 'function'>",
69
+ ":serialized:": "gASVngMAAAAAAACMF2Nsb3VkcGlja2xlLmNsb3VkcGlja2xllIwNX2J1aWx0aW5fdHlwZZSTlIwKTGFtYmRhVHlwZZSFlFKUKGgCjAhDb2RlVHlwZZSFlFKUKEsBSwBLAUsCSxNDCHwAiAAUAFMAlIyECiAgICAgICAgUHJvZ3Jlc3Mgd2lsbCBkZWNyZWFzZSBmcm9tIDEgKGJlZ2lubmluZykgdG8gMAogICAgICAgIDpwYXJhbSBwcm9ncmVzc19yZW1haW5pbmc6IChmbG9hdCkKICAgICAgICA6cmV0dXJuOiAoZmxvYXQpCiAgICAgICAglIWUKYwScHJvZ3Jlc3NfcmVtYWluaW5nlIWUjFovdm9sdW1lL1VTRVJTVE9SRS9yYWZmX2FuL3Byb2plY3RzL2V4cGVyaW1lbnRzL3JlbGVhc2UxLjAvcmwtYmFzZWxpbmVzMy16b28vdXRpbHMvdXRpbHMucHmUjARmdW5jlEv+QwIABpSMDWluaXRpYWxfdmFsdWWUhZQpdJRSlH2UKIwLX19wYWNrYWdlX1+UjAV1dGlsc5SMCF9fbmFtZV9flIwLdXRpbHMudXRpbHOUjAhfX2ZpbGVfX5SMWi92b2x1bWUvVVNFUlNUT1JFL3JhZmZfYW4vcHJvamVjdHMvZXhwZXJpbWVudHMvcmVsZWFzZTEuMC9ybC1iYXNlbGluZXMzLXpvby91dGlscy91dGlscy5weZR1Tk5oAIwQX21ha2VfZW1wdHlfY2VsbJSTlClSlIWUdJRSlIwcY2xvdWRwaWNrbGUuY2xvdWRwaWNrbGVfZmFzdJSMEl9mdW5jdGlvbl9zZXRzdGF0ZZSTlGghfZR9lChoGGgPjAxfX3F1YWxuYW1lX1+UjB1saW5lYXJfc2NoZWR1bGUuPGxvY2Fscz4uZnVuY5SMD19fYW5ub3RhdGlvbnNfX5R9lCiMEnByb2dyZXNzX3JlbWFpbmluZ5SMCGJ1aWx0aW5zlIwFZmxvYXSUk5SMBnJldHVybpRoLnWMDl9fa3dkZWZhdWx0c19flE6MDF9fZGVmYXVsdHNfX5ROjApfX21vZHVsZV9flGgZjAdfX2RvY19flGgKjAtfX2Nsb3N1cmVfX5RoAIwKX21ha2VfY2VsbJSTlEc/T3UQTVUdaYWUUpSFlIwXX2Nsb3VkcGlja2xlX3N1Ym1vZHVsZXOUXZSMC19fZ2xvYmFsc19flH2UdYaUhlIwLg=="
70
+ },
71
+ "tensorboard_log": null,
72
+ "lr_schedule": {
73
+ ":type:": "<class 'function'>",
74
+ ":serialized:": "gASVngMAAAAAAACMF2Nsb3VkcGlja2xlLmNsb3VkcGlja2xllIwNX2J1aWx0aW5fdHlwZZSTlIwKTGFtYmRhVHlwZZSFlFKUKGgCjAhDb2RlVHlwZZSFlFKUKEsBSwBLAUsCSxNDCHwAiAAUAFMAlIyECiAgICAgICAgUHJvZ3Jlc3Mgd2lsbCBkZWNyZWFzZSBmcm9tIDEgKGJlZ2lubmluZykgdG8gMAogICAgICAgIDpwYXJhbSBwcm9ncmVzc19yZW1haW5pbmc6IChmbG9hdCkKICAgICAgICA6cmV0dXJuOiAoZmxvYXQpCiAgICAgICAglIWUKYwScHJvZ3Jlc3NfcmVtYWluaW5nlIWUjFovdm9sdW1lL1VTRVJTVE9SRS9yYWZmX2FuL3Byb2plY3RzL2V4cGVyaW1lbnRzL3JlbGVhc2UxLjAvcmwtYmFzZWxpbmVzMy16b28vdXRpbHMvdXRpbHMucHmUjARmdW5jlEv+QwIABpSMDWluaXRpYWxfdmFsdWWUhZQpdJRSlH2UKIwLX19wYWNrYWdlX1+UjAV1dGlsc5SMCF9fbmFtZV9flIwLdXRpbHMudXRpbHOUjAhfX2ZpbGVfX5SMWi92b2x1bWUvVVNFUlNUT1JFL3JhZmZfYW4vcHJvamVjdHMvZXhwZXJpbWVudHMvcmVsZWFzZTEuMC9ybC1iYXNlbGluZXMzLXpvby91dGlscy91dGlscy5weZR1Tk5oAIwQX21ha2VfZW1wdHlfY2VsbJSTlClSlIWUdJRSlIwcY2xvdWRwaWNrbGUuY2xvdWRwaWNrbGVfZmFzdJSMEl9mdW5jdGlvbl9zZXRzdGF0ZZSTlGghfZR9lChoGGgPjAxfX3F1YWxuYW1lX1+UjB1saW5lYXJfc2NoZWR1bGUuPGxvY2Fscz4uZnVuY5SMD19fYW5ub3RhdGlvbnNfX5R9lCiMEnByb2dyZXNzX3JlbWFpbmluZ5SMCGJ1aWx0aW5zlIwFZmxvYXSUk5SMBnJldHVybpRoLnWMDl9fa3dkZWZhdWx0c19flE6MDF9fZGVmYXVsdHNfX5ROjApfX21vZHVsZV9flGgZjAdfX2RvY19flGgKjAtfX2Nsb3N1cmVfX5RoAIwKX21ha2VfY2VsbJSTlEc/T3UQTVUdaYWUUpSFlIwXX2Nsb3VkcGlja2xlX3N1Ym1vZHVsZXOUXZSMC19fZ2xvYmFsc19flH2UdYaUhlIwLg=="
75
+ },
76
+ "_last_obs": null,
77
+ "_last_episode_starts": null,
78
+ "_last_original_obs": {
79
+ ":type:": "<class 'numpy.ndarray'>",
80
+ ":serialized:": "gASVjQwAAAAAAACMFW51bXB5LmNvcmUubXVsdGlhcnJheZSMDF9yZWNvbnN0cnVjdJSTlIwFbnVtcHmUjAduZGFycmF5lJOUSwCFlEMBYpSHlFKUKEsBSxBLGIaUaAOMBWR0eXBllJOUjAJmOJSJiIeUUpQoSwOMATyUTk5OSv////9K/////0sAdJRiiUIADAAAAAAAQPN/Zj8zMzMzswjaPh+F61EfhUq/9ihcj1BikL8AAADARaS3PwAAAEA/yF4/AAAAkKKE6z9VVVVVCc0XvwAAAAAAAPA/AAAAoIS+oD8AAAAg+8dePwAAABi7UOs/VVVV1ffxVL8AAAAAAADwPwAAACBKNtw/AAAAQE+I3D8AAACA8YfdPwAAAEDIVN8/AAAAoF0X4T8AAAAgW0fjPwAAAIBXseY/AAAAwItZ7D8AAAAAAADwPwAAAAAAAPA/AAAAYICBZj9I4XoUrhf5vj0K16PEf14/XI/C9UJikL8AAACgnoG3PwAAAODMH2S/AAAA4PqH6z8AAAAAYFxqPwAAAAAAAPA/AAAAoAWAoD8AAADAcR9kvwAAADgmU+s/AAAAAM12XD8AAAAAAADwPwAAAMBMNtw/AAAA4FGI3D8AAABA9IfdPwAAACDLVN8/AAAAQF8X4T8AAADgXEfjPwAAAKBZseY/AAAAgI5Z7D8AAAAAAADwPwAAAAAAAPA/AAAAQDN+Zj8pXI/CjSnnPtejcD2SmFe/9ihcj1JikL8AAAAAXa63PwAAAABtY2s/AAAAgESD6z8AAAAASJJUvwAAAAAAAPA/AAAA4EPNoD8AAABgHmNrPwAAAPgYUOs/VVVVVZiiYr8AAAAAAADwPwAAAGBJNtw/AAAAgE6I3D8AAACg8IfdPwAAAGDHVN8/AAAAIF0X4T8AAACgWkfjPwAAAOBWseY/AAAAAItZ7D8AAAAAAADwPwAAAAAAAPA/AAAAIIWAZj/Xo3A95m7TPnoUrkfIy0O/KVyPwk9ikL8AAABAAKG3PwAAAGA/+lY/AAAAgBWF6z9VVVVVH+AyPwAAAAAAAPA/AAAA4Ky5oD8AAACAE/pWPwAAAMjwUOs/q6qqKrZET78AAAAAAADwPwAAACBKNtw/AAAAQE+I3D8AAACA8YfdPwAAAEDIVN8/AAAAoF0X4T8AAAAgW0fjPwAAAIBXseY/AAAAwItZ7D8AAAAAAADwPwAAAAAAAPA/AAAA4AyCZj+PwvUoINWpvmZmZmZ6ZQ8/w/UoXE1ikL8AAAAAbZa3PwAAAICItxS/AAAAcH2G6z+rqqqqOPNXPwAAAAAAAPA/AAAA4NCpoD8AAABg6bYUvwAAAKChUes/AAAAAB5SDT8AAAAAAADwPwAAAKBLNtw/AAAAwFCI3D8AAAAA84fdPwAAAODJVN8/AAAAgF4X4T8AAAAgXEfjPwAAAMBYseY/AAAAYI1Z7D8AAAAAAADwPwAAAAAAAPA/AAAAAA2BZj/D9ShcL7HIPo/C9ShDJzm/MzMzM09ikL8AAAAAUJ23PwAAAOArMk0/AAAAmJeF6z8AAACAtsRGPwAAAAAAAPA/AAAAIGW0oD8AAACg6jFNPwAAAGArUes/AAAAwF3dQ78AAAAAAADwPwAAAEBLNtw/AAAAYFCI3D8AAACg8ofdPwAAAIDJVN8/AAAAQF4X4T8AAADgW0fjPwAAAGBYseY/AAAAAI1Z7D8AAAAAAADwPwAAAAAAAPA/AAAAAJWBZj/2KFyPJsW6PpqZmZleRCu/rkfhek5ikL8AAABAkZq3PwAAAAAYpj8/AAAAoPWF6z+rqqqqarNQPwAAAAAAAPA/AAAAoCawoD8AAACAEKY/PwAAAFBcUes/AAAAgGCINb8AAAAAAADwPwAAAEBLNtw/AAAAYFCI3D8AAACg8ofdPwAAAIDJVN8/AAAAQF4X4T8AAADgW0fjPwAAAGBYseY/AAAAAI1Z7D8AAAAAAADwPwAAAAAAAPA/AAAAoHR/Zj8K16NwxXbfPuF6FK7GBlC/exSuR1FikL8AAABA4aa3PwAAAGAummI/AAAAIEqE6z9VVVVVlWE6vwAAAAAAAPA/AAAA4GTCoD8AAABA+pliPwAAAHiRUOs/q6qqql1QWb8AAAAAAADwPwAAACBKNtw/AAAAQE+I3D8AAACA8YfdPwAAAEDIVN8/AAAAoF0X4T8AAAAgW0fjPwAAAIBXseY/AAAAwItZ7D8AAAAAAADwPwAAAAAAAPA/AAAAgB1+Zj8pXI/C7cPnPuxRuB7BNVi/zczMzFJikL8AAABA7663PwAAAIDaGWw/AAAAEC+D6z9VVVVVW7RVvwAAAAAAAPA/AAAAICrOoD8AAACgkRlsPwAAAOgNUOs/VVVVFbweY78AAAAAAADwPwAAAGBJNtw/AAAAgE6I3D8AAACg8IfdPwAAAGDHVN8/AAAAIF0X4T8AAACgWkfjPwAAAOBWseY/AAAAAItZ7D8AAAAAAADwPwAAAAAAAPA/AAAAoNJ9Zj89CtejDOTpPkjhehQOYFq/exSuR1NikL8AAAAAI7G3PwAAAKA9nW4/AAAAUOGC6z9VVVVVmbNZvwAAAAAAAPA/AAAA4H3RoD8AAABA7pxuPwAAAJDmT+s/VVVVVXjUZL8AAAAAAADwPwAAAABJNtw/AAAAIE6I3D8AAABA8IfdPwAAAADHVN8/AAAA4FwX4T8AAABAWkfjPwAAAKBWseY/AAAAoIpZ7D8AAAAAAADwPwAAAAAAAPA/AAAAQNCAZj+kcD0K/yzNPsH1KFyKuD2/CtejcE9ikL8AAAAAcJ63PwAAAMCuP1E/AAAAEHGF6z+rqqqqEo1CPwAAAAAAAPA/AAAAYNm1oD8AAAAgiT9RPwAAAJAeUes/VVVVVQF5R78AAAAAAADwPwAAAOBKNtw/AAAAAFCI3D8AAABA8ofdPwAAACDJVN8/AAAAIF4X4T8AAACgW0fjPwAAACBYseY/AAAAoIxZ7D8AAAAAAADwPwAAAAAAAPA/AAAA4B9/Zj/D9Shc6/PhPuF6FK7YSVK/4XoUrlFikL8AAADgRqm3PwAAAIBKOmU/AAAA6POD6z8AAAAAe4tFvwAAAAAAAPA/AAAAYMnFoD8AAACAGjplPwAAAJBrUOs/VVVVVRrjXL8AAAAAAADwPwAAAMBJNtw/AAAA4E6I3D8AAAAg8YfdPwAAAODHVN8/AAAAYF0X4T8AAADgWkfjPwAAAEBXseY/AAAAYItZ7D8AAAAAAADwPwAAAAAAAPA/AAAAgCeBZj8UrkfhuhjGPuxRuB4Egja/XI/C9U5ikL8AAAAgrJy3PwAAAID9H0o/AAAAyK2F6z8AAAAAADZJPwAAAAAAAPA/AAAA4EazoD8AAABA4B9KPwAAAAA6Ues/AAAAwMTGQb8AAAAAAADwPwAAAEBLNtw/AAAAYFCI3D8AAACg8ofdPwAAAIDJVN8/AAAAQF4X4T8AAADgW0fjPwAAAGBYseY/AAAAAI1Z7D8AAAAAAADwPwAAAAAAAPA/AAAAgNmBZj+amZmZCZLjvqRwPQppyUc/UrgehUlikL8AAABAvo63PwAAAID1Y0+/AAAA0AiH6z9VVVVVAEthPwAAAAAAAPA/AAAAIDaaoD8AAACAXWNPvwAAABAzUus/q6qqqhUzRj8AAAAAAADwPwAAAABMNtw/AAAAIFGI3D8AAABg84fdPwAAAGDKVN8/AAAAwF4X4T8AAABgXEfjPwAAAABZseY/AAAAwI1Z7D8AAAAAAADwPwAAAAAAAPA/AAAAwJeBZj89CtejSCf4vuxRuB6TW10/CtejcENikL8AAAAAfYK3PwAAAIAAX2O/AAAAMOeH6z8AAACAA85pPwAAAAAAAPA/AAAAIL+BoD8AAAAAqV5jvwAAAJgTU+s/q6qqqhpmWz8AAAAAAADwPwAAAGBMNtw/AAAAgFGI3D8AAADg84fdPwAAAMDKVN8/AAAAAF8X4T8AAACgXEfjPwAAAGBZseY/AAAAII5Z7D8AAAAAAADwPwAAAAAAAPA/AAAAoFt/Zj8K16NwX2vgPh+F61ESulC/MzMzM1FikL8AAABAlqe3PwAAAOBMamM/AAAAWDGE6z+rqqqqhI4/vwAAAAAAAPA/AAAAYG7DoD8AAACgFGpjPwAAAHiFUOs/VVVVVZprWr8AAAAAAADwPwAAACBKNtw/AAAAQE+I3D8AAACA8YfdPwAAAEDIVN8/AAAAoF0X4T8AAAAgW0fjPwAAAIBXseY/AAAAwItZ7D8AAAAAAADwPwAAAAAAAPA/lHSUYi4="
81
+ },
82
+ "_episode_num": 0,
83
+ "use_sde": true,
84
+ "sde_sample_freq": -1,
85
+ "_current_progress_remaining": 0.5480064,
86
+ "ep_info_buffer": {
87
+ ":type:": "<class 'collections.deque'>",
88
+ ":serialized:": "gASVgBAAAAAAAACMC2NvbGxlY3Rpb25zlIwFZGVxdWWUk5QpS2SGlFKUKH2UKIwBcpSMFW51bXB5LmNvcmUubXVsdGlhcnJheZSMBnNjYWxhcpSTlIwFbnVtcHmUjAVkdHlwZZSTlIwCZjiUiYiHlFKUKEsDjAE8lE5OTkr/////Sv////9LAHSUYkMIZw5JLRS/ckCUhpRSlIwBbJRNBQOMAXSUR0Clc9xJ2+wldX2UKGgGaAloD0MIpu81BEfBckCUhpRSlGgVTf0CaBZHQKV0p+F10T11fZQoaAZoCWgPQwjknxnEB8NyQJSGlFKUaBVN8wJoFkdApYaRWxQizXV9lChoBmgJaA9DCLLxYIvdqXJAlIaUUpRoFU0GA2gWR0ClhuHyNGVidX2UKGgGaAloD0MIXOSeri7gckCUhpRSlGgVTdoCaBZHQKWIMkt29td1fZQoaAZoCWgPQwgNpmH4iLhPQJSGlFKUaBVNvAFoFkdApYhc/D+BH3V9lChoBmgJaA9DCM9r7BKVrHJAlIaUUpRoFU0gA2gWR0CliGKQaJhwdX2UKGgGaAloD0MIwQKYMnDbckCUhpRSlGgVTd0CaBZHQKWIlFtKqXF1fZQoaAZoCWgPQwiNmxpovrhyQJSGlFKUaBVN9wJoFkdApYjHuqm0mnV9lChoBmgJaA9DCLDkKhY/hnJAlIaUUpRoFU0pA2gWR0CliNWYOUdJdX2UKGgGaAloD0MI5BHcSFmickCUhpRSlGgVTQwDaBZHQKWJ9MIu5Bl1fZQoaAZoCWgPQwj7A+W2vcByQJSGlFKUaBVN9AJoFkdApYnQh8pkPXV9lChoBmgJaA9DCBu7RPWW2XJAlIaUUpRoFU3nAmgWR0ClioIPkJa8dX2UKGgGaAloD0MIlBRYANPSckCUhpRSlGgVTekCaBZHQKWKT4nndO91fZQoaAZoCWgPQwgWGLK6ldByQJSGlFKUaBVN8AJoFkdApYp83VCoj3V9lChoBmgJaA9DCOaTFcPVyHJAlIaUUpRoFU3zAmgWR0ClipcPWhAXdX2UKGgGaAloD0MI6SlyiDjZckCUhpRSlGgVTfICaBZHQKWK2LZSNwR1fZQoaAZoCWgPQwgxzt+EAs5yQJSGlFKUaBVN6wJoFkdApZ1ra0x/NXV9lChoBmgJaA9DCGkZqfcUsXJAlIaUUpRoFU39AmgWR0ClneU+kgwHdX2UKGgGaAloD0MIbypSYSzRckCUhpRSlGgVTfoCaBZHQKWeLsDW9UV1fZQoaAZoCWgPQwg/dEF9C9FyQJSGlFKUaBVN6wJoFkdApZ9b1h9b5nV9lChoBmgJaA9DCPopjgNv0HJAlIaUUpRoFU3dAmgWR0Cln3LThHbzdX2UKGgGaAloD0MIaafmcsOvckCUhpRSlGgVTQEDaBZHQKWfrw2l2vB1fZQoaAZoCWgPQwiNs+kI4MRyQJSGlFKUaBVN9QJoFkdApZ/RUaQ3gnV9lChoBmgJaA9DCPWdX5RguXJAlIaUUpRoFU0AA2gWR0CloBje9Ba+dX2UKGgGaAloD0MIiujX1k+rckCUhpRSlGgVTQcDaBZHQKWgMq2jO9p1fZQoaAZoCWgPQwi7Cik/qUNdQJSGlFKUaBVNJQJoFkdApaA8/wAlwHV9lChoBmgJaA9DCBcOhGTB23JAlIaUUpRoFU3XAmgWR0CloP1pTMq0dX2UKGgGaAloD0MIF9NM9zodYUCUhpRSlGgVTUUCaBZHQKWg0bo8p1B1fZQoaAZoCWgPQwgd5ssLsCFNQJSGlFKUaBVNwwFoFkdApaEhllK9PHV9lChoBmgJaA9DCHXpX5KK9HJAlIaUUpRoFU3NAmgWR0ClslqjzqbCdX2UKGgGaAloD0MIELBW7ZqnckCUhpRSlGgVTQcDaBZHQKWyEtRvWH11fZQoaAZoCWgPQwgW+IpuvbBeQJSGlFKUaBVNMgJoFkdApbJZF9a2W3V9lChoBmgJaA9DCAUx0LVvu3JAlIaUUpRoFU3vAmgWR0ClsmfA0sOHdX2UKGgGaAloD0MIMGe2KzTdckCUhpRSlGgVTdsCaBZHQKWyh+DvmYB1fZQoaAZoCWgPQwhoWIy6Vo1yQJSGlFKUaBVNCgNoFkdApbSjd8Aq/nV9lChoBmgJaA9DCLYsX5fhv3JAlIaUUpRoFU3YAmgWR0CltZASvkimdX2UKGgGaAloD0MI56kOudmsckCUhpRSlGgVTfICaBZHQKW1qWAPNFB1fZQoaAZoCWgPQwiduvJZXsRyQJSGlFKUaBVN4QJoFkdApbXc4T9KmXV9lChoBmgJaA9DCCzvqgeM0HJAlIaUUpRoFU3XAmgWR0ClteoN/e+FdX2UKGgGaAloD0MIIGCt2jWqckCUhpRSlGgVTeoCaBZHQKW2VtWMju91fZQoaAZoCWgPQwjq6o7Fdq5yQJSGlFKUaBVN8AJoFkdApbZ4+2VmjHV9lChoBmgJaA9DCPHYz2Jpv3JAlIaUUpRoFU3oAmgWR0ClyO3HzYmLdX2UKGgGaAloD0MIOWBXkyfCckCUhpRSlGgVTdkCaBZHQKXIpT1kDp11fZQoaAZoCWgPQwgWwf9WsmNyQJSGlFKUaBVNLQNoFkdApcisx7AtWnV9lChoBmgJaA9DCIoe+BjsynJAlIaUUpRoFU3bAmgWR0ClyPmDL8rJdX2UKGgGaAloD0MIbLJGPYSxckCUhpRSlGgVTe0CaBZHQKXJK/j81oB1fZQoaAZoCWgPQwjyzTY35q1yQJSGlFKUaBVN7gJoFkdApcl2yJKraXV9lChoBmgJaA9DCJeuYBtxxnJAlIaUUpRoFU3nAmgWR0ClyXkPtlZpdX2UKGgGaAloD0MIHQHcLN7IckCUhpRSlGgVTesCaBZHQKXJnzzVc2R1fZQoaAZoCWgPQwi+9zdor4RyQJSGlFKUaBVNGANoFkdApcnHm/336HV9lChoBmgJaA9DCL4ViQnq5WZAlIaUUpRoFU2yAmgWR0Cly1a+vhZRdX2UKGgGaAloD0MIFqQZi+a+ckCUhpRSlGgVTe8CaBZHQKXMrJ5E+gV1fZQoaAZoCWgPQwi610l9mchyQJSGlFKUaBVN6gJoFkdApczvbblA/3V9lChoBmgJaA9DCG5t4XnprnJAlIaUUpRoFU0FA2gWR0ClzPFMh5gPdX2UKGgGaAloD0MILnB5rBnEckCUhpRSlGgVTekCaBZHQKXeFIHTqjd1fZQoaAZoCWgPQwh72AsFrNNyQJSGlFKUaBVN5QJoFkdApd5vai9Iw3V9lChoBmgJaA9DCMalKm2xx3JAlIaUUpRoFU3qAmgWR0Cl3pch9srNdX2UKGgGaAloD0MIfqzgtyHAckCUhpRSlGgVTeYCaBZHQKXe9eJpFkR1fZQoaAZoCWgPQwjUEFX4c7dyQJSGlFKUaBVN8AJoFkdApd8Q5eZ5RnV9lChoBmgJaA9DCICeBgzSoHJAlIaUUpRoFU0LA2gWR0Cl33+o1k1/dX2UKGgGaAloD0MI8Il1qvzPckCUhpRSlGgVTeECaBZHQKXftFKCg9N1fZQoaAZoCWgPQwh9eJYg451yQJSGlFKUaBVNEwNoFkdApd+W8ujASHV9lChoBmgJaA9DCHoAi/w6xnJAlIaUUpRoFU3sAmgWR0Cl338QZn+RdX2UKGgGaAloD0MInfF9cWnAckCUhpRSlGgVTfQCaBZHQKXfzabnX/Z1fZQoaAZoCWgPQwiZR/5goIlyQJSGlFKUaBVNHANoFkdApeAyQRwqAnV9lChoBmgJaA9DCJENpIuNmnJAlIaUUpRoFU0JA2gWR0Cl4DOwgTysdX2UKGgGaAloD0MIyoegavS5ckCUhpRSlGgVTfUCaBZHQKXheqebutx1fZQoaAZoCWgPQwjoo4y4ALFUQJSGlFKUaBVNEAJoFkdApfL2skpqh3V9lChoBmgJaA9DCMmvH2LDwXJAlIaUUpRoFU3tAmgWR0Cl80Xl0YCRdX2UKGgGaAloD0MIglfLnVm2ckCUhpRSlGgVTe8CaBZHQKXzhgCwKSh1fZQoaAZoCWgPQwgmHeVg9sZyQJSGlFKUaBVN5wJoFkdApfN4TIvJzXV9lChoBmgJaA9DCDpbQGg9n3JAlIaUUpRoFU0JA2gWR0Cl87xXfZVXdX2UKGgGaAloD0MIvJS6ZNzOckCUhpRSlGgVTeICaBZHQKXz2FXaJyh1fZQoaAZoCWgPQwh7Mv/o25dyQJSGlFKUaBVNBwNoFkdApfQ8rbxmTXV9lChoBmgJaA9DCNh9x/CYlHJAlIaUUpRoFU0NA2gWR0Cl9LuKfnOjdX2UKGgGaAloD0MIQrRWtDmbckCUhpRSlGgVTQgDaBZHQKX1It7KJVN1fZQoaAZoCWgPQwhE+BdBI7JyQJSGlFKUaBVN7wJoFkdApfUttdiUgXV9lChoBmgJaA9DCEQ2kC72r3JAlIaUUpRoFU31AmgWR0Cl9Rve54GEdX2UKGgGaAloD0MIu5o8ZTWkckCUhpRSlGgVTfQCaBZHQKX1AtdRiw11fZQoaAZoCWgPQwgeFf93RJRyQJSGlFKUaBVN/wJoFkdApfVmBMBZIXV9lChoBmgJaA9DCD9wlSfQtXJAlIaUUpRoFU3vAmgWR0Cl9bILgGbDdX2UKGgGaAloD0MIHjNQGb+hckCUhpRSlGgVTfcCaBZHQKX1v/WDpTx1fZQoaAZoCWgPQwgiqYWSCbVyQJSGlFKUaBVN5QJoFkdApggMM1CPZXV9lChoBmgJaA9DCM4WEFpPsnJAlIaUUpRoFU3yAmgWR0CmCXJSrHU+dX2UKGgGaAloD0MImIqNeV2QckCUhpRSlGgVTf4CaBZHQKYJOw35vcd1fZQoaAZoCWgPQwh1IVZ/RKByQJSGlFKUaBVN+wJoFkdApgnK2fChvnV9lChoBmgJaA9DCOIC0CidmHJAlIaUUpRoFU0JA2gWR0CmCdUZvUBodX2UKGgGaAloD0MIfcoxWRzEckCUhpRSlGgVTeQCaBZHQKYJ3ApKBd51fZQoaAZoCWgPQwjikXh5OqxyQJSGlFKUaBVN9wJoFkdApgoZtUGVzXV9lChoBmgJaA9DCMHmHDyTvHJAlIaUUpRoFU3sAmgWR0CmCnWAf+0gdX2UKGgGaAloD0MIB2Fu93ItXsCUhpRSlGgVS0VoFkdApgrwy44IbHV9lChoBmgJaA9DCDAsf74tunJAlIaUUpRoFU3lAmgWR0CmCvaqbSZ0dX2UKGgGaAloD0MIHAjJAmbGckCUhpRSlGgVTekCaBZHQKYLZ2ki2Ul1fZQoaAZoCWgPQwjMCG8PgspyQJSGlFKUaBVN4AJoFkdApgtPk7wKB3V9lChoBmgJaA9DCA1xrIsbw3JAlIaUUpRoFU3fAmgWR0CmCzZFw1iwdX2UKGgGaAloD0MIT5SERJq/ckCUhpRSlGgVTfECaBZHQKYLhsWO6up1fZQoaAZoCWgPQwjbhlEQvJdyQJSGlFKUaBVNAwNoFkdApgvlGy5ZsHV9lChoBmgJaA9DCAN7TKR0qHJAlIaUUpRoFU30AmgWR0CmDBp4rz5HdX2UKGgGaAloD0MIWMUbmQe4ckCUhpRSlGgVTegCaBZHQKYMGCI1tO51ZS4="
89
+ },
90
+ "ep_success_buffer": {
91
+ ":type:": "<class 'collections.deque'>",
92
+ ":serialized:": "gASVIAAAAAAAAACMC2NvbGxlY3Rpb25zlIwFZGVxdWWUk5QpS2SGlFKULg=="
93
+ },
94
+ "_n_updates": 17656,
95
+ "n_steps": 8,
96
+ "gamma": 0.99,
97
+ "gae_lambda": 0.9,
98
+ "ent_coef": 0.0,
99
+ "vf_coef": 0.4,
100
+ "max_grad_norm": 0.5,
101
+ "normalize_advantage": false,
102
+ "_last_dones": {
103
+ ":type:": "<class 'numpy.ndarray'>",
104
+ ":serialized:": "gASVmAAAAAAAAACMFW51bXB5LmNvcmUubXVsdGlhcnJheZSMDF9yZWNvbnN0cnVjdJSTlIwFbnVtcHmUjAduZGFycmF5lJOUSwCFlEMBYpSHlFKUKEsBSxCFlGgDjAVkdHlwZZSTlIwCYjGUiYiHlFKUKEsDjAF8lE5OTkr/////Sv////9LAHSUYolDEAAAAAAAAAAAAAAAAAAAAACUdJRiLg=="
105
+ }
106
+ }
a2c-BipedalWalker-v3/policy.optimizer.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:268efb6b46ae61c88c8dcfe070f6074046132c70799d0c39ace71818dbc4f28f
3
+ size 52030
a2c-BipedalWalker-v3/policy.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8227e543498d31c3b74832f8794efeaae4c630d4ddb8bd1d51c333326143afa1
3
+ size 52670
a2c-BipedalWalker-v3/pytorch_variables.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d030ad8db708280fcae77d87e973102039acd23a11bdecc3db8eb6c0ac940ee1
3
+ size 431
a2c-BipedalWalker-v3/system_info.txt ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ OS: Linux-5.13.0-44-generic-x86_64-with-debian-bullseye-sid #49~20.04.1-Ubuntu SMP Wed May 18 18:44:28 UTC 2022
2
+ Python: 3.7.10
3
+ Stable-Baselines3: 1.5.1a8
4
+ PyTorch: 1.11.0
5
+ GPU Enabled: True
6
+ Numpy: 1.21.2
7
+ Gym: 0.21.0
args.yml ADDED
@@ -0,0 +1,59 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ !!python/object/apply:collections.OrderedDict
2
+ - - - algo
3
+ - a2c
4
+ - - env
5
+ - BipedalWalker-v3
6
+ - - env_kwargs
7
+ - null
8
+ - - eval_episodes
9
+ - 10
10
+ - - eval_freq
11
+ - 10000
12
+ - - gym_packages
13
+ - []
14
+ - - hyperparams
15
+ - null
16
+ - - log_folder
17
+ - rl-trained-agents/
18
+ - - log_interval
19
+ - -1
20
+ - - n_evaluations
21
+ - 20
22
+ - - n_jobs
23
+ - 1
24
+ - - n_startup_trials
25
+ - 10
26
+ - - n_timesteps
27
+ - -1
28
+ - - n_trials
29
+ - 10
30
+ - - num_threads
31
+ - -1
32
+ - - optimize_hyperparameters
33
+ - false
34
+ - - pruner
35
+ - median
36
+ - - sampler
37
+ - tpe
38
+ - - save_freq
39
+ - -1
40
+ - - save_replay_buffer
41
+ - false
42
+ - - seed
43
+ - 3598227847
44
+ - - storage
45
+ - null
46
+ - - study_name
47
+ - null
48
+ - - tensorboard_log
49
+ - ''
50
+ - - trained_agent
51
+ - ''
52
+ - - truncate_last_trajectory
53
+ - true
54
+ - - uuid
55
+ - false
56
+ - - vec_env
57
+ - subproc
58
+ - - verbose
59
+ - 1
config.yml ADDED
@@ -0,0 +1,31 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ !!python/object/apply:collections.OrderedDict
2
+ - - - ent_coef
3
+ - 0.0
4
+ - - gae_lambda
5
+ - 0.9
6
+ - - gamma
7
+ - 0.99
8
+ - - learning_rate
9
+ - lin_0.00096
10
+ - - max_grad_norm
11
+ - 0.5
12
+ - - n_envs
13
+ - 16
14
+ - - n_steps
15
+ - 8
16
+ - - n_timesteps
17
+ - 5000000.0
18
+ - - normalize
19
+ - true
20
+ - - normalize_advantage
21
+ - false
22
+ - - policy
23
+ - MlpPolicy
24
+ - - policy_kwargs
25
+ - dict(log_std_init=-2, ortho_init=False)
26
+ - - use_rms_prop
27
+ - true
28
+ - - use_sde
29
+ - true
30
+ - - vf_coef
31
+ - 0.4
env_kwargs.yml ADDED
@@ -0,0 +1 @@
 
 
1
+ {}
replay.mp4 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:61c185675a76aac58f416dbdc9498499d4b504cd74b4ea3c11c2f8fe85ceba6c
3
+ size 434169
results.json ADDED
@@ -0,0 +1 @@
 
 
1
+ {"mean_reward": 300.8528041999999, "std_reward": 1.4428884079717876, "is_deterministic": true, "n_eval_episodes": 10, "eval_datetime": "2022-06-02T18:06:00.128574"}
train_eval_metrics.zip ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:73b8b2e1247db5816f83a3c811ccb80f71fc87132e41ed4be3ea930353619467
3
+ size 132197
vec_normalize.pkl ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:32daf605898f4b73839fd068d54c4535d603ea9233cbde2c56e1657348640d28
3
+ size 8055