Hugging Face's logo

justinlangseth
/
dqn-Sailing-v1c

Reinforcement Learning
stable-baselines3
Sailing-v1
deep-reinforcement-learning
Eval Results
Model card Files Community
justinlangseth commited on
Commit
cd2460e
·
1 Parent(s): e7b04f6

Initial commit

Browse files
Files changed (13) hide show
  1. .gitattributes +1 -0
  2. README.md +77 -0
  3. args.yml +105 -0
  4. config.yml +31 -0
  5. dqn-Sailing-v1.zip +3 -0
  6. dqn-Sailing-v1/_stable_baselines3_version +1 -0
  7. dqn-Sailing-v1/data +127 -0
  8. dqn-Sailing-v1/policy.optimizer.pth +3 -0
  9. dqn-Sailing-v1/policy.pth +3 -0
  10. dqn-Sailing-v1/pytorch_variables.pth +3 -0
  11. dqn-Sailing-v1/system_info.txt +7 -0
  12. env_kwargs.yml +1 -0
  13. train_eval_metrics.zip +3 -0
.gitattributes CHANGED
@@ -30,3 +30,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
30
  *.zip filter=lfs diff=lfs merge=lfs -text
31
  *.zst filter=lfs diff=lfs merge=lfs -text
32
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
30
  *.zip filter=lfs diff=lfs merge=lfs -text
31
  *.zst filter=lfs diff=lfs merge=lfs -text
32
  *tfevents* filter=lfs diff=lfs merge=lfs -text
33
+ *.mp4 filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,77 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: stable-baselines3
3
+ tags:
4
+ - Sailing-v1
5
+ - deep-reinforcement-learning
6
+ - reinforcement-learning
7
+ - stable-baselines3
8
+ model-index:
9
+ - name: DQN
10
+ results:
11
+ - task:
12
+ type: reinforcement-learning
13
+ name: reinforcement-learning
14
+ dataset:
15
+ name: Sailing-v1
16
+ type: Sailing-v1
17
+ metrics:
18
+ - type: mean_reward
19
+ value: 100.00 +/- 100.00
20
+ name: mean_reward
21
+ verified: false
22
+ ---
23
+
24
+ # **DQN** Agent playing **Sailing-v1**
25
+ This is a trained model of a **DQN** agent playing **Sailing-v1**
26
+ using the [stable-baselines3 library](https://github.com/DLR-RM/stable-baselines3)
27
+ and the [RL Zoo](https://github.com/DLR-RM/rl-baselines3-zoo).
28
+
29
+ The RL Zoo is a training framework for Stable Baselines3
30
+ reinforcement learning agents,
31
+ with hyperparameter optimization and pre-trained agents included.
32
+
33
+ ## Usage (with SB3 RL Zoo)
34
+
35
+ RL Zoo: https://github.com/DLR-RM/rl-baselines3-zoo<br/>
36
+ SB3: https://github.com/DLR-RM/stable-baselines3<br/>
37
+ SB3 Contrib: https://github.com/Stable-Baselines-Team/stable-baselines3-contrib
38
+
39
+ ```
40
+ # Download model and save it into the logs/ folder
41
+ python -m rl_zoo3.load_from_hub --algo dqn --env Sailing-v1 -orga justinlangseth -f logs/
42
+ python enjoy.py --algo dqn --env Sailing-v1 -f logs/
43
+ ```
44
+
45
+ If you installed the RL Zoo3 via pip (`pip install rl_zoo3`), from anywhere you can do:
46
+ ```
47
+ python -m rl_zoo3.load_from_hub --algo dqn --env Sailing-v1 -orga justinlangseth -f logs/
48
+ rl_zoo3 enjoy --algo dqn --env Sailing-v1 -f logs/
49
+ ```
50
+
51
+ ## Training (with the RL Zoo)
52
+ ```
53
+ python train.py --algo dqn --env Sailing-v1 -f logs/
54
+ # Upload the model and generate video (when possible)
55
+ python -m rl_zoo3.push_to_hub --algo dqn --env Sailing-v1 -f logs/ -orga justinlangseth
56
+ ```
57
+
58
+ ## Hyperparameters
59
+ ```python
60
+ OrderedDict([('batch_size', 128),
61
+ ('buffer_size', 125000),
62
+ ('exploration_final_eps', 0.01),
63
+ ('exploration_fraction', 0.25),
64
+ ('gamma', 0.99),
65
+ ('gradient_steps', 1),
66
+ ('learning_rate', 0.00063),
67
+ ('learning_starts', 0),
68
+ ('n_envs', 24),
69
+ ('n_timesteps', 50000),
70
+ ('policy', 'MlpPolicy'),
71
+ ('policy_kwargs',
72
+ 'dict(net_arch=[512, 256, 128, 128], activation_fn=nn.ReLU)'),
73
+ ('target_update_interval', 250),
74
+ ('train_freq', 4),
75
+ ('vec_env_wrapper', 'stable_baselines3.common.vec_env.VecMonitor'),
76
+ ('normalize', False)])
77
+ ```
args.yml ADDED
@@ -0,0 +1,105 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ !!python/object/apply:collections.OrderedDict
2
+ - - - account
3
+ - sfsenorthamerica_cleanroom01
4
+ - - algo
5
+ - dqn
6
+ - - data_table
7
+ - sail_train_log
8
+ - - database
9
+ - sailing
10
+ - - device
11
+ - auto
12
+ - - env
13
+ - Sailing-v1
14
+ - - env_kwargs
15
+ - null
16
+ - - eval_episodes
17
+ - 15
18
+ - - eval_freq
19
+ - 25000
20
+ - - gym_packages
21
+ - []
22
+ - - host_ip
23
+ - null
24
+ - - host_port
25
+ - null
26
+ - - hyperparams
27
+ - n_timesteps: 50000
28
+ - - log_folder
29
+ - logs
30
+ - - log_interval
31
+ - -1
32
+ - - max_total_trials
33
+ - null
34
+ - - model_file_prefix
35
+ - sail_train_
36
+ - - model_stage
37
+ - sail_models
38
+ - - n_eval_envs
39
+ - 15
40
+ - - n_evaluations
41
+ - null
42
+ - - n_jobs
43
+ - 1
44
+ - - n_startup_trials
45
+ - 10
46
+ - - n_timesteps
47
+ - -1
48
+ - - n_trials
49
+ - 500
50
+ - - no_optim_plots
51
+ - false
52
+ - - node_num
53
+ - 0
54
+ - - num_threads
55
+ - 2
56
+ - - optimization_log_path
57
+ - null
58
+ - - optimize_hyperparameters
59
+ - false
60
+ - - password
61
+ - Anow03773hdhdhs
62
+ - - progress
63
+ - false
64
+ - - protocol
65
+ - https
66
+ - - pruner
67
+ - median
68
+ - - sampler
69
+ - tpe
70
+ - - save_freq
71
+ - -1
72
+ - - save_replay_buffer
73
+ - true
74
+ - - schema
75
+ - public
76
+ - - seed
77
+ - 2152775171
78
+ - - storage
79
+ - null
80
+ - - study_name
81
+ - null
82
+ - - tensorboard_log
83
+ - ''
84
+ - - track
85
+ - false
86
+ - - trained_agent
87
+ - ''
88
+ - - truncate_last_trajectory
89
+ - true
90
+ - - username
91
+ - jlangseth
92
+ - - uuid
93
+ - false
94
+ - - vec_env
95
+ - dummy
96
+ - - verbose
97
+ - 1
98
+ - - wandb_entity
99
+ - null
100
+ - - wandb_project_name
101
+ - sb3
102
+ - - warehouse
103
+ - app_wh
104
+ - - yaml_file
105
+ - null
config.yml ADDED
@@ -0,0 +1,31 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ !!python/object/apply:collections.OrderedDict
2
+ - - - batch_size
3
+ - 128
4
+ - - buffer_size
5
+ - 125000
6
+ - - exploration_final_eps
7
+ - 0.01
8
+ - - exploration_fraction
9
+ - 0.25
10
+ - - gamma
11
+ - 0.99
12
+ - - gradient_steps
13
+ - 1
14
+ - - learning_rate
15
+ - 0.00063
16
+ - - learning_starts
17
+ - 0
18
+ - - n_envs
19
+ - 24
20
+ - - n_timesteps
21
+ - 50000
22
+ - - policy
23
+ - MlpPolicy
24
+ - - policy_kwargs
25
+ - dict(net_arch=[512, 256, 128, 128], activation_fn=nn.ReLU)
26
+ - - target_update_interval
27
+ - 250
28
+ - - train_freq
29
+ - 4
30
+ - - vec_env_wrapper
31
+ - stable_baselines3.common.vec_env.VecMonitor
dqn-Sailing-v1.zip ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c5a45290a4c318dd45d5e862a15893e93f808b3816159ee226f58c0e0137f742
3
+ size 3038214
dqn-Sailing-v1/_stable_baselines3_version ADDED
@@ -0,0 +1 @@
 
 
1
+ 1.6.2
dqn-Sailing-v1/data ADDED
@@ -0,0 +1,127 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "policy_class": {
3
+ ":type:": "<class 'abc.ABCMeta'>",
4
+ ":serialized:": "gAWVMAAAAAAAAACMHnN0YWJsZV9iYXNlbGluZXMzLmRxbi5wb2xpY2llc5SMCURRTlBvbGljeZSTlC4=",
5
+ "__module__": "stable_baselines3.dqn.policies",
6
+ "__doc__": "\n Policy class with Q-Value Net and target net for DQN\n\n :param observation_space: Observation space\n :param action_space: Action space\n :param lr_schedule: Learning rate schedule (could be constant)\n :param net_arch: The specification of the policy and value networks.\n :param activation_fn: Activation function\n :param features_extractor_class: Features extractor to use.\n :param features_extractor_kwargs: Keyword arguments\n to pass to the features extractor.\n :param normalize_images: Whether to normalize images or not,\n dividing by 255.0 (True by default)\n :param optimizer_class: The optimizer to use,\n ``th.optim.Adam`` by default\n :param optimizer_kwargs: Additional keyword arguments,\n excluding the learning rate, to pass to the optimizer\n ",
7
+ "__init__": "<function DQNPolicy.__init__ at 0x14928d160>",
8
+ "_build": "<function DQNPolicy._build at 0x14928d1f0>",
9
+ "make_q_net": "<function DQNPolicy.make_q_net at 0x14928d280>",
10
+ "forward": "<function DQNPolicy.forward at 0x14928d310>",
11
+ "_predict": "<function DQNPolicy._predict at 0x14928d3a0>",
12
+ "_get_constructor_parameters": "<function DQNPolicy._get_constructor_parameters at 0x14928d430>",
13
+ "set_training_mode": "<function DQNPolicy.set_training_mode at 0x14928d4c0>",
14
+ "__abstractmethods__": "frozenset()",
15
+ "_abc_impl": "<_abc_data object at 0x14928c5d0>"
16
+ },
17
+ "verbose": 1,
18
+ "policy_kwargs": {
19
+ ":type:": "<class 'dict'>",
20
+ ":serialized:": "gAWVVQAAAAAAAAB9lCiMCG5ldF9hcmNolF2UKE0AAk0AAUuAS4BljA1hY3RpdmF0aW9uX2ZulIwbdG9yY2gubm4ubW9kdWxlcy5hY3RpdmF0aW9ulIwEUmVMVZSTlHUu",
21
+ "net_arch": [
22
+ 512,
23
+ 256,
24
+ 128,
25
+ 128
26
+ ],
27
+ "activation_fn": "<class 'torch.nn.modules.activation.ReLU'>"
28
+ },
29
+ "observation_space": {
30
+ ":type:": "<class 'gym.spaces.box.Box'>",
31
+ ":serialized:": "gAWVxwEAAAAAAACMDmd5bS5zcGFjZXMuYm94lIwDQm94lJOUKYGUfZQojAVkdHlwZZSMBW51bXB5lGgFk5SMAmY0lImIh5RSlChLA4wBPJROTk5K/////0r/////SwB0lGKMBl9zaGFwZZRLDIWUjANsb3eUjBJudW1weS5jb3JlLm51bWVyaWOUjAtfZnJvbWJ1ZmZlcpSTlCiWMAAAAAAAAAAAAAAAAAAAAAAAAMAAAADAAAAAAAAAAAAAAIC/AACAvwAAAMAAAAAAAACFwwAAFsOUaApLDIWUjAFDlHSUUpSMBGhpZ2iUaBIoljAAAAAAAAAAAACFQwAAFkMAAABAAAAAQAAAhUMAABZDAACAPwAAgD8AAABAAADQQwAAhUMAABZDlGgKSwyFlGgVdJRSlIwNYm91bmRlZF9iZWxvd5RoEiiWDAAAAAAAAAABAQEBAQEBAQEBAQGUaAeMAmIxlImIh5RSlChLA4wBfJROTk5K/////0r/////SwB0lGJLDIWUaBV0lFKUjA1ib3VuZGVkX2Fib3ZllGgSKJYMAAAAAAAAAAEBAQEBAQEBAQEBAZRoIUsMhZRoFXSUUpSMCl9ucF9yYW5kb22UTnViLg==",
32
+ "dtype": "float32",
33
+ "_shape": [
34
+ 12
35
+ ],
36
+ "low": "[ 0. 0. -2. -2. 0. 0. -1. -1. -2. 0. -266. -150.]",
37
+ "high": "[266. 150. 2. 2. 266. 150. 1. 1. 2. 416. 266. 150.]",
38
+ "bounded_below": "[ True True True True True True True True True True True True]",
39
+ "bounded_above": "[ True True True True True True True True True True True True]",
40
+ "_np_random": null
41
+ },
42
+ "action_space": {
43
+ ":type:": "<class 'gym.spaces.discrete.Discrete'>",
44
+ ":serialized:": "gAWVLwsAAAAAAACME2d5bS5zcGFjZXMuZGlzY3JldGWUjAhEaXNjcmV0ZZSTlCmBlH2UKIwBbpRLA4wGX3NoYXBllCmMBWR0eXBllIwFbnVtcHmUaAeTlIwCaTiUiYiHlFKUKEsDjAE8lE5OTkr/////Sv////9LAHSUYowKX25wX3JhbmRvbZSMFG51bXB5LnJhbmRvbS5fcGlja2xllIwSX19yYW5kb21zdGF0ZV9jdG9ylJOUjAdNVDE5OTM3lIWUUpR9lCiMDWJpdF9nZW5lcmF0b3KUaBOMBXN0YXRllH2UKIwDa2V5lIwSbnVtcHkuY29yZS5udW1lcmljlIwLX2Zyb21idWZmZXKUk5QolsAJAAAAAAAAAAAAgFPCs5yHA7WTcuyrW+jmsvLPtoHa1QbvYaExTaBrtczZE+YYn3SceS/IpRuAxHkBSZ4PQ+Rd4BiTkhNVzNRKKWTSCJW+NNCeRHJ6A/Ctvwpzm3s+6WCxBtp1A0ULbn3WFmrrBDRmg7fz9aUvae9CY0O1XPfCd1LMRkQ3LQiJbtCOrnf7GLaAT3ue+U8y7BLYuY5ehakZyq49di1nK0KAnsXuxx/1IgFdS88kD8wZUmREV5hwnQr1Ehe16VreO8T/Qc86sV+2h90z2FiJqqLNf3s/yZS3bA9DHzGZfRtgkKu3Bx0ZTN7I+466APXPqRreJf4gBqUW/NB248FO9cpD0wFaed9QV63NqpiFOs/RYeEwD8e32dZFRfi9SK5aLvuApJMu2LOfZweypHIkyPjeY5W+tsm2bdPmVoCAOiyi73cf5k0LQeJNWqZU/wuy/f8myghZ9qrjf+2JSJMaB9VNMXWmxuq4Dq0fkhzJr1ML7SgftfSG29O5koFUMozQL58gAzyX96ZMcpWbZ+3/zlaVhGln5egXC8MtIK6xIFCvh/vD/F1jLgYIp14MM597MuPmTpa+OaKek7bql9Cp8/0skhg5QSCvCaijm8wenxrfqLyRxDPCpS+L3isJC5LrjFgWnsdxQXVrJK8uaobJcTIJ5NrYYfA2l5gH27iPI9EqhzFtZJXiE4vXpH18f3kouYV9RowPzOtmYsbmstR/Mx/VY7E0XBmnMidL4dYTUXgxaDbFxWy3y6miL2yw0I2O09vPWV7LqbwMbthlU26lrLfnJDz88B+7y4pFCwvsHgCsMWq6pvroAF4Ms+++JnhzrL0GLrtfJ7667p42Vg78GirmKIRMFH0p6aLRPV4V/fclE3PLBj0InU315M5v7fDFj/IreJeFUhIAy5/BlvLdgwIfBMBWvyjhqGRBo2KmdiDAO9BPhdi6oGmZUCxTDjUyMd8rjeRdntTE+L9sHQUSvUfpRjKcSIjZXcjsMMSwwU0QzIXewD4nZg8EAZ72iHnChWveW7cB5EQRFE+YPvvyNAw4OvORF+DjDtmLUz6jNz0JbkuKbhIQEEVbpDRkoeIGA4HATymZeE4TX/hvhOxJfX5liXD9/Uon8OjJ/hhIrDNUYR2NOV2R1L+VuK/JA9o0izGPiCC9vi7UK1u4re8tvPrBInC6BH2DpMvWSRepdH0frFxGsH/kRB/S77USeMs38a+yorDaa9Wsb9WnuQg10vl087jhvwSZx77fGq52NV1h/UnZAjnqJOjCPBgQrY0wsSeSW6SOSdItAOupiWkVvSzHz+gopd/3FUUPbjdik7Vd6O6ycGOFwp4+wyZqq1MKHww5exJND6De8NB4fGBzsvKoT8O1fAC3Y2Z+3mLYwkCJXULx2zNSO90bQi0L0BhTF1AidNtqRkak72V+V1kos6m4F+kyZiOiwxfcHKWHmpQe3A8HpX6nacx0Zso0WyF/W/sOLo//2g0YD9koIjgKvw//f9Em4CvxNBUMnJYTVQJGysF4QMShnM6byaMNJhRVALf+X0+SgLJkEwLfKdBbKxjA035OEc2YKmUpvooVUUt+9U8d7cKRgNYKKPwCHyYfLJSQt4ZEdThgjeywDxgsGzPax5SklOLroSYn39feofatZDzJXevfPTHyi1ZLwpe6Hwkbqz1FuOnHiShPedbEA9b/HCtBytSZrgUwlwHpQlAiBTSxpN7TFzVZrrhRd6N8W+FeE9SAGCHwXchFR4SCGThPyRGO/XWkMPJ42BLUOmMGepDQgWH646tjoJSE3EXcA5iDS+Nq6Oh521oC2UPAnvxIj1QbVh8IbxlBytRTTjc0c14E9cyhIPlgIoHriuEFSMXSEzOGL1MmL6UCbiXfsRg9Z6OwWCCl3VeGg5bEZ1kjJkvs08k7wtPk4ATAjaTL3QoY2gf106zFbJtL4D5gmLMJ9OuzE2Fn5uaAqqpjXIqqEXxS9jtpsRU9VTHCg68RwXQVIUhuVJgHq8fOigBMrW7Am5+jjo/GNNlcFcp813dFiXy4qHhjGSEjNXp3ln03NZkOgqXQ8SalJlPOvyrAS9wW9EtjQKhcrBSWSsQ8C3o/Mc/sR/CMRB85ZIojR/tiCKtOutxQMIusIOnYHK8g6kPpTQ8J/PfJ8pa3GEoYoA67axQTXsysd2Y6ZDwpz1HkAeISVK2AlCcuQssrQv8dVLAcins/2kjRFp5Vp82HSX9j6Ci9GH5mkdyqV84vWsdwRz4JNXHZoHVZKnrxSdA1HUwRUI/5oWiqnGk9KyiS4Mv9dQIVluJ1+/pAHYEdG9YgLHiNE2zA7aIQbqqlGX6jH1CrHSPL9mnlHdPiKjozwRXu02UQuzlGJn+/PUkU6cPYLLeLc8e7S2qfCZxbdpHioand7wYKqb5bMb8dA3Dwvm6P6iJXSogJ+Q+0z43li8ydYJqZNZlSjsljr/2c5UU33vMhLDLEXIHT8WzrsS45TsKNbhYfTYx6Ds/8W5yOtiOPWc0+fRlAjbQC++FKo5UaMl2eRCxI4U5/heX2HxJGNk50rICjuFsG/8Q+NUuJgS+y3FMhe+sm1e2MdC+ldkBqEn8oRxajECbP4Wizz1tfJliW/1A5fdGTfUxM3HV72bJgnqswmoAumAx6d36KfuZwEEp0/wcrdo+8/unJ5f2mYeqCrOcaxDJrs6SxW1zVaH/YTZl+RNA0NjPLgaqnlveaes/MkpzsVEQDtvKGFrG1cnmGjZVi2azrSDGQ0Y423nEksDC5awYcOJmVYbeA4DkMSNfj+7Dx2SzH/PVPuXLX9aw9K9QF4Ml48zsSrwsVjIa8+gIdffs2pf2wCKcPtFez6vOT0UDuFHknJjMDg0fI3DnyC7jJqO8V4XpmPyarTp3JJRGhmqTHhpZInn70JMfS+RFry5+rLSOM0T+KWV8fYjs9eyCiZijlR4AiADooXm9G8JIzZCLZX2Dty83iyz7gQzSxYO7ULuTT1stvGuJwbBP4LMhLXkbxdhAmBSDiYNOnc3O+yFsO6Ps9UOQD8S4Pbr8hZ4mFjbicpO635SwpmHINYDeuewln3/GHz69LpCjmpnKPeF9ZxXcq6MR4kJUV2j/dQzqjLniNaQmrMkULdI7W1sMXRFcsz9xs1GVwVqmtMVws8HtvXMYNmosCrrgAFX2ghPz7dXCV6vML5YhfNbDAzzG6MHffrslrhMav3vtlt8Fnld4VaH6IhMkowayT1lSVvfvlKHCWwtKaTcOZrR5LZGalJOpFbVIFUOAo+LnY/25bmc3KloyLzgiTudjPsXEGPNPBIvE/5cMEvU4Lrs0N3tCke4abYDXF9f14QrwLlGgJjAJ1NJSJiIeUUpQoSwNoDU5OTkr/////Sv////9LAHSUYk1wAoWUjAFDlHSUUpSMA3Bvc5RNcAJ1jAloYXNfZ2F1c3OUSwCMBWdhdXNzlEcAAAAAAAAAAHVidWIu",
45
+ "n": 3,
46
+ "_shape": [],
47
+ "dtype": "int64",
48
+ "_np_random": "RandomState(MT19937)"
49
+ },
50
+ "n_envs": 1,
51
+ "num_timesteps": 50016,
52
+ "_total_timesteps": 50000,
53
+ "_num_timesteps_at_start": 0,
54
+ "seed": 0,
55
+ "action_noise": null,
56
+ "start_time": 1665781016202374000,
57
+ "learning_rate": {
58
+ ":type:": "<class 'function'>",
59
+ ":serialized:": "gAWV4QIAAAAAAACMF2Nsb3VkcGlja2xlLmNsb3VkcGlja2xllIwOX21ha2VfZnVuY3Rpb26Uk5QoaACMDV9idWlsdGluX3R5cGWUk5SMCENvZGVUeXBllIWUUpQoSwFLAEsASwFLAUsTQwSIAFMAlE6FlCmMAV+UhZSMVy9Vc2Vycy9qbGFuZ3NldGgvUHk0L3ZlbnYvbGliL3B5dGhvbjMuOC9zaXRlLXBhY2thZ2VzL3N0YWJsZV9iYXNlbGluZXMzL2NvbW1vbi91dGlscy5weZSMBGZ1bmOUS4BDAgABlIwDdmFslIWUKXSUUpR9lCiMC19fcGFja2FnZV9flIwYc3RhYmxlX2Jhc2VsaW5lczMuY29tbW9ulIwIX19uYW1lX1+UjB5zdGFibGVfYmFzZWxpbmVzMy5jb21tb24udXRpbHOUjAhfX2ZpbGVfX5SMVy9Vc2Vycy9qbGFuZ3NldGgvUHk0L3ZlbnYvbGliL3B5dGhvbjMuOC9zaXRlLXBhY2thZ2VzL3N0YWJsZV9iYXNlbGluZXMzL2NvbW1vbi91dGlscy5weZR1Tk5oAIwQX21ha2VfZW1wdHlfY2VsbJSTlClSlIWUdJRSlIwcY2xvdWRwaWNrbGUuY2xvdWRwaWNrbGVfZmFzdJSMEl9mdW5jdGlvbl9zZXRzdGF0ZZSTlGgffZR9lChoFmgNjAxfX3F1YWxuYW1lX1+UjBljb25zdGFudF9mbi48bG9jYWxzPi5mdW5jlIwPX19hbm5vdGF0aW9uc19flH2UjA5fX2t3ZGVmYXVsdHNfX5ROjAxfX2RlZmF1bHRzX1+UTowKX19tb2R1bGVfX5RoF4wHX19kb2NfX5ROjAtfX2Nsb3N1cmVfX5RoAIwKX21ha2VfY2VsbJSTlEc/RKTSsr/bTYWUUpSFlIwXX2Nsb3VkcGlja2xlX3N1Ym1vZHVsZXOUXZSMC19fZ2xvYmFsc19flH2UdYaUhlIwLg=="
60
+ },
61
+ "tensorboard_log": null,
62
+ "lr_schedule": {
63
+ ":type:": "<class 'function'>",
64
+ ":serialized:": "gAWV4QIAAAAAAACMF2Nsb3VkcGlja2xlLmNsb3VkcGlja2xllIwOX21ha2VfZnVuY3Rpb26Uk5QoaACMDV9idWlsdGluX3R5cGWUk5SMCENvZGVUeXBllIWUUpQoSwFLAEsASwFLAUsTQwSIAFMAlE6FlCmMAV+UhZSMVy9Vc2Vycy9qbGFuZ3NldGgvUHk0L3ZlbnYvbGliL3B5dGhvbjMuOC9zaXRlLXBhY2thZ2VzL3N0YWJsZV9iYXNlbGluZXMzL2NvbW1vbi91dGlscy5weZSMBGZ1bmOUS4BDAgABlIwDdmFslIWUKXSUUpR9lCiMC19fcGFja2FnZV9flIwYc3RhYmxlX2Jhc2VsaW5lczMuY29tbW9ulIwIX19uYW1lX1+UjB5zdGFibGVfYmFzZWxpbmVzMy5jb21tb24udXRpbHOUjAhfX2ZpbGVfX5SMVy9Vc2Vycy9qbGFuZ3NldGgvUHk0L3ZlbnYvbGliL3B5dGhvbjMuOC9zaXRlLXBhY2thZ2VzL3N0YWJsZV9iYXNlbGluZXMzL2NvbW1vbi91dGlscy5weZR1Tk5oAIwQX21ha2VfZW1wdHlfY2VsbJSTlClSlIWUdJRSlIwcY2xvdWRwaWNrbGUuY2xvdWRwaWNrbGVfZmFzdJSMEl9mdW5jdGlvbl9zZXRzdGF0ZZSTlGgffZR9lChoFmgNjAxfX3F1YWxuYW1lX1+UjBljb25zdGFudF9mbi48bG9jYWxzPi5mdW5jlIwPX19hbm5vdGF0aW9uc19flH2UjA5fX2t3ZGVmYXVsdHNfX5ROjAxfX2RlZmF1bHRzX1+UTowKX19tb2R1bGVfX5RoF4wHX19kb2NfX5ROjAtfX2Nsb3N1cmVfX5RoAIwKX21ha2VfY2VsbJSTlEc/RKTSsr/bTYWUUpSFlIwXX2Nsb3VkcGlja2xlX3N1Ym1vZHVsZXOUXZSMC19fZ2xvYmFsc19flH2UdYaUhlIwLg=="
65
+ },
66
+ "_last_obs": null,
67
+ "_last_episode_starts": {
68
+ ":type:": "<class 'numpy.ndarray'>",
69
+ ":serialized:": "gAWViwAAAAAAAACMEm51bXB5LmNvcmUubnVtZXJpY5SMC19mcm9tYnVmZmVylJOUKJYYAAAAAAAAAAEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAZSMBW51bXB5lIwFZHR5cGWUk5SMAmIxlImIh5RSlChLA4wBfJROTk5K/////0r/////SwB0lGJLGIWUjAFDlHSUUpQu"
70
+ },
71
+ "_last_original_obs": {
72
+ ":type:": "<class 'numpy.ndarray'>",
73
+ ":serialized:": "gAWV9QQAAAAAAACMEm51bXB5LmNvcmUubnVtZXJpY5SMC19mcm9tYnVmZmVylJOUKJaABAAAAAAAAASEB0Pvr1dBvbWYu4ONx7oAAAdDAAC+QmRk176ePmg/bB6IOdgKo0L6AwQ/AgqjwlqnCUPDaeZBrwjGuqs4fr0AAPhCAADKQjddKL4ahHw/wHt0usb0kkKidVpBj2WQwnE77UKbi2BBmgmivIy7yjwAAPpCAADWQv3dAr+yBVw/ZGQrOqReukLuSMzAje65woxY7EIg4A9BlqKPvAVLlTwAAPxCAADEQjG/Br/4qlk/BJcbOtmzskJBd/rA/AOywjAUAEO1vcFBIQcbvUIXFT0AAP5CAADWQokBKb+YSEA/SfmIOiyUpUIhGIo/k5ClwozH+UJ3MxJANuN6vMUceTwAAABDAADiQhKICL+IjVg/KtQvOsGE3UJ2DkfAZG7dwl/d/EKuao1BjeMSva0TFj0AAPxCAADSQvDNIr/aj0U/9umFOuGlrkJcX90+VaWuwt9E9ULN7upBqtkEudZfnrsAAPZCAADcQtNZur4FcW4/ph+jublEoUK6ILu+TUShwmTSBEMIZa9B8SGtuXBpir0AAApDAADCQqpxpb3PKX8/kDSAuhGClkKQs6XAviaWwovbDUNWcaZBc5xQOzCxdL0AAA5DAADWQkoeDL6Rl30/diFxurpjrEIf1RG+qmOswrL/BkNL21FC2sKXOWkVvbwAAAJDAADKQqr4uLxL738/KCNSupwrQ0I/9p9AtSRCwmna9UI+c0xC9PMJu/MQP70AAABDAADOQs+hpL3pK38/YQ1QuiSKUEJqWaLAwoxPwrbf6UJnZ6hB7RCFvHOhuDwAAABDAADiQj0O8764UGE/jjweOq45uUJOAjHBJua3wo6x8UJdNzxCSb4Lu5zp27gAAPxCAADmQsrVsb7SD3A/P1/gODpIiEIi56TAUeSHwtfs+kKzMjFBimTSvAZQ8TwAAPhCAADWQoY9FL9WtlA/4pBrOl/fv0KiNbs/qtm/wgcD70Iz/ao+Sjrju8XcpTgAAAdDAADaQlfa6b61vGM/FcSHObmH20LH53fBA1XZwvOiC0P0zg5Cj0BcvKF+UDwAAAFDAADSQne/+r60Ml8/Qy4EOgk4jEI3LypBhpiKwtUKBkNva8FAE7K0vGwjwDwAAPxCAADGQhthEL/MZlM/KD9VOhSbukJPrQBBSem5wuT2AUOCrqRA3V5AvIYBEjwAAA1DAADkQk6dAr8eLFw/K873OcXS2kK9kTDBGLXZwiif7UJzp5pB6DvJvGgd0TwAAANDAADUQrJaEr8DClI/sAU4OtkKr0K9BkPBI1atwuL8CkPBJ61Bm+/+OkXcRr0AAARDAADWQqNan77zSHM/fSlUukRIq0I+nN9AELaqwphG8kKbZHJBswEUvTdzBD0AAPZCAADYQl2iKr8T1z4/pd+DOvy8uULzWe6/bbO5wsWJBkOZwihCytAovEXHXTwAAAVDAADiQlZF1r73gGg/oePwOUGnjUJK4sQ/s56NwixyCEPpZg5C9QlQOU5WVr0AAApDAADcQrRsp74W7XE/vl9UutrUlELO6ca/i8yUwpSMBW51bXB5lIwFZHR5cGWUk5SMAmY0lImIh5RSlChLA4wBPJROTk5K/////0r/////SwB0lGJLGEsMhpSMAUOUdJRSlC4="
74
+ },
75
+ "_episode_num": 30,
76
+ "use_sde": false,
77
+ "sde_sample_freq": -1,
78
+ "_current_progress_remaining": -0.000320000000000098,
79
+ "ep_info_buffer": {
80
+ ":type:": "<class 'collections.deque'>",
81
+ ":serialized:": "gAWVXQYAAAAAAACMC2NvbGxlY3Rpb25zlIwFZGVxdWWUk5QpS2SGlFKUKH2UKIwBcpSMFW51bXB5LmNvcmUubXVsdGlhcnJheZSMBnNjYWxhcpSTlIwFbnVtcHmUjAVkdHlwZZSTlIwCZjSUiYiHlFKUKEsDjAE8lE5OTkr/////Sv////9LAHSUYkMEyKBMxJSGlFKUjAFslGgJaAyMAmk0lImIh5RSlChLA2gQTk5OSv////9K/////0sAdJRiQwT0AgAAlIaUUpSMAXSUR0AKLefqX4TLdX2UKGgGaAloD0MEGsZRxJSGlFKUaBVoCWgYQwT6AgAAlIaUUpRoHUdACmZ9/jKgZnV9lChoBmgJaA9DBBDZU8SUhpRSlGgVaAloGEMEBwMAAJSGlFKUaB1HQAr5s9B8hLZ1fZQoaAZoCWgPQwSeFE/ElIaUUpRoFWgJaBhDBAsDAACUhpRSlGgdR0ALHpfQa72+dX2UKGgGaAloD0MErN1ZxJSGlFKUaBVoCWgYQwQMAwAAlIaUUpRoHUdAC0yad+Xqq3V9lChoBmgJaA9DBPvHWsSUhpRSlGgVaAloGEMEGgMAAJSGlFKUaB1HQAvsOG0u14R1fZQoaAZoCWgPQwRHW1TElIaUUpRoFWgJaBhDBBwDAACUhpRSlGgdR0AL+y7f51vEdX2UKGgGaAloD0MEiildxJSGlFKUaBVoCWgYQwQsAwAAlIaUUpRoHUdADJiWE9Mbm3V9lChoBmgJaA9DBMIYV8SUhpRSlGgVaAloGEMEMwMAAJSGlFKUaB1HQAzYISlFc6h1fZQoaAZoCWgPQwTwmmjElIaUUpRoFWgJaBhDBEwDAACUhpRSlGgdR0ANs0SAYpDvdX2UKGgGaAloD0MEkzhnxJSGlFKUaBVoCWgYQwRSAwAAlIaUUpRoHUdADeoegctGu3V9lChoBmgJaA9DBIxXccSUhpRSlGgVaAloGEMEhwMAAJSGlFKUaB1HQA/ECNjslcB1fZQoaAZoCWgPQwTcolTElIaUUpRoFWgJaBhDBJQDAACUhpRSlGgdR0AQG0kWykbhdX2UKGgGaAloD0MEwFpxxJSGlFKUaBVoCWgYQwSaAwAAlIaUUpRoHUdAEDbayrxRVXV9lChoBmgJaA9DBHy5c8SUhpRSlGgVaAloGEMEmwMAAJSGlFKUaB1HQBA6dpZfUnZ1fZQoaAZoCWgPQwTt4HLElIaUUpRoFWgJaBhDBLEDAACUhpRSlGgdR0AQndadMCcPdX2UKGgGaAloD0ME4mBVxJSGlFKUaBVoCWgYQwTAAwAAlIaUUpRoHUdAEOBYmsvIwXV9lChoBmgJaA9DBJhzdsSUhpRSlGgVaAloGEMEzQMAAJSGlFKUaB1HQBEfG2kSElF1fZQoaAZoCWgPQwRMJ3vElIaUUpRoFWgJaBhDBNUDAACUhpRSlGgdR0ARQsDnvDxcdX2UKGgGaAloD0MEYHl+xJSGlFKUaBVoCWgYQwTlAwAAlIaUUpRoHUdAEZOkLx7RfHV9lChoBmgJaA9DBJ06e8SUhpRSlGgVaAloGEMELAQAAJSGlFKUaB1HQBtNo8IRh+h1fZQoaAZoCWgPQwTUQmzElIaUUpRoFWgJaBhDBFEEAACUhpRSlGgdR0Ab8p6QeV9ndX2UKGgGaAloD0MEQuBQxJSGlFKUaBVoCWgYQwTcAgAAlIaUUpRoHUdAIW08/2TPjXV9lChoBmgJaA9DBGgJUMSUhpRSlGgVaAloGEMEJQMAAJSGlFKUaB1HQCJnmYBvJil1fZQoaAZoCWgPQwSErk7ElIaUUpRoFWgJaBhDBFoDAACUhpRSlGgdR0AivBzFMqSYdX2UKGgGaAloD0MEvT5exJSGlFKUaBVoCWgYQwRKAwAAlIaUUpRoHUdAIw4/3WWhRXV9lChoBmgJaA9DBHPoV8SUhpRSlGgVaAloGEMEYgMAAJSGlFKUaB1HQCM29FnZkCp1fZQoaAZoCWgPQwTp5nzElIaUUpRoFWgJaBhDBBYEAACUhpRSlGgdR0AkAWGh24d7dX2UKGgGaAloD0ME2MpmxJSGlFKUaBVoCWgYQwTUAwAAlIaUUpRoHUdAJYjf3vhIfHV9lChoBmgJaA9DBAY1c8SUhpRSlGgVaAloGEMEZAQAAJSGlFKUaB1HQCYO+M6zVtp1ZS4="
82
+ },
83
+ "ep_success_buffer": {
84
+ ":type:": "<class 'collections.deque'>",
85
+ ":serialized:": "gAWVIAAAAAAAAACMC2NvbGxlY3Rpb25zlIwFZGVxdWWUk5QpS2SGlFKULg=="
86
+ },
87
+ "_n_updates": 521,
88
+ "buffer_size": 1,
89
+ "batch_size": 128,
90
+ "learning_starts": 0,
91
+ "tau": 1.0,
92
+ "gamma": 0.99,
93
+ "gradient_steps": 1,
94
+ "optimize_memory_usage": false,
95
+ "replay_buffer_class": {
96
+ ":type:": "<class 'abc.ABCMeta'>",
97
+ ":serialized:": "gAWVNQAAAAAAAACMIHN0YWJsZV9iYXNlbGluZXMzLmNvbW1vbi5idWZmZXJzlIwMUmVwbGF5QnVmZmVylJOULg==",
98
+ "__module__": "stable_baselines3.common.buffers",
99
+ "__doc__": "\n Replay buffer used in off-policy algorithms like SAC/TD3.\n\n :param buffer_size: Max number of element in the buffer\n :param observation_space: Observation space\n :param action_space: Action space\n :param device: PyTorch device\n :param n_envs: Number of parallel environments\n :param optimize_memory_usage: Enable a memory efficient variant\n of the replay buffer which reduces by almost a factor two the memory used,\n at a cost of more complexity.\n See https://github.com/DLR-RM/stable-baselines3/issues/37#issuecomment-637501195\n and https://github.com/DLR-RM/stable-baselines3/pull/28#issuecomment-637559274\n Cannot be used in combination with handle_timeout_termination.\n :param handle_timeout_termination: Handle timeout termination (due to timelimit)\n separately and treat the task as infinite horizon task.\n https://github.com/DLR-RM/stable-baselines3/issues/284\n ",
100
+ "__init__": "<function ReplayBuffer.__init__ at 0x149261c10>",
101
+ "add": "<function ReplayBuffer.add at 0x149261ca0>",
102
+ "sample": "<function ReplayBuffer.sample at 0x149261d30>",
103
+ "_get_samples": "<function ReplayBuffer._get_samples at 0x149261dc0>",
104
+ "__abstractmethods__": "frozenset()",
105
+ "_abc_impl": "<_abc_data object at 0x14925bfc0>"
106
+ },
107
+ "replay_buffer_kwargs": {},
108
+ "train_freq": {
109
+ ":type:": "<class 'stable_baselines3.common.type_aliases.TrainFreq'>",
110
+ ":serialized:": "gAWVYQAAAAAAAACMJXN0YWJsZV9iYXNlbGluZXMzLmNvbW1vbi50eXBlX2FsaWFzZXOUjAlUcmFpbkZyZXGUk5RLBGgAjBJUcmFpbkZyZXF1ZW5jeVVuaXSUk5SMBHN0ZXCUhZRSlIaUgZQu"
111
+ },
112
+ "actor": null,
113
+ "use_sde_at_warmup": false,
114
+ "exploration_initial_eps": 1.0,
115
+ "exploration_final_eps": 0.01,
116
+ "exploration_fraction": 0.25,
117
+ "target_update_interval": 10,
118
+ "_n_calls": 2084,
119
+ "max_grad_norm": 10,
120
+ "exploration_rate": 0.01,
121
+ "exploration_schedule": {
122
+ ":type:": "<class 'function'>",
123
+ ":serialized:": "gAWVhQMAAAAAAACMF2Nsb3VkcGlja2xlLmNsb3VkcGlja2xllIwOX21ha2VfZnVuY3Rpb26Uk5QoaACMDV9idWlsdGluX3R5cGWUk5SMCENvZGVUeXBllIWUUpQoSwFLAEsASwFLBEsTQyxkAXwAGACIAWsEchCIAFMAiAJkAXwAGACIAIgCGAAUAIgBGwAXAFMAZABTAJROSwGGlCmMEnByb2dyZXNzX3JlbWFpbmluZ5SFlIxXL1VzZXJzL2psYW5nc2V0aC9QeTQvdmVudi9saWIvcHl0aG9uMy44L3NpdGUtcGFja2FnZXMvc3RhYmxlX2Jhc2VsaW5lczMvY29tbW9uL3V0aWxzLnB5lIwEZnVuY5RLbkMGAAEMAQQClIwDZW5klIwMZW5kX2ZyYWN0aW9ulIwFc3RhcnSUh5QpdJRSlH2UKIwLX19wYWNrYWdlX1+UjBhzdGFibGVfYmFzZWxpbmVzMy5jb21tb26UjAhfX25hbWVfX5SMHnN0YWJsZV9iYXNlbGluZXMzLmNvbW1vbi51dGlsc5SMCF9fZmlsZV9flIxXL1VzZXJzL2psYW5nc2V0aC9QeTQvdmVudi9saWIvcHl0aG9uMy44L3NpdGUtcGFja2FnZXMvc3RhYmxlX2Jhc2VsaW5lczMvY29tbW9uL3V0aWxzLnB5lHVOTmgAjBBfbWFrZV9lbXB0eV9jZWxslJOUKVKUaB0pUpRoHSlSlIeUdJRSlIwcY2xvdWRwaWNrbGUuY2xvdWRwaWNrbGVfZmFzdJSMEl9mdW5jdGlvbl9zZXRzdGF0ZZSTlGgjfZR9lChoGGgNjAxfX3F1YWxuYW1lX1+UjBtnZXRfbGluZWFyX2ZuLjxsb2NhbHM+LmZ1bmOUjA9fX2Fubm90YXRpb25zX1+UfZQoaAqMCGJ1aWx0aW5zlIwFZmxvYXSUk5SMBnJldHVybpRoL3WMDl9fa3dkZWZhdWx0c19flE6MDF9fZGVmYXVsdHNfX5ROjApfX21vZHVsZV9flGgZjAdfX2RvY19flE6MC19fY2xvc3VyZV9flGgAjApfbWFrZV9jZWxslJOURz+EeuFHrhR7hZRSlGg3Rz/QAAAAAAAAhZRSlGg3Rz/wAAAAAAAAhZRSlIeUjBdfY2xvdWRwaWNrbGVfc3VibW9kdWxlc5RdlIwLX19nbG9iYWxzX1+UfZR1hpSGUjAu"
124
+ },
125
+ "batch_norm_stats": [],
126
+ "batch_norm_stats_target": []
127
+ }
dqn-Sailing-v1/policy.optimizer.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c3209d618916d9b07eafbdb3d738275518793768ced4a211c7441ad68e07d5fd
3
+ size 1509963
dqn-Sailing-v1/policy.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e7ff3847d5d6058421f90d17ff3d01ad68891562a03423e3674ade7f6466914e
3
+ size 1508521
dqn-Sailing-v1/pytorch_variables.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d030ad8db708280fcae77d87e973102039acd23a11bdecc3db8eb6c0ac940ee1
3
+ size 431
dqn-Sailing-v1/system_info.txt ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ OS: macOS-12.6-x86_64-i386-64bit Darwin Kernel Version 21.6.0: Mon Aug 22 20:17:10 PDT 2022; root:xnu-8020.140.49~2/RELEASE_X86_64
2
+ Python: 3.8.9
3
+ Stable-Baselines3: 1.6.2
4
+ PyTorch: 1.12.1
5
+ GPU Enabled: False
6
+ Numpy: 1.23.3
7
+ Gym: 0.21.0
env_kwargs.yml ADDED
@@ -0,0 +1 @@
 
 
1
+ {}
train_eval_metrics.zip ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b7fb69eff21cb1a3db9afffcb0429d2a4c3dcb99f1425e1327d8c2aba3bc99b6
3
+ size 5928