| license: mit | |
| license_link: https://huggingface.co/TheoVincent/Atari_i-QN/blob/main/LICENSE | |
| tags: | |
| - reinforcement-learning | |
| - jax | |
| - atari | |
| co2_eq_emissions: | |
| emissions: 3000000 | |
| # Model parameters trained with `i-DQN` and `i-IQN` | |
| This repository contains the model parameters trained with `i-DQN` on [56 Atari games](#i-DQN_games) and trained with `i-IQN` on [20 Atari games](#i-IQN_games) 🎮 5 seeds are available for each configuration which makes a total of **380 available models** 📈 | |
| The [evaluate.ipynb](./evaluate.ipynb) notebook contains a minimal example to evaluate to model parameters 🧑🏫 It uses JAX 🚀 The hyperparameters used during training are reported in [config.json](./config.json) 🔧 | |
| To the training code 👉[💻](https://github.com/theovincent/i-DQN) | |
| ps: The set of [20 Atari games](#i-DQN_games) is included in the set of [56 Atari games](#i-IQN_games). | |
| ### Model performances | |
| | <div style="width:300px; font-size: 30px; font-family:Serif; font-name:Times New Roman" > **i-DQN** and **i-IQN** are improvements of [DQN](https://www.nature.com/articles/nature14236.pdf) and [IQN](https://arxiv.org/abs/1806.06923). <br> Published at [TMLR](https://arxiv.org/abs/2403.02107)✨ </br> <div style="font-size: 16px"> <details> <summary id=i-DQN_games>List of games trained with `i-DQN` </summary> *Alien, Amidar, Assault, Asterix, Asteroids, Atlantis, BankHeist, BattleZone, BeamRider, Berzerk, Bowling, Boxing, Breakout, Centipede, ChopperCommand, CrazyClimber, DemonAttack, DoubleDunk, Enduro, FishingDerby, Freeway, Frostbite, Gopher, Gravitar, Hero, IceHockey, Jamesbond, Kangaroo, Krull, KungFuMaster, MontezumaRevenge, MsPacman, NameThisGame, Phoenix, Pitfall, Pong, Pooyan, PrivateEye, Qbert, Riverraid, RoadRunner, Robotank, Seaquest, Skiing, Solaris, SpaceInvaders, StarGunner, Tennis, TimePilot, Tutankham, UpNDown, Venture, VideoPinball, WizardOfWor, YarsRevenge, Zaxxon.* </details> <details> <summary id=i-IQN_games>List of games trained with `i-IQN`</summary> *Alien, Assault, BankHeist, Berzerk, Breakout, Centipede, ChopperCommand, DemonAttack, Enduro, Frostbite, Gopher, Gravitar, IceHockey, Jamesbond, Krull, KungFuMaster, Riverraid, Seaquest, Skiing, StarGunner.* </details> </div> </div> | <img src="performances.png" alt="drawing" width="600px"/> | | |
| | :-: | :-: | | |
| ## User installation | |
| Python 3.10 is recommended. Create a Python virtual environment, activate it, update pip and install the package and its dependencies in editable mode: | |
| ```bash | |
| python3.10 -m venv env | |
| source env/bin/activate | |
| pip install --upgrade pip | |
| pip install numpy==1.23.5 # to avoid numpy==2.XX | |
| pip install -r requirements.txt | |
| pip install --upgrade "jax[cuda12_pip]==0.4.13" -f https://storage.googleapis.com/jax-releases/jax_cuda_releases.html | |
| ``` | |
| ## Citing `iterated Q-Network` | |
| ``` | |
| @article{vincent2024iterated, | |
| title={Iterated $ Q $-Network: Beyond the One-Step Bellman Operator}, | |
| author={Vincent, Th{\'e}o and Palenicek, Daniel and Belousov, Boris and Peters, Jan and D'Eramo, Carlo}, | |
| journal={Transactions on Machine Learning Research}, | |
| year={2025} | |
| } | |
| ``` |