０．はじめに

先日，仮想通貨の板情報を推定する，相関0.9超えのモデルを構築しました．

しかし，取引するモジュールがなく，現在このモデルを活かしきれていません．そこで，取引するモジュールを強化学習を使ってエンドツーエンドで構築していこうと思います．

ただ，強化学習を使ったことがないので，環境構築から始まり，サンプルプログラムの実行などからやっていこうと思います．

１．ライブラリ「OpenAI Gym」のインストール

この辺の記事を参考にOpenAI Gymのインストールをしてみました

確認のためにサンプルプログラムを実行してみました．

ちゃんとレンダリングされてますね！

参考までに実行したコードを記載しておきます．

C:\Users\masaki>conda info -e
# conda environments:
#
base                  *  C:\Users\masaki\anaconda3

C:\Users\masaki>conda create -n py_gym_v1
Collecting package metadata (current_repodata.json): done
Solving environment: done

==> WARNING: A newer version of conda exists. <==
 current version: 4.10.1
 latest version: 4.10.3
Please update conda by running
   $ conda update -n base -c defaults conda
## Package Plan ##
 environment location: C:\Users\masaki\anaconda3\envs\py_gym_v1
Proceed ([y]/n)? y
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
#     $ conda activate py_gym_v1
#
# To deactivate an active environment, use
#
#     $ conda deactivate

C:\Users\masaki>conda activate py_gym_v1
(py_gym_v1) C:\Users\masaki>pip install gym
Collecting gym
 Downloading gym-0.21.0.tar.gz (1.5 MB)
    |████████████████████████████████| 1.5 MB 2.2 MB/s
Requirement already satisfied: numpy>=1.18.0 in c:\users\masaki\anaconda3\lib\site-packages (from gym) (1.20.1)
Requirement already satisfied: cloudpickle>=1.2.0 in c:\users\masaki\anaconda3\lib\site-packages (from gym) (1.6.0)
Building wheels for collected packages: gym
 Building wheel for gym (setup.py) ... done
 Created wheel for gym: filename=gym-0.21.0-py3-none-any.whl size=1616826 sha256=e32cfc7e91a07ac7b2e03e3ee00341b819fbd11edd5793d6d2641272705953f6
 Stored in directory: c:\users\masaki\appdata\local\pip\cache\wheels\27\6d\b3\a3a6e10704795c9b9000f1ab2dc480dfe7bed42f5972806e73
Successfully built gym
Installing collected packages: gym
Successfully installed gym-0.21.0
(py_gym_v1) C:\Users\masaki>pip install 'gym[atari]'
ERROR: Invalid requirement: "'gym[atari]'"
(py_gym_v1) C:\Users\masaki>pip install gym[atari]
Requirement already satisfied: gym[atari] in c:\users\masaki\anaconda3\lib\site-packages (0.21.0)
Requirement already satisfied: numpy>=1.18.0 in c:\users\masaki\anaconda3\lib\site-packages (from gym[atari]) (1.20.1)
Requirement already satisfied: cloudpickle>=1.2.0 in c:\users\masaki\anaconda3\lib\site-packages (from gym[atari]) (1.6.0)
Collecting ale-py~=0.7.1
 Downloading ale_py-0.7.2-cp38-cp38-win_amd64.whl (925 kB)
    |████████████████████████████████| 925 kB 1.3 MB/s
Collecting importlib-resources
 Downloading importlib_resources-5.2.2-py3-none-any.whl (27 kB)
Requirement already satisfied: zipp>=3.1.0 in c:\users\masaki\anaconda3\lib\site-packages (from importlib-resources->ale-py~=0.7.1->gym[atari]) (3.4.1)
Installing collected packages: importlib-resources, ale-py
Successfully installed ale-py-0.7.2 importlib-resources-5.2.2
(py_gym_v1) C:\Users\masaki>python
Python 3.8.8 (default, Apr 13 2021, 15:08:03) [MSC v.1916 64 bit (AMD64)] :: Anaconda, Inc. on win32
Warning:
This Python interpreter is in a conda environment, but the environment has
not been activated.  Libraries may fail to load.  To activate this environment
please see https://conda.io/activation
Type "help", "copyright", "credits" or "license" for more information.
>>> from gym import envs
>>> envids = [spec.id for spec in envs.registry.all()]
>>> print(envids)
['ALE/Tetris-v5', 'ALE/Tetris-ram-v5', 'Adventure-v0', 'AdventureDeterministic-v0', 'AdventureNoFrameskip-v0', 'Adventure-v4', 'AdventureDeterministic-v4', 'AdventureNoFrameskip-v4', 'Adventure-ram-v0', 'Adventure-ramDeterministic-v0', 'Adventure-ramNoFrameskip-v0', 'Adventure-ram-v4', 'Adventure-ramDeterministic-v4', 'Adventure-ramNoFrameskip-v4', 'AirRaid-v0', 'AirRaidDeterministic-v0', 'AirRaidNoFrameskip-v0', 'AirRaid-v4', 'AirRaidDeterministic-v4', 'AirRaidNoFrameskip-v4', 'AirRaid-ram-v0', 'AirRaid-ramDeterministic-v0', 'AirRaid-ramNoFrameskip-v0', 'AirRaid-ram-v4', 'AirRaid-ramDeterministic-v4', 'AirRaid-ramNoFrameskip-v4', 'Alien-v0', 'AlienDeterministic-v0', 'AlienNoFrameskip-v0', 'Alien-v4', 'AlienDeterministic-v4', 'AlienNoFrameskip-v4', 'Alien-ram-v0', 'Alien-ramDeterministic-v0', 'Alien-ramNoFrameskip-v0', 'Alien-ram-v4', 'Alien-ramDeterministic-v4', 'Alien-ramNoFrameskip-v4', 'Amidar-v0', 'AmidarD
．．．．．

２．サンプルプログラム

インスタに映っている山登りのサンプルプログラムです．

import gym
env = gym.make('CartPole-v0') # select game
observation = env.reset() # initialize
for i in range(100):
 env.render()
 observation, reward, done, info = env.step(1) # go right
env.env.close()

３．おわりに

次回はGoogle Colabを使った環境構築を行っていきます．

▼▼▼▼▼▼次回記事▼▼▼▼▼▼

▲▲▲▲▲▲▲▲▲▲▲▲▲▲▲▲▲

#強化学習
#自動売買強化学習プロジェクト
#Python
#OpenAIGym

#自動売買

#仮想通貨自動売買