{"push_id":2810452908,"size":1,"distinct_size":1,"ref":"refs/heads/master","head":"dcbd553fbb56cb2b9ecd1954c0f07df147ab1aa5","before":"3628ab9fbbf218015d400cff5e11cd09eabb304a","commits":[{"sha":"dcbd553fbb56cb2b9ecd1954c0f07df147ab1aa5","author":{"name":"Marko Grdinić","email":"5622e3ec1a18094ea0e746d23e3f804b2589765e@gmail.com"},"message":"As per plan, I think for the rest of the day I will just gather my thoughts like I did in the last commit.\n\nThe next goal would definitely be RNNs. When I tried them months ago with raw PG, it worked like crap, so I am going to have to take a look at whether the combination of PRONG and the MC critic is enough to get it into the stable range.\n\nI might do well to revisit the recurrent K-FAC paper again and try and figure out what it is doing.\n\nI might also bring back the LN+Relu activation, but it is not a priority.\n\nHonestly, I've benefited quite a bit from simplyifying PRONG to the level where it is usable. I think that studying the recurrent K-FAC and higher order methods might make a worthwhile investment of my time.\n\nSo it goes roughly like this.\n\nIt does not feel good to me to just assume that K-FAC made for feedforward nets that I have now will be enough to do it.\n\nTo be 100% sure, I should master the recurrent variant, and pick up that thread on the tridiagonal one.\n\nThis means more time spent on studying. Yeah, yeah I know. But this is something I believe in and is of real importance in RL.\n\nNobody is really using this stuff apart from the OpenAI guys working on ACKTR. In the latest Dota 2 bout, the LSTM they are using is just plain PPO. I am guessing another group from within OpenAI is doing that.\n\nI think that in the worst case, the recurrent K-FAC should be enough to get RNNs fully functional in the RL domain.\n\nAt that point I will have advanced in my education and will have another neglected asset in my toolbox.\n\nThat will put me in a prime position to tackle Rudder. After I get that I will be absolute.\n\nThe 'make an agent' phase will be done with. At that point the goal will change to 'scale them' and 'make use of them in the real world' after that.\n\nIt would be good if I could get through both RNN and Rudder in the next two months. I know that this will take me a while as usual.\n\nSince it is 4pm and I definitely do not feel like programming again today, what I will do is pick up the recurrent K-FAC paper and do some studying.\n\nAs an aside, both the recurrent K-FAC and the tridiagonal update use Bayesian reasoning to get at the updates. At this point, rather than ask Reddit, I am thinking that I will try asking Martens directly if I really need to.","distinct":true,"url":"https://api.github.com/repos/mrakgr/The-Spiral-Language/commits/dcbd553fbb56cb2b9ecd1954c0f07df147ab1aa5"}]}
{"actor":{"display_login":"mrakgr"}}