Deep Reinforcement Learning with Python: Master classic RL, deep RL, distributional RL, inverse RL, and more with OpenAI Gym and TensorFlow, 2nd Edition Paperback – September 30, 2020

Much better than the first version.

I can highly recommend this second edition of the book "Deep Reinforcement Learning with Python". The book gives a good introduction of reinforcement learning for practitioners and researchers. It is updated with Tensorflow 2.0, OpenAI Gym and Stable Baselines training and examples. The book is enjoyable to read, with nice illustrations and it explains why certain concepts are introduced, rather than just throwing math equations at the reader. For example, the expected return value is explain in simpler terms using the weighted average of a small discrete distribution, before going into more abstract notation and concepts.The second chapter introduces OpenAI Gym, helps installing it on your computer and shows a few simple self-contained examples how to create your own Gym environment from scratch. Chapter 3 introduces the Bellman equation, Q function value and policy iteration, applied to the Gym environments created in the 2nd chapter for better intuition.It also shows how to use popular Gym environments with Atari games, Box 2D and MuJoCo. Note that you can replace MuJoCo by the open source PyBullet environments, it is integrated with Stable Baselines and the Baselines Zoo has pre-trained policies for PyBullet environments such as Walker2D, HalfCheetah, Ant, Reacher, Hopper, Humanoid, Minitaur robot and more. Chapter 7 shows how to build a neural network from scratch with forward and back-propagation of the gradients using pure Numpy, which gives a better intuition than just using existing large frameworks such as Tensorflow and PyTorch. Chapter 8 is a good introduction to Tensorflow, again with hands-on code and updated examples for the latest version of TF2 and Keras.The later chapters 9-13 discusses algorithms that our robotics research group uses, such as PPO and SAC (soft actor critic). Chapter 16 goes in detail on the popular implementation of such algorithms using the open source Stable Baselines, we also use for our recent motion imitation research.Each chapter finishes with questions and further reading material, it makes it suitable for self-study or (under)graduate classroom material.

This could be a great introductory book on reinforcement learning for both students and professionals. It has a great mix of theory, examples, and implementations of RL algorithms. The flow of concepts is gradual, starting from basic RL concepts like MDPs, Monte Carlo, Q Learning, and moves to Deep RL and a few advanced concepts towards the later chapters like model-based RL, imitation learning, etc. Another positive note is the use of TensorFlow 2.0 in this book which is much easier to understand than its previous versions.While many RL algorithms are covered, the book does miss out on topics like multi-agent reinforcement learning, reward shaping, and hyperparameter selection and tuning. One area of improvement could be to split this book into basic and advanced versions. This might improve engagement from either group (just a suggestion, 700 pages may be too long for some readers)!Overall, I thought the book is a great read.

I have read the first edition of this book. Compared to the first edition, this one is unbelievably good with extreme details. What I see is this edition seems to be completely rewritten with a very detailed explanation. I couldn’t find anything similar to first edition much. Author has included a section called math essentials before every algorithm and this helps to understand the underlying math behind RL algorithms in a very easy way. Next what I like the most is the flow of concepts and how they are interconnected.Right from very basic Q learning to advanced algorithms like PPO, ACKTR, GAIL, categorical DQN, max entropy inverse RL, this book covers everything with special importance given to math. I also liked how the code is very clearly explained with line by line explanation. Line by line code explanation I haven’t seen anywhere. I’m pretty sure you won’t regret buying this masterpiece.

The most important aspect of most programming books or courses is how well they support learners in writing the code themselves.Ravichandiran’s book cleverly utilizes tools provided by Open AI Gym, along with TensorFlow, to provide lots of short hands-on exercises.

I own and have read pretty much all of the DRL books that were published in the past 3 years, and I can with certainty say that this book is by far the best on the subject. An amazing clarity of explanation combined with the vast scope. Thank you so very much Sudharsan!

So bought a book with a publication of Sep 2020, thought it might actually be compatible with up to date software (Sep 2020). Well ....Requires Tensorflow 1.X - EVERYTHING being done now is tensorflow 2.0, including any recent release of Python (needs to be < 3.6), Visual studio, (must be old). First example, third line:module 'tensorflow' has no attribute 'Session'because, Session isn't in tensorflow 1.X.So, to run ANY examples, you have to downgrade to old software, which is a huge pain. And even if you do, no one write anything now with tensorflow 1.X.So, this book is really completely irrelevant to anything anyone wants to so in 2021 in tensorflow. Was hopelessly out of date before it was published.

Libro con mucho nivel sobre aprendizaje por refuerzo. Recomendable aun siendo de2020

I give full marks for ease and elegance with which the topic is dealt with. I had so much struggle learning from the other popular ones. However nothing registered in my mind. This book makes it really easy.Highly recommended.

This book has a promising table of contents. But after having wasted an entire day solving error after error trying to get the code in chapter 2 running, I've thrown in the towel.The problem is probably in the environment, not the code itself, but the installation instructions are insufficien to get a correct environment.The code download contains a number of jupyter notebooks, but these are not complete.

Ce livre couvre bien la plupart des algos et problématiques du RL, mais par contre il est bourré de passages redondants. On à l'impression que le livre est très gros, mais c'est principalement parcequ'il répète inutilement la plupart des explications et équations. De plus il y a un problème : c'est censé être basé sur Tensorflow 2.0, mais pour beaucoup d'algorithmes il s'est contenté de reprendre du code Tensorflow 1.0 en mettant TF 2.0 en mode compatibilité 1.0 !!! Donc pour TF 2.0 vous repasserez... dommage.

Molto dettagliato e approfondito, non superficiale. Interesse personale per l'argomento