Science

Reinforcement Learning: An Introduction

Reinforcement Learning: An Introduction

  • Downloads：5180
  • Type：Epub+TxT+PDF+Mobi
  • Create Date：2021-06-23 09:53:39
  • Update Date：2025-09-06
  • Status：finish
  • Author：Richard S. Sutton
  • ISBN：0262039249
  • Environment：PC/Android/iPhone/iPad/Kindle
  • Download

Summary

The significantly expanded and updated new edition of a widely used text on reinforcement learning， one of the most active research areas in artificial intelligence。

Reinforcement learning， one of the most active research areas in artificial intelligence， is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives while interacting with a complex， uncertain environment。 In Reinforcement Learning， Richard Sutton and Andrew Barto provide a clear and simple account of the field's key ideas and algorithms。 This second edition has been significantly expanded and updated， presenting new topics and updating coverage of other topics。

Like the first edition， this second edition focuses on core online learning algorithms， with the more mathematical material set off in shaded boxes。 Part I covers as much of reinforcement learning as possible without going beyond the tabular case for which exact solutions can be found。 Many algorithms presented in this part are new to the second edition， including UCB， Expected Sarsa， and Double Learning。 Part II extends these ideas to function approximation， with new sections on such topics as artificial neural networks and the Fourier basis， and offers expanded treatment of off-policy learning and policy-gradient methods。 Part III has new chapters on reinforcement learning's relationships to psychology and neuroscience， as well as an updated case-studies chapter including AlphaGo and AlphaGo Zero， Atari game playing， and IBM Watson's wagering strategy。 The final chapter discusses the future societal impacts of reinforcement learning。

Download

Reviews

David Tao ，

Everytime I go back and read this I learn many new things

Steven ，

I came across this book as the textbook for Reinforcement Learning Specialization on Coursera。 However， I continued to read it for fun afterwards。 It serves as an excellent introduction to reinforcement learning （RL） providing great insight into not only the techniques of RL， but also the fundamental motivations and underlying ideas behind RL。 It explains concepts simple and jargon free manner that is easy to understand。 While at times it is wordy， I was sufficiently interested to continue readi I came across this book as the textbook for Reinforcement Learning Specialization on Coursera。 However， I continued to read it for fun afterwards。 It serves as an excellent introduction to reinforcement learning （RL） providing great insight into not only the techniques of RL， but also the fundamental motivations and underlying ideas behind RL。 It explains concepts simple and jargon free manner that is easy to understand。 While at times it is wordy， I was sufficiently interested to continue reading regardless。 I found particularly interesting the last section which elaborates on the connections of RL with psychology and neuroscience， as well as， applications， case studies， and frontiers。 I would recommend this book to anyone interested in getting into RL。 。。。more

C。 Hinsley ，

Absolutely the best textbook I have ever read。 This thing is pretty much perfect。

Farhad Da ，

I read this book along "Reinforcement Learning Specialization" in Coursera:https://www。coursera。org/specializati。。。This book is compelling， complete， and it explains concepts in a way that is very understandable。 I read this book along "Reinforcement Learning Specialization" in Coursera:https://www。coursera。org/specializati。。。This book is compelling， complete， and it explains concepts in a way that is very understandable。 。。。more

Tolga Karahan ，

It's an excellent book to learn theory behind RL and reading it is really fun。 Just read it， think about it， ask about it， and implement what you learned from it。 It deepens someone's understanding of RL。 It's an excellent book to learn theory behind RL and reading it is really fun。 Just read it， think about it， ask about it， and implement what you learned from it。 It deepens someone's understanding of RL。 。。。more

Denis Vasilev ，

Введение в теорию обучения с подкреплением。 Истоки， принципы， теория， методы， практические примеры。 Читал второе издание， там уже есть и AlphaGo и DOTA。 Книга все же о теории， не руководство

Shyam Poovaiah ，

This is the book I read while following my RL course in IISc。A good supplemental material to this book would be David Silver's course on the same topic on YouTube。This book is succinct and provides the required intuitions。Along with the Math and the Algorithms the initial chapters provide the breakthroughs in Behavioral Psychology which led to progress in RL。 This is the book I read while following my RL course in IISc。A good supplemental material to this book would be David Silver's course on the same topic on YouTube。This book is succinct and provides the required intuitions。Along with the Math and the Algorithms the initial chapters provide the breakthroughs in Behavioral Psychology which led to progress in RL。 。。。more

indemidelo ，

Ancora in studio

Claude ，

C'est incontestablement la référence sur le reinforcement learning。 Et un gros pavé。La présentation des différentes techniques est claire et progressive， avec beaucoup de math et assez aride。 Un bon complément est le MOOC Coursera réalisé par l'université de Toronto qui s'appuie sur ce livre pour la structure du cours et des labs。 Ça rend le process un peu moins aride et focalise sur les techniques principales。 La difficulté est inhérente au domaine。 Le RL c'est beaucoup de techniques différente C'est incontestablement la référence sur le reinforcement learning。 Et un gros pavé。La présentation des différentes techniques est claire et progressive， avec beaucoup de math et assez aride。 Un bon complément est le MOOC Coursera réalisé par l'université de Toronto qui s'appuie sur ce livre pour la structure du cours et des labs。 Ça rend le process un peu moins aride et focalise sur les techniques principales。 La difficulté est inhérente au domaine。 Le RL c'est beaucoup de techniques différentes， provenant d'origine diverses et donc beaucoup de vocabulaire。 Les maths ne sont pas extrêmement complexes quand les formules des dérivées sont fournies。Je recommanderais aussi de commencer par balayer la 3ième partie du livre qui met le sujet en perspective : les rapports avec l'apprentissage humain ou animal， les applications （Go pour la plus connu）， les problématiques associées， le futur de ces techniques。 Cette partie est très vivante et passionnante。 Bien sûr à ce stade une moitié des explications sera incompréhensible。 Mais en reprenant depuis le début le contexte d'usage des techniques présentées sera plus clair et les termes importants ressortiront （value function， reward。。。）。 A la seconde lecture ce sera comme un révision pour tout bien remettre à la bonne place。 。。。more

Satyaki Upadhyay ，

Goes nicely with David Silver's lectures on youtube。 Goes nicely with David Silver's lectures on youtube。 。。。more

Adam ，

Dense and informative。 It would be helpful if there was a greater focus on building basic intuition with descriptive figures before diving into the technical details and heavy math。 Like many textbooks the exercises felt like enormous jumps from the material and a better guide into them would be nice。 Overall， though the book does give a nice tour-de-force of all the learning strategies developed to date and explores nuances of their behavior。

Gourav Sengupta ，

okay， no one can review this book， this is a book based on which the entire subject has evolved。 Its wonderfully written， lucid， clear， and helps to create the basis for evolving the subject for the next generations。

Nguyen ，

The first time， I read one chapter per day， like cramming。 Now， I am re-reading it for a second time， more carefully， and solving problems as well。

Tinwerume ，

Not exactly modern， but I think it provides a nice story of how various reinforcement learning algorithms fit together。

Hamed mansoury ，

این کتاب دروازه ای هست برای ورود به دنیای یادگیری تقویتییادگیری تقویتی نوعی یادگیری بر پایه پاداش و تنبیه ئه، شبیه به روشی که انسان یاد میگیرهیعنی کاری که براش سود داره رو بیشتر انجام میده و کاری که بهش آسیب میرسونه رو کمتر انجام میدهویژگی خوب این کتاب فراگیر بودنشه。 در هر ویرایش مباحث جدیدی بهش اضافه میشه。 کتاب سه قسمت داره。بخش اول از صفر شروع میکنه به گفتن مباحث و دانش پیشین خاصی نمیخواد。 یکم ریاضی و یکم آمار و احتمالاتمباحثی مطرح میشه مثل اینکه پاداش چیه، هدف رو چطوری پیدا کنیم، آینده نگر بود این کتاب دروازه ای هست برای ورود به دنیای یادگیری تقویتییادگیری تقویتی نوعی یادگیری بر پایه پاداش و تنبیه ئه، شبیه به روشی که انسان یاد میگیرهیعنی کاری که براش سود داره رو بیشتر انجام میده و کاری که بهش آسیب میرسونه رو کمتر انجام میدهویژگی خوب این کتاب فراگیر بودنشه。 در هر ویرایش مباحث جدیدی بهش اضافه میشه。 کتاب سه قسمت داره。بخش اول از صفر شروع میکنه به گفتن مباحث و دانش پیشین خاصی نمیخواد。 یکم ریاضی و یکم آمار و احتمالاتمباحثی مطرح میشه مثل اینکه پاداش چیه، هدف رو چطوری پیدا کنیم، آینده نگر بودن یا نبودن عامل و الگوریتمای ابتدایی یادگیری تقویتی توضیح داده میشنبخش دومتوی بخش اول همه اطلاعات توی جداول ذخیره میشه که عملا فقط برای مسائل کوچک مثل پیدا کردن مسیر توی یه جدول میشه ازشون استفاده کردبرای استفاده از یادگیری تقویتی توی مسائل بزرگتر （مثل بازیهای آتاری） نیازه که مقادیر با پارامترهایی تخمین زده بشن。 برای تخمین زدن این مقادیر باید از روشهای هوش مصنوعی （برای مثال، شبکه عصبی） استفاده کرد。 پس فصل دوم مثل اول ساده نیست و داشتن دانشی متوسط از هوش مصنوعی و روشهاش لازمهبخش سوم توی قسمت سوم کتاب هم کاربرد ها و چالشهای پیش رو برای این نوع یادگیری رو نویسنده توضیح دادهمزیت این روش نسبت به یادگیری ماشین اینه که نیاز به دانش و اطلاعات زیادی از محیط نیست و عامل طی تعامل با محیط ،فقط با پاداش و تنبیه میفهمه که چه کاری خوبه و چه کاری بدبرای یادگیری بازیهای کامپیوتری هم از یادگیری تقویتی استفاده میشهدر کل به همه دانشجوهای هوش مصنوعی پیشنهاد میکنم این کتاب رو بخونن 。。。more

Jaakko ，

It’s a good book but I wasn’t looking for something this theoretical right now。

Andrew ，

Read portions for CS7642 - Reinforcement Learning for Georgia Institute of Technology。 Great book， but very dense and requires patience to understand the nuances of what is going on。Merged review:Read portions for CS7642 - Reinforcement Learning for Georgia Institute of Technology。 Great book， but very dense language and requires patience to understand the nuances of what is going on。

Cristian Vasquez ，

This rating is for the second edition。

Will Dorrell ，

A really excellent textbook serving as an introduction to the field of Reinforcement Learning。Does a really great job in a lot of ways:i） very clear descriptions of almost all conceptsii） a very systematic description that allows you to build a framework for understanding the field， permitting you to fit new bits of information into that frameworkiii） Shows current work， extensions and touches on some recent exciting developmentsNot so good:i） trys very hard to be light on the maths， to the exte A really excellent textbook serving as an introduction to the field of Reinforcement Learning。Does a really great job in a lot of ways:i） very clear descriptions of almost all conceptsii） a very systematic description that allows you to build a framework for understanding the field， permitting you to fit new bits of information into that frameworkiii） Shows current work， extensions and touches on some recent exciting developmentsNot so good:i） trys very hard to be light on the maths， to the extent that it can limit understanding （especially on eligibility traces）ii） Can seem to waffle on a little at timesBut overall really truly excellent - a great guide to the uninitiated who are armed with the requisite background。 。。。more

John Doe ，

worth re-reading。great illustration on fundamental conceptual ideas。needs some time to internalize all the methods and tricks about RL。once you really got the idea， reinforcement learning becomes very intuitive。 yep， that's the most sensible way to build an automatic learning/optimizing robot。 worth re-reading。great illustration on fundamental conceptual ideas。needs some time to internalize all the methods and tricks about RL。once you really got the idea， reinforcement learning becomes very intuitive。 yep， that's the most sensible way to build an automatic learning/optimizing robot。 。。。more

☘Misericordia☘ ⚡ϟ⚡⛈⚡☁ ❇️❤❣ ，

Takeouts: thoughts on - Mone-Karlo methods，- dynamic programming， - learning from temporal differences。

Lara Thompson ，

Constantly building from simple algorithms to more complex ones and their variations。 A beautiful taxonomy of RL is built。 The examples are simple enough to try oneself and yet complex enough to distinguish the different approaches。 Highly recommend。

Aris ，

The absolute bible from the RL guru。 RL beautifully explained。 Heavy stuff， full of maths as you would expect from a textbook， but still quite approachable。

Abdullah Shams ，

Really natural progression and base covreage along the book， as you develpe from the broad to central， to the perticular， to the future。 Really recommend it to everyone who wants to build a knowledge around reinforcment lrearning and have a strong clear foundations。

Budi Prawira ，

Best reference book for Reinforcement LearningI used the first edition of this book as one of the key reference for my graduation thesis back in the 90s。 This second edition brings everything up to date。 It reminds me why I love RL so much。

Silvia Tulli ，

Reinforcement Learning Introduction

p-adic vacation ，

I read the first two parts。 Reinforcement learning is the real machine learning。 --me

Gresa ，

Concise introduction to the field that is fueling the development of autonomous agents。

Jean Martins ，

"Also related to TD learning are Holland's （1975， 1976） early ideas about consistency among value predictions。 These influenced one of the authors （Barto）， who was a graduate student from 1970 to 1975 at the University of Michigan， where Holland was teaching。 Holland's ideas led to a number of TD-related systems。。。" "Also related to TD learning are Holland's （1975， 1976） early ideas about consistency among value predictions。 These influenced one of the authors （Barto）， who was a graduate student from 1970 to 1975 at the University of Michigan， where Holland was teaching。 Holland's ideas led to a number of TD-related systems。。。" 。。。more

Alex Telfar ，

Really good textbook。I was surprised about how well we understand much of RL。 Coming from ML this was a welcome novelty。Although， I would have like to see a few more of the proofs and for there to be exercises。

Reference

google book https://www.google.com/search?tbm=bks&q=Reinforcement Learning: An Introduction

hathitrust digital library https://babel.hathitrust.org/cgi/ls?q1=Reinforcement Learning: An Introduction&field1=ocr&a=srchls&ft=ft&lmt=ft

The British Library https://bll01.primo.exlibrisgroup.com/discovery/search?query=any,contains,Reinforcement Learning: An Introduction&tab=LibraryCatalog&search_scope=Not_BL_Suppress&vid=44BL_INST:BLL01&lang=en&offset=0