The Book of Why: The New Science of Cause and Effect
- Downloads：8654
- Type：Epub+TxT+PDF+Mobi
- Create Date：2021-06-06 08:55:08
- Update Date：2025-09-06
- Status：finish
- Author：Judea Pearl
- ISBN：0141982411
- Environment：PC/Android/iPhone/iPad/Kindle
- Download
Summary
'Wonderful 。。。 Illuminating 。。。 Fun to read' Daniel Kahneman， author of Thinking， Fast and Slow
A pioneer of artificial intelligence shows how the study of causality revolutionized science and the world
'Correlation does not imply causation。' This mantra was invoked by scientists for decades in order to avoid taking positions as to whether one thing caused another， such as smoking and cancer and carbon dioxide and global warming。 But today， that taboo is dead。 The causal revolution， sparked by world-renowned computer scientist Judea Pearl and his colleagues， has cut through a century of confusion and placed cause and effect on a firm scientific basis。 Now， Pearl and science journalist Dana Mackenzie explain causal thinking to general readers for the first time， showing how it allows us to explore the world that is and the worlds that could have been。 It is the essence of human and artificial intelligence。 And just as Pearl's discoveries have enabled machines to think better， The Book of Why explains how we can think better。
A pioneer of artificial intelligence shows how the study of causality revolutionized science and the world
'Correlation does not imply causation。' This mantra was invoked by scientists for decades in order to avoid taking positions as to whether one thing caused another， such as smoking and cancer and carbon dioxide and global warming。 But today， that taboo is dead。 The causal revolution， sparked by world-renowned computer scientist Judea Pearl and his colleagues， has cut through a century of confusion and placed cause and effect on a firm scientific basis。 Now， Pearl and science journalist Dana Mackenzie explain causal thinking to general readers for the first time， showing how it allows us to explore the world that is and the worlds that could have been。 It is the essence of human and artificial intelligence。 And just as Pearl's discoveries have enabled machines to think better， The Book of Why explains how we can think better。
Reviews
Serg Masís
，
At times long-winded but very well explained and comprehensive overview of causal inference and its history， and it's connection to real-world problems。 At times long-winded but very well explained and comprehensive overview of causal inference and its history， and it's connection to real-world problems。 。。。more
Rubén
，
Es muy interesante pero al final es un monólogo muy largo del autor explicando por qué todos son tontos menos él， que es muy listo - siempre lo ha sido - y ha entendido bien el problema y cómo resolverlo。 Aún así el concepto de inferencia causal es brutal， fascinante， y como punto de entrada está muy bien: es divulgativo y sencillo。
Cristián S
，
I think this book covers a really interesting topic， that is how causal thinking works， and how counterfactual questions are answered， showing that the classical statistical tools are not sufficient to do so， and then showing that the tools to do it are available through a new way of thinking。 Unfortunately， what I didn't like is the tone of the author during his explanations， it is full of ego， so instead of just making his point clear， it is constantly talking bad things about statisticians。 T I think this book covers a really interesting topic， that is how causal thinking works， and how counterfactual questions are answered， showing that the classical statistical tools are not sufficient to do so， and then showing that the tools to do it are available through a new way of thinking。 Unfortunately， what I didn't like is the tone of the author during his explanations， it is full of ego， so instead of just making his point clear， it is constantly talking bad things about statisticians。 This is funny at the beginning but after a while you get tired of that。 。。。more
Dana King
，
There's a lot going on in this book with complex connections between history， statistics， and philosophy of science。 After reading couple chapters， I realized I could find out what I wanted to know from this good review of it: http://bostonreview。net/science-natur。。。 There's a lot going on in this book with complex connections between history， statistics， and philosophy of science。 After reading couple chapters， I realized I could find out what I wanted to know from this good review of it: http://bostonreview。net/science-natur。。。 。。。more
Leonardo
，
A ver， no es que el libro esa malo， pero la verdad no me gustó。 Me costó seguirlo porque el tema no es sencillo， porque lo leí en inglés y porque en esta modalidad de trabajo estoy leyendo poco por día entonces lo leí muy salteado。 Es un libro de divulgación sobre los desarrollos de Judea Pearl en inferencia causal。 Creo que podría haber sido más emocionante porque la cuestión de poder extraer causalidad de la información es un tema super importante e interesante， más en este contexto de superab A ver， no es que el libro esa malo， pero la verdad no me gustó。 Me costó seguirlo porque el tema no es sencillo， porque lo leí en inglés y porque en esta modalidad de trabajo estoy leyendo poco por día entonces lo leí muy salteado。 Es un libro de divulgación sobre los desarrollos de Judea Pearl en inferencia causal。 Creo que podría haber sido más emocionante porque la cuestión de poder extraer causalidad de la información es un tema super importante e interesante， más en este contexto de superabundancia de datos （big data）， pero este libro no lo es。 Y creo que en un punto no lo es porque la gran cuestión es que no se puede extraer causalidad de los datos， aunque si de un modelo y se puede chequear con datos que el modelo tenga sentido。 Al menos esto es lo que entendí。 。。。more
Ben Sauk
，
Great book for understanding causality。
Arne Gevaert
，
Goed wi
Hendrik
，
What a read。 I never enjoyed reading a pop-scientific book （or any book， for that matter） such much as this。 It communicated in an enjoyable and clear way thoughts and ideas I always seemed to have but never succeeded in explaining neither to myself nor others。 Even though it goes quite deep into niches of science， it never fails to remain clear and engaging。 Judea Pearl manages to integrate anecdotes， jokes， and examples in a masterful way。 They never appear set-up， cheesy， or too self-referent What a read。 I never enjoyed reading a pop-scientific book （or any book， for that matter） such much as this。 It communicated in an enjoyable and clear way thoughts and ideas I always seemed to have but never succeeded in explaining neither to myself nor others。 Even though it goes quite deep into niches of science， it never fails to remain clear and engaging。 Judea Pearl manages to integrate anecdotes， jokes， and examples in a masterful way。 They never appear set-up， cheesy， or too self-referential。 I would love to forgot all about this book just to be able to read it for the first time again。 In any case， I am looking forward to the next time I will read it again。 And this time will come。 。。。more
Mike
，
Excellent！ Hopefully causal models will become integrated into our education system。
Manny
，
Well， I am not an expert on statistics， so maybe I'm missing something important， but I really don't understand all the negative criticism that I see in other reviews of this book。 Pearl， who has spent a long career working in an area which spans statistical reasoning， philosophy and AI， set himself an extremely ambitious goal: he wanted to establish a clear， logically consistent foundation for the notions of causality （"A makes B happen"） and counterfactuals （"B would have happened if A had hap Well， I am not an expert on statistics， so maybe I'm missing something important， but I really don't understand all the negative criticism that I see in other reviews of this book。 Pearl， who has spent a long career working in an area which spans statistical reasoning， philosophy and AI， set himself an extremely ambitious goal: he wanted to establish a clear， logically consistent foundation for the notions of causality （"A makes B happen"） and counterfactuals （"B would have happened if A had happened"）。 As he says， both statisticians and philosophers had been deeply mistrustful of both concepts， preferring only to talk about associations。Pearl gives coherent reasons to believe that that this is overcautious。 Human language is packed full of causality and counterfactuals: it's the fundamental substrate of our common worldview， we can't do without it。 To take just one of many flagrant examples， it's impossible to make sense of fundamental legal and moral concepts like "responsibility" or "guilt" without using this language。 If the prosecutor wants to convince the court that X is guilty of murdering Y， he needs to demonstrate that Y would have been alive had it not been for X's actions。 To say that this is philosophically or mathematically inadmissible is to deny the validity of the entire field of legal reasoning。 If you're familiar with the philosophical tradition， your knee-jerk response at this point may be to object "but what about Hume？"。 Pearl looks at what Hume actually says on the subject， and points out that his revised definition of causality is not just phrased in terms of associations。 He also realised that he needed to add counterfactuals。At least on his own account， Pearl and his students appear to have made a great deal of progress in attacking these thorny problems。 They have developed a way of thinking about them where the central construct is a "causal diagram"， a graph where different factors are connected by arrows representing hypothesized causal links。 Causal diagrams are a good match to people's intuitions about causality， and Pearl gives many examples showing how they support different kinds of reasoning。 Some of this reasoning is obvious， some of it is very subtle; some of it becomes obvious only after looking at the diagrams。 For example， a pattern which comes up many times in different forms is so-called "collider bias"， where two causal arrows meet at the same point: if you condition on the joining concept， you'll create a spurious association。 Pearl gives a cute illustration from the world of dating， where the folk wisdom is that the good-looking dates tend to be jerks。 His explanation is as follows。 Being good-looking and having a pleasant personality are both features that make someone more attractive。 It is reasonable to suppose that these two things may not actually be correlated。 But if your sample is drawn from the people you've dated， you're conditioning on the "attractiveness" variable: you're only looking at people who were attractive enough that you dated them。 This creates a spurious negative association between "good-looking" and "pleasant personality"。 So if someone is good-looking， they are more likely to have an unpleasant personality。 "Collider bias" is very simple compared to some of the things covered in the book。 Particularly impressive items are the "do-calculus" （an axiomatic framework for estimating the effect of performing an intervention）， and a set of formulas for measuring direct and indirect effects when one factor operates on another through an intermediary; for example， smoking causing cancer through the intermediary of tar。 Pearl describes the reasoning that led him to these ideas， where in many cases a deceptively simple formula is the product of several years of careful thought。Well， it's possible that I'm a sucker who's been taken in by good marketing。 But Pearl has an excellent reputation: he's published hundreds of widely cited papers and picked up just about every award going。 To me， he looks like the real deal。 I think I need to read his 2009 book Causality and download a causal inference package。 。。。more
Joao Pedro Martins
，
Too much of the author in it， several were all too idiosyncratic for me。 Also found several explanations hard to follow。 Ended up just skimming the latter part of the book。 Topic is， however， essential。 Just wasn't very well explained， imo。 Too much of the author in it， several were all too idiosyncratic for me。 Also found several explanations hard to follow。 Ended up just skimming the latter part of the book。 Topic is， however， essential。 Just wasn't very well explained， imo。 。。。more
Mario
，
An amazing book regarding Causality， and how causality is an integral part of any attempt to create a General Artificial Intelligence。 It also explains many probability， statistical， inferential， and epistemological concepts relevant to causality in a masterful， easy to grasp， and rich way。 A must read for anyone doing RCTs， policy research， AI， statistics or any other area of science。 It also includes， historical context of the genesis of causality science， many examples and graphical vignette An amazing book regarding Causality， and how causality is an integral part of any attempt to create a General Artificial Intelligence。 It also explains many probability， statistical， inferential， and epistemological concepts relevant to causality in a masterful， easy to grasp， and rich way。 A must read for anyone doing RCTs， policy research， AI， statistics or any other area of science。 It also includes， historical context of the genesis of causality science， many examples and graphical vignettes that really help the reader understand the concepts。Nevertheless， the author's seem to characterize econometrics and statistics in an extremely radical point of view。 I think the authors need to give more credit to these fields。 Also， I found surprising that they do not mention Jaynes's a Logic Of Science in the book。 The book of why seems to follow an extremely similar path。A must read！ 。。。more
Aaron Nielsen
，
Read as a part of book club hosted by the Statistics Department at the university。 I appreciated his discussion of confounding variables and how controlling for every possible confounder can actually be counterproductive in some scenarios。 It’s a reasonable introduction to causal analysis that is becoming more popular in academic departments。 The down side of the book is Pearl is a pompous ass and seems to think he belongs in the company of Newton， Copernicus， and Galileo。 He spent seemingly hal Read as a part of book club hosted by the Statistics Department at the university。 I appreciated his discussion of confounding variables and how controlling for every possible confounder can actually be counterproductive in some scenarios。 It’s a reasonable introduction to causal analysis that is becoming more popular in academic departments。 The down side of the book is Pearl is a pompous ass and seems to think he belongs in the company of Newton， Copernicus， and Galileo。 He spent seemingly half of the book complementing himself。 3。5*。 Enlightening yet irritating。 。。。more
Zach Goldstein
，
The historical parts， especially about cigarettes， are great。 Pearl is unusually optimistic about our ability to determine cause and effect。 The methods explained here are fundamental。
Daniel O'Brien
，
An interesting and informative book on a fascinating field that needs to be better understood by a wider range of people。 Sadly， the pedagogical value of the book is completely undermined by the author's anger and resentment at basically the entire field of statistics。 An interesting and informative book on a fascinating field that needs to be better understood by a wider range of people。 Sadly， the pedagogical value of the book is completely undermined by the author's anger and resentment at basically the entire field of statistics。 。。。more
Caurisa Marks
，
Fascinating book。 It was fairly easy for me to understand。
Simon Riley
，
The work of Pearl et al is the most important epistemological advancement in science in generations， the possibilities it opens are exhilarating， and this work is reasonably approachable。 I encourage you to read it。 But having attempted Pearl's Causality and found it impenetrable before discovering Shipley's Cause and Correlation in Biology （a MUCH better resource for applied researchers in the natural sciences） I have to say this still isn't the introduction I want for myself nor the introducti The work of Pearl et al is the most important epistemological advancement in science in generations， the possibilities it opens are exhilarating， and this work is reasonably approachable。 I encourage you to read it。 But having attempted Pearl's Causality and found it impenetrable before discovering Shipley's Cause and Correlation in Biology （a MUCH better resource for applied researchers in the natural sciences） I have to say this still isn't the introduction I want for myself nor the introduction I want to offer to more lay audiences。 The author admits in the introduction what a challenge it is to write for a general audience， and it is an admirable effort， but unfortunately I still find it somewhat lacking。 。。。more
James
，
A decent book on a fantastic topic。 Causal modelling and its implications are incredibly useful and important for statistics， epistemology， and AI。 The author does quite a good job making this case。 The content density isn't the best， though。 At times the author can digress or drag on too much。 I am very interested in this material， but I plan to continue by reading the textbook - that seems like a better way to glean the useful material from the Causal Revolution thus far。Notes:t• Clearly causa A decent book on a fantastic topic。 Causal modelling and its implications are incredibly useful and important for statistics， epistemology， and AI。 The author does quite a good job making this case。 The content density isn't the best， though。 At times the author can digress or drag on too much。 I am very interested in this material， but I plan to continue by reading the textbook - that seems like a better way to glean the useful material from the Causal Revolution thus far。Notes:t• Clearly causality is a useful way to model the world， and clearly we have some innate capacity for it。 Traditional statistics has just failed （or refused） to formalize it。 t• You do not really understand a topic until you can make a machine to do itt• Inference engine takes in assumptions， queries， and data。 It produces three outputs: assessment of whether the question is answerable， formula for answer for arbitrary data， and actual answer given data。t• Causal diagrams are a special case of Bayes netst• "Chain" junctions screen off information from either side。 If we have A -> B -> C， then p（A） gives me information about p（C）， and vice versa。 But if I know p（B）， then p（A） and p（C） don't tell me anything about the other anymore。 p（A） and p（C） are correlated， but p（A|B） and p（C|B） are not。t• "Fork" junctions create confounding correlations， until they are taken into account。 If I have A <- B -> C， then p（A） and p（C） will be correlated， until I know p（B）。 p（A） and p（C） are correlated， but p（A|B） and p（C|B） are not。t• "Collider" junctions are the opposite of forks。 If I have A -> B <- C， then p（A） and p（C） are uncorrelated。 But if you give me p（B）， that gives me information about both of them。 p（A） and p（C） are uncorrelated， but p（A|B） and p（A|C） are not。t• This means that raw data allows us to distinguish non-relations， colliders， and chains/forks， but does not allow us to distinguish between chains and forkstt○ Author says model-building from data is very hardt• The current statistical methods for identifying confounders vary， but the often end up looking like "control for the influence of any variable you can except the one you're interested in"。 This， however， can end up controlling for mediating variables （in A->B->C， B is a mediator， not a confounder）， which will produce a false negative for A's effect on Ct• In a causal diagram， arrows allow information to flow both ways。 This means that in a long bridge like A->B->C-E->F<-G<-H， A and H can still give us information about each other if all the junctions are "open"。 Chain and bridge junctions are open by default， collider junctions have to be "opened" by controlling for the middle variable。 So in this case， there is no flow of information by default， and controlling for C and F would allow flow！t• To deconfound two variables and determine the relationship X->Y， block all paths connecting X and Y that start with an arrow leading into X， without blocking any paths that start with an arrow out of X。 These are called "back door paths"， and they are the paths that have the potential to affect p（Y|X） and p（Y|do（X）） differently， since do（X） erases all arrows leading into X。 By doing this， we make sure at p（Y|X） = p（Y|do（X））。t• Another way is the "front door" adjustment， which sort of sums up all the effects over causal pathways leading from X to Y。 This is useful when we don't have info on confounders （so we can't do back-door）， but we do have info on mediatorst• Do-calculustt○ p（Y|do（X）， Z， W） = p（Y|do（X）， Z） if W is "screened off" by Z as discussed earliertt○ p（Y|do（X）， Z） = p（Y|X， Z） if Z satisfies the "back door" criteriontt○ p（Y|do（X）） = P（Y） if there is no path from X to Y with only forward-directed arrowst• Do-calculus is proved complete！ If it is possible to identify the causality from the given data， then the do-calculus is sufficient。 Otherwise， either gather more data or RCT it outt• There is a polynomial time decision algorithm for do-calculus proofs！t• Counterfactuals provably cannot be represented with do-calculust• Probability of necessity （PN） and probability of sufficiency （PS） are very different。 Something that is necessary has high PN - meaning in a scenario where the outcome is 1， removing the factor with high PN leads to it not happening。 Something that is sufficient has high PS - meaning that in a scenario where the outcome is zero， adding the factor that has high PS leads to the outcome being 1。tt○ PS（X） = P（Y_{X=1}=1 | X = 0， Y = 0）tt○ PN（X） = P（Y_{X=0} | X=1， Y=1）。t• Statisticians seem to keep making philosophical claims about what kind of statements are "allowed" or "not allowed"。 Ex: Robins and Greenland think that the mediation formula is out-of-bounds because it requires mixing information from two different counterfactual worlds， and that is not allowed。 On what grounds？ Our standards of what are and are not allowed are supposed to be tuned to predict the world。 If it works it works， no more to be said。 Just test itI still disagree with the author's persistent claims that causality cannot be inferred from data alone。 He keeps saying that you need to provide a model， and I agree with that - but where does he think models come from？ We build our models based on our observations。 We don't know how to teach a computer how we do that yet， but that doesn't mean we attribute model-building to "exogenous domain knowledge" and leave it at that！ I have a feeling I must be misunderstanding author's claim that causality cannot be inferred from data alone because this seems like too silly a mistake， but the point is important to write down regardless 。。。more
Leib Mitchell
，
3。0 out of 5 stars Sometimes Really Technical Things can't be made accesible for hoi polloiReviewed in the United States on July 20， 2019I found this book when I was visiting somebody for a dinner date。 It was the Basic Books label， and so I assumed that it had something to say。I cannot say that it didn't have anything to say， but I can say that it didn't say anything that I was interested in or could understand as presented。Sometimes Really Technical Fields really do need to be studied by going 3。0 out of 5 stars Sometimes Really Technical Things can't be made accesible for hoi polloiReviewed in the United States on July 20， 2019I found this book when I was visiting somebody for a dinner date。 It was the Basic Books label， and so I assumed that it had something to say。I cannot say that it didn't have anything to say， but I can say that it didn't say anything that I was interested in or could understand as presented。Sometimes Really Technical Fields really do need to be studied by going to lectures and solving problems out of a textbook。 It is not enough to try to merely put it into words that a reader can understand。 There have been other books that were published on the Basic Books label that were just like this one。 （I have in mind "The First 3 Minutes，" by Weinberg。）This author is talking about something called "Path Analysis" （among other things） and I'm pretty sure that if I sat through an undergraduate course on this I could understand it。But， would it really be worth it？I just have too many other things to study and so this book is not worth the time that it would take to understand it。I made it to page 130 before I gave the book back。Verdict: Not recommended。 。。。more
James Chapman
，
Learnt a lot and great as an intro to causality
Dan Raghinaru
，
Pearl explains why the theory and practice of statistics developed almost exclusively around correlation。 Moreover， when big data and AI fields started to employ more and more statistics， the need for causal explanations turned this limitation into a big problem。 For example， in order to show causality， statistics needs to add/employ concepts and practices like: randomization， intervention， control groups， prior specifications and limitations， hypotheses， prospective studies， adjustments， and so Pearl explains why the theory and practice of statistics developed almost exclusively around correlation。 Moreover， when big data and AI fields started to employ more and more statistics， the need for causal explanations turned this limitation into a big problem。 For example， in order to show causality， statistics needs to add/employ concepts and practices like: randomization， intervention， control groups， prior specifications and limitations， hypotheses， prospective studies， adjustments， and so on。 Pearl claims that his new theory of causality may bypass all of these limitations and difficulties; if a causal diagram can be provided along with the basic observational/correlation data。 In the end， it can be argued that causality is a metaphysical concept that cannot be discovered in the data; but only functions as a human category that directs the collection， organization， explanation， and so on of the data。 It feels to me that Pearl is trying to go beyond this in order to provide a scientific foundation for causality as if it really exists in the world。 His proposal in the last chapter to add free will， and thus a potential ability to make/assess counterfactual statements and to understand causality， to the current AI approaches seems not at all fundamental， practical， or significant to me。 In order to defend the introduction of causal diagrams and to defend their scientific status， Pearl stated in this book that: “Logic void of representation is metaphysics”。 This struck me as a strange statement; since logic understood as representation is metaphysics。 。。。more
Minervas Owl
，
This review has been hidden because it contains spoilers。 To view it， click here。 I am surprised that I hadn't heard of the causal diagram and the "do" operator before reading this book。 Based on such tools， Pearl and his collaborators developed important theoretical work such as the do-calculus。 But even from the non-academic's perspective， showing causal relationships in a diagram and separating P（Y|X） and P（Y|do（X）） are super powerful。 They make the sets of assumptions transparent and bring clarity to thinking and discussion。 For example， I was exhilarated when I read that I am surprised that I hadn't heard of the causal diagram and the "do" operator before reading this book。 Based on such tools， Pearl and his collaborators developed important theoretical work such as the do-calculus。 But even from the non-academic's perspective， showing causal relationships in a diagram and separating P（Y|X） and P（Y|do（X）） are super powerful。 They make the sets of assumptions transparent and bring clarity to thinking and discussion。 For example， I was exhilarated when I read that the concept of confounding can be expressed elegantly using the "block the backdoor path" criterion。 I find the five games of deconfounding in chapter 4 entertaining。 Then， I was amazed that every game used a causal diagram from an actual academic paper。 So the backdoor path criterion does help academic study！Among the many other exciting topics Pearl discussed， I am most impressed by one episode of the great debate about whether smoking causes lung cancer。 People have observed correlation， but statisticians such as Fisher hypothesize that a "smoking gene" might completely account for smokers' cancer risk。 A randomized trial is unethical， and it was long before the era of gene sequencing。 But self-taught statistician Cornfield published a solid rebuttal to Fisher: given that smokers have nine times the risk of developing lung cancer， if 11% of non-smokers have the "smoking gene，" then 99% of smokers should have such gene。 It is inconceivable that the choice of smoking， which is subject to individual wills， peer pressure， and cultural influence， can be so tightly determined by the gene。Near the end of the book， Pearl writes， "the culture of 'external validity' is totally preoccupied with listing and categorizing the threats to validity rather than fighting them。" I think this sentence speaks true of causal studies in general， and the story he told about Cornfield's study epitomizes this spirit of fighting。 For many questions， I often find a massive no man's land between two poles。 At one end， we have agnosticism。 At the other end， we have （1） pundits who readily wield any numbers in their favor， （2） business analysts who decide that practicality mandates that they should shun away from discussions of spurious correlation and "let the data speak。" Ideally， for matters cannot be readily settled by random experiments， we should sow in the middle land: reach a reasonable conclusion with humility but not be paralyzed by it。 Readers with training in economics have learned tools that help with this purpose， such as matching， instrumental variables， the difference in difference， and panel data analysis。 However， they can still benefit a lot from Pearl's book。 First， Pearl provides new tools such as the front-door method。 Second， he shows us how to avoid misusing standard tools， such as controlling and matching。Judea Pearl mentioned in an endnote that he avoided discussing cyclic models in the book， although they are important for economics。 I wonder how big challenge cycles create for causal studies and whether this is one reason why economists do not use causal diagrams。Overall， this is a brilliant and absorbing book。 I am not sure I understand everything， especially in the chapters about counterfactual and mediation analysis。 Worth re-reading later。 。。。more
Elizaveta
，
The book is a masterpiece: so many terms and theorems from the causal inference explained easy and with colorful examples。 It was a great helper along my learning way of climbing the ladder of causation。I was fascinated by the fact that many great things I knew before was actually developed by Judea Pearl （bayesian networks， belief propagation， do-calculus）。
Nafis Faizi
，
The book of why by Prof Judea Pearl and science writer Dana Mackenzie is one book that has profoundly affected me in many ways。 Before I write any further， it is important to underline my two personal biases- 1。 I am a student of Epidemiology， and 2。 I have reservations against this ‘data revolution’， the obsession that data alone will solve all the problems of the world。 So， the book is written as a quest to the understanding of causality starting with the oftly oversimplified cliché- ‘correlat The book of why by Prof Judea Pearl and science writer Dana Mackenzie is one book that has profoundly affected me in many ways。 Before I write any further， it is important to underline my two personal biases- 1。 I am a student of Epidemiology， and 2。 I have reservations against this ‘data revolution’， the obsession that data alone will solve all the problems of the world。 So， the book is written as a quest to the understanding of causality starting with the oftly oversimplified cliché- ‘correlation is not causation’ to discussing the evolution of causal theories since time， their successes and failures that has led to the current 'causal revolution' of our era and reflections on its profound implications in AI。 Despite being on a technical topic， the writers have brilliantly written it in the form of a quest with fascinating snapshots in the lives of the different scientists who have played a role in the quest。 Karl Pearson， Galton， Fisher， Barabra Burks， Thomas Bayes， Greenland， Neyman， Rubin， Van derWheele， Kruskal and many more epidemiologists and statisticians work around the topic alonwith their rivalries and their obsessions have been discussed amazingly well around this quest to know ‘why’。Part of the reason why this book is such an engaging read is the infectious enthusiasm of Judea Pearl。 Chapter after chapter， readers can almost visualize this infective enthusiasm and sincerity of Prof Pearl’s own quest towards the topic。 The enthusiasm helps readers sail through some of the mathematical jargons as well。Read this one to know humankind’s approach to the most basic problem of all times- ‘the cause’。 Read it to appreciate the fascinating causal revolution that has happened in the past couple of decades after Pearl’s own work in bayesianism。 Read it for the fascinating snapshots into the scientists we all know of （with some of their controversies and intense rivalries）。 Read this to know the sincere life of the most important mathematician alive on the face of the earth right now。 - Pearl's new fan。 。。。more
Marisa
，
Definitely a solid book， but I was often bored and had a hard time getting through it because of how much thorough explanation was provided for the basic stats that were foundational to understanding the newer concepts
Kinshuk Kashyap
，
I found it extremely boring to actually keep reading。 I'm familiar with the author's works in the sense that by diffusion his ideas have found their way to me， but the way it is all dragged on about in the book just didn't do it for me。 Maybe I'm just not the target audience。。。 I found it extremely boring to actually keep reading。 I'm familiar with the author's works in the sense that by diffusion his ideas have found their way to me， but the way it is all dragged on about in the book just didn't do it for me。 Maybe I'm just not the target audience。。。 。。。more
Roozbeh Daneshvar
，
This book was a must-read for me， although I admit that I could not understand many parts of it （despite stopping mid-way， going back a few chapters and re-reading them）。 I believe that the book could have better utilized examples to convey the concepts （many more examples were needed， as sometimes the narration became too abstract）。I still need to talk about some concepts with someone who has read the book （and I have arranged with two friends to talk about them some time in the future）。 Below This book was a must-read for me， although I admit that I could not understand many parts of it （despite stopping mid-way， going back a few chapters and re-reading them）。 I believe that the book could have better utilized examples to convey the concepts （many more examples were needed， as sometimes the narration became too abstract）。I still need to talk about some concepts with someone who has read the book （and I have arranged with two friends to talk about them some time in the future）。 Below I am bringing a few pieces of the book。 If students take two different standardized tests on the same material， the ones who scored high on the first test will usually score higher than average on the second test but not as high as they did the first time。 This phenomenon of regression to the mean is ubiquitous in all facets of life， education， and business。 Sons of tall men tend to be taller than average—but not as tall as their fathers。 Sons of short men tend to be shorter than average—but not as short as their fathers。 P（L | D） may be totally different from P（L | do（D））。 This difference between seeing and doing is fundamental and explains why we do not regard the falling barometer to be a cause of the coming storm。 Seeing the barometer fall increases the probability of the storm， while forcing it to fall does not affect this probability。 The ability to reflect on one’s past actions and envision alternative scenarios is the basis of free will and social responsibility。 the connection between imagining and causal relations is almost self-evident。 It is useless to ask for the causes of things unless you can imagine their consequences。 We say that one event is associated with another if observing one changes the likelihood of observing the other。 The goal of strong AI is to produce machines with humanlike intelligence， able to converse with and guide humans。 Deep learning has instead given us machines with truly impressive abilities but no intelligence。 The difference is profound and lies in the absence of a model of reality。 Intervention ranks higher than association because it involves not just seeing but changing what is。 “Whig history” was the epithet used to mock the hindsighted style of history writing， which focused on successful theories and experiments and gave little credit to failed theories and dead ends。 The modern style of history writing became more democratic， treating chemists and alchemists with equal respect and insisting on understanding all theories in the social context of their own time。 Unlike correlation and most of the other tools of mainstream statistics， causal analysis requires the user to make a subjective commitment。 She must draw a causal diagram that reflects her qualitative belief—or， better yet， the consensus belief of researchers in her field of expertise—about the topology of the causal processes at work。 She must abandon the centuries-old dogma of objectivity for objectivity’s sake。 Where causation is concerned， a grain of wise subjectivity tells us more about the real world than any amount of objectivity。 In addition， in many cases it can be proven that the influence of prior beliefs vanishes as the size of the data increases， leaving a single objective conclusion in the end。 A Bayesian network is literally nothing more than a compact representation of a huge probability table。 Confounding bias occurs when a variable influences both who is selected for the treatment and the outcome of the experiment。 Nature is like a genie that answers exactly the question we pose， not necessarily the one we intend to ask。 Fisher realized that an uncertain answer to the right question is much better than a highly certain answer to the wrong question。 Confounding， then， should simply be defined as anything that leads to a discrepancy between the two: P（Y | X） ≠ P（Y | do（X））。 I define confounding as anything that makes P（Y | do（X）） differ from P（Y | X）。 a back-door path is any path from X to Y that starts with an arrow pointing into X。 X and Y will be deconfounded if we block every back-door path （because such paths allow spurious correlation between X and Y）。 If we do this by controlling for some set of variables Z， we also need to make sure that no member of Z is a descendant of X on a causal path; otherwise we might partly or completely close off that path。 I consider the complete solution of the confounding problem one of the main highlights of the Causal Revolution because it ended an era of confusion that has probably resulted in many wrong decisions in the past。 “dose-response effect”: if substance A causes a biological effect B， then usually （though not always） a larger dose of A causes a stronger response B。 the cultural shocks that emanate from new scientific findings are eventually settled by cultural realignments that accommodate those findings—not by concealment。 A prerequisite for this realignment is that we sort out the science from the culture before opinions become inflamed。 Paradoxes arise when we misapply the rules we have learned in one realm to the other。 The lesson is quite simple: the way that we obtain information is no less important than the information itself。 This is a general theme of Bayesian analysis: any hypothesis that has survived some test that threatens its validity becomes more likely。 The greater the threat， the more likely it becomes after surviving。 In my opinion， a true resolution of a paradox should explain why we see it as a paradox in the first place。 conditioning on a collider creates a spurious association We live our lives as if the common cause principle were true。 Whenever we see patterns， we look for a causal explanation。 In fact， we hunger for an explanation， in terms of stable mechanisms that lie outside the data。 The most satisfying kind of explanation is direct causation: X causes Y。 When that fails， finding a common cause of X and Y will usually satisfy us。 Simpson’s paradox alerts us to cases where at least one of the statistical trends （either in the aggregated data， the partitioned data， or both） cannot represent the causal effects。 Path coefficients are fundamentally different from regression coefficients， although they can often be computed from the latter。 Rule 1 says when we observe a variable W that is irrelevant to Y （possibly conditional on other variables Z）， then the probability distribution of Y will not change。 We know that if a set Z of variables blocks all back-door paths from X to Y， then conditional on Z， do（X） is equivalent to see（X）。 We can， therefore， write P（Y | do（X）， Z） = P（Y | X， Z） if Z satisfies the back-door criterion。 We adopt this as Rule 2 of our axiomatic system。 Rule 3 is quite simple: it essentially says that we can remove do（X） from P（Y | do（X）） in any case where there are no causal paths from X to Y。 That is， P（Y | do（X）） = P（Y） if there is no path from X to Y with only forward-directed arrows。 From the point of view of causal analysis， this teaches us a good lesson: in any study of interventions， we need to ask whether the variable we’re actually manipulating （lifetime LDL levels） is the same as the variable we think we are manipulating （current LDL levels）。 This is part of the “skillful interrogation of nature。” Responsibility and blame， regret and credit: these concepts are the currency of a causal mind。 To make any sense of them， we must be able to compare what did happen with what would have happened under some alternative hypothesis。 our ability to conceive of alternative， nonexistent worlds separated us from our protohuman ancestors and indeed from any other creature on the planet。 Every other creature can see what is。 Our gift， which may sometimes be a curse， is that we can see what might have been。 mistaking a mediator for a confounder is one of the deadliest sins in causal inference and may lead to the most outrageous errors。 The latter invites adjustment; the former forbids it。 In fact he had the right idea when he distinguished between bias and discrimination。 Bias is a slippery statistical notion， which may disappear if you slice the data a different way。 Discrimination， as a causal concept， reflects reality and must remain stable。 Anytime you see a paper or a study that analyzes the data in a model-free way， you can be certain that the output of the study will merely summarize， and perhaps transform， but not interpret the data。 Data interpretation means hypothesizing on how things operate in the real world。 。。。more
Eve
，
I did not finish。。。
Danielle
，
Does not work well as an audiobook （too many figures that must be referenced）。 Enjoyable overall， interesting examples， and the last chapter was very thought-provoking。
Nader Hajj Shehadeh
，