Amy Chua and the Reflection Problem

I saw Amy Chua speak some years ago (at the World Bank, I think) about her book, World on Fire. She was an excellent, witty and provocative speaker, so much so that I read the book as a result. And while the book was indeed thought-provoking, it was not convincing. Her basic point was that democracies can produce instability: minorities in democracies can amass economic wealth, which in turn leads to resentment, which in turn leads to political scapegoating and instability and all other kinds of bad things. The problem is she never presented to me a convincing counterfactual; she never showed me how a world devoid of democracy would also be devoid of resentment and instability.

As a result of reading World on Fire, I will not read Battle Hymn beyond the excerpt in the Wall Street Journal (although I understand that the excerpt is not representative of the book). If the book is only a memoir, then it is almost certainly fine, but this is not how it is being represented. Instead, it is being characterized as a comparison between "Chinese parenting" and its results and "Western parenting" and its results. Again, this may be unfair to Ms. Chua, but the book has spurred myriad commentary about the virtues and deficiencies of various parenting styles.

What is lost in all of this is how difficult it is to actually draw inferences about the effects of parenting styles on outcomes. Charles Manski calls this "the reflection problem." Here is Manski:

Here is an identification problem from everyday life: Suppose that you observe the almost simultaneous movements of a person and of his image in a mirror. Does the mirror image cause the person's movements, does the image reflect the person's movements, or do the person and image move together in response to a common external stimulus? Empirical observations alone cannot answer this question. Even if you were able to observe innumerable instances in which persons and their mirror images move together, you would not be able to logically deduce the process at work. To reach a conclusion requires that you understand something of optics and of human behavior.

A like inferential problem, which I have called the reflection problem (Manski 1993a), arises if you try to interpret the common observation that individuals belonging to the same group tend to behave similarly. Two hypotheses often advanced to explain this phenomenon are endogenous effects, wherein the propensity of an individual to behave in some way varies with the prevalence of that behavior in the group; and correlated effects, wherein individuals in the same group tend to behave similarly because they face similar environments and have similar individual characteristics.

Similar behavior within groups could stem from endogenous effects (e.g., group members could experience pressure to conform to group norms) or group similarities might reflect correlated effects (e.g., persons with similar characteristics might choose to associate with one another). Empirical observations of the behavior of individuals in groups, even innumerable such observations, cannot per se distinguish between these hypotheses. To draw conclusions requires that empirical evidence be combined with sufficiently strong maintained assumptions about the nature of individual behavior and social interactions.

Why might you care whether observed patterns of behavior are generated by endogenous effects, by correlated effects, or in some other way? A good practical reason is that different processes have differing implications for public policy. For example, understanding how students interact in classrooms is critical to the evaluation of many aspects of educational policy, from ability tracking to class size standards to racial integration programs.

Suppose that, unable to interpret observed patterns of behavior, you seek the expert advice of two social scientists. One, perhaps a sociologist, asserts that pressure to conform to group norms makes the individuals in a group tend to behave similarly. The other, perhaps an economist, asserts that persons with similar characteristics choose to associate with one another. Both assertions are consistent with the empirical evidence. The data alone cannot reveal whether one assertion or the other is correct. Perhaps both are. This is an identification problem.

Whatever one thinks about Ms. Chua's parenting, we have no firm evidence whether her kids' outcomes are a function of Chinese parenting, Chua-specific parenting, or just her kids' endemic talents. It is a serious problem when we forget that.