recent pieces, bad social science

on some of my methodological pet peeves <3

Hi, all!

I’ve written two pieces in recent weeks—I write one every week these days, so, go figure—that I hope you’ll read.

The first is about the filmmaker Werner Herzog, whom I love:

https://www.washingtonpost.com/books/2023/10/05/werner-herzog-every-man-himself-god-against-all-memoir-review/

The second is about The Two-Parent Privilege, a recent book arguing that, because children in two-parent households have better outcomes later in life, society should foster and promote marriage:

https://www.washingtonpost.com/books/2023/10/14/two-parent-privilege-melissa-kearney-review/

If you’ll allow it—and you could scarcely stop it!—I’ll expand briefly on the argument I made in the piece, which is about the perils of overly reductive and hubristic social science. What I say targets The Two-Parent Privilege in particular, because it’s always useful to have a specific example, but I think and hope you’ll be able to see how my points end up applying to a wider range of social scientific forays (though—and I can’t emphasize this enough—most certainly not to all social scientific forays).

Let’s begin with the many things that I’m not saying in the piece, because the sorts of arguments I’m making are so easily (and dare I say, so often willfully) misread. I’m not saying that anyone in favor of marriage is irretrievably biased; I’m not saying that social science in general is irretrievably biased; I’m not saying that no one should get married (I myself am married—to a quantitative social scientist!); and I’m certainly not saying that children in two-parent households aren’t advantaged in consequential ways (“It is not hard to guess why two-parent households might be more conducive to adult success,” I write).

Here are some of the things I am saying (some of which I was also saying in my original piece, some of which are elaborations). “Bias” of a certain kind is probably inevitable in the social sciences (as in all fields of human inquiry), but biases are not created equal. Some biases—generally the uninterrogated ones—doom inquiry; others do not. The bias displayed by Melissa Kearney, the author of The Two-Parent Privilege and an economist at the University of Maryland, is not benign. It calls her conclusions into question—not her quantitative conclusions, which are straightforward implications of her calculations, but her qualitative conclusions, none of which are necessitated by raw statistics.

I use the word “bias” advisedly here. The reason I think “bias” in the relevant sense is inescapable—and yet sometimes benign—is because I don’t think of it as a matter of individual human frailty. Instead, I think of it as an inexorable feature of the very act of modeling. Models precede statistical findings, so, to state the obvious, they are not themselves statistical findings. The way we set the models up--which sorts of cases we choose to compare, how we choose to define the phenomena we’re studying, and so on—is not itself determined by the math. On the contrary, it determines what statistical strategies we pursue. That models are always a product of choices and therefore rest on contentious philosophical assumptions is emphatically not a reason to give up on modeling. Models are valuable tools, but they are at most one piece of a richer picture. There’s no shame in being one piece of a richer picture! Every method is at most one piece of a richer picture. The way to respond to the limitations of our native methods is to opt for collaboration and methodological promiscuity—to turn to other methods (and other thinkers) to fill in the gaps. Unfortunately, the way that Kearney responds is by doubling down on the insistence that she is neutrality personified, that she is just helplessly following where the numbers lead her. Because she refuses to countenance that the set-up of her models is ideological, she doesn’t take the time to justify what are obviously philosophical, and not merely statistical, choices.

Again, let me stress: there is nothing wrong with using economic models to understand a limited piece of the enormous, fraught puzzle that is the question of marriage and human flourishing. Problems arise only when a limited method develops inflated ambitions, presenting itself as comprehensive. It must be admitted that delusions of disciplinary grandeur afflict thinkers of all sorts, not just social scientists. Humanists, for instance, are prone to the self-serving and misguided belief that arm-chair hand-waving is a suitable replacement for fieldwork. Appeals to vibes in contexts that demand data are rampant. My objection to Freddie DeBoer’s recent book, for instance, was precisely that he was making empirical claims for which he did not have empirical evidence. But social scientists are certainly not immune to disciplinary jingoism, and Kearney has committed the cardinal sin of imagining that an inquiry of definite but limited utility can answer a gigantic question about a complex human good.

Let’s be more specific. (The following will probably make more sense if you’ve read my review, which begins by summarizing the book. You should ~~~click on the Washington Post link and then subscribe to the paper~~~~.) Kearney is importing ideology into her work at three points.

1. When she chooses to evaluate whether marriage benefits children by comparing children in households with two happily married parents to children in households with single parents, rather than comparing children in households with single parents to children in households with unhappily married parents. (True, her studies just compare children with two parents to children with one. But generally, the reason why parents tend to stay together is that they’re at least minimally happy together. So the relevant comparison is counter-factual, I would think, and is to unhappily married parents.) The data do not compel her to make one comparison rather than the other. The question of which cases it is profitable or informative to compare arises well before any numbers are crunched, so it is not decided by the numbers in question.

2. When she defines marriage as a resource-sharing institution. Again, this definition is not compelled by the data. Again, it precedes her quantitative inquiries and therefore cannot be the product of them.

3. When she makes normative prescriptions on the basis of her descriptive findings. (Note: I use “descriptive” here in the philosophical, and not the statistical, sense.) Kearney frames her prescriptions—among them, that we ought to foster a norm of marriage—as mere appanages of her descriptive findings. But the fact children in two-parent households tend to have higher incomes and educational attainments later in life does not necessarily mean that we should promote marriage, absent an additional assumption (for instance, that we should do everything we can to ensure that children end up with large incomes and many educational attainments, no matter what other costs we incur or which other prohibitions we violate, or that there are no other factors to weigh). She has identified a disparity that I happen to agree is in want of remedy, but her findings do not recommend any particular course of action by themselves.

Let me be clear about what, exactly, I object to in all this. Again, the problem is not that she is doing social science, with its resident methodological contortions. The problem is that she is not supplementing her findings with a healthy dose of critical thinking, and, to make matters worse, she is pretending that the numbers do all the moralizing by themselves. To run through my gripes in turn:

1. I simply think that Kearney makes a bad comparison. There is no point in determining whether children with two happy parents do better than children with one parent—unless one-parent families have a real chance of becoming happy two-parent families. So, do they? If Kearney acknowledged that her comparison makes sense only given certain background assumptions, she could have justified those background assumptions, or at least addressed that they are background assumptions; she could have explained why she thinks (if she does) that most one-parent families could easily become happy two-parent families. Admittedly, her book contains an anecdote about a cab driver she meets and chats with briefly. He tells her that he is not married to his daughter’s mother, even though he loves both the woman and the child. It’s a sweet anecdote, but it’s hardly an argument. I’m skeptical that this case is representative; I suspect that, on the contrary, the relevant alternative almost all of the time is an abusive, unstable, or deeply unhappy household. But I’m willing to be proven wrong! If Kearney devised a study showing that most single parents could marry their estranged partners without serious frictions or negative consequences for either the children or either of the adults, then I’d find her initial comparison more fruitful.

2. I think this is a false but useful assumption, and Kearney acknowledges as much. It’s certainly good for us to learn about the ways in which marriage, as a resource-sharing institution, confers resources on children. But if we want to understand whether marriage, the actual thing, is good, overall, we’ll have to understand it as more than a resource-sharing institution.

3. But--and here (2) relates to (3)— we cannot recommend marriage on the basis of the benefits conferred by marriage qua resource-sharing institution, because marriage is, in reality, much more than a resource-sharing institution. Kearney’s data may demonstrate pretty neatly that, if we want to optimize for resource-conferral to children, and we do not care about anything else, then we should optimize marriage. But first of all, do we want to optimize for resource-conferral, and if so, why? And more importantly, what about all the other things that marriage offers (or threatens) when we zoom out and consider it in its entirety, and not in a form truncated for the purpose of a model? (Again, to really belabor this often misunderstood point: the problem is not that it was truncated for the purpose of modeling; the problem is that Kearney uncritically infers claims about the real thing from findings about the truncated article.)

Shouldn’t the many other relevant factors at least count as costs to be weighed, even if they’re ultimately outweighed? It isn’t obvious how should we weigh the cost of two adults living in misery with romantic partners they loathe against the benefits that a child with two parents enjoys (even assuming these resources aren’t negated by the tensions in the household). Nor is it obvious how to weigh the resources of time and money a child with unhappy parents enjoys against the more nebulous but probably palpable disadvantages of fielding emotional turmoil, etc. Maybe parents should sacrifice their happiness for the purpose of ensuring that their children have higher educational attainments later in life. But I have trouble seeing how numbers alone could ever decide the matter. Kearney asks and answers one question: “what should we do if we want to maximize the resources each child receives?” She does not answer, because she does not ask, “What should we do, all things considered?”

A related—albeit more complex—issue: when Kearney makes policy recommendations, she seems to be hewing pretty closely to reality. The assumption seems to be that we should hold reality fixed, for the most part, but make some minor changes at the margins. (To her credit, she stops short of claiming we should mandate marriage or proscribe divorce.) But if we’re in the business of imagining a different world, why can’t we imagine a really different world? Why can’t we demand anything better than our bad world, exactly as is, except more people are married and one or two mentorship programs have been instituted? And moreover, if Kearney is willing to call for us to “work to restore and foster a norm of two-parent homes for children” (which, what does that even mean or entail?), why isn’t she willing to call for a return to a more communal form of child-rearing, in which neighborhoods, extended family, and the like had some hand? Again, this choice is not mandated by the numbers. After all, she identifies two mechanisms by which two-parent households benefit children: they have larger incomes (two people earn more than one), and they devote more time and attention to their children (two people have more time and emotional energy than one). Both of these things can be supplied by any number of institutions! If there is some reason to prefer marriage as a solution, then what is it? Simply that marriage is the solution that we already have? But we’re in the business of making normative demands—that is, in deviating from the reality we already have—so why should the mere conventionality of marriage recommend it? I’m sure there are arguments (maybe not ones I’d accept, but I’d have to hear them to know if I accept them, wouldn’t I?) about why marriage is preferable to alternatives—about why marriage is good—but Kearney doesn’t make any, because again, she glibly acts as if the statistics have simply done this work for her.

TL;DR It’s fine that social science is—necessarily—saturated with values, but it isn’t fine if you pretend it isn’t and don’t take the time to justify those values.