The Power of Data, as Told Via An Economist and Medicine
A few months ago I was chatting with a friend, who is pregnant, about a new newsletter she was enjoying. She told me how Brown University Economics Professor Emily Oster has written a lot about her experiences with pregnancy and parenthood, and, more recently, the COVID-19 pandemic, through the lens of data transparency. Though these particular topics have been the emphases of her most recent work, she argues that lessons learned can be applied to all areas of medical interactions.
As someone who has had plenty of experience with medical professionals, recommendations, and diagnoses, I was intrigued. Oster’s main thesis in her articles (in outlets ranging from the New York Times, to the Wall Street Journal, the Atlantic, and many others) and books is that we deserve to know the data, or at least have the option to hear the data, behind the recommendations we hear from medical professionals so that we can make more informed decisions. Using an example regarding pregnancy, there is a big difference between “don’t drink a drop of alcohol while you’re pregnant,” “a glass of wine every once in a while is ok,” and “here are the studies behind this recommendation and here are the exact risks involved with each serving of alcohol you ingest.” Different people will have different preferences regarding which of those they want to hear, but very few medical professionals present example #3 as an option.
Now, I know how comforting a simple instruction can be, especially regarding something as emotional, overwhelming, and difficult-to-understand as health. Oster herself stipulates in her writing that her personal penchant for wanting every minute detail of her medical recommendations is unique. But most people would benefit from understanding a little bit more where their doctors are coming from, in terms of data that doesn’t necessitate a medical or statistical doctorate to understand. And in a lot of cases, that means that people may come to different conclusions than their doctors and each other in regards to what is best for them. This is certainly not to say that patients should not follow advice from their doctor or other medical professionals! But there are certain medical situations, and pregnancy is a really common one, in which there is more grey area than you might expect.
Most of Oster’s writing centers around two central claims:
- Much medical data (especially in the area of pregnancy, in which randomized trials are nearly impossible given ethical obstacles) is either flawed, mis-quoted, or both.
- Some recommendations based on said medical data is cut-and-dry. Some are not and are largely dependent on a patient’s preferences and risk-aversion.
I think the best example of both of these theses is evidenced in Oster’s first book, Expecting Better, which focuses on pregnancy. There are certain tests that can be performed to learn more about the genetic makeup of a fetus and the likelihood of certain chromosomal conditions like Down’s Syndrome. Two of these tests, a CVS and an amniocentesis also have risk of miscarriage associated with the procedures themselves. So, many parents have the make the decision whether or not to have these tests performed. This is obviously a very personal decision, which is complicated by the facts that data on the miscarriage risks of these procedures are very hard to quantify and that many sources quote a figure that is certainly inaccurate. So we run into our two issues here: quoting incorrect data and making cut-and-dry decisions on behalf of a patient.
In reality, the risk of miscarriage associated with either of these procedures is, while difficult to quantify, estimated about approximately 1 in 800. Most pregnant women are told the risk is 1 in 200, which is based on 2 studies from the 1970s and 1980s that have since been proven extremely flawed and therefore very likely inaccurate. One of them had outcomes that were statistically insignificant in the first place, but also completely disappeared once maternal age was taken into account. And yet, that 1-in-200 quote still makes its way around. So there’s problem #1: patients are not working with accurate data when making their own decisions. But problem #2 comes in when patients aren’t even encouraged to make that decision for themselves. This has improved in recent years, but for a long time, women over the age of 35 were simply told to have one of these invasive tests performed, and women under 35 were told not to. Why? Because for a 35+-year-old mother, the likelihood of finding an abnormality on one of these tests is higher than the likelihood of miscarriage, and for mothers sub-35, it is not. Now, setting aside the fact that even this assumption is flawed because the statistics used include the 1-in-200 figure that is straight-up incorrect. More importantly, this recommendation makes a huge assumption in the priorities of the parent(s). As much as I love relying on numbers to tell me what to do, when assessing risk, the raw numbers are only half the equation. The other, very important, half is accounting for the priorities and preferences of the people the decision affects.
A mother who is over 35 has a higher likelihood of carrying a child with, for example, Down’s Syndrome, than her 30-year-old counterpart. But as a medical professional, data scientist, economist, or anyone else, I cannot know how large that likelihood looms for her as compared to her (questionably) lower risk of miscarriage. It is up to her to decide which risk she wants to weigh more heavily.
It is important for medical professionals and patients alike to recognize both of Oster’s assertions and to make sure that they are working with correct information and making the right decisions for themselves (or, in a medical professional’s case, helping their patient come to the right conclusion for them with the medical expertise you can provide).
This is also an important thought exercise in the role of data in our lives. Question where your data is coming from. The most stark example of the issue of unreliable data I saw in Oster’s writing has to do with alcohol consumption. As I briefly mention above, randomized studies are very hard to come by in pregnancy. No one will (or should) ask 1,000 pregnant women to abstain from drinking completely in pregnancy, another 1,000 to have a glass of wine a day, and another to go on drinking binges several times a week. For this reason, most studies in pregnancy rely on observing women who are already engaging in, or have engaged in, certain behaviors without encouragement from the researchers. This makes it difficult to have true control groups. There is an often-cited study that concluded that one drink per day by a pregnant woman can cause behavioral problems for a child later in life. This conclusion relied on a study population in which 45% of the women who self-reported as having one drink per day also reported cocaine use during their pregnancies. Sure, maybe the alcohol caused some issues. But maybe it was the cocaine, which is ignored in the often-cited study conclusions. Again, this is a study that has heavily influenced the “not-even-one-drink” recommendation many patients receive.
Ok, I promise I’m almost done, but please indulge me in one more example: research shows a slight correlation in maternal weight with a baby’s birth weight — a mother who has a higher BMI is likely to have a baby with a higher birth weight, and vice versa. But complications associated with a low birth weight are much more severe than those associated with a high one. But in most medical recommendations, gaining-too-much and gaining-too-little in pregnancy are weighed (no pun intended) equally. If anything, many doctors harp more on instances of gaining “too much” than “too little”. The combinations of facts that this emphasis is misguided based on the data and reinforces messages women get all the dang time all around them every day, is demoralizing. This is an example in which preconceived notions of weight gain generally being unhealthy can override situation-specific medical data when it comes to medical recommendations. In this situation it is not the data that is flawed, but the communication of the data.
Again, while this blog post is focused on pregnancy-related examples, as is a hefty portion of Oster’s work, these ideas can be applied to other medical areas as well. Earlier this year my doctor recommended a change in my medication regimen and I didn’t fully understand why. Many late-night medical-paper-reading hours later, I felt a lot more comfortable with the new plan because, even though nothing was different than what she recommended, I just understood it more. For me, knowing what I was putting into my body and why put me at ease in a way that may not be the case for others.
A more serious example may include a patient with a serious cancer diagnosis, who should be given very accurate data to decide, for themselves, if they want to undergo treatment to try to fight it, or enjoy the time they have left without being weakened by chemotherapy or living their remaining months in a hospital room. Two people with identical prognoses may choose different routes based on their personal preferences and priorities. Our best friend data is a key component in making this decision, but not the be-all-end-all.
I will end with the two caveats I mentioned earlier but I believe bear repeating. First, please do not read this (or Oster’s work, for that matter), as permission to disregard what your medical professional recommends or to assume any statistics they quote you as incorrect. This all just serves as a reminder to ask questions and to make sure you’re working with correct information.
Second, knowing and assessing all the risks involved with any medical decision is not for everyone. There is nothing wrong with just wanting your doctor to tell you what to do, especially if that guidance is going to be the most medically conservative. Drink no alcohol or caffeine, and gain the exact recommended amount of weight during your pregnancy and you won’t be giving your baby or yourself any increased risk. But there are some situations, like the invasive prenatal tests for example, where decisions are necessary. And there are some people for whom that one glass of wine a week will make all the difference for their mental health and they don’t know that there is very little (if any) data to support the notion that they shouldn’t. People who gain a few pounds more than the recommend pregnancy range may worry much more than they have to. Even worse, they may try so hard to avoid gaining “too much” that they actually gain too little, which is statistically more likely to be harmful. Or maybe there are people like Oster (and me!) who feel much more at ease knowing what medical recommendations are based in, even if we are likely to follow them in the end. This is all to say that there is no one-size-fits-all method to communicating medical data to patients. But just as the assumption that they’ll want all the data in the world isn’t a fair one, neither is the assumption that they won’t have any interest or need for it. And, most importantly, that data has to be accurate, or at least properly communicated if it is inconclusive.
