Isn't trying a plausible sociological intervention then measuring its effectiveness a win regardless of how effective it ended up being? Either way, we know more about what's likely to work for future interventions. It's the scientific method.
These interventions target very complex systems. It's not unexpected that we need to try a bunch of things that don't work before we find something that does, to the extent that most of the things we try are ineffective, right?
Edit: I wrote the above before reaching the end of the article. By the end, the article reaches a similar conclusion, albiet with less science and more mysticism.
Maybe that means it's a good article, leading me to reach its conclusion before taking the stance itself. On the other hand, the article's leading sections gave such a strong impression that the author thought something needs to be drastically changed about the current system — the opposite stance from its conclusion — on seemingly shakey grounds, that I almost didn't finish reading it. I'm not sure what to think, aside from please leave innocent goats alone.
GrantMoyer
One very successful social intervention in the UK was the push to reduce drink-driving. Starting around the 80s I think, maybe earlier. There was a big media campaign with TV ads as well as implementing and enforcing stricter laws. What I find interesting is that many in the older generation still found it more acceptable to have a few drinks and drive, whereas those growing up during this period largely find it very socially unacceptable.
ryan_glass
> One of my favorite genres of psychology experiment is called something like: “the study is probably bullshit but the effect is probably real”. At the top of that heap is The Illusion of Control, a classic from 1975 and a favorite of Psych 101 classes everywhere. Usually, this study gets summarized as “people value their lottery tickets more when you let them pick the numbers vs. when you pick the numbers for them.” It’s as if people think that the mere act of picking imbues their ticket with a bit of juju, which makes them less willing to part with it.
> That would be a beautiful study, but it wasn’t the study that was actually conducted. The “lottery tickets” in question were in fact football cards, each one with a picture of a player, their team, etc. Some participants were given the opportunity to choose their football card and some had a card handed to them. Later, just before the drawing, all of the participants were asked how much it would take to buy their card back. On average, the pickers asked for $8.67, while the not-pickers said $1.86.4
> I’m sure you can see the obvious and less exciting explanation for these results: when people were given the chance, they picked their favorite players. The people who didn’t get to pick ended up with a card they liked less. This “illusion of control” effect is merely a “reality of people having preferences and acting on them” effect.5
Alternatively, people who are knowledgeable of a subject will have selfishly biased opinions about it, which they value more than unknowledgeable people will value an arbitrary stance on the subject. From experience, this seems to affect things beyond situations of chance like lotteries, and infests collaborations like businesses in ways such as Not Invented Here Syndrome or generally poor management choices that fail to consider situational factors (e.g. overarching decrees that are ineffective in certain situations/locations/applications).
shit_game
The article talks about measurement of impact as though it's a totally value-neutral idea, but it's absolutely not:
* Measurement requires substantial logistical overhead; both up-front in creating a plan for measurement, and during intervention due to the extra record keeping involved. This is a (probably frequently large) drain on program efficacy. Just listen to any schoolteacher's take on standardized testing, evaluation rubrics, etc.
* Some contexts, and some types of intervention are more amenable to measurement than others. Insisting that nothing which is not measured can be valuable biases our decision-making towards the most measurable actions, which are likely not the best. Strong correlation with simplistic actions, short timelines, and lack of responsiveness to individual context.
* Goodhart's law.
Author (and everyone) would benefit from the book Seeing Like a State.
mwhitfield
this reminds me of the thought that a country run by scientists and engineers should be better. but there's none, and i think this can easily be understood by Daniel Kahneman's fast/slow thinking.
in other words, government is time-constrained, so when a problem is sufficiently complex, there's just not enough time to really flesh out a good solution.
also, humans can be selfish and will occasionally try to game the system, which changes your problem into an active adversary. and when you have active adversaries, you're playing a completely different game (indeed, when designing the policy in the first place you may not have even considered it a game with opponents!).
yowayb
tough article. 90% is a tongue in cheek ridicule, with lots of data points, and then the bait and switch at the end, with far less argumentation.
mainly, interventions are experiments, and therefore we need more of them, you never know. but they are not experiments, otherwise we wouldn't repeat failed ones over and over, or let existing failing ones run on and on.
maybe the answer really is more simple, that most programs are indeed politically motivated endeavors, indeed largely signals of a worldview, less programs to bring about material improvement.
fwiw - i dont know the answer, but having your cake and eat it, being perceptive of the absurdity, yet, not daring to call it out, is an author's disposition, less a truth. so maybe, if we do the one hand other hand game, accept the bleak truth glimpsed in the facts, but also accept that there's nothing to be done about it within existing frameworks, and not point at it as something desirable.
trgn
> Plain packaging — where you strip cigarette boxes of their attractive branding — seems like a no brainer, but in Australia, it looks like the policy caused smokers to switch to cheaper brands and to smoke more
This example provides an easily understood illustration of why so many social programs fail. Simply put, the people proposing and creating the programs have zero understanding of the people whose lives they are trying to influence and control. Like generals ensconced in their offices located not even on the same continent as the front line they study maps and tables and hear the advice of experts with even less direct experience than themselves and then dispatch orders that arrive too late and attempt to command a reality that does not exist. The reports they receive back show nothing has changed or the situation has got worse and yet they never adapt or see their own fault. Why should they? They've suffered no consequence.
pigpop
Lots of examples in this article, but none of them go deep enough to matter. Just like people bringing up the cigarette issue, I see other factors in sone of the other topics, such as medical school (residency slots weren't mentioned, default assumption was motivators were solely economic, etc).
I'm not sure what to even discuss about this article due to the low quality. I guess there overall point is that policies and interventions don't necessarily scale, or perform the same across cultures/cohorts, or even accomplish their intended goals. I would largely agree with those points, but not due to the way the article got there.
giantg2
In most of the examples given the aim is to be seen to be doing something. It is cheaper and/or avoids unwanted change than doing something that actually works.
For example, lots of organisations will have diversity officers and training. Very few do things such as taking names, languages spoken (obviously you would leave any that are relevant to the job) and other clues to sex and race from job applications before shortlisting people for interview.
graemep
> When Arkansas added a work requirement to Medicaid, the number of Arkansans with health insurance went down without any increase in employment.
The author is assuming good faith on behalf of the Arkansas state government that the purpose of the requirement was to increase employment, rather than merely cutting down on the number of citizens successfully receiving Medicaid.
comments (10)
These interventions target very complex systems. It's not unexpected that we need to try a bunch of things that don't work before we find something that does, to the extent that most of the things we try are ineffective, right?
Edit: I wrote the above before reaching the end of the article. By the end, the article reaches a similar conclusion, albiet with less science and more mysticism.
Maybe that means it's a good article, leading me to reach its conclusion before taking the stance itself. On the other hand, the article's leading sections gave such a strong impression that the author thought something needs to be drastically changed about the current system — the opposite stance from its conclusion — on seemingly shakey grounds, that I almost didn't finish reading it. I'm not sure what to think, aside from please leave innocent goats alone.
GrantMoyer
ryan_glass
> That would be a beautiful study, but it wasn’t the study that was actually conducted. The “lottery tickets” in question were in fact football cards, each one with a picture of a player, their team, etc. Some participants were given the opportunity to choose their football card and some had a card handed to them. Later, just before the drawing, all of the participants were asked how much it would take to buy their card back. On average, the pickers asked for $8.67, while the not-pickers said $1.86.4
> I’m sure you can see the obvious and less exciting explanation for these results: when people were given the chance, they picked their favorite players. The people who didn’t get to pick ended up with a card they liked less. This “illusion of control” effect is merely a “reality of people having preferences and acting on them” effect.5
Alternatively, people who are knowledgeable of a subject will have selfishly biased opinions about it, which they value more than unknowledgeable people will value an arbitrary stance on the subject. From experience, this seems to affect things beyond situations of chance like lotteries, and infests collaborations like businesses in ways such as Not Invented Here Syndrome or generally poor management choices that fail to consider situational factors (e.g. overarching decrees that are ineffective in certain situations/locations/applications).
shit_game
mwhitfield
in other words, government is time-constrained, so when a problem is sufficiently complex, there's just not enough time to really flesh out a good solution.
also, humans can be selfish and will occasionally try to game the system, which changes your problem into an active adversary. and when you have active adversaries, you're playing a completely different game (indeed, when designing the policy in the first place you may not have even considered it a game with opponents!).
yowayb
mainly, interventions are experiments, and therefore we need more of them, you never know. but they are not experiments, otherwise we wouldn't repeat failed ones over and over, or let existing failing ones run on and on.
maybe the answer really is more simple, that most programs are indeed politically motivated endeavors, indeed largely signals of a worldview, less programs to bring about material improvement.
fwiw - i dont know the answer, but having your cake and eat it, being perceptive of the absurdity, yet, not daring to call it out, is an author's disposition, less a truth. so maybe, if we do the one hand other hand game, accept the bleak truth glimpsed in the facts, but also accept that there's nothing to be done about it within existing frameworks, and not point at it as something desirable.
trgn
This example provides an easily understood illustration of why so many social programs fail. Simply put, the people proposing and creating the programs have zero understanding of the people whose lives they are trying to influence and control. Like generals ensconced in their offices located not even on the same continent as the front line they study maps and tables and hear the advice of experts with even less direct experience than themselves and then dispatch orders that arrive too late and attempt to command a reality that does not exist. The reports they receive back show nothing has changed or the situation has got worse and yet they never adapt or see their own fault. Why should they? They've suffered no consequence.
pigpop
I'm not sure what to even discuss about this article due to the low quality. I guess there overall point is that policies and interventions don't necessarily scale, or perform the same across cultures/cohorts, or even accomplish their intended goals. I would largely agree with those points, but not due to the way the article got there.
giantg2
For example, lots of organisations will have diversity officers and training. Very few do things such as taking names, languages spoken (obviously you would leave any that are relevant to the job) and other clues to sex and race from job applications before shortlisting people for interview.
graemep
The author is assuming good faith on behalf of the Arkansas state government that the purpose of the requirement was to increase employment, rather than merely cutting down on the number of citizens successfully receiving Medicaid.
AdmiralAsshat