"if the people being evaluated are kept in the dark, the thinking goes, they'll be less likely to attempt to game the system. Instead, they'll simply have to work hard, follow the rules, and pray that the model registers and appreciates their efforts."
"So thanks to a highly questionable model, a poor school lost a good teacher, and a rich school, which didn't fire people on the basis of their students' scores, gained one."
"That's how trustworthy models operate. They maintain a constant back-and-forth with whatever in the world they're trying to understand or predict. Conditions change, and so must the model."
"Our own values and desires influence our choices, from the data we choose to collect to the questions we ask. Models are opinions embedded in mathematics."
"... sentencing models that profile a person by his of her circumstances help to create the environment that justifies their assumptions. This destructive model loop goes round and round, and in the process the model becomes more and more unfair."
"A young suburbanite with every advantage - the prep school education, the exhaustive coaching for college admissions tests, the overseas semester in Paris or Shanghai - still flatters himself that it is his skill, hard work, and prodigious problem-solving abilities that have lifted him into a world of privilege. Money vindicates all doubts."
"The top 20 percent of the population controls 89 percent of the wealth in the country, and the bottom 40 percent controls none of it. Their assets are negative: the average household in this enormous and struggling underclass has a net debt of $14,800."
"So even if our model is color blind, the result is anything but. In our largely segregated cities, geography is a highly effective proxy for race."
"The result is that we criminalize poverty, believing all the while that our tools are not only scientific but fair."
"...we've seen time and again that mathematical models can sift through data to locate people who are likely to face great challenges, whether from crime, poverty, or education. It's up to society whether to use that intelligence to reject and punish them - or to reach out to them with the resources they need. We can use the efficiency that make WMDs so pernicious in order to help people."
"... Simpson's Paradox: when a whole body of data displays one trend, yet when broken into subgroups, the opposite trend comes into view for each of those subgroups."
"While that campaign launched into public view, hundreds of others continue to hover below the surface, addressing individual voters."
"As this happens, it will become harder to access the political message our neighbors are seeing - and as result, to understand why they believe what they do, often passionately."
"As is often the case with WMDs, the very same models that inflict damage could be used to humanity's benefit. Instead of targeting people in order to manipulate them, it could line them up for help."
"Many of these models, like some of the WMDs we've discussed, will arrive with the best intentions. But they must also deliver transparency, disclosing the input data they're using as well as the results of their targeting. And they must be open to audits."
No comments:
Post a Comment