Many techniques that depend on AI and machine learning effectively conduct trial-and-error testing at a massive scale. They help us identify statistically significant correlations between problems and solutions. However, they aren't necessarily able to explain why the solution works. As Jonathan Zittrain puts it, they provide answers without explanations.

In some cases this is fine; the benefits outweigh whatever risk that comes with this "intellectual debt" (Zittrain's phrase). But in other cases, algorithms may be responsible for massive societal impacts with no transparency to help explain why, and with no answer to those who may be negatively affected by these data-driven decisions.


Related

Citation

Zittrain, Jonathan. “The Hidden Costs of Automated Thinking.” The New Yorker, July 23, 2019. https://www.newyorker.com/tech/annals-of-technology/the-hidden-costs-of-automated-thinking.