I described this to a colleague as the disappointing power of statistics. With traditional rule-based systems, you had been forced to build models that emerged from careful understanding of the problem. You had to articulate the structure of the domain before you could write code that operated on it, and the act of articulation was where most of the learning happened. Data-driven methods broke that requirement. You could now process large datasets with techniques that surfaced patterns and produced solutions without any of that prior knowledge work having been done. The result was capability without the comprehension that traditionally came hand-in-hand.
What I see in AI today is the same problem, accelerated and scaled to a level my MSc students could not have imagined ten years ago. The basic questions I started raising in supervisions have become questions about the constitution of knowledge itself.
The Disappointing Power of Statistics - by David Millard
This feels so spot on. Driving on,y with data, only with AI abstracts away the human component. That part holders judgement, experience, choice.