AI Is Changing Schema Matching in Ways Rule-Based Systems Couldn't
Every AI project assumes the data underneath it is already aligned. It almost never is.
Before a model trains, before a dashboard loads, before a retrieval pipeline returns anything useful, someone has to answer a question that sounds trivial and is not: do these two columns actually mean the same thing?
Right now a data engineer is staring at two spreadsheets trying to work that out. One column is called cust_id. Another calls it customer_number. A third file, inherited from a team that got acquired last year, went with CustomerRef. None of them agree on whether the value is an integer, a string, or a zero-padded code, and one stores it as a float because Excel got involved at some point, which is usually how these stories begin.
That is the job: working out that the three columns with three names and three types are probably the same thing. It...
Copyright of this story solely belongs to hackernoon.com. To see the full text click HERE
