The Data Mining Debate
1 comment(s)
It goes without saying that banks have an abundance of data. It is the job of the fraud analysts and investigators to sift through various bits of data to find anomalies and resolve fraudulent activity. These fraud subject-matter-experts usually can develop their own ad-hoc queries to find the information they are looking for. Surely, with access to disparate data and the ability to run queries, they can uncover fraud without deploying someone else’s software….right?
Most of the time, the answer is a resounding “no”. Very few banks have the time, money and expertise to develop an effective fraud detection solution singlehandedly. In its purest form, banking involves accepting deposits, and making loans. Why allocate resources to building a fraud detection engine? Most importantly, how many bank employees are statisticians that are experienced in mining data for fraud? I would guess not many…
Why is it so important to have advanced fraud statistics knowledge? Many years ago, one of my clients launched a project to centralize the company’s data in one data warehouse. The justification for doing so was in part based on the amount of fraud that could be by prevented by mining the data.
Over the course of the next couple of months, they used the collective wisdom of the fraud department to develop literally thousands of queries that generated pages, and pages of output. They then had the team test the accuracy of each report by randomly selecting transactions to investigate further. The vast majority of the time, the fraud suspects turned out to be normal transactions. The team did uncover some fraud, but given the time and effort expended, the return on investment was abysmal.
Why did they fail so miserably at finding fraud? Just like most banks, they had the data, a dedicated team of fraud investigators and the time to fully investigate each report. In hindsight, their approach contained three fatal flaws:
- No one on the team had any experience applying advanced statistics to large data sets. When applied correctly, statistics can uncover seemingly “hidden” relationships, reduce guesswork and provide positive proof that a report is of value.
- They assumed that their fraud investigator’s knowledge was all that they needed to uncover additional fraud. The investigators, although skilled, had not investigated every single type of fraud that the company could experience. The queries uncovered more of the fraud that they already knew took place. They failed to uncover new fraud schemes that the team had yet to investigate.
- By the time they realized that they lacked a broader, unfiltered view of fraud as well as advanced statistical techniques to uncover new fraud suspects, the political support for the data warehousing project had waned. The fraud team was asked to stand down and return to their normal functions.
Capturing the data, leveraging investigator experience and developing a fancy data mining engine does not mean that you’ll end up with an effective fraud detection engine. In fact, as the scenario above shows, you may end up portraying your department as inept and incapable of protecting the bank.