Data Mining Process: Creation of the Analytical File with External Data Sources

Today, virtually all organizations conducting activities in data mining purchase external data sources. These data sources represent information a given company is unable to collect solely on the basis of its internal activities. Another perspective on this is that this type of data represents information that is available to the public either free or at some cost. The level of data mining sophistication will determine the extent of these external data source purchases. The reason for purchasing this data is to augment existing information about customers. How can this external data augment a company’s data about individual customers (especially when most of this external data is at a less granular or a more aggregate level, such as data about postal walks or census information)? This can be better understood if the kind of data available is considered at the level of the individual customer. For the most part, a very substantial portion of this data concerns transactions or purchase behavior. External data sources that are valuable to most organizations deal primarily with demographics that are in limited supply in an organization’s own database. For example, demographic variables at an individual level, such as age, income, and gender, may be missing more than 50% of their values when researchers try to extract this information from a customer database. The use of external overlay data can provide information related to these characteristics based on where the individual lives.