Review of Big Data Pre-processing

The massive growth in the scale of data has been observed in recent years being a key factor of the Big Data scenario. Big Data can be defined as high volume, velocity and variety of data that require a new high-performance processing. Addressing big data is a challenging and time-demanding task that requires a large computational infrastructure to ensure successful data processing and analysis. The presence of data pre-processing methods for data mining in big data is reviewed in this paper. The definition, characteristics, and categorization of data pre-processing approaches in big data are introduced. The connection between big data and data pre-processing throughout all families of methods and big data technologies are also examined, including a review of the state-of-the-art. In addition, research challenges are discussed, with focus on developments on different big data framework, such as Hadoop, Spark and Flink and the encouragement in devoting substantial research efforts in some families of data pre-processing methods and applications on new big data learning paradigms.