Cloud computing platform based big data processing method

Fecha de publicación: 07/09/2016
Fuente: Wipo "BigData"
The present invention relates to the technical field of Internet information processing, and particularly to a cloud computing platform based big data processing method. The method comprises according to the same information elements contained in different data records, using a big data processing framework under a cloud computing platform to abstract associated information of target information. The big data processing framework of the cloud computing platform, such as a MapReduce, a SparkIn and the like under a Hadoop system assigns the task to a plurality of nodes to be processed by automatic segmentation of the task, and the division and autonomy manner can implement parallel processing on massive data and greatly improves data processing efficiency. The method disclosed by the present invention, through the big data processing framework, can analyze associated information hidden behind massive target information in the complex Internet information according to requirements, and provides a reliable and convenient way for deepen excavation and application of data information, and provides effective technical support for target background analysis, marketing, market segmentation, risk prediction, risk prevention and control, and the like.