Big data governance platform based on Internet

Fecha de publicación: 20/01/2023
Fuente: Wipo "BigData"
The invention provides a big data governance platform based on the Internet, and relates to the technical field of big data. According to the big data management platform based on the Internet, the big data management platform based on the Internet comprises big data cluster node planning, disk array carrying, big data cluster hardware planning, big data cluster directory planning, KAFKA disk planning, KAFKA TOPIC partition planning, standby KAFKA TOPIC partition planning and HBase table Region planning, and the big data cluster node planning, the disk array carrying, the big data cluster hardware planning, the big data cluster directory planning, the KAFKA disk planning, the KAFKA TOPIC partition planning, the standby KAFKA TOPIC partition planning and the HBase table Region planning are included. The big data cluster node planning comprises the following steps of S1, planning HDFS cluster nodes, performing planning according to daily increment of a service system, determining a system data volume retention period, estimating the number of nodes according to disk storage capacity, obtaining a minimum value of the number of the nodes of the DATANODE, and properly increasing the number of the nodes of the DATANODE according to actual use requirements. By constructing the big data platform model, the problems that the big data platform does not have a forming mode for nesting, and only relevant hardware and software can be reconfigured during construction are solved.