BRANCH OPTIMIZATION METHOD FOR EXECUTION OF BIG DATA ETL (EXTRACT-TRANSFORM-LOAD)

Fecha de publicación: 02/06/2022
Fuente: Wipo "BigData"
The present invention discloses a branch optimization method for execution of a big data ETL model. The necessity of model execution can be analyzed according to the update characteristics of raw data sets and the characteristics of the ETL model; and optimization judgment is carried out on a plurality of operator branches of the ETL model, and for branches with lower update frequency, a middle repeated calculation process is skipped in a manner of reconstructing a cache table, so that the repeated execution rate is reduced from the operator aspect, the execution efficiency of the ETL model is improved, and the big data analysis is carried out more efficiently.