Algorithmization of repeated query optimization in cloud databases with the aid of computer training

In cloud environments, hardware configuration, data usage, and workload distribution are constantly changing. These changes make it difficult for the query optimizer of the cloud database management system to choose the optimal query execution plan (QEP). In scientific literature, it was proposed to re-optimize the query during its execution for the purpose of optimizing it with a more accurate cost estimate. However, some of these optimizations cannot provide performance gains in terms of query response time or monetary costs, which are the two optimization goals for cloud databases, and may have a negative impact on performance due to overhead. This raises the question of how to determine when the optimization is efficient. The aim of the study is to develop a method of repeated query optimization that uses computer training. The key idea of the algorithm is to employ past query executions to learn how to predict the effectiveness of query re-optimization, and this is done in order to help the query optimizer avoid unnecessary re-optimization of queries for future ones. The method runs the query step-by-step, utilizing a computer training model, to predict whether re-optimization of the query will be useful after the stage is completed, and calls the query optimizer to automatically perform re-optimization. An experimental evaluation of the effectiveness is to be carried out.

1. Bruno N., Jain S., Zhou J. Continuous cloud-scale query optimization and processing. VLDB Endow. 2013;11(6):961–972.

2. Wolf F., May N., Willems R., Sattler K.-U. On the Calculation of Optimality Ranges for Relational Query Execution Plans. 2018 International Conference on Management of Data (SIGMOD '18). 2018;663–675.

3. Deshpande A., Ives Z., Raman V. Adaptive Query Processing. Foundations and Trends in Databases. 2017;1(1):1–140.

4. Markl V., Raman V., Simmen D., Lohman G., Pirahesh H., Cilimdzic M. Robust query processing through progressive optimization. ACM SIGMOD International Conference on Management of data (SIGMOD '04). 2004;659–670.

5. Details omitted for double-blind reviewing.

6. Bankole A.A., Ajila S.A. Predicting cloud resource provisioning using machine learning techniques. 26th IEEE Canadian Conference on Electrical and Computer Engineering (CCECE). 2013;1–4.

7. Breiman L. Random Forests. Machine Learning. 2001;45(5):5–32.

8. Schmidhuber J. Deep Learning in Neural Networks: An Overview. Neural Networks. 2014;61(10):85–117.

9. Corinna C., Varnik V.N. Support-vector networks. Machine Learning. 1995;20:273–297.

10. Liu H., Xu M., Yu Z., Corvinelli V., Zuzarte C. Cardinality Estimation Using Neural Networks. 25th Annual International Conference on Computer Science and Software Engineering (CASCON '15). 2015;53–59.

11. Kipf A., Kipf T., Radke B., Leis V., Boncz P., Kemper A. Learned Cardinalities: Estimating Correlated Joins with Deep Learning. VLDB Endow. 2019;15(1):85–97.

12. Saravanan T., Shohedul H., Nick K., Gautam D. Approximate Query Processing for Data Exploration using Deep Generative Models. 36th International Conference on Data Engineering (ICDE). 2020;1309–1320.

13. Helff F., Gruenwald L., d'Orazio L. Weighted Sum Model for Multi-Objective Query Optimization for Mobile-Cloud Database Environments. EDBT/ICDE Workshops. 2016;1558:751–760.

14. Barata M., Bernardino J., Furtado P. An Overview of Decision Support Benchmarks: TPCDS, TPC-H and SSB. Advances in Intelligent Systems and Computing. 2015;353:619–628.