Leveraging Kubernetes for Scalable Data Processing and Automation in Cloud DevOps

• Author(s): Rajkumar Kyadasu ; Sandhyarani Ganipaneni ; Sivaprasad Nadukuru ; Om Goel ; Niharika Singh; Prof. (Dr.) Arpit Jain
• Paper ID: 1705127
• Page: 546-571
• Published Date: 09-11-2024
• Published In: Iconic Research And Engineering Journals
• Publisher: IRE Journals
• e-ISSN: 2456-8880
• Volume/Issue: Volume 7 Issue 3 September-2023

Abstract

The adoption of Kubernetes has revolutionized the landscape of cloud-based DevOps by offering a robust framework for automating the deployment, scaling, and management of containerized applications. This paper explores how Kubernetes can be leveraged to enhance data processing capabilities and streamline automation in cloud DevOps environments. With the exponential growth of data, enterprises face challenges in ensuring scalability, efficiency, and reliability. Kubernetes addresses these challenges by orchestrating workloads across distributed environments, enabling seamless data pipeline execution and dynamic resource allocation. Furthermore, this study delves into the role of Kubernetes in automating DevOps workflows, facilitating continuous integration and continuous deployment (CI/CD) processes, and enabling auto-scaling based on real-time workloads. Key topics covered include the integration of Kubernetes with cloud-native tools, the use of operators and custom resource definitions (CRDs) to enhance automation, and strategies for managing stateful and stateless applications effectively. The findings highlight the benefits of using Kubernetes for building resilient, scalable data ecosystems and empowering DevOps teams to achieve faster release cycles with minimal downtime. The research concludes by emphasizing the potential of Kubernetes as a catalyst for innovation in cloud DevOps, fostering operational agility and accelerating the adoption of data-driven practices.

Keywords

Kubernetes, scalable data processing, cloud DevOps, automation, container orchestration, CI/CD pipelines, workload orchestration, cloud-native tools, auto-scaling, custom resource definitions (CRDs), stateful applications, continuous integration, continuous deployment, operational agility.

Citations

IRE Journals:
Rajkumar Kyadasu , Sandhyarani Ganipaneni , Sivaprasad Nadukuru , Om Goel , Niharika Singh; Prof. (Dr.) Arpit Jain "Leveraging Kubernetes for Scalable Data Processing and Automation in Cloud DevOps" Iconic Research And Engineering Journals Volume 7 Issue 3 2023 Page 546-571

IEEE:
Rajkumar Kyadasu , Sandhyarani Ganipaneni , Sivaprasad Nadukuru , Om Goel , Niharika Singh; Prof. (Dr.) Arpit Jain "Leveraging Kubernetes for Scalable Data Processing and Automation in Cloud DevOps" Iconic Research And Engineering Journals, 7(3)