Complete DevOps Project in 2025 | How I Built A Fully Automated DevOps Pipeline from Scratch

Introduction

In this tutorial, we will walk through the process of building a fully automated DevOps pipeline using modern tools and technologies. This project is designed to enhance your skills in DevOps practices, with a focus on automation, GitOps, and Kubernetes. By the end of this guide, you'll have a comprehensive understanding of how to create a Python web application, containerize it, and deploy it to a Kubernetes environment using Terraform, GitHub Actions, Helm, and ArgoCD.

Step 1: Create Terraform Configuration Files

Begin by creating the necessary Terraform configuration files to set up your Kubernetes environment.

1. Create the following files:

   • main.tf: Main configuration file.
   • provider.tf: Define the provider details.
   • backend.tf: Specify the backend for storing the state.
   • variables.tf: Define any variables needed for your configuration.

2. In provider.tf, specify the Minikube provider and configure your Kubernetes cluster resource.

3. In backend.tf, set up the Terraform state to manage your infrastructure effectively.

Step 2: Initialize and Apply Terraform Configuration

After creating the configuration files, you need to initialize and apply them.

1. Run the following command to initialize Terraform:

   terraform init
   

2. Apply your configuration to create the Minikube cluster:

   terraform apply
   

3. Verify the cluster setup with:

   kubectl get nodes
   

Step 3: Build the Python Application

Now, let’s develop a simple Python web application using Flask.

1. Create a new file named app.py and write the following code:

   from flask import Flask
   
   app = Flask(__name__)
   
   @app.route('/')
   def hello():
       return "Hello, DevOps!"
   
   if __name__ == '__main__':
       app.run(host='0.0.0.0', port=5000)
   

2. Test the application locally to ensure it’s functioning correctly.

Step 4: Containerize the Application with Docker

Next, we will create a Dockerfile to containerize the Python application.

1. Create a file named Dockerfile and add the following content:

   FROM python:3.8-slim
   
   WORKDIR /app
   
   COPY requirements.txt .
   RUN pip install -r requirements.txt
   
   COPY app.py .
   
   CMD ["python", "app.py"]
   

2. Create a requirements.txt file and include any dependencies (e.g., Flask).

Step 5: Set Up CI Pipeline with GitHub Actions

To automate the building and pushing of Docker images, set up a CI pipeline using GitHub Actions.

1. Create a .github/workflows/main.yml file and define the workflow:

   name: CI
   
   on:
     push:
       branches:
         - main
   
   jobs:
     build:
       runs-on: ubuntu-latest
       steps:
         - name: Checkout code
           uses: actions/checkout@v2
   
         - name: Build Docker image
           run: docker build . -t your-dockerhub-username/your-image-name
   
         - name: Push Docker image
           run: docker push your-dockerhub-username/your-image-name
   

2. Replace your-dockerhub-username and your-image-name with your actual Docker Hub username and desired image name.

Step 6: Create a Helm Chart for Deployment

Now, we will create a Helm chart to manage the Kubernetes deployment.

1. Use the Helm command to create a new chart:

   helm create myapp
   

2. Modify the values.yaml file to set the image name and tag based on your Docker image.

Step 7: Deploy ArgoCD and Configure GitOps

Set up ArgoCD to manage your Kubernetes resources using GitOps principles.

1. Deploy ArgoCD using Helm:

   helm install argocd argo/argo-cd
   

2. Access the ArgoCD dashboard and log in using the default credentials.

3. Create an application YAML file argocd-app.yaml linking your Helm chart to the ArgoCD application.

4. Configure ArgoCD to monitor your GitHub repository using a Personal Access Token.

Step 8: Sync and Test the Pipeline

Finally, enable auto-sync in ArgoCD and test your pipeline.

1. Apply the ArgoCD application configuration and enable auto-sync to automatically deploy changes.

2. To test the pipeline, make an update to your Python application and push the changes to GitHub.

3. Access the updated application using port forwarding:

   kubectl port-forward svc/myapp 5000:80
   

Conclusion

In this tutorial, we covered the steps to create a fully automated DevOps pipeline using a Python web application, containerization, and deployment to Kubernetes with Terraform, GitHub Actions, Helm, and ArgoCD. By implementing these practices, you can streamline your CI/CD workflows and gain valuable experience in modern DevOps methodologies. Consider exploring additional features and enhancements to further improve your pipeline. Happy coding!