Skip to main content

Command Palette

Search for a command to run...

DevOps Week 8 – Advanced Kubernetes Management, Observability & Monitoring β˜ΈοΈπŸ“Š

Master Kubernetes Custom Resources (CRDs), Custom Controllers, ConfigMaps, Secrets, Observability, Prometheus, Grafana Monitoring, Metrics Collection, and Production-Ready Kubernetes Operations.

Updated
β€’27 min read
DevOps Week 8 – Advanced Kubernetes Management, Observability & Monitoring β˜ΈοΈπŸ“Š
H
πŸ‘‹ Hi, I’m Hritik Ranjan β€” a B.Tech CSE student and a passionate tech enthusiast focused on Quality Engineering, AI/ML, Cybersecurity, and DevOps. πŸ’‘ I enjoy building and testing scalable, secure, and intelligent systems that solve real-world problems. My expertise and interests include: πŸ”Ή Quality Assurance & Testing Hands-on experience in manual and automation testing using Selenium & Java, ensuring high-quality and reliable applications. πŸ”Ή Artificial Intelligence & Machine Learning Exploring advanced algorithms and developing intelligent systems for practical use cases. πŸ”Ή Cybersecurity Focused on vulnerability assessment, security testing, and system hardening. πŸ”Ή Web Development Building responsive and user-friendly applications using modern technologies. πŸ”Ή Data Science Analyzing complex data to extract actionable insights. πŸ’Ό Key Projects: πŸš€ Blindness Detection System Applied computer vision techniques to detect blindness-related conditions. πŸš€ AI-Powered Rail Madad Enhancement Developed an intelligent complaint management system to improve railway customer service. πŸš€ Interactive Applications Built multiple projects like quiz apps, calculators, and productivity tools. 🌱 I’m continuously learning and improving my skills in DevOps, Cloud, and Automation to become a well-rounded engineer. 🀝 Open to collaborations, internships, and opportunities in QA, DevOps, AI/ML, and Cybersecurity. πŸ“« Let’s connect: hritikranjan1408@gmail.com

Kubernetes Custom Resources (CR), CRD & Custom Controller ☸️


πŸ“Œ Why Do We Need Custom Resources in Kubernetes?

Kubernetes already provides built-in resources like:

  • Pod

  • Deployment

  • Service

  • ConfigMap

  • Secret

But in real-world production environments, companies need extra functionality such as:

  • Service Mesh (Istio)

  • GitOps (ArgoCD)

  • Monitoring (Prometheus)

  • Security Tools

  • Database Operators

Kubernetes cannot manage all these specialized tools by default.

πŸ‘‰ So Kubernetes provides Custom Resources to extend its capabilities.


🌍 What is a Custom Resource (CR)?

A Custom Resource (CR) is a new object type created by users inside Kubernetes.

It works like native Kubernetes objects.

Example:

Instead of creating only Pods or Deployments, we can create:

  • Database

  • Backup

  • Monitoring

  • VirtualService

  • Application

as Kubernetes resources.


πŸ“– What is a CRD (Custom Resource Definition)?

A CRD is a YAML configuration that tells Kubernetes:

β€œHey Kubernetes, a new resource type exists.”

It defines:

βœ… Resource name
βœ… API version
βœ… Structure/schema
βœ… Validation rules


πŸ’‘ Simple Real-Life Example

Imagine Kubernetes is a smartphone.

Built-in apps = Native Kubernetes resources.

CRD = Installing a new app.

CR = Using that app.

Controller = Background service that makes the app work.


πŸ› οΈ Example of CRD

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  name: databases.mycompany.com

spec:
  group: mycompany.com

  names:
    kind: Database
    plural: databases

  scope: Namespaced

  versions:
    - name: v1
      served: true
      storage: true

πŸ‘‰ This CRD creates a new Kubernetes resource called:

Database

πŸ“¦ What is a Custom Resource (CR)?

After CRD creation, users can create actual objects using that resource type.

Example:

apiVersion: mycompany.com/v1
kind: Database

metadata:
  name: mysql-db

spec:
  engine: mysql
  size: small

This is called a:

βœ… Custom Resource (CR)


πŸŽ›οΈ What is a Custom Controller?

A Custom Controller continuously watches Kubernetes resources.

It checks:

β€œHas a new custom resource been created?”

If yes:

βœ… It performs actions automatically.


πŸ”„ How Custom Controller Works?

Example:

User creates:

kind: Database

Controller sees it and automatically:

βœ… Creates Pods
βœ… Creates Storage
βœ… Configures networking
βœ… Sets permissions
βœ… Monitors database

This automation is the core power of Kubernetes Operators.


βš™οΈ Complete Workflow Explained

Step 1 β€” Deploy CRD

DevOps Engineer deploys:

kubectl apply -f crd.yaml

Kubernetes now understands the new resource type.


Step 2 β€” Deploy Controller

Controller is deployed inside the cluster.

It watches the custom resources.


Step 3 β€” User Creates CR

User creates:

kind: Database

Step 4 β€” Controller Takes Action

Controller automatically performs required operations.


🏒 Real-World Tools Using CRDs & Controllers

Tool Purpose
Istio Service Mesh
ArgoCD GitOps
Prometheus Operator Monitoring
Cert-Manager SSL Certificates
Crossplane Cloud Infrastructure
Elastic Operator Elasticsearch Management

πŸ”₯ Why Are Custom Controllers Important?

Without controller:

❌ CRD is only a schema/template
❌ No actual automation happens

Controller is the β€œbrain” behind custom resources.


🧠 Beginner-Friendly Analogy

Component Real-Life Example
CRD Form Template
CR Filled Form
Controller Employee processing the form

πŸ›‘οΈ Benefits of CRDs & Controllers

βœ… Extend Kubernetes functionality
βœ… Full automation
βœ… Reduce manual work
βœ… Infrastructure as Code
βœ… Easy scaling
βœ… Production-ready operations


πŸ’» Which Language is Used for Controllers?

Most Kubernetes controllers are written in:

Golang (Go)

Because Kubernetes itself is written in Go.

Popular frameworks:

  • client-go

  • controller-runtime

  • Kubebuilder


πŸ“š Important Interview Questions


❓ What is a CRD in Kubernetes?

βœ… Answer:

CRD allows users to create new custom resource types inside Kubernetes.


❓ What is the difference between CRD and CR?

CRD CR
Defines new resource type Actual instance/object
Works like template Works like real object

❓ Why is Controller required?

βœ… Answer:

Controller watches resources and performs automation tasks based on desired state.

Without controller, CRD alone does nothing.


❓ What is an Operator in Kubernetes?

βœ… Answer:

An Operator is an advanced custom controller that automates application management inside Kubernetes.


πŸ”„ Beginner-Friendly Architecture Flow

DevOps Engineer
        ↓
Deploy CRD
        ↓
Deploy Controller
        ↓
User Creates Custom Resource
        ↓
Controller Watches Resource
        ↓
Automation Happens
        ↓
Pods / Services / Storage Created

πŸ“˜ Kubernetes ConfigMaps & Secrets Explained β˜ΈοΈπŸ”


πŸ”Ή Why ConfigMaps & Secrets Are Important? πŸ€”

In real-world applications, many values change depending on the environment.

Examples:

  • Database URL

  • Database Port

  • API Endpoints

  • Passwords

  • API Keys

  • Tokens

If these values are written directly inside application code:
❌ Application becomes difficult to manage
❌ Security risks increase
❌ Updating configuration requires rebuilding application


πŸ”Ή Real-World Example πŸ’‘

Suppose your application runs in:

  • Development

  • Testing

  • Production

Each environment uses:

  • Different database

  • Different API URLs

  • Different credentials

Instead of changing code every time, Kubernetes provides:
βœ… ConfigMaps
βœ… Secrets

to manage configurations separately.


πŸ”Ή What is a ConfigMap? βš™οΈ

ConfigMap is a Kubernetes object used to store:

Non-sensitive configuration data

Examples:
βœ… Database host
βœ… Application mode
βœ… Port numbers
βœ… Feature flags
βœ… Connection types


πŸ”Ή Why Use ConfigMaps? πŸš€

ConfigMaps help:
βœ… Separate configuration from code
βœ… Reuse same application image in multiple environments
βœ… Simplify updates
βœ… Improve maintainability


πŸ”Ή Beginner-Friendly Example πŸ’‘

Instead of writing:

DATABASE_PORT = 5432

inside application code, store it in:

Kubernetes ConfigMap

Application reads it dynamically.


πŸ”Ή What is a Secret? πŸ”

Secrets are Kubernetes objects used to store:

Sensitive information securely

Examples:
βœ… Passwords
βœ… API keys
βœ… Tokens
βœ… Certificates
βœ… Database credentials


πŸ”Ή Why Secrets Are Important? ⚠️

Sensitive data should never be:
❌ Hardcoded in code
❌ Stored publicly
❌ Shared insecurely

Secrets help protect:

Confidential application data

πŸ”Ή Difference Between ConfigMap & Secret βš”οΈ

ConfigMap Secret
Non-sensitive data Sensitive data
Plain configuration Passwords/API keys
Easier visibility Base64 encoded
Lower security requirements Higher security requirements

πŸ”Ή How Kubernetes Stores Secrets? πŸ—‚οΈ

Kubernetes stores Secrets inside:

etcd database

By default: Secrets are:

Base64 encoded

πŸ”Ή Important Beginner Note ⚠️

Base64 encoding is:

NOT strong encryption

It only hides data in encoded format.

For production environments, additional tools are recommended:
βœ… HashiCorp Vault
βœ… Sealed Secrets
βœ… External Secret Managers


πŸ”Ή RBAC & Secrets Security πŸ”’

Secrets should always be protected using:

RBAC (Role-Based Access Control)

Only authorized users should access:

  • Passwords

  • Tokens

  • Certificates


πŸ”Ή Ways to Use ConfigMaps Inside Pods πŸ“¦

The instructor demonstrated two major methods:

βœ… Environment Variables
βœ… Volume Mounts


πŸ”Ή ConfigMaps as Environment Variables 🌍

ConfigMap values can be injected directly into containers as:

Environment variables

Applications can read values dynamically.


πŸ”Ή Real-Life Example πŸ’‘

Instead of changing code, developers can update:

Kubernetes ConfigMap

and application automatically uses new configuration.


πŸ”Ή Example ConfigMap YAML πŸ“„

apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config
data:
  DATABASE_HOST: postgres-service
  DATABASE_PORT: "5432"

πŸ”Ή Using ConfigMap in Pod as Environment Variable βš™οΈ

env:
- name: DATABASE_HOST
  valueFrom:
    configMapKeyRef:
      name: app-config
      key: DATABASE_HOST

πŸ”Ή Benefits of Environment Variables βœ…

βœ… Easy to configure
βœ… Beginner friendly
βœ… Good for small configurations


πŸ”Ή Limitation of Environment Variables ⚠️

If ConfigMap changes:

Container restart is usually required

to reflect updated values.


πŸ”Ή ConfigMaps as Volume Mounts πŸ“‚

Another powerful approach is:

Mounting ConfigMaps as files inside Pods

πŸ”Ή Biggest Advantage of Volume Mounts πŸš€

If ConfigMap changes:

Files update automatically inside Pod

without restarting container.


πŸ”Ή Why This is Important? πŸ’‘

This helps:
βœ… Avoid downtime
βœ… Dynamically update configuration
βœ… Improve production stability


πŸ”Ή Real-World Example 🌍

Suppose: Application uses:

nginx.conf

Instead of rebuilding container, you update ConfigMap.

Kubernetes automatically updates configuration file inside Pod.


πŸ”Ή Example Volume Mount YAML πŸ“„

volumeMounts:
- name: config-volume
  mountPath: /etc/config

volumes:
- name: config-volume
  configMap:
    name: app-config

πŸ”Ή Why Volume Mounts Are Preferred in Production? πŸš€

Because they provide:
βœ… Dynamic updates
βœ… No downtime
βœ… Better scalability
βœ… Easier maintenance


πŸ”Ή Managing Kubernetes Secrets πŸ”

The instructor demonstrated:
βœ… Creating Secret
βœ… Storing password securely
βœ… Accessing Secret inside Pod


πŸ”Ή Example Secret YAML πŸ“„

apiVersion: v1
kind: Secret
metadata:
  name: db-secret
type: Opaque
data:
  password: cGFzc3dvcmQxMjM=

πŸ”Ή Why Secret Value Looks Strange? πŸ€”

Because Kubernetes stores Secret values in:

Base64 encoded format

πŸ”Ή Decoding Secret Value πŸ§ͺ

Example:

echo cGFzc3dvcmQxMjM= | base64 --decode

Output:

password123

πŸ”Ή Important Production Recommendation πŸš€

For enterprise environments, companies usually use:
βœ… HashiCorp Vault
βœ… AWS Secrets Manager
βœ… Azure Key Vault
βœ… Sealed Secrets

for stronger security.


πŸ”Ή Real-World Production Use Cases 🏒

ConfigMaps are used for:
βœ… Application configuration
βœ… Feature toggles
βœ… Database endpoints
βœ… Environment settings

Secrets are used for:
βœ… Passwords
βœ… API keys
βœ… Certificates
βœ… Tokens


πŸ”Ή Beginner-Friendly Architecture ☸️

ConfigMap / Secret
          ↓
      Kubernetes Pod
          ↓
 Environment Variable / Volume Mount
          ↓
      Application Uses Config

πŸ”Ή Important kubectl Commands βš™οΈ


βœ” View ConfigMaps

kubectl get configmaps

βœ” View Secrets

kubectl get secrets

βœ” Describe ConfigMap

kubectl describe configmap app-config

βœ” Describe Secret

kubectl describe secret db-secret

βœ” Create ConfigMap

kubectl create configmap app-config --from-literal=PORT=8080

βœ” Create Secret

kubectl create secret generic db-secret --from-literal=password=password123

πŸ”₯ Real-World Scenario Based Questions


❓ Why should configurations not be hardcoded?

βœ… Answer:

Because configurations change across environments and hardcoding makes applications difficult to maintain and insecure.


❓ Why use ConfigMaps?

βœ… Answer:

ConfigMaps separate configuration from application code and allow dynamic configuration management.


❓ Why use Secrets instead of ConfigMaps for passwords?

βœ… Answer:

Secrets are designed for sensitive information and provide better security controls.


❓ Why are Volume Mounts preferred in production?

βœ… Answer:

Because ConfigMap updates automatically reflect inside Pods without restarting containers.


❓ Is Base64 encoding secure encryption?

βœ… Answer:

No. Base64 is only encoding, not strong encryption.

Additional security tools are recommended for production.


πŸ”₯ Interview Tip πŸš€

If interviewer asks:

Difference between ConfigMap & Secret?

Best answer:

ConfigMaps store non-sensitive configuration data, while Secrets securely store sensitive information like passwords and API keys.

πŸ’‘ Introduction to Observability

  • Observability is the ability to understand the internal state of a system by analyzing the data it produces, including logs, metrics, and traces.

  • Monitoring(Metrics): involves tracking system metrics like CPU usage, memory usage, and network performance. Provides alerts based on predefined thresholds and conditions

    • Monitoring tells us what is happening.
  • Logging(Logs): involves the collection of log data from various components of a system.

    • Logging explains why it is happening.
  • Tracing(Traces): involves tracking the flow of a request or transaction as it moves through different services and components within a system.

    • Tracing shows how it is happening.

πŸ€” Why Monitoring?

  • Monitoring helps us keep an eye on our systems to ensure they are working properly.

  • Perpose: maintaining the health, performance, and security of IT environments.

  • It enables early detection of issues, ensuring that they can be addressed before causing significant downtime or data loss.

  • We use monitoring to:

    • Detect Problems Early

    • Measure Performance:

    • Ensure Availability:

πŸ€” Why Observability?

  • Observability helps us understand why our systems are behaving the way they are.

  • It’s like having a detailed map and tools to explore and diagnose issues.

  • We use observability to:

    • Diagnose Issues:

    • Understand Behavior:

    • Improve Systems:

πŸ†š What is the Exact Difference Between Monitoring and Observability?

  • πŸ”₯ Monitoring is the when and what of a system error, and observability is the why and how
Category Monitoring Observability
Focus Checking if everything is working as expected Understanding why things are happening in the system
Data Collects metrics like CPU usage, memory usage, and error rates Collects logs, metrics, and traces to provide a full picture
Alerts Sends notifications when something goes wrong Correlates events and anomalies to identify root causes
Example If a server's CPU usage goes above 90%, monitoring will alert us If a website is slow, observability helps us trace the user's request through different services to find the bottleneck
Insight Identifies potential issues before they become critical Helps diagnose issues and understand system behavior

πŸ”­ Does Observability Cover Monitoring?

  • Yes!! Monitoring is subset of Observability

  • Observability is a broader concept that includes monitoring as one of its components.

  • monitoring focuses on tracking specific metrics and alerting on predefined conditions

  • observability provides a comprehensive understanding of the system by collecting and analyzing a wider range of data, including logs, metrics, and traces.

πŸ–₯️ What Can Be Monitored?

  • Infrastructure: CPU usage, memory usage, disk I/O, network traffic.

  • Applications: Response times, error rates, throughput.

  • Databases: Query performance, connection pool usage, transaction rates.

  • Network: Latency, packet loss, bandwidth usage.

  • Security: Unauthorized access attempts, vulnerability scans, firewall logs.

πŸ‘€ What Can Be Observed?

  • Logs: Detailed records of events and transactions within the system.

  • Metrics: Quantitative data points like CPU load, memory consumption, and request counts.

  • Traces: Data that shows the flow of requests through various services and components.

πŸ†š Monitoring on Bare-Metal Servers vs. Monitoring Kubernetes

  • Bare-Metal Servers:

    • Direct Access: Easier access to hardware metrics and logs.

    • Fewer Layers: Simpler environment with fewer abstraction layers.

  • Kubernetes:

    • Dynamic Environment: Challenges with monitoring ephemeral containers and dynamic scaling.

    • Distributed Nature: Requires tools that can handle distributed systems and correlate data from multiple sources.

πŸ†š Observing on Bare-Metal Servers vs. Observing Kubernetes

  • Bare-Metal Servers:

    • Simpler Observability: Easier to collect and correlate logs, metrics, and traces due to fewer components and layers.
  • Kubernetes:

    • Complex Observability: Requires sophisticated tools to handle the dynamic and distributed nature of containers and microservices.

    • Integration: Necessitates the integration of multiple observability tools to get a complete picture of the system.

βš’οΈ What are the Tools Available?

  • Monitoring Tools: Prometheus, Grafana, Nagios, Zabbix, PRTG.

  • Observability Tools: ELK Stack (Elasticsearch, Logstash, Kibana), EFK Stack (Elasticsearch, FluentBit, Kibana) Splunk, Jaeger, Zipkin, New Relic, Dynatrace, Datadog.

πŸ“˜ Kubernetes Monitoring Using Prometheus & Grafana β˜ΈοΈπŸ“Š


πŸ”Ή Why Monitoring is Important in Kubernetes? πŸ€”

Imagine you have deployed an application successfully.

After a few days:

  • Pods start crashing

  • CPU usage becomes very high

  • Memory gets exhausted

  • Users report application downtime

Without monitoring: ❌ You won't know what went wrong.

Monitoring helps DevOps teams:
βœ… Detect issues early
βœ… Track cluster health
βœ… Analyze performance
βœ… Troubleshoot failures
βœ… Improve availability


πŸ”Ή Real-Life Example πŸ’‘

Think of Kubernetes like an airplane.

Even if the airplane is flying smoothly, pilots continuously monitor:

  • Fuel level

  • Engine temperature

  • Speed

  • Weather conditions

Similarly, DevOps engineers monitor:

  • CPU usage

  • Memory usage

  • Pod health

  • Network traffic

  • Application performance

to ensure everything runs smoothly.


πŸ”Ή What is Prometheus? πŸ“ˆ

Prometheus is an open-source monitoring and alerting tool designed for cloud-native applications and Kubernetes environments.

Its primary job is:

Collect and store metrics from Kubernetes and applications

Prometheus continuously gathers data such as:

  • CPU utilization

  • Memory consumption

  • Pod status

  • Node health

  • Application metrics


πŸ”Ή Why Prometheus is Popular? πŸš€

Prometheus is widely used because:

βœ… Open Source
βœ… Kubernetes Native
βœ… Easy Integration
βœ… Powerful Query Language (PromQL)
βœ… Alerting Support
βœ… Large Community Support


πŸ”Ή What are Metrics? πŸ“Š

Metrics are numerical measurements that tell us how systems are performing.

Examples:

Metric Example
CPU Usage 75%
Memory Usage 3 GB
Running Pods 15
Request Count 10,000
Network Traffic 500 Mbps

Metrics help us understand system health.


πŸ”Ή How Prometheus Works? βš™οΈ

Prometheus follows a pull-based model.

Instead of applications sending data, Prometheus periodically collects data from targets.

Workflow:

Kubernetes Cluster
        ↓
 Prometheus Server
        ↓
 Stores Metrics
        ↓
 Grafana Dashboard

πŸ”Ή Prometheus Architecture Explained πŸ—οΈ

πŸš€ Prometheus

  • Prometheus is an open-source systems monitoring and alerting toolkit originally built at SoundCloud.

  • It is known for its robust data model, powerful query language (PromQL), and the ability to generate alerts based on the collected time-series data.

  • It can be configured and set up on both bare-metal servers and container environments like Kubernetes.

🏠 Prometheus Architecture

  • The architecture of Prometheus is designed to be highly flexible, scalable, and modular.

  • It consists of several core components, each responsible for a specific aspect of the monitoring process.

πŸ”₯ Prometheus Server

  • Prometheus server is the core of the monitoring system. It is responsible for scraping metrics from various configured targets, storing them in its time-series database (TSDB), and serving queries through its HTTP API.

  • Components:

    • Retrieval: This module handles the scraping of metrics from endpoints, which are discovered either through static configurations or dynamic service discovery methods.

    • TSDB (Time Series Database): The data scraped from targets is stored in the TSDB, which is designed to handle high volumes of time-series data efficiently.

    • HTTP Server: This provides an API for querying data using PromQL, retrieving metadata, and interacting with other components of the Prometheus ecosystem.

  • Storage: The scraped data is stored on local disk (HDD/SSD) in a format optimized for time-series data.

🌐 Service Discovery

  • Service discovery automatically identifies and manages the list of scrape targets (i.e., services or applications) that Prometheus monitors.

  • This is crucial in dynamic environments like Kubernetes where services are constantly being created and destroyed.

  • Components:

    • Kubernetes: In Kubernetes environments, Prometheus can automatically discover services, pods, and nodes using Kubernetes API, ensuring it monitors the most up-to-date list of targets.

    • File SD (Service Discovery): Prometheus can also read static target configurations from files, allowing for flexibility in environments where dynamic service discovery is not used.

πŸ“€ Pushgateway

  • The Pushgateway is used to expose metrics from short-lived jobs or applications that cannot be scraped directly by Prometheus.

  • These jobs push their metrics to the Pushgateway, which then makes them available for Prometheus to scrape(pull).

  • Use Case:

    • It's particularly useful for batch jobs or tasks that have a limited lifespan and would otherwise not have their metrics collected.

🚨 Alertmanager

  • The Alertmanager is responsible for managing alerts generated by the Prometheus server.

  • It takes care of deduplicating, grouping, and routing alerts to the appropriate notification channels such as PagerDuty, email, or Slack.

🧲 Exporters

  • Exporters are small applications that collect metrics from various third-party systems and expose them in a format Prometheus can scrape. They are essential for monitoring systems that do not natively support Prometheus.

  • Types of Exporters:

    • Common exporters include the Node Exporter (for hardware metrics), the MySQL Exporter (for database metrics), and various other application-specific exporters.

πŸ–₯️ Prometheus Web UI

  • The Prometheus Web UI allows users to explore the collected metrics data, run ad-hoc PromQL queries, and visualize the results directly within Prometheus.

πŸ“Š Grafana

  • Grafana is a powerful dashboard and visualization tool that integrates with Prometheus to provide rich, customizable visualizations of the metrics data.

πŸ”Œ API Clients

  • API clients interact with Prometheus through its HTTP API to fetch data, query metrics, and integrate Prometheus with other systems or custom applications.

πŸ”Ή Kubernetes Monitoring Architecture ☸️

Monitoring workflow:

Kubernetes Cluster
        ↓
 API Server
        ↓
 Prometheus
        ↓
 Time Series Database
        ↓
 AlertManager
        ↓
 Grafana Dashboard

πŸ”Ή Setting Up Kubernetes Monitoring πŸš€

The instructor used:

Minikube Cluster

Minikube provides a local Kubernetes environment for learning.

Recommended configuration:

minikube start --memory=4096

This allocates 4 GB RAM.


πŸ”Ή Why Use Helm? βš“

Installing monitoring tools manually is difficult.

Helm simplifies deployment.

Helm is often called:

Package Manager for Kubernetes

Similar to:

  • apt for Ubuntu

  • yum for CentOS

  • npm for Node.js


πŸ”Ή Installing Prometheus Using Helm πŸ“ˆ

The video demonstrates installing Prometheus using Helm Charts.

Benefits:

βœ… Faster installation
βœ… Easy upgrades
βœ… Consistent deployment
βœ… Production-ready setup


πŸ”Ή What is Grafana? πŸ“Š

Grafana is a visualization tool used to display monitoring data.

Prometheus collects metrics.

Grafana displays them beautifully.


πŸ”Ή Why Grafana is Important? 🎨

Raw metrics are difficult to understand.

Grafana converts them into:

βœ… Charts
βœ… Graphs
βœ… Dashboards
βœ… Visual Reports

This makes monitoring easier.


πŸ”Ή Real-Life Example πŸ’‘

Imagine Prometheus as:

Data Collector

and Grafana as:

Data Visualization Expert

Prometheus gathers information.

Grafana presents it in an easy-to-understand format.


πŸ”Ή Connecting Grafana with Prometheus πŸ”—

After Grafana installation:

Prometheus is configured as:

Data Source

Grafana then retrieves metrics from Prometheus.


πŸ”Ή Community Dashboards in Grafana 🌍

The instructor imported Dashboard:

ID 3662

This is a popular Kubernetes monitoring dashboard.

Benefits:

βœ… Ready-made dashboard
βœ… No manual configuration
βœ… Cluster overview instantly available


πŸ”Ή Metrics Visible in Grafana πŸ“Š

The dashboard displays:

  • CPU utilization

  • Memory usage

  • Pod status

  • Node status

  • Network traffic

  • Cluster health

all in one place.


πŸ”Ή What is Kube State Metrics? πŸ“¦

Kube State Metrics is a special Kubernetes service that provides detailed cluster information.

It exposes metrics about:

βœ… Pods
βœ… Deployments
βœ… ReplicaSets
βœ… Services
βœ… Nodes


πŸ”Ή Why Kube State Metrics is Needed? πŸ€”

Prometheus can collect basic metrics.

However, for Kubernetes-specific information:

Kube State Metrics is required

It provides deeper visibility into cluster resources.


πŸ”Ή Examples of Kube State Metrics πŸ“ˆ

You can monitor:

  • Number of replicas

  • Pod restart count

  • Deployment status

  • Node readiness

  • Resource requests

This is extremely useful in production.


πŸ”Ή Monitoring Application Health 🩺

Cluster monitoring is important.

Application monitoring is equally important.

Developers expose:

/metrics endpoint

Prometheus then collects application-specific metrics.


πŸ”Ή Custom Application Monitoring πŸš€

Example metrics:

  • API response time

  • Login requests

  • Database connections

  • Error rates

  • Transaction counts

This provides deep visibility into application performance.


πŸ”Ή How Prometheus Collects Custom Metrics? βš™οΈ

Workflow:

Application
      ↓
Metrics Endpoint
      ↓
Prometheus Scraping
      ↓
Grafana Dashboard

πŸ”Ή What is Scraping? πŸ”

Scraping means:

Prometheus periodically collecting metrics

from applications or Kubernetes resources.

Example:

Every 15 seconds:

  • Read metrics

  • Store metrics

  • Update dashboards


πŸ”Ή Benefits of Kubernetes Monitoring πŸš€

Monitoring provides:

βœ… Faster troubleshooting
βœ… Improved reliability
βœ… Better performance optimization
βœ… Reduced downtime
βœ… Capacity planning


πŸ”Ή Real Production Use Cases 🏒

Organizations use Prometheus & Grafana for:

Infrastructure Monitoring

Monitor:

  • Nodes

  • CPU

  • Memory

  • Disk


Application Monitoring

Monitor:

  • Response times

  • Errors

  • Requests


Capacity Planning

Monitor growth trends to determine when infrastructure upgrades are needed.


Incident Detection

Alerts automatically notify teams when problems occur.


πŸ”Ή Important kubectl Commands βš™οΈ

View Pods:

kubectl get pods

View Services:

kubectl get svc

View Deployments:

kubectl get deployments

View Namespaces:

kubectl get namespaces

πŸ”₯ Real-World Scenario Based Questions

❓ Why is monitoring important in Kubernetes?

βœ… Answer:

Monitoring helps identify failures, performance issues, and resource bottlenecks before they impact users.


❓ What is Prometheus?

βœ… Answer:

Prometheus is an open-source monitoring and alerting tool that collects and stores metrics from Kubernetes and applications.


❓ What is Grafana?

βœ… Answer:

Grafana is a visualization platform used to create dashboards and graphs from Prometheus metrics.


❓ What is Kube State Metrics?

βœ… Answer:

Kube State Metrics exposes detailed Kubernetes object metrics such as Pods, Deployments, Services, and ReplicaSets.


❓ Why use Helm for Prometheus installation?

βœ… Answer:

Helm simplifies installation, upgrades, and management of Kubernetes applications using prebuilt charts.


❓ What is AlertManager?

βœ… Answer:

AlertManager sends notifications when Prometheus detects issues like high CPU usage, application failures, or node outages.


πŸ”₯ Interview Tip πŸš€

If an interviewer asks:

"How would you monitor a Kubernetes cluster?"

A strong answer is:

"I would use Prometheus to collect metrics, Grafana to visualize dashboards, Kube State Metrics for Kubernetes-specific insights, and AlertManager for automated notifications and incident response."



πŸš€ Continue Your Learning Journey

Thank you for taking the time to read this article.

Technology is evolving rapidly, and continuous learning is one of the most valuable investments you can make in your career. Whether you're exploring DevOps, Cloud Computing, Artificial Intelligence, Cybersecurity, Software Development, Data Science, or Career Growth, the resources below can help you deepen your knowledge and stay ahead in the industry.


πŸŽ“ Recommended Learning Platforms

πŸš€ Coursera

Learn from world-renowned universities and industry leaders including Google, IBM, Stanford, Microsoft, Meta, and many more.

βœ” Professional Certificates βœ” Career-focused Learning Paths βœ” AI & Machine Learning Programs βœ” Cloud & DevOps Certifications βœ” Business & Leadership Courses

πŸ”— https://imp.i384100.net/k0KvbV


πŸ’» Udemy

One of the largest online learning platforms with practical, hands-on courses covering:

βœ” DevOps & Kubernetes βœ” Docker & Cloud Computing βœ” AWS, Azure & GCP βœ” Programming & Development βœ” Cybersecurity & Ethical Hacking

πŸ”— https://trk.udemy.com/MAL2MY


πŸ“Š DataCamp

A great platform for anyone interested in:

βœ” Python Programming βœ” SQL & Databases βœ” Data Analytics βœ” Machine Learning βœ” Artificial Intelligence

Interactive learning paths and hands-on projects make it ideal for beginners and professionals alike.

πŸ”— https://datacamp.pxf.io/nX4kER


πŸŽ“ edX

Access high-quality courses and certifications from leading institutions such as:

βœ” Harvard University βœ” MIT βœ” Berkeley βœ” Microsoft

Perfect for learners seeking university-level education online.

πŸ”— https://edx.sjv.io/POvVeN


🎨 Domestika

Enhance your creative skills with courses on:

βœ” Graphic Design βœ” Video Editing βœ” Animation βœ” Digital Marketing βœ” Content Creation

πŸ”— https://domestika.sjv.io/dynKAW


πŸ› οΈ Recommended Tools & Resources

πŸ”₯ AppSumo

Discover exclusive lifetime deals on:

βœ” AI Tools βœ” Productivity Software βœ” Developer Utilities βœ” Marketing Platforms βœ” Business Applications

A must-have resource for developers, creators, freelancers, and entrepreneurs looking to save money while accessing premium tools.

πŸ”— https://appsumo.8odi.net/L04a33


πŸ›’ Shopify

Looking to start an online business or launch an eCommerce store?

Shopify provides everything you need to build, manage, and scale an online business.

βœ” Online Store Builder βœ” Payment Integration βœ” Inventory Management βœ” Marketing Tools

πŸ”— https://shopify.pxf.io/Vxv09k


🌐 WordPress, WooCommerce & Jetpack

Create professional websites, blogs, and online stores with one of the most trusted web ecosystems in the world.

Ideal for:

βœ” Personal Blogs βœ” Portfolio Websites βœ” Business Websites βœ” eCommerce Stores

πŸ”— https://automattic.pxf.io/Z6vR5W


🌍 Language Learning Resources

πŸ—£οΈ Preply

Learn English and other languages through personalized one-on-one tutoring sessions with experts from around the world.

πŸ”— https://preply.sjv.io/o4gBDY


πŸ“š British Council English Online

Improve your professional communication skills and English fluency through structured learning programs.

πŸ”— https://englishonline.sjv.io/9VOGa4


🧠 Rosetta Stone

One of the most recognized language-learning platforms for immersive language acquisition.

πŸ”— https://aff.rosettastone.com/X4OyqG


πŸ§ͺ Science & Educational Resources

πŸ”¬ MEL Science

Interactive science kits and educational experiences designed to make STEM learning engaging and practical.

πŸ”— https://imp.i328067.net/bk2beg


πŸ“– Carson Dellosa Education

Educational materials and learning resources for students, teachers, and lifelong learners.

πŸ”— https://carsondellosaeducation.sjv.io/E0JbjW


❀️ Support My Work

Creating detailed technical content, tutorials, guides, and learning resources takes significant time and effort.

If you find my articles helpful and would like to support my work, you can do so through the following platforms:

⭐ Become a GitHub Sponsor

Support my open-source contributions, technical content, and community projects.

πŸ”— https://github.com/sponsors/hritikranjan1


β˜• Buy Me a Chai

Enjoying my content? Consider buying me a chai and supporting future tutorials, guides, and educational resources.

πŸ”— https://www.chai4.me/hritikranjan


πŸ‘¨β€πŸ’» Connect With Me

Hritik Ranjan

πŸ’‘ AI Enthusiast ☁️ DevOps Learner πŸ” Cybersecurity Advocate πŸ’» Software Developer

Connect & Follow

πŸ”— GitHub: https://github.com/hritikranjan1

πŸ”— LinkedIn: https://linkedin.com/in/hritikranjan1


πŸ“’ Found This Article Helpful?

If this article added value to your learning journey:

βœ… Share it with your network
βœ… Bookmark it for future reference
βœ… Follow for more DevOps, AI, Cloud, Cybersecurity, and Software Engineering content

Thank you for reading and being part of this learning journey.

Keep Learning. Keep Building. Keep Growing. πŸš€

DevOps Learning Journey πŸš€

Part 10 of 14

Documenting my step-by-step journey of learning DevOps β€” from basics to advanced concepts. In this series, I’ll share weekly notes, hands-on practice, tools, and real-world insights as I grow in DevOps.

Up next

πŸš€ DevOps Week 9.1 – Complete CI/CD Journey with Jenkins & GitHub Actions

Master the foundations of CI/CD, learn Jenkins from Zero to Hero, build real-world pipelines with Docker & Kubernetes, and automate software delivery using GitHub Actions and Self-Hosted Runners.