Kubernetes has become the de facto standard for container orchestration, offering scalability, flexibility, and automation for modern cloud-native applications. However, while Kubernetes provides powerful deployment capabilities, its default configuration leaves several security responsibilities to cluster administrators.

For organizations operating under strict regulatory and compliance requirements—such as healthcare providers—protecting sensitive data requires far more than simply deploying applications successfully. Network isolation, workload identity, runtime threat detection, encrypted communication, and centralized monitoring all become essential components of a secure production environment.

This project presents the design and implementation of a layered Kubernetes security architecture built entirely with open-source technologies. By integrating Cilium, Falco, Istio, Prometheus, and Grafana, the solution demonstrates how multiple security mechanisms can work together to provide defense in depth while maintaining production-ready performance.

Introduction

The security challenge

A default Kubernetes cluster is designed to be flexible rather than secure by default.

Pods are typically able to communicate freely across namespaces, service-to-service traffic is transmitted without encryption unless explicitly configured, and runtime attacks occurring inside containers often remain invisible until after an incident has already occurred.

For environments handling sensitive healthcare information, these limitations present significant security risks. The objective of this project was therefore to design a practical Zero Trust architecture capable of protecting workloads while remaining simple to deploy and maintain.

System Architecture

The proposed architecture follows a layered defense strategy, where each security component addresses a different attack surface. Instead of relying on a single protection mechanism, network security, runtime monitoring, encrypted communication, and observability complement each other to provide comprehensive cluster protection.

Layer 1 – Network Security (Cilium)

The first security layer is implemented using Cilium, which replaces unrestricted pod-to-pod communication with a default-deny security model.

Using eBPF-powered Network Policies, only explicitly authorized traffic is permitted between workloads. In addition, Cilium provides Layer 7 filtering and real-time network visibility through Hubble, allowing administrators to monitor communication flows across the cluster.

Layer 2 – Runtime Threat Detection (Falco)

While network policies restrict communication, they cannot detect malicious behavior occurring inside running containers.

To address this limitation, Falco continuously monitors Linux system calls using eBPF and detects suspicious activities such as:

• Shell execution inside containers

• Privilege escalation attempts

• Unauthorized file modifications

• Access to sensitive system directories

Custom Falco rules were developed alongside the default rule set to improve detection accuracy for the implemented environment.

Layer 3 – Service Mesh Security (Istio)

Secure communication between services is provided through Istio.

By automatically injecting Envoy sidecars into application pods, Istio enables Mutual TLS (mTLS) without requiring application changes. Every service is authenticated through certificate-based identities while all internal traffic is encrypted, significantly reducing the risk of unauthorized communication within the cluster.

Layer 4 – Monitoring and Observability

The final layer focuses on centralized monitoring.

Metrics generated by Cilium, Falco, and Istio are collected using Prometheus and visualized through Grafana, providing administrators with a unified dashboard displaying:

• Network traffic

• Runtime security events

• Service communication

• mTLS status

• Overall cluster health

This centralized observability simplifies both security monitoring and incident response.

Implementation Challenges

During development, several implementation challenges were encountered that are often overlooked in official documentation.

• Bidirectional Cilium policies are required for successful communication, as both ingress and egress rules must be explicitly defined.

• Default-deny policies also restrict infrastructure components such as Istio and Prometheus, making additional allow rules necessary for proper functionality.

• Falco's ServiceMonitor configuration requires correct placement within the Helm chart to expose metrics successfully to Prometheus.

Addressing these issues was essential for achieving a fully operational and production-ready deployment.

Performance Evaluation

The benchmark demonstrates that the additional CPU utilization introduced by the security stack remains low and suitable for production workloads. Memory usage remained within acceptable limits, confirming that the deployed components introduce only moderate resource overhead.

The measured latency introduced by Mutual TLS remained minimal, demonstrating that encrypted communication can be enabled without significantly affecting application performance.

Results & Conclusions

Testing confirmed that the layered security architecture successfully prevented or detected every evaluated attack scenario.

Network segmentation effectively blocked unauthorized lateral movement, Falco detected runtime threats in real time, while Istio ensured encrypted and authenticated communication between services. At the same time, performance measurements demonstrated low CPU and memory overhead together with minimal latency, making the solution suitable for production environments requiring both strong security and operational efficiency.

Securing Kubernetes requires a combination of complementary technologies rather than relying on a single security solution.

By integrating network segmentation, runtime threat detection, encrypted service communication, and centralized observability, this project demonstrates a practical implementation of a layered security architecture suitable for highly regulated environments such as healthcare.

The results show that strong Kubernetes security can be achieved using open-source technologies while maintaining low resource consumption and production-ready performance.

Open Source Implementation

To simplify adoption and encourage reproducibility, the complete implementation has been published as an open-source project on GitHub.

The repository includes:

• Kubernetes manifests

• Helm configuration files

• Installation scripts

• Custom Falco detection rules

• Cilium Network Policies

• Grafana dashboards

• Documentation required to deploy the complete security stack on a fresh Kubernetes cluster.

GitHub Repository URL: https://github.com/DitmirSpahiu/k8s-SecStack

Location

Bulevardi Bajram Curri, Nr. 1, Tirana, Albania, 1001

Contacts

support@cloudgravitysolutions.com

Social Media