In a context where on-premise resources and cloud services are combined, I wonder what criteria to prioritize when choosing between a serverless architecture and orchestrated containers. What are the impacts on latency, state management, billing, and portability? And in your opinion, which scenarios clearly justify opting for one over the other?
Hybrid cloud: serverless or containers, which choice for scalability?
👁️ 255 views💬 6 replies❤️ 0 likes
6 Replies
In my hybrid cloud migration project, I initially deployed high-concurrency, externally facing APIs using AWS Lambda (or Azure Functions) for their serverless architecture. The pay-per-use billing model significantly reduces costs for burst traffic, and the cold start latency—around 100ms—is acceptable for non-latency-sensitive workloads like user profile queries or log aggregation. State management is handled by external databases like DynamoDB or CosmosDB, keeping the functions stateless, which simplifies portability and testing.
For services requiring long-running processes, fine-grained resource control, or extreme sensitivity to network latency—such as real-time video transcoding, database middleware, or high-frequency internal trading gateways—I prefer container orchestration platforms like Kubernetes or OpenShift. Containers can be pre-warmed to maintain warm instances, achieving near-millisecond response times. State can be directly mounted to local disks or managed via StatefulSets to ensure data locality, avoiding additional latency from cross-network calls. While costs are fixed for node resources, granular scaling (vertical/horizontal) and fine-tuned node pool scheduling help maintain cost efficiency under heavy loads.
In summary, serverless is ideal for stateless, bursty, cost-driven frontends or lightweight backends, while containers excel in scenarios demanding persistent connections, ultra-low latency, complex state management, or custom runtime environments. By delineating boundaries based on workload characteristics, you can balance scalability and control within a single hybrid cloud setup.
In my experience with a client using a hybrid cloud for a video streaming platform, I first tested both approaches before making a decision. If ultra-low latency (under 30 ms) must be maintained between the on-premise data center and the front-end, orchestrated containers (Kubernetes with GPU nodes) are generally more reliable: you have exact control over where the pod runs, can place nodes close to local storage, and avoid the "cold start" issue of serverless. On the other hand, for purely stateless functions—event processing, API triggers, cleanup jobs—the serverless model (e.g., AWS Lambda or Azure Functions) drastically cuts costs since you only pay for actual execution time, and scaling is instant without cluster management.
In practice, I recommend a hybrid approach: keep services requiring persistent state (databases, Redis cache, long-running tasks) in containers deployed on your on-premise infrastructure or dedicated VMs, and offload stateless microservices and event pipelines to serverless. This setup gives you good portability—containers work across any cloud—while leveraging serverless’s pay-per-use model for load spikes. If your workload is highly variable and you need predictable billing, stick with containers and use an autoscaler with CPU/memory metrics; otherwise, let serverless handle horizontal scaling and focus on state management with containers.
For a hybrid setup, I always start by separating "stateless" workloads from "stateful" ones. In my latest on-premise + AWS project, Lambda functions (or equivalent GCF) handled API entry points, processing/transformations that don’t require maintaining sessions between calls well; latency averaged 30-50ms thanks to VPC Peering, and billing was linear—we only paid per invocation and execution time. However, as soon as I needed to maintain persistent sessions (local caches, long-lived DB connections, or batch processing with context), I switched to Kubernetes (EKS/GKE) with pods orchestrated via Istio. Containers provide fine-grained network control, leading to more predictable latency (~10-15ms intra-cluster) and, most importantly, the ability to mount shared volumes or use StatefulSets for persistence. Billing-wise, costs remain more stable since you pay for nodes even when idle, but you gain portability: the same Helm manifest runs in our on-premise datacenter with a K3s server and in the cloud with minimal adjustments.
In practice, I recommend: if your workload is purely event-driven, highly variable, and doesn’t need local storage, go serverless—you get pay-per-use billing and instant scaling. If you have strict latency requirements, need to store state, or have dependencies that don’t translate well to ephemeral functions (native libraries, special drivers, licenses), deploy orchestrated containers. A hybrid approach—keeping stateless APIs serverless while grouping stateful services in a Kubernetes cluster that spans both on-premise and cloud—has worked very well for me.
I recently migrated an image-processing microservice from an on-premise data center to a hybrid environment. Initially, I chose Kubernetes containers because I needed to maintain persistent state (local cache and access to a shared volume) and control latency—the serverless functions introduced a 200–300 ms cold start that slowed down the pipeline when traffic was irregular. By orchestrating the containers, I could place the pods close to the internal storage node and leverage a low-latency internal network while still scaling horizontally using Kubernetes auto-scalers.
However, once I added a new endpoint for on-demand calls (generating thumbnails on request), I shifted that part to serverless. The pay-as-you-go billing model reduced the total cost: rare invocations didn’t justify dedicated VMs, and portability between AWS Lambda and Azure Functions was seamless thanks to a small abstraction wrapper.
In short: if you need state management, fine-tuned latency control, and a stable infrastructure, containers are still the best choice; for ephemeral workloads without persistent state where per-millisecond billing matters, serverless often wins.
How significantly does the cold start time of serverless functions impact overall latency compared to already running containers in a hybrid environment? Are there empirical data on cost differences between sporadic versus constant workloads? And what best practice approaches do you recommend for maintaining state across serverless invocations?
In my hybrid experience, I first prioritize the nature of the traffic: for functions that run sporadically and where response time can tolerate a few milliseconds of cold start, the serverless model (AWS Lambda, Azure Functions, etc.) remains very attractive thanks to pay-per-use billing and instant scalability. I’ve noticed that as soon as the request rate exceeds a few hundred per second, latency starts to climb, and an orchestrated container (Kubernetes or Docker Swarm) becomes more predictable. You retain control over startup time, pod sizing, and avoid the "cold start penalty."
When it comes to state management, I generally opt for containers when persistent sessions or local caches need to be maintained. The serverless model, even with an external state store, always introduces a network round trip that increases latency and complicates the code (synchronization, locking, etc.). On the billing side, serverless is advantageous for spiky workloads, whereas containers offer better long-term cost visibility thanks to VM or reserved node billing. Lastly, portability: Docker images are universal and can be deployed both on-premise and across different clouds, whereas each serverless platform has its own APIs and limitations, creating a certain lock-in.
In practice, I use serverless for event processing pipelines (log ingestion, S3 triggers) and containers for stateful services, critical APIs, and microservices requiring fine-grained control over networking or performance.