🦍 ECS NETWORKING BATTLE 🦍

🎯

Sidecars

Isolation 95%

Simplicity 85%

Resources 40%

✓ PROS

Task-level isolation via IAM roles

Battle-tested pattern

No cluster changes needed

Works on existing setup

Per-customer secrets/config

✗ CONS

~100-150MB per task overhead

Extra CPU per task

More containers to manage

🌐

awsvpc Mode

Isolation 100%

Simplicity 90%

Resources 70%

✓ PROS

Perfect network isolation

Security groups per task

Unique IP per task

AWS-native solution

No custom networking

✗ CONS

Requires ENI trunking enable

Instance replacement needed

ENI limits per instance

IP address exhaustion risk

Lower task density

🔄

Shared Envoy

Isolation 30%

Simplicity 25%

Resources 95%

✓ PROS

Minimal resource overhead

Single proxy to manage

Works on existing cluster

High task density maintained

✗ CONS

No real security isolation

Port-based routing only

Containers can access wrong ports

Complex iptables for enforcement

Custom AMI required for safety

Fragile at scale

Single point of failure

🪜 IMPLEMENTATION LADDER 🪜

🎯 SIDECARS

⭐ Update task definitions

⭐ Add Envoy container config

⭐ Set per-customer env vars

⭐ Deploy and test

🎮 DIFFICULTY: EASY

✅ Works on current cluster!

🌐 AWSVPC MODE

⭐ Enable ENI trunking on cluster

⭐⭐ Drain existing instances

⭐⭐ Launch new instances

⭐⭐ Update all task definitions

⭐⭐ Configure security groups

⭐⭐ Verify IP capacity

🎮 DIFFICULTY: MEDIUM

⚠️ Requires cluster migration!

🔄 SHARED ENVOY

⭐⭐ Design port allocation scheme

⭐⭐⭐ Build custom AMI with iptables

⭐⭐⭐ Create dynamic config system

⭐⭐⭐ Implement service discovery

⭐⭐⭐ Deploy per-container rules

⭐⭐⭐ Test cross-customer isolation

⭐⭐⭐ Debug inevitable edge cases

🎮 DIFFICULTY: HARD

💀 Ongoing maintenance burden!

🪙 WINNER: SIDECARS 🪙

BEST BALANCE OF
SECURITY × SIMPLICITY × COMPATIBILITY