VPC segmentation

Identity-referenced SGs, private subnets, blast radius.

A leaked password gets an attacker onto one machine. Whether that becomes one bad afternoon or a company-ending breach depends almost entirely on what that one machine can reach next. Identity decides who you are. The network decides where you can go once you're inside, and a good layout means a stolen credential drops the attacker into a small locked room instead of the whole building.

A well-run office building handles this with zones. The lobby is open to the street. The office floors need a badge to enter. The vault in the basement takes a second badge that almost nobody carries, and its door only opens for people already inside the building, never for someone walking in off the pavement. AWS lets you build the same zoning inside a VPC (a virtual private cloud, your own private slice of the AWS network). The whole job here is setting those zones up so they stay closed until you deliberately open one narrow path.

The doorman who remembers you

A security group is a small firewall wrapped around a single network card. Every server or load balancer in AWS sits behind a virtual network interface (an ENI, or elastic network interface), and its security group is the list of who may talk to that interface. The word that matters is stateful. Once you allow a connection in, the reply is allowed back out automatically, the way a doorman who watched you walk in won't stop you on the way out. You write the rule for the direction you care about and the return path takes care of itself.

Here's the part teams get wrong. You can write a rule that allows a range of addresses (a CIDR block, the 10.0.1.0/24 style notation for a group of IPs), but on a live network those addresses shift constantly as servers scale up and down. So don't allow addresses. Allow a badge. A rule that says accept database connections from anything wearing the app-server badge keeps working no matter how many app servers come and go, and it never accidentally admits a machine that happened to inherit an old IP. In AWS terms, the database's security group allows the app's security group by ID.

aws ec2 authorize-security-group-ingress \
  --group-id sg-0db1a2c3d4e5f6a7b \
  --ip-permissions 'IpProtocol=tcp,FromPort=5432,ToPort=5432,UserIdGroupPairs=[{GroupId=sg-0a9b8c7d6e5f4a3b2,Description="postgres from app tier only"}]'
 
{
    "Return": true,
    "SecurityGroupRules": [
        {
            "SecurityGroupRuleId": "sgr-041c2e3f4a5b6c7d8",
            "GroupId": "sg-0db1a2c3d4e5f6a7b",
            "IsEgress": false,
            "IpProtocol": "tcp",
            "FromPort": 5432,
            "ToPort": 5432,
            "ReferencedGroupInfo": {
                "GroupId": "sg-0a9b8c7d6e5f4a3b2"
            },
            "Description": "postgres from app tier only"
        }
    ]
}

Read the rule back and the distinction is visible at a glance: the inbound source is a group, and the list of allowed IP ranges is empty. There is no address path into this database, only a badge.

aws ec2 describe-security-groups \
  --group-ids sg-0db1a2c3d4e5f6a7b \
  --query 'SecurityGroups[0].IpPermissions'
 
[
    {
        "IpProtocol": "tcp",
        "FromPort": 5432,
        "ToPort": 5432,
        "IpRanges": [],
        "Ipv6Ranges": [],
        "PrefixListIds": [],
        "UserIdGroupPairs": [
            {
                "GroupId": "sg-0a9b8c7d6e5f4a3b2",
                "UserId": "123456789012",
                "Description": "postgres from app tier only"
            }
        ]
    }
]

The bouncer with no memory

A network ACL, or NACL (network access control list), does a related job one level up, guarding the edge of an entire subnet instead of a single interface. The catch that trips almost everyone is that a NACL is stateless. It has no memory. It judges every packet going in and every packet coming out as separate decisions, so allowing a request in does not allow the answer back out. You have to write both directions yourself. Replies come back on high-numbered temporary ports (ephemeral ports, which AWS suggests covering as the whole 1024 to 65535 range), so a NACL that allows only port 5432 inbound will accept the database query and then quietly drop the response. The connection just hangs.

Because of that, don't reach for NACLs to express fine-grained rules. Keep security groups for per-workload allow lists, and use NACLs as a blunt, subnet-wide backstop, the sort of thing that says no traffic from this banned range touches this subnet at all, whatever the security groups permit. NACL rules are also numbered and evaluated lowest first, stopping at the first match, so ordering matters in a way security group rules (a pure set of allows) never do. And a NACL can express an explicit deny, which a security group cannot, so it's the right tool when you need a hard block rather than a simple allow.

aws ec2 describe-network-acls \
  --network-acl-id acl-08f5e4d3c2b1a0f9e \
  --query 'NetworkAcls[0].Entries[?RuleNumber!=`32767`]'
 
[
    {
        "RuleNumber": 100,
        "Protocol": "6",
        "RuleAction": "allow",
        "Egress": false,
        "CidrBlock": "10.20.10.0/24",
        "PortRange": { "From": 5432, "To": 5432 }
    },
    {
        "RuleNumber": 120,
        "Protocol": "6",
        "RuleAction": "allow",
        "Egress": true,
        "CidrBlock": "10.20.0.0/16",
        "PortRange": { "From": 1024, "To": 65535 }
    }
]

Protocol 6 is TCP. Rule 100 lets the database query in on 5432. Rule 120 is the one people forget: it lets the reply back out on the ephemeral port range. Leave rule 120 out and every allowed connection stalls, even though the inbound rule looks perfect. The default catch-all rule (number 32767, a deny) is filtered out of the view above so the two real entries stand on their own.

Three keycard zones

The rules only contain a breach if the layout underneath them is right. The standard shape is three tiers, each in its own subnet. Public subnets hold only what must face the internet: load balancers, and the NAT gateway (network address translation, a one-way outbound door) that lets private servers fetch updates without being reachable themselves. Private subnets hold your application servers, with no route to the internet gateway at all, so nothing outside can dial them directly. Data subnets hold databases and caches, and get neither an internet gateway nor a NAT route, so a database can't be reached from outside and can't quietly phone out either. Each subnet is a keycard zone, and the badge rules from earlier decide which zone may knock on the next.

This is what bounds the damage. An attacker who lands on a public load balancer finds it can reach the app tier on exactly one port and nothing else. From the app tier, only the database port into the data tier. The vault never sees the street. For real separation between environments, give each one its own AWS account so production and staging share no network by default. Nothing routes between two separate accounts unless you deliberately build a path between them, so a slip in staging can't leak into production. That gap is a wall, not a badge door someone might forget to lock.

One thing this design deliberately can't do: it governs who may start a connection to your servers, not where your servers may reach out. Security groups and NACLs don't inspect the contents of traffic or filter by domain name, so a compromised app server can still open an outbound connection to anywhere it likes. Closing that path is egress control, and it takes a different tool.

Try this

Work through “Three keycard zones” yourself on a sandbox you can throw away, following the commands above in order. Then break one step deliberately and re-run, so you have seen the failure before it finds you.

Takeaway

The trap worth remembering here: new security groups allow all outbound by default. Check that on your own systems before you need to, because it is cheaper to find on a quiet afternoon than during an incident.

Related