This is a follow up on part 2. If you are not familiar with Cloudformation, you might want to read part 1 and 2.
The VPC side has been set up. We can verify the public subnet works as intended by deploying a bastion host in the public network.
Funny name but unless we connect to the private part of the VPC via site to site VPN or Direct Connect we will a way to connect to EC2s deployed in the private subnets somehow. That « somehow » is an EC2 instance in the public subnet that only allows SSH connections from specific IPs. We do not want the internet to try and brute force them selves into that host.
OK, so, we’ll need and EC2 and a security group. Let’s add them to our VPC yaml file.
Resources: [...] BastionSecurityGroup: Type: 'AWS::EC2::SecurityGroup' Properties: VpcId: !Ref myVPC GroupDescription: Enable SSH access via port 22 SecurityGroupIngress: - IpProtocol: tcp FromPort: '22' ToPort: '22' CidrIp: !Ref SSHRemoteAccessCIDR Tags: - Key: Name Value: ProjectXBastionSG
Things to note here:
The RemoteAccessCIDR is new and we reference it. We want this to be a parameter for easy setup and re-use.
I skip the metadata part for now, here is the parameter part:
Parameters: [...] SSHRemoteAccessCIDR: AllowedPattern: ^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(\/([0-9]|[1-2][0-9]|3[0-2]))$ ConstraintDescription: CIDR block parameter must be in the form x.x.x.x/x Description: Allowed CIDR block for external SSH access to the bastions Type: String
No default value this time! That means we will be forced to give this parameter a value.
The ingress rule is only allowing SSH traffic on port 22 if it comes from the SSHRemoteAccessCIDR. Any other traffic is rejected.
Off to the Bastion host where we will start by hardcoding a few things.
Resources: [...] BastionHost: Type: AWS::EC2::Instance Properties: ImageId: ami-0ff760d16d9497662 InstanceType: t2.micro KeyName: !Ref BastionKeyPairName SubnetId: !Ref VPCPublicSubnet1 SecurityGroupIds: - !Ref BastionSecurityGroup Tags: - Key: Name Value: ProjectXBastionHost
The AMI ID is for Centos 7. There is plenty of choices, use what floats your boat.
The instance size is a t2.micro. We really do not need more for this type of host that is going to be idle most of the time.
The security group and subnet are already known. But we need a key pair. So go to the AWS console into the EC2 section and create a new key pair: I called mine « cfntest ».
keypaircreation
We reference a BastionKeyPairName, so again, we want this to be a parameter. This is a little different from the previous parameters as there is a construct that will give us a list of all key pairs in a drop down list for easy selection in the Cloudformation interface.
Parameters: [...] BastionKeyPairName: Description: Amazon EC2 Key Pair Type: "AWS::EC2::KeyPair::KeyName" ConstraintDescription: must be a valid Key Pair
We are ready for a first run. If you had difficulties completing the metadata part of the file, you can use the file from the github repository.
Notice the 2 new fields in the Cloudformation web interface:
cfnnewfields
In the drop down menu, you will find the key pair we created earlier.
In the bastion access CIDR, I will limit the CIDR to my outgoing work address IP. The screenshot IP is of course invented for security reasons. Use your outgoing IP. If you wonder what it is, open your browser and google « what’s my ip ».
Let’s create our stack …
All should be going well and you should have a bastion host created in subnet 1. If you wonder what’s its IP, you can get it in the EC2 section. But we will use an Output to make our life easier. Add the outputs at the end of the file.
[...] Outputs: BastionHostDNS: Description: Bastion host public DNS Value: !GetAtt BastionHost.PublicDnsName BastionPublicIPAddress: Description: Bastion host public IP Value: !GetAtt BastionHost.PublicIp
We use the GetAtt intrinsic function to get the resource attributes we want: PublicDnsName and PublicIp.
Now, maybe you have already updated your stack. If so kudos! But in case you didn’t, now is the time.
cfnupdate
Choose « Replace current template » and re-upload the file. Our parameters have not changed, we can « next » our way to the stack creation and this time update the stack.
The bastion host public DNS and public IP are now displayed in the « Outputs » tab in Cloudformation.
cfnoutputsbastion
Remember the key pair we created earlier. It should be in your downloads folder.
~/Downloads ➜ ll cfntest.pem -rw-r--r-- 1 orobert staff 1696 Jul 31 10:20 cfntest.pem
We need to change the rights on it or our ssh connection will not succeed.
~/Downloads ➜ chmod 600 cfntest.pem ~/Downloads ➜ ll cfntest.pem -rw------- 1 orobert staff 1696 Jul 31 10:20 cfntest.pem
Now we are ready to check if we can log into the bastion host.
➜ ssh -i cfntest.pem centos@34.250.141.122 The authenticity of host '34.250.141.122 (34.250.141.122)' can't be established. ECDSA key fingerprint is SHA256:IzF8sTEGjeuew8XBo96TNpu5M+QOTedZVms7ymfaCGs. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added '34.250.141.122' (ECDSA) to the list of known hosts. [centos@ip-10-0-1-219 ~]$ ping google.com PING google.com (209.85.202.113) 56(84) bytes of data. 64 bytes from dg-in-f113.1e100.net (209.85.202.113): icmp_seq=1 ttl=40 time=1.02 ms 64 bytes from dg-in-f113.1e100.net (209.85.202.113): icmp_seq=2 ttl=40 time=1.04 ms ^C --- google.com ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 1001ms rtt min/avg/max/mdev = 1.025/1.032/1.040/0.033 ms [centos@ip-10-0-1-219 ~]$
Outgoing internet works as intended. Looks good. Try to log in from another IP (tethered from your phone for example). It won’t work thanks to the security group we configured.
We could deploy an EC2 in a private subnet and connect to it as well.
PrivateHostSecurityGroup: Type: 'AWS::EC2::SecurityGroup' Properties: VpcId: !Ref myVPC GroupDescription: Enable SSH access via port 22 SecurityGroupIngress: - IpProtocol: tcp FromPort: '22' ToPort: '22' SourceSecurityGroupId: !Ref BastionSecurityGroup Tags: - Key: Name Value: ProjectXPrivateHostSG PrivateHost: Type: AWS::EC2::Instance Properties: ImageId: ami-0ff760d16d9497662 InstanceType: t2.micro KeyName: !Ref BastionKeyPairName SubnetId: !Ref VPCPrivateSubnet1 SecurityGroupIds: - !Ref PrivateHostSecurityGroup Tags: - Key: Name Value: ProjectXPrivateHost
cfnprivatehost
The private host has only an internal IP. So, there is no login into it from the internet. This is exactly what we wanted. The private security group is different from the previous security group. We don’t use a CIDR for authorised connections. We use source security group ID from the BastionSecurityGroup. This means that if the connection originates from that security group, it is authorised.
The only way we can connect to the private host is through the bastion host. Before you continue, read this: Securely Connect to Linux Instances Running in a Private Amazon VPC. Done? Good, we can now connect to the private host via the bastion host.
~/Downloads ➜ ssh -A centos@34.250.141.122 Last login: Wed Jul 31 09:36:06 2019 from 194.154.207.138 [centos@ip-10-0-1-219 ~]$ ssh centos@10.0.10.253 Last login: Wed Jul 31 09:36:23 2019 from ip-10-0-1-219.eu-west-1.compute.internal [centos@ip-10-0-10-253 ~]$
The first connection is to the bastion host, the second to the internal IP of the private host. The private host has internet access via the NAT Gateway, meaning it can initiate connections to internet services, but there is no way to access the private host from the internet. To test the private host connectivity to the internet, update the OS for example.
[centos@ip-10-0-10-253 ~]$ sudo yum update
Now, be aware, that there are no outbound restrictions in this setup, neither on the security group, nor via network ACLs, so outbound, everything is possible. What should be the right security configuration depends on what you want to do and how you want to do it. There are no stone written rules: make your own. Rule of thumb: open as much as needed and as little as possible.
We validated our setup works. In Part 4, we will add a little more flexibility to our template and deploy an nginx to the private host that we will access from the internet via a load balancer. “But you said the only way to access the private host is via the bastion host, …”. Yeah, well, the load balancer is another way 😉
Anyway, the template gets bigger. I already mentioned it but, from now on, you can use the Github repository to follow along.