Design and Deploy a Completely Private and Secured Networking Architecture on Cloud Using Automation

In today's cloud world, security has taken precedence over resiliency and high availability. With the introduction of Virtual Private Cloud, network security has become even more critical. In this article, you will learn the best practices to make a virtual private cloud more private and secure than ever before. By the end of this article, you will learn how to: Design and build a secured and private Virtual Private Cloud network architecture. This is important for Financial institutions that are planning to move to Cloud. The role of bastion hosts in a private cloud network architecture and securing the bastion hosts using network rules. Build a highly available Virtual Private Network and Bastion host setup keeping the promise of cloud resiliency. The importance of session recording for auditing and compliance. Set up session recording in Red Hat Enterprise Linux using built-in packages. Learn how automation using Terraform and Ansible helps in building the Cloud infrastructure, high availability, session recording etc., Let's explore best practices for designing and securing a Virtual Private Cloud (VPC) network in Cloud VPC Design and Architecture: Subnet Isolation: Divide your VPC into multiple subnets based on different tiers and security requirements. For example, have separate subnets for public-facing services and internal resources. Use of Network Access Control Lists (NACLs): Implement NACLs to control traffic at the subnet level. They act as a firewall for controlling inbound and outbound traffic to and from subnets. Network Security: Security Groups: Leverage security groups to control traffic at the instance level. Security groups act as a virtual firewall for your instance to control inbound and outbound traffic. VPC Peering / Transit Gateways: Use VPC peering / transit gateways to establish private connectivity between VPCs, allowing secure communication between them. VPN Connections: Establish Virtual Private Network (VPN) connections to securely connect your on-premises network to your VPC. High Availability: Multi-AZ Deployment: Distribute resources across multiple Availability Zones (AZs) to ensure high availability. In case of a failure in one AZ, traffic can be automatically routed to a healthy AZ. Auto Scaling Groups: Implement Auto Scaling Groups to automatically adjust the number of instances (Virtual Machines or Virtual Server Instances) based on demand. This ensures availability and fault tolerance. VPN and Bastion Hosts: Bastion Hosts: Use bastion hosts for secure access to instances in private subnets. These hosts act as an intermediary for secure access to your private instances. High Availability VPN: Set up VPN connections with redundancy across multiple endpoints for increased availability. Automation with Terraform: Infrastructure as Code (IaC): Use Terraform to define and provision your infrastructure as code. This allows for consistent and repeatable deployments. Version Control: Store your Terraform scripts in a version control system (e.g., Git) to track changes, collaborate with a team, and roll back to previous versions if needed. Monitoring and Logging: Logs and Metrics: Implement monitoring like CloudWatch to monitor your VPC. Set up alerts for key metrics and use Logs from the logging service for centralized logging. VPC Flow Logs: Enable VPC Flow Logs to capture information about the IP traffic going to and from network interfaces in your VPC. Regular Security Audits: Security Audits: Conduct regular security audits to identify and address potential vulnerabilities. This includes reviewing configurations, access controls, and monitoring logs for suspicious activities. By following these best practices, you can create a highly secure and resilient VPC. Regularly review and update your security measures to adapt to evolving threats and ensure ongoing protection of your cloud infrastructure. Implementing automation with tools like Terraform further enhances efficiency and consistency in managing your cloud resources. Secured way to connect to a Private VPC Network This article uses IBM Cloud to deploy the VPC network using Terraform as the infrastructure as code automation. You can use any cloud provider to deploy the architecture as VPC is available on all the leading cloud vendors for network isolation. Client-to-site VPN for VPC is one of the two VPN options available on IBM Cloud that provides a client-to-site VPN solution that allows users to connect to IBM Cloud resources through secure, encrypted connections. What Does this Architecture Address? The bastion hosts and the client-to-site VPN servers are highly available. The VPN creates a secure tunnel to connect from a local machine to the Cloud. The bastion host accepts SSH connections from client to site VPN (192.168.x.x). The last (final) rule in the stateless access control lists is denying all the inbound and outbound traffic. No public gateways are attached to the subnets, and no floating IPs on the virtual server instances (VSIs). The security groups attached to the VSIs allow traffic only on specific required ports and private IPs. Before you Begin ● IBM Cloud API Key to provision cloud resources. ● Certificate's CRN from Secrets Manager to store VPN certificates. ● OpenVPN client to connect to client-to-site VPN from your local machine. ● Install and set up Terraform ● Install and set up Ansible Provision the Private VPC Network Using Terraform Once you have the secrets manager secret with the certificate, launch your terminal and set the following Terraform variables. Secrets Manager export TF_VAR_ibmcloud_api_key= export TF_VAR_secrets_manager_certificate_crn= Clone the GitHub repository with Terraform and Ansible code. git clone https://github.com/VidyasagarMSC/private-vpc-network cd terraform Run the Terraform commands to provision the VPC resources: subnets, bastion hosts (VSIs), VPN, etc., terraform init terraform plan terraform apply Connect to Client-To-Site VPN ● Once the VPC resources are successfully provisioned, you need to download the VPN client profile. ● Click the client to site tab and then on the name of the VPN ● Download the profile from the Clients tab ● The VPN provisioned through Terraform uses certificates. Follow the instructions here to connect to the ovpn client. ● You should see the successful connection on your OVPN client. 4. Verify the SSH Connection On a terminal, add an SSH private key to the SSH agent with the following command ssh-add Example: ssh-add ~/.ssh/ Run the following command to SSH into the RHEL VSI through a bastion host. You will be using the private IP address of the bastion in Zone 1 ssh -J root@10.10.0.13 root@10.10.128.13 Command to connect from a bastion host in Zone 2 is ssh -J root@10.10.65.13 root@10.10.128.13 Remember, you should be connected to the client-to-site VPN to SSH into the RHEL VSI through the bastion host. Try disconnecting the VPN and SSHing into the RHEL VSI. What Is Session Recording and Why Is It Required? A Bastion host and a jump server are both security mechanisms used in network and server environments to control and enhance security when connecting to remote systems. They serve similar purposes but have some differences in their implementation and use cases. The Bastion host is placed in front of the private network to take SSH requests from public traffic and pass the request to the downstream machine. Bastion hosts and jump servers are vulnerable to intrusion as they are exposed to public traffic. Session recording helps an administrator of a system to audit user SSH sessions and comply with regulatory requirements. In the event of a security breach, you as an administrator would like to audit and analyze the user sessions. This is critical for a security-sensitive system. Before deploying the session recording solution, you need to provision a private VPC network. Alternatively, if you are planning to use your own VPC infrastructure, you need to attach a floating IP to the virtual server instance and a public gateway to each of the subnets. Additionally, you need to allow network traffic from public internet access. Deploy Session Recording Using Ansible To be able to deploy the Session Recording solution you need to have the following packages installed on the RHEL VSI: tlog SSSD cockpit-session-recording The packages will be installed through Ansible automation on all the VSIs, both bastion hosts and RHEL VSI. Move to the Ansible folder. cd ansible 2. Create hosts.ini from the template file. cp hosts_template.ini hosts.ini Update the hosts.ini entries as per your VPC IP addresses. [bastions] 10.10.0.13 10.10.65.13 [servers] 10.10.128.13 [bastions:vars] ansible_port=22 ansible_user=root ansible_ssh_private_key_file=/Users/vmac/.ssh/ssh_vpc packages="['tlog','cockpit-session-recording','systemd-journal-remote']" [servers:vars] ansible_port=22 ansible_user=root ansible_ssh_private_key_file=/Users/vmac/.ssh/ssh_vpc ansible_ssh_common_args='-J root@10.10.0.13' packages="['tlog','cockpit-session-recording','systemd-journal-remote']" 4. Run the Ansible playbook to install the packages from an IBM Cloud private mirror/repository. ansible-playbook main_playbook.yml -i hosts.ini --flush-cache You can see in the image that after you SSH into the RHEL machine now, you will see a note saying that the current session is being recorded. Check the Session Recordings, Logs, and Reports If you closely observe the messages post SSH, you will see a URL to the web console that can be accessed using the machine name or private IP over port 9090. To allow traffic on port 9090, in the Terraform code, Change the value of the allow_port_9090 variable to true and run terraform apply. The latest terraform apply will add ACL and security group rules to allow traffic on port 9090. Now, open a browser and navigate to http://10.10.128.13:9090 . To access using the VSI name, you need to set up a private DNS (out of scope for this article). You need a root password to access the web console. Navigate to session recording to see the list of session recordings. Along with session recordings, you can check the logs, diagnostic reports, etc. What’s next Go through the deployed VPC network architecture and check the ACL and security group inbound and outbound rules. Check the client-to-site VPN routes (deliver and translate). Though the architecture shows one VPC, this can be expanded to have multiple VPCs, with each VPC playing a different role to bring in the separation of traffic like edge, management, and workload VPCs, respectively. Conclusion This article covered why session recording is required in bastion hosts for auditing and compliance and how session recording can be set up with the built-in RHEL packages using Ansible Automation. While designing a secured virtual private cloud network, you learned the best practices in architecting a VPC private network. We also covered the need to build highly available VPN servers and bastion hosts. With the provisioning of cloud infrastructure using Terraform and Ansible for session recording, you got hands-on experience. If you like my content, please check my website and give a follow on LinkedIN. Happy reading. In today's cloud world, security has taken precedence over resiliency and high availability. With the introduction of Virtual Private Cloud, network security has become even more critical. In this article, you will learn the best practices to make a virtual private cloud more private and secure than ever before. By the end of this article, you will learn how to: Design and build a secured and private Virtual Private Cloud network architecture. This is important for Financial institutions that are planning to move to Cloud. The role of bastion hosts in a private cloud network architecture and securing the bastion hosts using network rules. Build a highly available Virtual Private Network and Bastion host setup keeping the promise of cloud resiliency. The importance of session recording for auditing and compliance. Set up session recording in Red Hat Enterprise Linux using built-in packages. Learn how automation using Terraform and Ansible helps in building the Cloud infrastructure, high availability, session recording etc., Design and build a secured and private Virtual Private Cloud network architecture. This is important for Financial institutions that are planning to move to Cloud. The role of bastion hosts in a private cloud network architecture and securing the bastion hosts using network rules. Build a highly available Virtual Private Network and Bastion host setup keeping the promise of cloud resiliency. The importance of session recording for auditing and compliance. Set up session recording in Red Hat Enterprise Linux using built-in packages. Learn how automation using Terraform and Ansible helps in building the Cloud infrastructure, high availability, session recording etc., Let's explore best practices for designing and securing a Virtual Private Cloud (VPC) network in Cloud VPC Design and Architecture: Subnet Isolation: Divide your VPC into multiple subnets based on different tiers and security requirements. For example, have separate subnets for public-facing services and internal resources. Use of Network Access Control Lists (NACLs): Implement NACLs to control traffic at the subnet level. They act as a firewall for controlling inbound and outbound traffic to and from subnets. Network Security: Security Groups: Leverage security groups to control traffic at the instance level. Security groups act as a virtual firewall for your instance to control inbound and outbound traffic. VPC Peering / Transit Gateways: Use VPC peering / transit gateways to establish private connectivity between VPCs, allowing secure communication between them. VPN Connections: Establish Virtual Private Network (VPN) connections to securely connect your on-premises network to your VPC. High Availability: Multi-AZ Deployment: Distribute resources across multiple Availability Zones (AZs) to ensure high availability. In case of a failure in one AZ, traffic can be automatically routed to a healthy AZ. Auto Scaling Groups: Implement Auto Scaling Groups to automatically adjust the number of instances (Virtual Machines or Virtual Server Instances) based on demand. This ensures availability and fault tolerance. VPN and Bastion Hosts: Bastion Hosts: Use bastion hosts for secure access to instances in private subnets. These hosts act as an intermediary for secure access to your private instances. High Availability VPN: Set up VPN connections with redundancy across multiple endpoints for increased availability. Automation with Terraform: Infrastructure as Code (IaC): Use Terraform to define and provision your infrastructure as code. This allows for consistent and repeatable deployments. Version Control: Store your Terraform scripts in a version control system (e.g., Git) to track changes, collaborate with a team, and roll back to previous versions if needed. Monitoring and Logging: Logs and Metrics: Implement monitoring like CloudWatch to monitor your VPC. Set up alerts for key metrics and use Logs from the logging service for centralized logging. VPC Flow Logs: Enable VPC Flow Logs to capture information about the IP traffic going to and from network interfaces in your VPC. Regular Security Audits: Security Audits: Conduct regular security audits to identify and address potential vulnerabilities. This includes reviewing configurations, access controls, and monitoring logs for suspicious activities. VPC Design and Architecture: Subnet Isolation: Divide your VPC into multiple subnets based on different tiers and security requirements. For example, have separate subnets for public-facing services and internal resources. Use of Network Access Control Lists (NACLs): Implement NACLs to control traffic at the subnet level. They act as a firewall for controlling inbound and outbound traffic to and from subnets. VPC Design and Architecture: Network Security: Security Groups: Leverage security groups to control traffic at the instance level. Security groups act as a virtual firewall for your instance to control inbound and outbound traffic. VPC Peering / Transit Gateways: Use VPC peering / transit gateways to establish private connectivity between VPCs, allowing secure communication between them. VPN Connections: Establish Virtual Private Network (VPN) connections to securely connect your on-premises network to your VPC. Network Security: High Availability: Multi-AZ Deployment: Distribute resources across multiple Availability Zones (AZs) to ensure high availability. In case of a failure in one AZ, traffic can be automatically routed to a healthy AZ. Auto Scaling Groups: Implement Auto Scaling Groups to automatically adjust the number of instances (Virtual Machines or Virtual Server Instances) based on demand. This ensures availability and fault tolerance. High Availability: VPN and Bastion Hosts: Bastion Hosts: Use bastion hosts for secure access to instances in private subnets. These hosts act as an intermediary for secure access to your private instances. High Availability VPN: Set up VPN connections with redundancy across multiple endpoints for increased availability. VPN and Bastion Hosts: Automation with Terraform: Infrastructure as Code (IaC): Use Terraform to define and provision your infrastructure as code. This allows for consistent and repeatable deployments. Version Control: Store your Terraform scripts in a version control system (e.g., Git) to track changes, collaborate with a team, and roll back to previous versions if needed. Automation with Terraform: Monitoring and Logging: Logs and Metrics: Implement monitoring like CloudWatch to monitor your VPC. Set up alerts for key metrics and use Logs from the logging service for centralized logging. VPC Flow Logs: Enable VPC Flow Logs to capture information about the IP traffic going to and from network interfaces in your VPC. Monitoring and Logging: Regular Security Audits: Security Audits: Conduct regular security audits to identify and address potential vulnerabilities. This includes reviewing configurations, access controls, and monitoring logs for suspicious activities. Regular Security Audits: By following these best practices, you can create a highly secure and resilient VPC. Regularly review and update your security measures to adapt to evolving threats and ensure ongoing protection of your cloud infrastructure. Implementing automation with tools like Terraform further enhances efficiency and consistency in managing your cloud resources. Secured way to connect to a Private VPC Network This article uses IBM Cloud to deploy the VPC network using Terraform as the infrastructure as code automation. You can use any cloud provider to deploy the architecture as VPC is available on all the leading cloud vendors for network isolation. Client-to-site VPN for VPC is one of the two VPN options available on IBM Cloud that provides a client-to-site VPN solution that allows users to connect to IBM Cloud resources through secure, encrypted connections. What Does this Architecture Address? The bastion hosts and the client-to-site VPN servers are highly available. The VPN creates a secure tunnel to connect from a local machine to the Cloud. The bastion host accepts SSH connections from client to site VPN (192.168.x.x). The last (final) rule in the stateless access control lists is denying all the inbound and outbound traffic. No public gateways are attached to the subnets, and no floating IPs on the virtual server instances (VSIs). The security groups attached to the VSIs allow traffic only on specific required ports and private IPs. The bastion hosts and the client-to-site VPN servers are highly available. The VPN creates a secure tunnel to connect from a local machine to the Cloud. The bastion host accepts SSH connections from client to site VPN (192.168.x.x). The last (final) rule in the stateless access control lists is denying all the inbound and outbound traffic. No public gateways are attached to the subnets, and no floating IPs on the virtual server instances (VSIs). The security groups attached to the VSIs allow traffic only on specific required ports and private IPs. Before you Begin ● IBM Cloud API Key to provision cloud resources. IBM Cloud API Key ● Certificate's CRN from Secrets Manager to store VPN certificates. Certificate's CRN from Secrets Manager ● OpenVPN client to connect to client-to-site VPN from your local machine. OpenVPN client ● Install and set up Terraform Install and set up Terraform ● Install and set up Ansible Install and set up Ansible Provision the Private VPC Network Using Terraform Once you have the secrets manager secret with the certificate, launch your terminal and set the following Terraform variables. Once you have the secrets manager secret with the certificate, launch your terminal and set the following Terraform variables. Secrets Manager export TF_VAR_ibmcloud_api_key= export TF_VAR_secrets_manager_certificate_crn= export TF_VAR_ibmcloud_api_key= export TF_VAR_secrets_manager_certificate_crn= Clone the GitHub repository with Terraform and Ansible code. Clone the GitHub repository with Terraform and Ansible code. git clone https://github.com/VidyasagarMSC/private-vpc-network cd terraform git clone https://github.com/VidyasagarMSC/private-vpc-network cd terraform Run the Terraform commands to provision the VPC resources: subnets, bastion hosts (VSIs), VPN, etc., Run the Terraform commands to provision the VPC resources: subnets, bastion hosts (VSIs), VPN, etc., terraform init terraform plan terraform apply terraform init terraform plan terraform apply Connect to Client-To-Site VPN ● Once the VPC resources are successfully provisioned, you need to download the VPN client profile . profile ● Click the client to site tab and then on the name of the VPN client to site ● Download the profile from the Clients tab ● The VPN provisioned through Terraform uses certificates. Follow the instructions here to connect to the ovpn client . Follow the instructions here to connect to the ovpn client ● You should see the successful connection on your OVPN client. 4. Verify the SSH Connection On a terminal, add an SSH private key to the SSH agent with the following command On a terminal, add an SSH private key to the SSH agent with the following command ssh-add ssh-add Example: ssh-add ~/.ssh/ Example: ssh-add ~/.ssh/ Run the following command to SSH into the RHEL VSI through a bastion host. You will be using the private IP address of the bastion in Zone 1 Run the following command to SSH into the RHEL VSI through a bastion host. You will be using the private IP address of the bastion in Zone 1 ssh -J root@10.10.0.13 root@10.10.128.13 ssh -J root@10.10.0.13 root@10.10.128.13 Command to connect from a bastion host in Zone 2 is ssh -J root@10.10.65.13 root@10.10.128.13 ssh -J root@10.10.65.13 root@10.10.128.13 root@10.10.65.13 root@10.10.128.13 Remember, you should be connected to the client-to-site VPN to SSH into the RHEL VSI through the bastion host. Try disconnecting the VPN and SSHing into the RHEL VSI. Remember, you should be connected to the client-to-site VPN to SSH into the RHEL VSI through the bastion host. Remember, you should be connected to the client-to-site VPN to SSH into the RHEL VSI through the bastion host. Try disconnecting the VPN and SSHing into the RHEL VSI. Try disconnecting the VPN and SSHing into the RHEL VSI. What Is Session Recording and Why Is It Required? A Bastion host and a jump server are both security mechanisms used in network and server environments to control and enhance security when connecting to remote systems. They serve similar purposes but have some differences in their implementation and use cases. The Bastion host is placed in front of the private network to take SSH requests from public traffic and pass the request to the downstream machine. Bastion hosts and jump servers are vulnerable to intrusion as they are exposed to public traffic. Session recording helps an administrator of a system to audit user SSH sessions and comply with regulatory requirements. In the event of a security breach, you as an administrator would like to audit and analyze the user sessions. This is critical for a security-sensitive system. Before deploying the session recording solution, you need to provision a private VPC network. Alternatively, if you are planning to use your own VPC infrastructure, you need to attach a floating IP to the virtual server instance and a public gateway to each of the subnets. Additionally, you need to allow network traffic from public internet access. you need to attach a floating IP to the virtual server instance and a public gateway to each of the subnets. Deploy Session Recording Using Ansible To be able to deploy the Session Recording solution you need to have the following packages installed on the RHEL VSI: tlog SSSD cockpit-session-recording tlog SSSD cockpit-session-recording The packages will be installed through Ansible automation on all the VSIs, both bastion hosts and RHEL VSI. Move to the Ansible folder. Move to the Ansible folder. cd ansible cd ansible 2. Create hosts.ini from the template file. hosts.ini cp hosts_template.ini hosts.ini cp hosts_template.ini hosts.ini Update the hosts.ini entries as per your VPC IP addresses. Update the hosts.ini entries as per your VPC IP addresses. hosts.ini [bastions] 10.10.0.13 10.10.65.13 [servers] 10.10.128.13 [bastions:vars] ansible_port=22 ansible_user=root ansible_ssh_private_key_file=/Users/vmac/.ssh/ssh_vpc packages="['tlog','cockpit-session-recording','systemd-journal-remote']" [servers:vars] ansible_port=22 ansible_user=root ansible_ssh_private_key_file=/Users/vmac/.ssh/ssh_vpc ansible_ssh_common_args='-J root@10.10.0.13' packages="['tlog','cockpit-session-recording','systemd-journal-remote']" [bastions] 10.10.0.13 10.10.65.13 [servers] 10.10.128.13 [bastions:vars] ansible_port=22 ansible_user=root ansible_ssh_private_key_file=/Users/vmac/.ssh/ssh_vpc packages="['tlog','cockpit-session-recording','systemd-journal-remote']" [servers:vars] ansible_port=22 ansible_user=root ansible_ssh_private_key_file=/Users/vmac/.ssh/ssh_vpc ansible_ssh_common_args='-J root@10.10.0.13' packages="['tlog','cockpit-session-recording','systemd-journal-remote']" 4. Run the Ansible playbook to install the packages from an IBM Cloud private mirror/repository. ansible-playbook main_playbook.yml -i hosts.ini --flush-cache ansible-playbook main_playbook.yml -i hosts.ini --flush-cache You can see in the image that after you SSH into the RHEL machine now, you will see a note saying that the current session is being recorded. Check the Session Recordings, Logs, and Reports If you closely observe the messages post SSH, you will see a URL to the web console that can be accessed using the machine name or private IP over port 9090. To allow traffic on port 9090, in the Terraform code, Change the value of the allow_port_9090 variable to true and run terraform apply . The latest terraform apply will add ACL and security group rules to allow traffic on port 9090. allow_port_9090 true terraform apply Now, open a browser and navigate to http://10.10.128.13:9090 . To access using the VSI name, you need to set up a private DNS (out of scope for this article). You need a root password to access the web console. Now, open a browser and navigate to http://10.10.128.13:9090 . To access using the VSI name, you need to set up a private DNS (out of scope for this article). You need a root password to access the web console. http://10.10.128.13:9090 http://10.10.128.13:9090 Navigate to session recording to see the list of session recordings. Along with session recordings, you can check the logs, diagnostic reports, etc. Navigate to session recording to see the list of session recordings. Along with session recordings, you can check the logs, diagnostic reports, etc. What’s next Go through the deployed VPC network architecture and check the ACL and security group inbound and outbound rules. Check the client-to-site VPN routes (deliver and translate). Though the architecture shows one VPC, this can be expanded to have multiple VPCs, with each VPC playing a different role to bring in the separation of traffic like edge, management, and workload VPCs, respectively. Go through the deployed VPC network architecture and check the ACL and security group inbound and outbound rules. Check the client-to-site VPN routes (deliver and translate). Though the architecture shows one VPC, this can be expanded to have multiple VPCs, with each VPC playing a different role to bring in the separation of traffic like edge, management, and workload VPCs, respectively. Conclusion This article covered why session recording is required in bastion hosts for auditing and compliance and how session recording can be set up with the built-in RHEL packages using Ansible Automation. While designing a secured virtual private cloud network, you learned the best practices in architecting a VPC private network. We also covered the need to build highly available VPN servers and bastion hosts. With the provisioning of cloud infrastructure using Terraform and Ansible for session recording, you got hands-on experience. If you like my content, please check my website and give a follow on LinkedIN . Happy reading. website LinkedIN