How to Configure an Airgap Repository for TCA 2.3: A Step-by-Step Guide

To ensure that your VMware Telco Cloud Automation environment is secure, you can isolate its network from unsecured public Internet or local area networks. This means, your system does not have any network interfaces connected to external networks. This topic introduces the steps for setting up a server in an internet-restricted (airgap) environment.

Prerequisites for Setting up the Airgap Repository

To set up an airgap server, you must prepare the following resources:

1. An environment with Internet connectivity with access to the following websites:

   • projects.registry.vmware.com

   • vmwaresaas.jfrog.io

   • packages.vmware.com

   • github.com

2. An FQDN to assign to the airgap server. Optionally, DNS service can be available for resolving FQDN names if the setup environment is not the target environment.

3. (Optional) Server certificates. To generate private CA-signed certificates automatically, the VMware Telco Cloud Automation airgap server provides setup scripts. However, you can prepare your own server certificate that is private CA-signed or public CA-signed. For a chained certificate, it must contain all the trusted CA certificates:

• Self-signed certificate or chained certificate file.

• Certificate key file.

• (Optional) CA certificate file for verifying the server certificate. It can be a self-signed certificate or a trusted private root CA file that has signed the certificate chain. For a public-signed server certificate, you need not specify the CA file but ensure that you include the CA certificates in the chained certificate file.

1. Download the airgap tarball file named VMware-Telco-Cloud-Automation-airgap-files-<release>-<build>.tar.gz from the VMware Customer Connect site. The tarball file is included in the VMware Telco Cloud Automation build.

2. Deploy Photon OS 3 OVA: A virtual machine with Photon OS 3.0 Rev 3 must be available for providing repository services in the airgap environment. Download the OVA template from here.

   1. Import the OVA template onto the ESXi host in an environment that has Internet connectivity.

   2. Right-click the ESXi host and select Deploy OVF Template.

   3. To complete the deployment, follow the wizard.

3. Edit the imported airgap virtual machine with the following disk requirements:

   1. Single disk with three partitions. Each partition is assigned to Harbor, Docker, and Photon. This approach applies to previous versions (1.9.5.x, 2.0.x) of VMware Telco Cloud Automation. A minimum of 500 GB is recommended for scaling-up considerations.

   2. Multiple disks and each disk is configured with a single partition. In the image synchronization phase (setup phase), three disks are required - one for the images, one for Photon, and one for Docker images. After completing the packaging and image synchronization, you must export the airgap server OVA. It is recommended to clear the Docker cache and remove the Docker cache disk to reduce the total size of the OVA to transfer. Multiple disks is the recommended way to configure airgap server. In case the airgap server is connected to the Internet for build-to-build updates and there is no need to export the OVA file, it is still recommended to use multiple disks where each disk can be resized separately on demand. The Docker cache disk can be reserved to accelerate the next synchronization. This is the default mode in the sample setup user input.

   3. VM with multiple disks (recommended):

      1. CPU - 4

      2. RAM - 8 GB

      3. NIC - 1. Wire to the network with Internet connectivity.

      4. Hard Disks - 4.

      5. Hard Disk 1: 16 GB for photon OS.

      6. Hard Disk 2: 100 GB for container images of Harbor repository, and Photon repository.

      7. Hard Disk 3: 200 GB for photon packages.

      8. Hard Disk 4: 200 GB for cached docker images.

   4. VM with a single disk:

      1. CPU - 4

      2. RAM - 8 GB

      3. NIC - 1. Wire to the network with Internet connectivity.

      4. Hard Disks - 2.

      5. Hard Disk 1: 16 GB for photon OS.

      6. Hard Disk 2: 500 GB for Docker repository, Harbor repository, and Photon repository.

4. Power on the airgap server.

5. Change the airgap VM root password. The default password is changeme. You must set up a new password on logging in for the first time.
   

Set up the Airgap Server

Before running the setup Ansible scripts, upload the required resources, make the airgap server accessible to the Internet, and install the required packages for running the scripts.

• Upload the resources airgap script tarball to the airgap vm.
  

• After the virtual machine is powered on, set up the eth0 network and set up either DHCP or static IP address in the guest operating system.

• Extract the tarball using tar command. For Photon VM, tar can be intalled by "tdnf install tar -y" command.

tar xfz VMware-Telco-Cloud-Automation-airgap-files-2.3.0-21563123.tar.gz

• Certificates: (Optional) If you use your own certificate, upload its relevant files into the /roots/certs folder. You can also use scripts for generating the certificate at runtime.

• Please make sure that your airgap virtual machine is able to access the Internet. If you are using a proxy server, please run the following script. If you are not using a proxy, you can skip this step. We will proceed without the proxy setup (refer to the official guide for instructions on setting up with a proxy and uploading proxy certificates).

root@photon-machine [ ~ ]# . airgap/scripts/bin/setup-proxy.sh <http-proxy-sever-url> <https-proxy-server-url> <airgap-server-fqdn>,<local-subnet>

• Prepare the user-inputs.yml file.

root@photon-machine [ ~/airgap/scripts/vars ]# ls
deploy-user-inputs.yml setup-user-inputs.yml
root@photon-machine [ ~/airgap/scripts/vars ]# cp setup-user-inputs.yml user-inputs.yml
root@photon-machine [ ~/airgap/scripts/vars ]# vi user-inputs.yml

• The user-inputs.yml file contains input parameter descriptions. You can change these values according to your environment.

• Note: In setup-user-inputs.yml, the default disk configuration is multiple disks. If you need a single disk setup, change the value of single_disk to single_disk: True.

• If you want the airgap server to hold multiple versions of VMware Telco Cloud Automation for the purpose of upgrading, change the products setting to specify all the required versions, separated by _. For example, the following configuration indicates that the airgap server syncs with both TCA 2.0.0 and 2.1.0 images.

products: 
  - name: "tca" 
     versions: "2.0.0_2.1.0"

• Now that we have updated our user-inputs.yml, Run the setup.yml Ansible Playbook. After you have updated the user-inputs.yml file, you can trigger ansible playbooks to setup the airgap server template:

root@photon-machine [ ~/airgap ]# scripts/bin/run.sh setup

• Ansible Playbook takes around three hours to run, depending on your network speed, and you can view the log file to monitor progress. If there is an error, run Ansible Playbook again with the same command.

Possible errors and measures

• Firstly, we need to ensure that all the required packages are up to date. Use the command below to complete this step.

tdnf -y update --nogpgcheck

• Use this command to view the progress.

tail -f ansible.log

• The repository sync may take several hours to complete, depending on the internet connectivity. As of today, 228 repositories are synced and around 28 GB of space is occupied the harbor. We will need to wait for the sync to finish.

• If the synchronization is not completed and times out, please run the command again to start the process. It will synchronize the remaining repositories.

scripts/bin/run.sh setup

• We might encounter this error related to community.general module. This is due to a bug in the ansible version.

fatal: [localhost]: FAILED! => {"reason": "couldn't resolve module/action 'community.general.pamd'. This often indicates a misspelling, missing collection, or incorrect module path.\n\nThe error appears to be in '/root/airgap/scripts/compliance/photon_stig/roles/photon3/tasks/photon.yml': line 113, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: PHTN-30-000002 - Delete rule for pam_tally2.so if it doesn't exist after the pam_unix.so auth rule in system-auth\n  ^ here\n"}

• We can either upgrade the Ansible core version to 2.14.0 or newer, or use the workaround command below to install the community.general module.

ansible-galaxy collection install community.general --force -vvv --no-cache

• Make sure there are no failed events in the ansible.log.

Validate Airgap Server Setup

After setting up the airgap server, validate it. Ansible Playbook performs these validations automatically, but you can also run them manually.

• Validate the TDNF version and the makecache function.

tdnf --version
tdnf clean all
tdnf makecache

• Note: If you encounter an error or a permission issue when running TDNF, remove the lock file at /var/run/.tdnf-instance-lockfile and try again.

• Validate certificate availability: Verify that the certificates are copied to /etc/docker/certs.d/{airgap server fqdn}:[{https port number}].

ls /etc/docker/certs.d/{airgap server fqdn}:[{https port number}]

• Validate Harbor login and image pulling.

docker login {airgap server fqdn}/registry
docker pull {airgap server fqdn}/registry/tkr-compatibility:v1

• Validate if the nginx service is running

nginx -T
systemctl status nginx

• Validate Photon OS repositories. Verify if you can access the four repositories that are used during cluster creation:

curl -k -I "https://$server_fqdn/updates/photon-updates/" 2>&1
curl -k -I "https://$server_fqdn/release/photon/" 2>&1
curl -k -I "https://$server_fqdn/updates/photon-telco-updates/" 2>&1
curl -k -I "https://$server_fqdn/updates/photon-telco-debuginfo/" 2>&1

• The airgap setup is now complete.