Setting up an OpenVPN Access Server cluster
A feature newly introduced in Access Server since version 2.6 is the ability to create a cluster of Access Servers. This feature is currently only available in the beta version of Access Server 2.6. Such a cluster is an answer to the requirement that our customers have expressed for a high-availability solution and it also provides the ability to spread the load across multiple servers. There are still challenges to overcome to make this an ideal solution for all situations, but for these particular use-cases, it works very well.
A single OpenVPN Access Server contains everything it needs to offer its services to connecting VPN clients. All the user credentials, certificates, services, and access rules, can all be present on one Access Server. This is also a single point of failure. To resolve this we have our LAN-model UCARP/VRRP based failover system to run a standby server that can take over if the primary fails. However, it is limited to running on a local network that supports VRRP/UCARP traffic to pass through. Some networks don’t allow this traffic. Amazon AWS for example also doesn’t, so this model doesn’t work there.
The solution is to separate the storage of certificates and credentials from the OpenVPN service daemons. Our cluster solution allows you to use a MySQL or MariaDB database system, or Amazon RDS, to store all the configuration of the Access Server. These database systems can be single servers or clusters. Amazon RDS for example can offer a fault-tolerant solution that automatically switches over to a backup server if the primary server fails. Multiple Access Servers can then attach to this central database in clustering mode, and offer its VPN services. In a simple setup, a DNS-based round-robin system can ensure that VPN clients will have a single address to connect to, and it will try these different servers in sequential order. If an OpenVPN Access Server server fails, it will lead to a temporary connection failure for the connected clients, until the clients connect to one of the other servers in the cluster. This should happen automatically. The failed server could then be removed from the DNS records and a new Access Server could be setup, attached to the cluster, and then added to replace the failed node.
As mentioned, this feature is currently in beta and is currently being developed. There are plans to enhance the clustering feature further. Currently each Access Server node in a cluster still manages its own licenses, and its own VPN client subnet. So, for example, a VPN client connected to node A cannot reach a VPN client connected to node B in a cluster. For situations where this is necessary, we will be making a solution in the future. And in regards to licensing, we are working on a better licensing system that can allow licenses to be shared across the cluster in a sensible manner. The use-case for a cluster solution is however ideal for large scale access to resources only reachable through the VPN server with high-availability, and/or to protect Internet access for large quantities of users with high-availability.
Currently, our existing LAN-model UCARP/VRRP type failover system cannot be upgraded directly to a cluster setup yet, this results in a failure. This will be addressed in a future release. Since the entire cluster feature is in beta currently, it is not advisable to upgrade a high-availability system to a beta release in any case. If you have such a failover system now and wish to test, a separate setup that does not affect your production systems is advisable.
Some important notes
Most of our customers will currently be using the default SQLite3 database files to store all the configuration of a single OpenVPN Access Server installation. When you install the Access Server from scratch, it will still default to SQLite3, even with the new versions of Access Server with clustering support built-in. You can upgrade an existing Access Server installation with the new version and it will continue to function as before, and retain all your settings. Please do note though that a failover setup cannot currently be updated to the newer version, so please be aware of that limitation. The cluster function is an on/off toggle that is not enabled by default, and with the cluster function switched off, it will just be a normal Access Server. But of course now with an updated Admin UI interface with all the usual functions and settings still present, and now with a new look and some additional functions. Only when you go into the Admin UI and enable the cluster function, do the new functions and settings come into play.
The cluster function requires that you migrate your settings to MySQL, MariaDB, or Amazon RDS. In theory, as long as it is a MySQL 5.6 or higher compatible system, it should work. Amazon Aurora available as an engine for an Amazon RDS database is such a compatible system. With Access Server 2.5.2 and older, the conversion from SQLite3 local storage to a MySQL, MariaDB, or Amazon RDS type system was already possible, and back then it had to be done on the command line manually. We now have a conversion tool built right into the Admin UI to handle that part. You do still need to have some database client support software installed for the connection to be made, and you do need to still set up the new database system yourself first. In our guide we are going to be assuming a situation where Amazon RDS is used, so this means we’re doing this cluster setup on Amazon AWS, but a MySQL or MariaDB setup can also be used for the database storage, and that means it’s not tied to Amazon AWS. To eliminate a single point of failure we do advise using a fault-tolerant setup for the database system, whatever solution you choose to use.
The conversion process from local SQLite3 to MySQL type database system currently has no safety checks in place, but these will of course be added. What this means is that if you convert an Access Server configuration to MySQL type database system, and then repeat that same act again from another Access Server, you will wipe the original settings, and the latest converted settings will be the only valid settings. So please do not repeat a database conversion to the same target MySQL database system.
Before you begin
Backup your settings first. You can use the backup commands on the command line found here to make a complete backup of the settings on your Access Server installation safely without having to stop operations on your server. If you’ve never made a backup before, now would be a good time to remind you that it would be a good idea to set up an automated backup plan.
Setting up Amazon RDS
It is important to note here that you are not required to use Amazon RDS. Another MySQL or MariaDB system also works, and this also means you are not tied to Amazon AWS. We just assume this is the easiest setup for our customers and serves as a general guide on how to set up an OpenVPN Access Server cluster. The term cluster is also used in database systems that are fault-tolerant. Please do not confuse a database cluster setup with an Access Server cluster setup, they are two different things. You need both a cluster database setup and an Access Server cluster setup with multiple nodes to be fully fault-tolerant.
- Log on to the Amazon AWS console.
- Under Services look under the Database header for the RDS option (Managed Relational Database Service).
- Click Create database and select the engine of choice: Amazon Aurora, MySQL, or MariaDB.
- Default options are usually fine here, click the Next button to continue.
- Specify instance size and fault-tolerance settings – this depends entirely on how heavily you intend to use the system.
- Specify the DB instance identifier, the Master username, and the Master password (twice).
- Take note of the username and password as you will need them later, and click the Next button to continue.
- Select which VPC and subnet this should be launched on. You can choose if it should be publicly reachable or not.
- It is not necessary to create a default database here. Review the other settings to set them as you prefer, defaults are usually fine.
- Complete the launch and then find your new Amazon RDS database in the Instances or Clusters overview.
- Wait until the status shows that it is Available before attempting to use it.
- Find the Cluster endpoint if you are using a cluster, or just the instance’s Endpoint when it’s just an instance.
- Take note of the endpoint name, you will need it together with the Master username and the Master password later.
This should result in an Amazon RDS instance or cluster that is still empty right now, but is capable of storing database information. A logical next step is to install one OpenVPN Access Server from scratch, or take an existing Access Server setup, and updating it to the latest beta release. Then the required MySQL client software can be installed, and a connection to this RDS system can be setup and tested. You are then ready to convert the database from local SQLite3 storage to the Amazon RDS database, and the cluster function can then be enabled, and additional Access Server nodes added.
Please keep in mind that Amazon RDS databases are protected by security groups. These are an Amazon specific security system that functions like a firewall. It is therefore necessary to adjust the security group settings so that your Access Server nodes that you intend to connect to this Amazon RDS database can actually reach this database.
Our example RDS database has these 3 important items that we will need later:
- Endpoint: astest-cluster.cluster-cw1zos4nytdr.us-west-1.rds.amazonaws.com
- Master username: adminuser
- Master password: Fw4MjHs5KPjScCkz7Yxyp5YKNh
Initial setup of first Access Server cluster node
Since the cluster build of Access Server currently is in beta, and not released as stable version yet, there is no appliance that can be deployed from scratch that has the cluster function built in. We are therefore going to assume that you have an Access Server already setup. If you do not know how to do this, we have various installation options and guides available on the page below. We are going to be assuming you have a working Access Server and that you can log in to the Admin UI from this point onwards. Please note that not all operating systems are currently being built for, so check the beta releases first before you make a choice on which OS to launch. Our appliances and our guides are currently all based around Ubuntu 16 x64 and that’s definitely available.
Once you have a working Access Server with a standard stable release, you can upgrade it to the latest beta release version. The latest beta release can be downloaded here:
When you choose to use an existing Access Server that you are already using now, and that already has users and settings configured, you can upgrade it to the latest version with the instructions below, and the licenses and users and settings will all be retained. As explained, the beta release is a newer version of the existing OpenVPN Access Server version, but with a new optional cluster feature added that does not necessarily have to be used. But in this guide of course we are going to explain how to enable it further on.
In order to upgrade the Access Server to this beta release, you will need to download the Access Server package from the page above and place it somewhere on the intended server host. You can do this via a roundabout way by using your desktop computer to download the installation package from our website, and then uploading it using a tool such as SCP or WinSCP. But an easier method is to use wget, which is a tool designed to retrieve files directly from the Internet and save it directly on the file system of the Linux operating system where you are upgrading the OpenVPN Access Server program.
You can right-click the download link and select “Copy Link Address” or “Copy target” or such. The exact wording depends on the browser used. The goal is having the link to the installation package in your copy/paste buffer. Next go to the Linux server where you want to install the OpenVPN Access Server program and use wget to download the installation package file directly to the server.
Type wget followed by the pasted URL:
wget <paste copied url>
For example for Ubuntu 16 x64 installation package:
Optional step for advanced users: it is possible to use https:// for the connection instead if you prefer a secure connection, and you can verify if the package file you have downloaded has been correctly downloaded, and that it is in fact the package file that we are distributing and not somehow a tainted copy. This is all very unlikely but still you can check with the tool sha256sum, which creates a hash for the downloaded file. You can then compare it with the SHA256sum hash mentioned on the beta release downloads page on our website, where you found the installer file. Use command line “sha256sum openvpn-as-2.6.beta.1-Ubuntu16.amd_64.deb” to generate the hash, and compare it to what is listed on the site. If they match you can be certain that you have the right file and it has downloaded correctly.
Now that the installation package file is downloaded to your system you can perform the upgrade with the following command:
Upgrade installation package on Debian/Ubuntu system:
dpkg -i openvpn-as-2.6.beta.2-Ubuntu16.amd_64.deb
Upgrade installation package on RedHat/CentOS/Fedora system:
rpm -Uvh openvpn-as-2.6.beta.2-CentOS7.x86_64.rpm
The upgrade process should then commence and finish. Afterwards you should reboot:
You should now have an Access Server that runs our latest beta release version, and you should be able to log in to the new Admin UI and see the new functions in the menu.
Firewall configuration – ports to open
These are the ports that need to be open. Our current appliances on Amazon AWS have most of these ports open, but port TCP 945 is a new one, it is the control port for the cluster system. These ports mentioned below assume standard configuration. If you know you have changed your ports then please adjust as necessary.
- TCP 22 – for SSH access
- TCP 443 – for web interface access, and OpenVPN TCP connections
- TCP 943 – for web interface access
- TCP 945 – for cluster control channel
- UDP 1194 – for OpenVPN UDP connections
Configure and test database server connection
From the steps in the section about setting up Amazon RDS we will take the endpoint name, the Master username, and the Master password, and use them to test if this connection works. When access Server can make a connection to the database, you can continue with the steps to setup a new cluster. To reiterate, these are the example settings we are using in this guide:
- Endpoint: astest-cluster.cluster-cw1zos4nytdr.us-west-1.rds.amazonaws.com
- Master username: adminuser
- Master password: Fw4MjHs5KPjScCkz7Yxyp5YKNh
As usual for all our command line tools documentation we are assuming you are logged on as root user and are in the /usr/local/openvpn_as/scripts/ folder. These instructions are for Ubuntu/Debian but by replacing apt-get with yum you should be able to achieve the correct results on CentOS and Red Hat.
Install the required software:
apt-get update apt-get install mysql-client libmysqlclient-dev
Next, connect to the RDS instance with MySQL command line tool:
mysql -h astest-cluster.cluster-cw1zos4nytdr.us-west-1.rds.amazonaws.com -u adminuser -p
When asked for a password, provide the Master password.
You should be seeing a message like this, and you can exit with the exit command:
Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 13 Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> exit Bye
This system is now able to make a successful connection to the Amazon RDS database. You can now proceed to the next step.
Setup a new cluster
Log on to the Admin UI of the Access Server, and go to Configuration and then go to Cluster. You will see a page where you can select to Setup a New Cluster. For our initial Access Server node, this is what we’ll choose. For any future nodes you want to add to an existing cluster setup, you can choose Join existing cluster instead.
If your database backend is on MySQL already, then you can just click Save to confirm that you want to convert your current Access Server node to a cluster node. It will then restart and you’re ready to start using your cluster setup. However, if your database backend is the default SQLite3, and not yet on the MySQL type database system, then you will see additional fields where you can enter the following information to do the conversion to MySQL type database. In the following steps we will assume you have yet to convert to a MySQL type database.
During this step of setting up a new cluster, any existing user certificates and settings will be converted and placed into the cluster configuration. This conversion can only be done once, so it’s not possible to repeat this to combine multiple different Access Servers into one cluster. The first Access Server you use to setup a cluster will be the master data set and any Access Server nodes that you add to the cluster later on will use that master data set, and changes made afterward to user configurations in a cluster will apply for all the nodes in the cluster.
- MySQL Username: enter the Master username here.
- MySQL Password: enter the Master password here.
- Hostname or IP: enter the endpoint name here.
- MySQL port: the default is port 3306.
Press the Save button to convert the local SQLite3 databases to the new MySQL type databases. The Access Server will now take a short while to convert things. If your user base is very large, it may take some time for this to complete fully. Once it is done, a restart of the Access Server will be done automatically. Once ready you will be back at the Admin UI login prompt, and you’re ready to start using your cluster setup.
Adding more nodes to the cluster
You can set up a new Access Server and install the beta build on it, and log on the Admin UI. Then you go to Cluster and you select Join existing cluster. You can then enter the database connection details and join the cluster. It is important to note that if this is an Access Server with existing users and configuration, then this node that you are adding will be wiped clean (but backups are made automatically just in case) and will get its user certificates and other information from the cluster instead.