Skip to main content

Troubleshooting an Access Server Failure

Abstract

When you have an Access Server installation that refuses to work, use this topic to troubleshoot known issues.

This topic provides troubleshooting tips for Access Server administrators dealing with a previously working server that is no longer functional. Below are logical steps that guide you through the process. With each step, if it didn’t help or doesn’t apply to you, move to the next step.

Step 1: Events leading up to the failure

Access Server is typically stable, and issues often arise after changes. Common scenarios to check:

Server stopped working after an in-place upgrade

  1. Reboot the server.

    • Ensure the server is restarted after the upgrade.

  2. Check installation compatibility.

    • Verify that the correct installation instructions were used. Installing software intended for different OS versions (e.g., Ubuntu 24 software on Ubuntu 22) can cause issues.

Related tutorial:

New server doesn't start after migrating data

  1. Check for an interface name mismatch.

    • Confirm that the interface names on the old and new servers match. Access Server binds to all interfaces by default but may be configured to bind to a specific interface that doesn't exist on the new server.

    • For example, the old server was set to bind to the interface, "eth0" and the new server's interface is named "ens192".

  2. Reset the OpenVPN web services and daemons.

Certificates were accidentally revoked

  1. If you took actions that revoked client certificates, restore a backup.

  2. If you don't have a backup, it's best to reprovision your VPN clients with new profiles.

Certificate verification failed on all VPN clients

Issue: If all VPN clients suddenly show a "certificate verification failed" error, it likely means the VPN certificate structure has expired.

Resolution: This usually happens when running Access Server before version 2.9 and the lifetime for the VPN certificate structure has expired (by default 10 years). You need to reprovision clients with new connection profiles after fixing the VPN certificate structure by following these steps:

  1. Upgrade Access Server to at least version 2.9 to detect and fix expired certificates automatically.

    Notice

    If an upgrade is not possible, you can reset the VPN certificate structure:

    1. Make a backup (this command resets the entire VPN certificate structure).

    2. Run as root:

      /usr/local/openvpn_as/scripts/sa --keysize 2048 init

  2. Download new connection profiles for your VPN clients and get connected again.

If you're unsure whether the VPN certificate authority or a client certificate has expired, follow these steps:

  1. Extract certificates:

    • Open the connection profile in a text editor.

    • Copy the CA certificate (between <ca> and </ca>) and save it as ca.crt.

    • Copy the client certificate (between <cert> and </cert>) and save it as client.crt.

  2. Check the expiry with OpenSSL:

    openssl x509 -noout -dates -in ca.crt

    openssl x509 -noout -dates -in client.crt

    • Sample output (for both ca and client):

      notBefore=Mar 18 03:56:25 2010 GMT
notAfter=Mar 23 03:56:25 2020 GMT

  3. If the certificate has expired (date shown in notAfter, as our sample output shows), it cannot be used. Both CA and client certificates must be valid.

Tip

Access Server, by default, generates CA certificates valid for ten years. You can manage these:

Access Server or OpenVPN Connect are not the latest versions

We recommend running the latest versions of Access Server and OpenVPN Connect as security best practices.

When you use older software, you may run into problems that have been resolved with newer versions. To update:

Check the release notes to see if the problem you’re experiencing matches any listed issues. This can help determine if updating to a newer version will resolve your problem.

Step 2: Confirm the reachability of the services

Connectivity issues are often related to the network or the internet connection between the VPN client and the VPN server. These issues may prevent you from connecting successfully while the server operates normally.

Verify that you can connect your VPN client to this server

You can use OpenVPN Connect v3 logs to help you verify your server connection:

  1. Launch OpenVPN Connect.

  2. Click the Connect-log-icon.jpglog icon.

  3. Check for errors like these:

    • Server poll timeout: The server address isn't responding or is unreachable.

    • Certificate verification failed: The certificate used is invalid or has other issues.

  4. If the VPN client connects, Access Server's VPN services are functioning.

  5. Ensure you use the latest OpenVPN Connect version.

Verify your own internet connection

  1. Try different internet connections:

    • Try connecting using another device or network, such as a smartphone hotspot.

    • If the VPN works on a different connection, your usual network might have a firewall issue or temporary problem.

  2. Check internal versus external connections:

    • If the VPN works internally but not externally, ensure proper external access setup.

    • Forward and allow these ports: TCP 443, TCP 943, TCP 945, and UDP 1194.

  3. Check network settings:

    • Verify the public address in the "Hostname or IP address" field in the Admin Web UI. (Configuration > Network Settings)

Verify that you can access the web interface of Access Server

  1. Open your Access Server's address in a web browser (to reach the Client or Admin Web UIs).

  2. If you receive an insecure certificate warning, proceed anyway.

    • If you see the Access Server login page, the web interface works correctly.

  3. If the web interface fails to load but VPN connectivity works, it might be an issue with the web services.

  4. Ensure TCP port 443 is allowed through any firewalls or systems between you and the Access Server.

  5. Restart the server to see if it resolves the problem temporarily.

  6. If the issue persists, consider updating your server OS and Access Server.

  7. For more help, refer to Web Interface Access.

Verify that the VPN server address resolves correctly

We recommend using a custom hostname, such as vpn.example.com, which resolves to the public IP address of your Access Server through a DNS record. This is the best way for users to download VPN clients and connection profiles:

Tip

If some users experience problems, use an online DNS checker tool to verify the DNS record's status from various global locations. This can help identify if the issue is localized or widespread.

If you continue to have DNS issues, try accessing the Access Server web interface by its public IP address in your browser. For example, if your server's IP address is 192.0.2.2, try https://192.0.2.2.

Verify that your server is still on the same IP address

Check to see if your server is still at the expected IP address. It's possible it could change if the server isn't set with a static IP address and the DHCP server has assigned a new one.

  1. Verify if your Access Server still has the expected IP address, especially if it's assigned by DHCP.

  2. If using a DNS record, ensure it points to the correct IP address. (Update to the new IP address if necessary.)

  3. Update the hostname in the Admin Web UI from the Network Settings page.

  4. If clients connect directly via IP and it's changed, reprovision them with the updated address.

Tip

Use a DNS record for easy central updates without needing to reprovision clients when IP changes.

Check network configuration

When adding or removing network adapters, configurations may misalign. Verify with ifconfig or ip addr show to match MAC addresses to network cards. If misconfigured, either swap configurations or adjust the virtual network settings.

Check the physical or virtual switch layer

Ensure your server is connected to a switch or router. On virtual platforms, verify virtual switch settings and connections. Enable promiscuous mode and MAC address spoofing if needed. For virtual machines, ensure the network type is set to bridging, not NAT.

Verify the firewall configuration

If your server is behind a firewall, ensure the firewall settings are correct. Temporarily disable the firewall to test connectivity. Check port forwarding rules and verify internal IP access within the same network.

If Access Server is behind a router with port forwarding set up, ensure the rule is pointing to the correct IP address.

Check default gateway settings

Ensure only one default gateway is configured. If an extra network card has a default gateway, remove it to avoid asymmetric routing issues. Add specific routes if needed for multiple subnets.

Check routing tables and subnetting

Verify that the Access Server's VPN subnet differs from your LAN subnet to avoid conflicts. Use the console command service openvpnas stop to temporarily stop the Access Server service and check for resolved connectivity issues.

Step 3: Confirm that the server is alive

After going through the two steps above, the next step is to determine if the server is offline. A server failure or Access Server configuration problem could stop your server from starting up properly. These sections help you troubleshoot getting your server up and connecting either via SSH or directly on the console.

Ping the server

  1. Open a terminal or command prompt.

  2. Run ping <server_ip_address>.

  3. If the server responds, proceed to the next step. If not, the server might be offline.

Attempt an SSH connection

  1. Use an SSH client like PuTTY (Windows) or terminal (Linux/Mac).

  2. Connect using SSH <username>@<server_ip_address>.

  3. If you get a login prompt, the server is up. If not, check network settings

  4. Try to get the necessary credentials/keys to gain access and sign in.

    Tip

    Some of our provided images use the default username, openvpnas to sign in.

Access via the (virtual) console

If you can't reach the server by ping or SSH, you can try to access the server console:

  1. Physical server: Connect a monitor and keyboard then check if the server is running and you can sign in.

  2. Virtual server: Access the virtual console through hypervisor management or cloud platform tools then check network settings and ensure the server is reachable.

Reboot if necessary

  1. If the server is unresponsive, reboot it.

  2. Check if it boots normally and then test connectivity.

Regain server access

If you've lost server access because you don't have the private key or you've lost credntials, you can attempt to reset access.

These steps show you how to reset the Linux root password on an Ubuntu/Debia system:

  1. Reboot the server.

  2. Hold Shift to access the GRUB menu.

  3. Press E to edit.

  4. Add rw init=/bin/bash at the end of the line starting with linux.

  5. Press Ctrl+x to boot.

  6. Run passwd to reset the password.

  7. Reboot the server.

    • You should now be able to sign in with the root user and new password. We recommend you replace the SSH keys for the openvpnas user, which is the default on our images.

Tip

Depending on cloud providers, you may have different steps to regain access. Refer to their documentation on how to regain access.

Recover data

If the server won't boot up, you may have to switch to data recovery.

  1. Refer to this tutorial on how to recover a server with SQLite3 dump backup files.

  2. If you don't have a backup of the configuration, try to restore the configuration files from the dead server in /usr/local/Openvpn_as/etc/db/.

  3. With virtual machines and cloud providers, you may be able to attach the virtual disk image of the virtual machine to another machine so you can at least recover the files from the disk image.

  4. If the filesystem is damaged, consider migrating to a new sever or repairing database files with SQLite3.

    Important

    With filesystem damage, other files may also be damaged. First, determine whether the issue is recoverable or if you should migrate to a new server. If needed, contact us for support.

Step 4: Sign in to the OS and check the status

After going through the steps above, if you can sign into the server's OS, but problems persist with your VPN or web services, check the following areas:

Check disk space

If the server's hard disk runs out of space, that will cause issues.

  1. Connect to the console and get root privileges.

  2. Run df.

    • Sample output:

      Filesystem     1K-blocks    Used Available Use% Mounted on
udev              241240       0    241240   0% /dev
tmpfs              49400    5556     43844  12% /run
/dev/sda1        8222648 8222648         0 100% /
tmpfs             246996       0    246996   0% /dev/shm
tmpfs               5120       0      5120   0% /run/lock
tmpfs             246996       0    246996   0% /sys/fs/cgroup
tmpfs              49396       0     49396   0% /run/user/0

  3. If the disk space is full, free up space or increase the disk size.

Check the Access Server service status

  1. Connect to your console and get root privileges.

  2. Run this command to check the Access Server service status:

    service openvpnas status

  3. If not running, start the service:

    service openvpnas start

  4. Run this command to check the status of Access Server components:

    /usr/local/openvpn_as/scripts/sacli status

  5. All components should be on. Any that's not has a problem.

Reset the interface and port configuration to default

If you rAccess Server isn't listening on the default ports, it may not be able to start.

Check logs for errors

  1. Check the log files:

    • View logs in /var/log/openvpnas.log for a standalone or cluster setup or /var/log/openvpnas-node.log for a failover setup.

  2. Run this command with root privileges:

    service openvpnas restart

  3. Then get the latest log file entires and check for errors.

  4. Contact support to send us a log file for analysis if needed.

In this section: