Logging and debug flag options for Access Server

Introduction

This document provides information about the log files and debugging flags for OpenVPN Access Server and OpenVPN Connect. These are helpful for troubleshooting problems and determining the routes and instructions your clients receive.

Where log information can be found for debugging

You can find log information in the following places:

  • In the VPN client app, OpenVPN Connect.
  • Saved on the client device.
  • In the Admin Web UI.
  • Saved on the VPN server.

Optionally, you can log additional information to the server log files for specific functions in Access Server using debug flags, activated in as.conf.

Locating the client log files

The client log files can help you figure out why a client has connection problems or which routes and instructions it’s receiving.

In OpenVPN Connect

You can export the log data from within OpenVPN Connect v3 directly.

  1. Open OpenVPN Connect.
  2. Click the log icon in the top right corner.
  3. From the Log File window, click the mail icon.
  4. Select a location to download the log file.

The log data for OpenVPN Connect v2 and v3 can also be retrieved directly from the filesystem.

On the client device

OpenVPN Connect v3 stores the log data locally on the client device:

  • Windows: <User Folder>\AppData\Roaming\OpenVPN Connect\log\openvpn.log
  • macOS: ~/Library/Application Support/OpenVPN Connect/log/ovpn.log

OpenVPN Connect v2 stores the log data locally in these locations:

  • Windows: C:\Program Files (x86)\OpenVPN Technologies\OpenVPN Client\etc\log\openvpn_(unique_name).log
  • macOS: /Library/Application Support/OpenVPN/log/openvpn_(unique_name).log

macOS may not show you the /Library in Finder. If it’s hidden:

  1. Open Finder, and in the menu at the top, click Go > Go to folder.
  2. Enter the path /Library to get into that directory.
  3. Navigate to the correct folder for the log file.

Note: OpenVPN Connect on macOS has permissions set on the log file, so you can’t usually open it. To bypass this, right-click the log file and click Get info. Then at the bottom, under Sharing & Permissions, use the yellow padlock icon to unlock the settings and to give everyone read access.

Locating the server log files

Access Server stores log files that contain technical and sensitive information. Most common sensitive data like login credentials are normally redacted from the log files, but if certain debug flags are enabled they can be visible in the logs. Also should errors occur it is possible that (partial) certificate data is in the log files. Therefore we recommend to treat the log data as sensitive. You can expand the technical information contained in the server logs to include various extra information using debug flags, explained further down.

You can send the log data to syslog locally. If you want it sent to a remote server, configure a rule in the local syslog daemon to redirect it to a networked syslog server.

In the Admin Web UI

Access Server displays log information in the Admin Web UI. To view it:

  1. Sign in to the Admin Web UI.
  2. Click Status > Log Reports.

With these logs, you can see:

  • When a user connects.
  • The connection duration.
  • Whether users connected to the VPN, to a web service, etc.
  • Their data usage.
  • Simple error messages from authentication or connection issues.

On the server

On the OpenVPN Access Server, there is the server-side log:

/var/log/openvpnas.log

/var/log/openvpnas.node.log (in case of a failover setup)

Creating a new, clean log file for troubleshooting

When troubleshooting issues, we recommend stopping Access Server, moving the log file to another location, and restarting Access Server, creating a new log file to make it easier to analyze the logs.

  1. Stop the Access Server service: service openvpnas stop
  2. Move and rename the log file: mv /var/log/openvpnas.log /var/log/openvpnas.log.old
  3. Restart the Access Server service: service openvpnas start
  4. Stop the Access Server service: service openvpnas stop
  5. Now you can get the log file from /var/log/openvpnas.log for analysis.
  6. Start the Access Server service again: service openvpnas start

Setting up log rotation for /var/log/openvpnas.log.*

OpenVPN Access Server normally keeps on logging until the disk is full and rotates log files, but the number of log files grows endlessly.

The threshold the log file must meet to be archived and replaced with a new log file is set to the default size of about one megabyte. Once it hits that size, it’s renamed openvpnas.log to openvpnas.log.1. Files are renamed .1, .2, .3, and so on. The archived logs are sequential, meaning that .1 is more recent than .2.

We provide instructions below for setting the allowable log file size, deleting old log files using a cron job, 

Adjust the allowable log file size

To adjust the size of the log file before it’s archived, change the setting in as.conf with the LOG_ROTATE_LENGTH parameter:

  1. Log on to the server with root privileges.
  2. Open as.conf for editing in a text editor (such as nano):
    nano /usr/local/openvpn_as/etc/as.conf
  3. Add the following line at the bottom (the number represents bytes and the default is around 1000000 bytes, about 1 megabyte):
    LOG_ROTATE_LENGTH=1000000
  4. Save and exit by pressing Ctrl+x, then y.
  5. Restart the Access Server service:
    service openvpnas restart

Access Server writes to the current log file until it reaches your new, specified file size.

How to delete old log files

You can set up a cron job that runs periodically to clear out old log files.

The number of files you choose to retain, times the file size of the log rotation setting, determines how much log data you retain in total, ensuring you never go over a certain number of bytes used for the OpenVPN Access Server's log files.

Note: You can also simply log to syslog, which is explained below, which should already have rotation rules set on it in the operating system, that clean it up regularly.

Here’s an example cron job for deleting old log files. Adjust the commands for your own limits and time of execution.

To set up a cron job that clears log files number .15 or higher at 4:00 a.m. each night:

  1. Open the crontab file for the account you are logged on as:
    crontab -e
  2. When doing this for the first time, you may be asked which text editor to use. We tend to advise nano as it's easy to use. At the bottom of the crontab file, add these two lines:
    SHELL=/bin/bash
    0 4 * * * /bin/rm /var/log/openvpnas.log.{15..1000} >/dev/null 2>&1
  3. Save and exit by pressing Ctrl+x, then y (if you use nano).

The script deletes files every night named /var/log/openvpnas.log.15 and higher (up to .1000). You should end up with the main log file and 14 archived log files. The account running this cron job does need permission to remove log files.

Logging to syslog instead of the standard log file

You have the option of logging to the local syslog daemon by changing a configuration setting. After making the change, Access Server logs to the syslog instead of logging to a file.

Suppose you want to redirect to another syslog server on the network. In that case, you can configure the operating system's syslog daemon to redirect any OpenVPN Access Server service syslog line to an external network syslog server. All syslog lines regarding Access Server contain the keyword openvpnas, so it’s possible to filter for this with a rule in the syslog daemon and forward only that information.

Configure Access Server to log to the syslog

  1. Open the as.conf file for editing:
    nano /usr/local/openvpn_as/etc/as.conf
  2. At the bottom, add this line; ensure it's CAPITALIZED:
    SYSLOG=1
  3. Save and exit by pressing Ctrl+x, then y.
  4. Restart the Access Server service:
    service openvpnas restart

Access Server will now log to the syslog daemon, which by default is logging to the file /var/log/syslog.

Redirecting to an external syslog server

The following instructions assume you’re using the Ubuntu operating system. You may need to look up documentation and make adjustments as needed if you’re using another OS.

  1. Create a file for the rsyslog daemon rule:
    nano /etc/rsyslog.d/openvpnas.conf
  2. The file should be a new, empty file. Add this line to log to an external UDP syslog system:
    if $programname == 'openvpnas' then @remote.syslog.server
    Or add this line if it is an external TCP syslog system:
    if $programname == 'openvpnas' then @@remote.syslog.server
  3. Save and exit by pressing Ctrl+x, then y.
  4. Restart the syslog daemon:
    service rsyslog restart

Note: You can optionally specify another file, like /var/log/myownfilename.log, instead of supplying a remote server address, like @remote.syslog.server.

Implementing a listed debug flag

Important note: If you don't know what you're doing, then the safest is to say, don't use these.

We recommend working with OpenVPN Inc. support personnel for using debugging flags when there’s a specific need to debug a particular problem. 

But we've decided to make some of the more useful debug flags available to the general public because some can be useful in gathering more Access Server data for purposes in addition to debugging.

Some of these debug flags can significantly increase the amount of logging data produced by Access Server, so beware of filling your hard drive with log data and running out of disk space. Not all flags produce a lot of information, but some do. And some of them even log password data or session data to the log, so beware of this. Therefore it's usually best to use some of these flags to pinpoint a problem, get log data, and then disable the debug flag.

Most debug flags are set in the /usr/local/openvpn_as/etc/as.conf file by adding it at the bottom of the file and cold restarting the Access Server service afterward with this command:

service openvpnas restart

Please note that all debug flags are case-sensitive.

DEBUG_AWSINFO=1

This flag logs extra information in the liman info output and the /var/log/openvpnas.log file regarding the licensing process when using an AWS pre-licensed tiered instance. This output helps troubleshoot the issue, especially when you’re experiencing problems reaching a license activation server. See also the troubleshooting section for the AWS tiered instance licensing system.

LOG_N_CLIENTS_CHANGE=1

This flag logs information whenever the internal, currently connected users count is altered. This can be useful if you suspect the connected user count is off for whatever reason. An example line from the log file:

0000-00-00 00:00:00+0000 [-] ***** N_CLIENTS CHANGE 0 -> 1

FAVOR_LZO=1

This is a debug flag to override the order in which compression algorithms are chosen for connecting clients. It forces the use of LZO. In extremely rare cases, this flag can help to resolve connectivity problems from iOS devices with very specific compression problems.

API_TRACE_SA=1

This flag logs all changes to the configuration settings by logging all activity between Access Server and the configuration databases.

It may be useful if you need to track if somebody or something is making alterations to your configuration settings without your knowledge, such as a colleague with access to the system, or a browser plugin that is supposed to only help you with some tasks like filling in forms or remembering passwords for you, but is instead messing with settings you didn't touch while you're working on other settings.

This example line from the log file shows that the user, openvpn, signs on to the Admin Web UI successfully:

2017-09-19 17:11:54+0200 [-] *** API CALL f=authenticate args=[{'username': 'openvpn', 'password': '[redacted]', 'client_ip_addr': '12.34.56.78'}, {'log_service_name': 'WEB_ADMIN', 'request_superuser_privileges': True}] time=0.012

DEBUG_LOGDB=1

This flag logs everything that goes into the log database.

Access Server provides details about user logins and bandwidth use in the Log Reports page in the Admin Web UI. This information is stored in the log.db database file, separate from the log files. The reasoning for this is that the log files help you track and resolve problems with the Access Server program itself, rather than keeping track of user actions like signing in and data usage.

However, use this flag if you want to log everything to the log files.

This example line shows that the user openvpn logged on to the Admin Web UI web service:

0000-00-00 00:00:00+0000 [-] LOG ERR: 'LOG_DB RECORD {"username": "openvpn", "node": "OPENVPNAS", "service": "WEB_ADMIN", "real_ip": "12.34.56.78", "timestamp": 1505833476, "start_time": 1505833476, "session_id": "u1OfDeOuagO1sGQg", "auth": 1}'

LOG_DB_XML_API_VERBOSE=1

This flag logs calls made to the XML API.

The Access Server has an XML-RPC interface that is typically limited to authentication and retrieving user-specific data like a user-locked profile. OpenVPN Connect for Windows and macOS uses the XML-RPC's limited set of commands for authentication and retrieving a user-locked profile, with other functions disabled by default. See the XML-RPC interface paragraph in the command line tools section for more details.

Once you activate this flag, you can use the logdba tool to query for XML-RPC API calls like so:

./logdba --csv --service_filt=XML_API --columns="+api_method"

And with API_TRACE_SA=1 this also gets dumped in openvpnas.log or syslog if the syslog function is enabled.

DEBUG_SUBSCRIPTION=2

This flag logs information for an activated subscription in /var/log/openvpnas.log. Specifically, it enables verbose debug subscription service logging. Once enabled, the communication between Access Server and the Subscription Tracking System is added to the log.