Logging and debug flag options for Access Server

Upgrade to Latest Version

Introduction

This document provides information about the log files and debugging flags for Access Server and OpenVPN Connect. These help troubleshoot problems and determine the routes and instructions your clients receive.

Where to find log information for debugging

You can find log information in the following places:

  • In the VPN client app, OpenVPN Connect.
  • Saved on the client device.
  • In the Admin Web UI.
  • Saved on the VPN server.

You can also log additional information to log files using debug flags, activated in as.conf.

Locating the client log files

The client log files can help you figure out the following:

  1. Why a client has connection problems.
  2. Which routes and instructions the client receives.

Locate the files in one of the following locations.

In OpenVPN Connect

Export the log data from within OpenVPN Connect v3 directly:

  1. Launch OpenVPN Connect.
  2. Click the log icon in the corner.
    • The Log File window displays.
  3. Click the mail icon.
    • The window opens to save the log file.
  4. Select a location and click Save.

On the client device

OpenVPN Connect v3 stores the log data locally on the client device:

  • Windows: <User Folder>\AppData\Roaming\OpenVPN Connect\log\openvpn.log
  • macOS: ~/Library/Application Support/OpenVPN Connect/log/ovpn.log

OpenVPN Connect v2 stores the log data locally in these locations:

  • Windows: C:\Program Files (x86)\OpenVPN Technologies\OpenVPN Client\etc\log\openvpn_(unique_name).log
  • macOS: /Library/Application Support/OpenVPN/log/openvpn_(unique_name).log

Note: OpenVPN Connect on macOS has permissions set on the log file, so you can’t usually open it. To bypass this, right-click the log file and click Get info. Then at the bottom, under Sharing & Permissions, use the yellow padlock icon to unlock the settings and to give everyone read access.

Locating the server log files

Access Server stores log files that contain technical and sensitive information. Most common sensitive data like login credentials are normally redacted from the log files, but if certain debug flags are enabled they can be visible in the logs. Also should errors occur it is possible that (partial) certificate data is in the log files. Therefore we recommend to treat the log data as sensitive. You can expand the technical information contained in the server logs to include various extra information using debug flags, explained further down.

You can send the log data to syslog locally. If you want it sent to a remote server, configure a rule in the local syslog daemon to redirect it to a networked syslog server.

In the Admin Web UI

Access Server displays log information in the Admin Web UI. To view it:

  1. Sign in to the Admin Web UI.
  2. Click Status > Log Reports.

With these logs, you can see the following:

  • When a user connects.
  • The connection duration.
  • If users connect to the VPN, to a web service, etc.
  • Their data usage.
  • Simple error messages from authentication or connection issues.

On the server

On the OpenVPN Access Server, there is the server-side log:

/var/log/openvpnas.log

/var/log/openvpnas.node.log (in case of a failover setup)

Creating a new, clean log file for troubleshooting

When troubleshooting issues, we recommend stopping Access Server, moving the log file to another location, and restarting Access Server, creating a new log file to make it easier to analyze the logs. You can do that by following these steps:

  1. Stop the Access Server service: service openvpnas stop
  2. Move and rename the log file: mv /var/log/openvpnas.log /var/log/openvpnas.log.old
  3. Restart the Access Server service: service openvpnas start
  4. Stop the Access Server service: service openvpnas stop
  5. Now you can get the log file from /var/log/openvpnas.log for analysis.
  6. Start the Access Server service again: service openvpnas start

Setting up log rotation for /var/log/openvpnas.log.*

Access Server adds information to the log files and rotates them until the disk is full, but the number of log files grows endlessly.

Access Server archives a log file when it reaches the default size of about one megabyte. Access Server renames the first archived log file openvpnas.log.1. It creates a new log file and names it openvpnas.log. As your archived log files increase, Access Server names them sequentially, meaning openvpnas.log.1 is more recent than openvpnas.log.2.

We provide instructions below for setting the allowable log file size and deleting old log files using a cron job. 

Adjust the allowable log file size

To adjust the size of the log file before it’s archived, change the setting in as.conf with the LOG_ROTATE_LENGTH parameter:

  1. Log on to the server with root privileges.
  2. Open as.conf for editing in a text editor (such as nano):
    nano /usr/local/openvpn_as/etc/as.conf
  3. Add the following line at the bottom (the number represents bytes and the default is around 1000000 bytes, about 1 megabyte):
    LOG_ROTATE_LENGTH=1000000
  4. Save and exit by pressing Ctrl+x, then y.
  5. Restart the Access Server service:
    service openvpnas restart

Access Server writes to the current log file until it reaches your new, specified file size.

How to delete old log files

You can set up a cron job that runs periodically to delete old log files.

The number of files you choose to retain, times the file size of the log rotation setting, determines how much log data you keep in total, ensuring you never go over a certain number of bytes used for Access Server's log files.

Note: You can also log to syslog, explained below, which should already have rotation rules set on it in the operating system that clean it up regularly.

Example cron job

Here’s an example cron job for deleting old log files. Adjust the commands for your limits and time of execution.

To set up a cron job that clears log files number .15 or higher at 4:00 a.m. each night:

  1. Sign in to the console with root privileges.
  2. Open the crontab file for the account you are logged on as:
    crontab -e
    • When doing this for the first time, select an editor with the prompt.
  3. At the bottom of the crontab file, add these two lines:
    SHELL=/bin/bash
    0 4 * * * /bin/rm /var/log/openvpnas.log.{15..1000} >/dev/null 2>&1
  4. Save and exit by pressing Ctrl+x, then y (if you use nano as your editor).

The script deletes files named /var/log/openvpnas.log.15 and greater every night. Your system keeps the main log file and 14 archived log files.

Logging to syslog instead of the standard log file

You can log to the local syslog daemon by changing a configuration setting. After making the change, Access Server logs to the syslog instead of logging to a file.

Suppose you want to redirect to another syslog server on the network. In that case, you can configure the operating system's syslog daemon to redirect any Access Server service syslog line to an external network syslog server. All syslog lines regarding Access Server contain the keyword openvpnas, so it’s possible to filter for this with a rule in the syslog daemon and forward only that information.

Configure Access Server to log to the syslog

  1. Open the as.conf file for editing:
    nano /usr/local/openvpn_as/etc/as.conf
  2. At the bottom, add this line; ensure it's CAPITALIZED:
    SYSLOG=1
  3. Save and exit by pressing Ctrl+x, then y.
  4. Restart the Access Server service:
    service openvpnas restart

Access Server now logs to the syslog daemon, /var/log/syslog, by default.

Redirecting to an external syslog server

The following instructions assume you’re using the Ubuntu operating system. You may need to look up documentation and make adjustments as needed if you’re using another OS.

  1. Create a file for the rsyslog daemon rule:
    nano /etc/rsyslog.d/openvpnas.conf
  2. The file should be a new, empty file. Add this line to log to an external UDP syslog system:
    if $programname == 'openvpnas' then @remote.syslog.server
    Or add this line if it is an external TCP syslog system:
    if $programname == 'openvpnas' then @@remote.syslog.server
  3. Save and exit by pressing Ctrl+x, then y.
  4. Restart the syslog daemon:
    service rsyslog restart

Note: (Optional) You can specify another file, like /var/log/myownfilename.log, instead of supplying a remote server address, like @remote.syslog.server.

Turn off audit and service logging

Access Server 2.12 and newer support turning off audit and service logging. You can turn them off separately or together. When you turn off logging, it doesn't delete previous log files or the records in log.db but no longer records new activity.

Note: If you decide to turn off logging, be aware that you won't have logs to refer to or send to support if you encounter issues.

Turn off audits:

  1. Open the as.conf file for editing:
    nano /usr/local/openvpn_as/etc/as.conf
  2. Comment out the log DB line:
    # log_db=...
  3. Save and exit by pressing Ctrl + x, then y.
  4. Restart the service:
    service openvpnas restart

Once you've turned off audits, the Log Reports page in the Admin Web UI displays, "Logging is disabled in as.conf."

Turn off service logging:

  1. Open the as.conf file for editing:
    nano /usr/local/openvpn_as/etc/as.conf
  2. Add the following line:
    log.disable_logging=true
  3. Save and exit by pressing Ctrl + x, then y.
  4. Restart the service:
    service openvpnas restart

If you want to turn off logging completely, you must apply both options above.

Implementing a listed debug flag

Important note: Use these debug flags at your own risk.

We recommend working with OpenVPN Inc. support personnel to use debugging flags for specific needs. 

Here we publish the most useful debug flags available to the general public for gathering more Access Server data.

Some of these debug flags can significantly increase the amount of logging data produced by Access Server, so beware of filling up your hard drive and running out of disk space. Not all flags create a lot of information, but some do, and some even log passwords or session data to the log, so beware of this.

We recommend using these flags to pinpoint a problem, get log data, and then turn off the debug flag.

Set a debug flag

You can set most debug flags in the /usr/local/openvpn_as/etc/as.conf file. You add the command to the bottom of the file and cold restart the Access Server service afterward with this command:

service openvpnas restart

Important: All debug flags are case-sensitive.

DEBUG_AWSINFO=1

For troubleshooting AWS tiered licenses, this flag logs extra information in the liman info output and the /var/log/openvpnas.log file regarding the licensing process. This output helps troubleshoot the issue, especially when experiencing problems reaching a license activation server. You can also refer to the troubleshooting section for the AWS tiered instance licensing system.

LOG_N_CLIENTS_CHANGE=1

For troubleshooting the number of connected users, this flag logs information whenever the internal, currently connected users count changes. This can be useful if you suspect the connected user count is off for whatever reason. An example line from the log file:

0000-00-00 00:00:00+0000 [-] ***** N_CLIENTS CHANGE 0 -> 1

FAVOR_LZO=1

Use this debug flag to override the order in which compression algorithms are chosen for connecting clients. It forces the use of LZO. In extremely rare cases, this flag can help to resolve connectivity problems from iOS devices with very specific compression problems.

API_TRACE_SA=1

This flag logs all changes to the configuration settings by logging all activity between Access Server and the configuration databases.

It may be useful for various use cases, such as the following:

  • Tracking if somebody or something is altering your configuration settings without your knowledge, such as a colleague with access to the system.
  • Determining whether a browser plugin that is supposed to perform tasks such as filling in forms or managing passwords somehow changes settings you didn't touch while you're working on other settings.

This example line from the log file shows that the user, openvpn, signs on to the Admin Web UI successfully:

2017-09-19 17:11:54+0200 [-] *** API CALL f=authenticate args=[{'username': 'openvpn', 'password': '[redacted]', 'client_ip_addr': '12.34.56.78'}, {'log_service_name': 'WEB_ADMIN', 'request_superuser_privileges': True}] time=0.012

DEBUG_LOGDB=1

This flag logs everything that goes into the log database.

Access Server displays details about user logins and bandwidth use on the Log Reports page in the Admin Web UI. This information comes from the log.db database file, separate from the log files, which helps you track and resolve Access Server problems rather than storing user actions like authentication and data usage.

However, use this flag to log everything to the log files.

This example line shows that the user openvpn logged on to the Admin Web UI web service:

0000-00-00 00:00:00+0000 [-] LOG ERR: 'LOG_DB RECORD {"username": "openvpn", "node": "OPENVPNAS", "service": "WEB_ADMIN", "real_ip": "12.34.56.78", "timestamp": 1505833476, "start_time": 1505833476, "session_id": "u1OfDeOuagO1sGQg", "auth": 1}'

LOG_DB_XML_API_VERBOSE=1

This flag logs calls made to the XML API.

The Access Server has an XML-RPC interface typically limited to authentication and retrieving user-specific data like a user-locked profile. OpenVPN Connect for Windows and macOS uses the XML-RPC's limited set of commands for authentication and retrieving a user-locked profile, with other functions disabled by default. For more details, refer to the XML-RPC interface paragraph in the command line tools section.

Once you activate this flag, you can use the logdba tool to query for XML-RPC API calls like so:

./logdba --csv --service_filt=XML_API --columns="+api_method"

And with API_TRACE_SA=1 this also gets dumped in openvpnas.log or syslog if the syslog function is enabled.

DEBUG_SUBSCRIPTION=2

This flag logs information for an activated subscription in /var/log/openvpnas.log. Specifically, it enables verbose debug subscription service logging. Once turned on, the communication between Access Server and the Subscription Tracking System is added to the log.