Starting with 1.2.6 PKCS#12 files are now imported in the iOS keychain as OpenVPN Connect specific key material. For this reason they are not visible in the iOS Settings.
To delete an imported PKCS#12 file touch Manage Certificates (or Certs on iPhones/iPods) at the top of the main app screen and then touch Delete. This will cause red "-" icons to appear to the left of all PKCS#12 files. Touch the "-" icon to actually delete a file.