Category Archives: Security

All roads lead to Rome – Importing custom TLS certificate into Oracle wallet

It appears that Oracle’s preferred way to get a certificate into a wallet is by generating a key pair and then create certificate signing request using “orapki” (that’s what you mostly see in the docs and on MOS). Once the request is singed by the CA you import the certificate into the wallet.
But, there are situations where you don’t get to create the signing request from the Oracle wallet. For instance, at one of my clients, they create everything at the CA and just send a PKCS#12 file to the DBAs. Since PKCS is a standard and the Oracle wallet itself is a PKCS#12 file one could (naively) assume that these things are compatible. Unfortunately, this is not always the case. When trying to generate an SSO file with “orapki” from an openssl created PKCS#12 file I get the following error:

Exception in thread "main" java.lang.NullPointerException
        at oracle.security.pki.OracleKeyStoreSpi.a(Unknown Source)
        at oracle.security.pki.OracleSSOKeyStoreSpi.a(Unknown Source)
        at oracle.security.pki.OracleFileWalletImpl.b(Unknown Source)
        at oracle.security.pki.OracleWallet.saveLSSO(Unknown Source)
        at oracle.security.pki.textui.OracleWalletTextUI.create(Unknown Source)
        at oracle.security.pki.textui.OracleWalletTextUI.command(Unknown Source)
        at oracle.security.pki.textui.OraclePKITextUI.main(Unknown Source)

This is just one example, I’ve encountered other problems when trying to work with PKCS#12 files that were created by tools other than “orapki” (or mkstore). Therefore you should always create the PKCS#12 file using “orapki” and then work from there.
I’m going to show a few practical ways how to import a “pre-created” custom TLS certificate into an Oracle wallet.

Assuming we get a PKCS#12 file containing everything we need:

  • Root certificate
  • Intermediate certificated
  • Private key
  • Signed certificate
ORACLE_WALLET_LOC=/home/oracle/wallet
ORACLE_WALLET_PWD=*****
SOURCE_PKCS12_FILE/home/oracle/custom-ca.p12
SOURCE_PKCS12_PWD=*****

orapki wallet create -wallet "${ORACLE_WALLET_LOC}" -compat_v12 -pwd "${ORACLE_WALLET_PWD}"
orapki wallet import_pkcs12 \
    -wallet "${ORACLE_WALLET_LOC}" \
   -pkcs12file "${SOURCE_PKCS12_FILE}" \
   -pkcs12pwd "${SOURCE_PKCS12_PWD}" \
   -pwd "${ORACLE_WALLET_PWD}"
orapki wallet create -wallet "${ORACLE_WALLET_LOC}" -compat_v12 -auto_login_local -pwd "${ORACLE_WALLET_PWD}"
orapki wallet display -wallet "${ORACLE_WALLET_LOC}"

Maybe we get a Java key store (JKS) containing everything:

ORACLE_WALLET_LOC=/home/oracle/wallet
ORACLE_WALLET_PWD=*****
SOURCE_JKS_FILE=/home/oracle/custom-ca.jks
SOURCE_JKS_PWD=*****

orapki wallet create -wallet "${ORACLE_WALLET_LOC}" -compat_v12 -pwd "${ORACLE_WALLET_PWD}"
orapki wallet jks_to_pkcs12 \
    -wallet "${ORACLE_WALLET_LOC}" \
    -keystore "${SOURCE_JKS_FILE}" \
    -jkspwd "${SOURCE_JKS_PWD}" \
    -pwd "${ORACLE_WALLET_PWD}"
orapki wallet create -wallet "${ORACLE_WALLET_LOC}" -compat_v12 -auto_login_local -pwd "${ORACLE_WALLET_PWD}"
orapki wallet display -wallet "${ORACLE_WALLET_LOC}"

Or, we might get everything as individual base64 encoded files:

  • root.cer
  • inter.cer
  • private-pwd.key (password protected)
  • custom.cer
CERT_FILES_LOC=/home/oracle
ORACLE_WALLET_LOC=/home/oracle/wallet
ORACLE_WALLET_PWD=*****
PRIVATE_KEY_PWD=*****

orapki wallet create -wallet "${ORACLE_WALLET_LOC}" -compat_v12 -pwd "${ORACLE_WALLET_PWD}"
orapki wallet add \
    -wallet "${ORACLE_WALLET_LOC}" \
    -trusted_cert \
    -cert "${CERT_FILES_LOC}/root.cer" \
    -pwd "${ORACLE_WALLET_PWD}"
orapki wallet add \
    -wallet "${ORACLE_WALLET_LOC}" \
    -trusted_cert \
    -cert "${CERT_FILES_LOC}/inter.cer" \
    -pwd "${ORACLE_WALLET_PWD}"
orapki wallet import_private_key \
    -wallet "${ORACLE_WALLET_LOC}" \
    -pvtkeyfile "${CERT_FILES_LOC}/private-pwd.key" \
    -pvtkeypwd "${PRIVATE_KEY_PWD}" \
    -cert "${CERT_FILES_LOC}/custom.cer" \
    -pwd "${ORACLE_WALLET_PWD}"
orapki wallet create -wallet "${ORACLE_WALLET_LOC}" -compat_v12 -auto_login_local -pwd "${ORACLE_WALLET_PWD}"
orapki wallet display -wallet "${ORACLE_WALLET_LOC}"

“Helper functions”
Extracting individual components from a PKCS#12 file:

CERT_FILES_LOC=/home/oracle
SOURCE_PKCS12_FILE/home/oracle/custom-ca.p12
SOURCE_PKCS12_PWD=*****
PRIVATE_KEY_PWD=*****

# extract private key from PKCS#12 file
openssl pkcs12 -nocerts -in "${SOURCE_PKCS12_FILE}" -out "${CERT_FILES_LOC}/private-pwd.key" -password pass:"${SOURCE_PKCS12_PWD}" -passout pass:"${PRIVATE_KEY_PWD}"
# cleanup file: remove Bag/Key Attribute information
sed -i -n '/-BEGIN ENCRYPTED PRIVATE KEY-/,/-END ENCRYPTED PRIVATE KEY-/p' "${CERT_FILES_LOC}/private-pwd.key"

# remove passphrase from private key file (!!!security risk!!!)
openssl rsa -in "${CERT_FILES_LOC}/private-pwd.key" -out "${CERT_FILES_LOC}/private-nopwd.key" -passin pass:"${PRIVATE_KEY_PWD}"

# extract certificate from PKCS#12 file
openssl pkcs12 -clcerts -nokeys -in "${SOURCE_PKCS12_FILE}" -out "${CERT_FILES_LOC}/custom.cer" -password pass:"${SOURCE_PKCS12_PWD}"

# extract root and intermediate certs from PKCS#12 file
#   manually split ca-chain.cer into root.cer and inter.cer
openssl pkcs12 -in "${SOURCE_PKCS12_FILE}" -cacerts -nokeys -chain -out "${CERT_FILES_LOC}/ca-chain.cer" -password pass:"${SOURCE_PKCS12_PWD}"

Create PKCS#12 file with “openssl”:

CERT_FILES_LOC=/home/oracle
TARGET_PKCS12_FILE/home/oracle/opennssl.p12
TARGET_PKCS12_PWD=*****
PRIVATE_KEY_PWD=*****

cat "${CERT_FILES_LOC}/root.cer" "${CERT_FILES_LOC}/inter.cer" > "${CERT_FILES_LOC}/ca-chain.cer"
openssl pkcs12 -export -out "${TARGET_PKCS12_FILE}" -inkey "${CERT_FILES_LOC}/private-pwd.key" -in "${CERT_FILES_LOC}/custom.cer" -certfile "${CERT_FILES_LOC}/ca-chain.cer" -password pass:"${TARGET_PKCS12_PWD}" -passin pass:"${PRIVATE_KEY_PWD}"

Convert a PKCS#12 file to Java key store (JKS):

OPENSSL_PKCS12_FILE=/home/oracle/wallet/openssl.p12
OPENSSL_PKCS12_PWD=*****
JKS_FILE=/home/oracle/wallet/keystore.jks
JKS_PWD=*****

keytool -v -importkeystore -srckeystore "${OPENSSL_PKCS12_FILE}" -srcstoretype pkcs12 -srcstorepass "${OPENSSL_PKCS12_PWD}" -destkeystore "${JKS_FILE}" -deststoretype jks -deststorepass "${JKS_PWD}"

Footnote: tested “orapki” from Oracle 19.5 database home

SQLcl – enabling MCS for JDBC thin driver

Let’s say you have configured TLS authentication for your database users using the Microsoft Certificate Store (MCS) on Windows clients.

For OCI based client this is pretty well documented and understood:

sqlnet.ora

SQLNET.AUTHENTICATION_SERVICES = (TCPS)
WALLET_LOCATION = (SOURCE = (METHOD = MCS))
SSL_SERVER_DN_MATCH = yes

tnsnames.ora

DEV1_TLS =
  (DESCRIPTION =
    (ADDRESS = (PROTOCOL = TCPS)(HOST = ol7ora19dev1.spotonoracle.com)(PORT = 2484))
    (CONNECT_DATA =
      (SERVICE_NAME = DEV1.spotonoracle.com)
    )
    (SECURITY =
      (SSL_SERVER_CERT_DN = "cn=dev1,ou=servers,ou=oracle,dc=spotonoracle,dc=com")
    )
  )

With above configuration I can connect to the database using SQL*Plus.

C:\Users\user1>sqlplus /@dev1_tls

SQL*Plus: Release 12.2.0.1.0 Production on Wed May 29 18:31:53 2019

Copyright (c) 1982, 2018, Oracle.  All rights reserved.

Last Successful login time: Wed May 29 2019 18:10:15 +02:00

Connected to:
Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production

Note that I use a 12.2 OCI client. This is because of bug 25809524 for which the fix has not made its way into the 18c release. I hope the soon to be released 19c client for Windows will have the fix.

In a previous post I’ve shown how to use a file based wallet (cwallet.sso) for JDBC thin applications,namely SQLcl. Unfortunately, we cannot simply change the WALLET_LOCATION parameter as we do in sqlnet.ora. This does not work:

set JAVA_TOOL_OPTIONS=-Doracle.net.wallet_location=(SOURCE=(METHOD=MCS)) -Doracle.net.authentication_services=(TCPS) -Doracle.net.ssl_server_dn_match=true

In order for JDBC thin applications to access the Microsoft Certificate Store we have to set Windows platform specific settings for trust store and key store properties:

set JAVA_TOOL_OPTIONS=-Djavax.net.ssl.trustStore=NONE -Djavax.net.ssl.trustStoreType=Windows-ROOT -Djavax.net.ssl.keyStore=NONE -Djavax.net.ssl.keyStoreType=Windows-MY -Doracle.net.authentication_services=(TCPS) -Doracle.net.ssl_server_dn_match=true

C:\Users\user1>sql /@dev1_tls
Picked up JAVA_TOOL_OPTIONS: -Djavax.net.ssl.trustStore=NONE -Djavax.net.ssl.trustStoreType=Windows-ROOT -Djavax.net.ssl.keyStore=NONE -Djavax.net.ssl.keyStoreType=Windows-MY -Doracle.net.authentication_services=(TCPS,KERBEROS5) -Doracle.net.ssl_server_dn_match=true

SQLcl: Release 19.1 Production on Wed May 29 18:45:21 2019

Copyright (c) 1982, 2019, Oracle.  All rights reserved.

Last Successful login time: Wed May 29 2019 18:45:22 +02:00

Connected to:
Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.3.0.0.0

This works for SQLcl, SQL Developer, and any reasonable JDBC thin application. 🙂

Bugs are annoying – Kerberos ticket issue

Update 08-JUN-2018
This bug has been fixed with the Oracle 19.3 client release for Windows. Below workaround should not be necessary anymore.

One cool feature of using Kerberos authentication is that when you have a TGT (Ticket Granting Ticket) in the ticket cache the Oracle client software can use that to get a service ticket and log you into the database without further asking for any credentials (single sing-on).

Here’s what it looks like with a Kerberos authenticated SSH session on Linux:

As you can see from the screenshot the “orasrv” service ticket is flagged “forwardable” and the database login is successful (“-f” tells “oklist” to show the ticket flags).

On Windows on the other hand the same fails with “ORA-12638: Credential retrieval failed”:

If we change to “non-forwardable” service tickets it works on both, Linux and Windows:

How do you change to “non-forwardable” service tickets?
1) Use “okinit” to manually re-initialize your ticket cache. By default it will get “non-forwardable” tickets (or use “-F” to be explicit).

2) You can configure the service principal in Active Directory so only “non-forwadable” tickets will be issued (even when you use “okinit -f” to explicitly ask for “forwardable” tickets)

Both workarounds do the trick but I don’t like neither of them for their obvious drawbacks. After a few weeks trying to convince the Oracle Support Engineer of the issue bug 28734494 has been filed on Oct 8, 2018 with no notable progress to date 🙁
Despite what the bug description says the issue is not MSLSA vs file ticket cache, it is the ticket flags that make or break.

Btw. I’ve tested client versions 12.2.0.1, 18.3, 18.5 and they all exhibit the same behaviour.

Tracing LDAP from CMU to AD

I know, how many acronyms can you use in a title?

This is a quick note about another tracing facility within Oracle. If you’re using Centrally Manager Users with Active Directory you can enable a trace for the LDAP searches Oracle performs.

Enable tracing:

alter system set events='trace[gdsi] disk low';

Disable tracing:

alter system set events='trace[gdsi] off';

Here are a few examples.

Failed Kerberos authentication

kzlg found dn in wallet
kzlg found pwd in wallet
kzlg found usr in wallet
kzlg found domain SPOTONORACLE; dc=spotonoracle,dc=com; 1 dirsrv
kzlg ldap_open win2012dc1.spotonoracle.com:636
kzlg DB-LDAP init SSL succeeded.
kzlg bind success
kzlg AD user name: user1@SPOTONORACLE.COM
kzlg default naming ctx: dc=spotonoracle,dc=com
kzlg search -s base -b dc=spotonoracle,dc=com
kzlg search filter: objectclass=*
kzlg AD lockout_duration: 18000000000
kzlg AD max_pwd_age: 36288000000000
kzlg search_ext -s sub -b dc=spotonoracle,dc=com
kzlg search filter: (&(objectclass=user)(userPrincipalName=user1@SPOTONORACLE.COM))
KZLG_ERR: failed the search err=28304.
kzlg number of entries: 0
KZLG_ERR: LDAPERR=28304, OER=28304
KZLG_ERR: error=28304
kzlg doing LDAP unbind 

Successful Kerberos authentication

kzlg found dn in wallet
kzlg found pwd in wallet
kzlg found usr in wallet
kzlg found domain SPOTONORACLE; dc=spotonoracle,dc=com; 1 dirsrv
kzlg ldap_open win2012dc1.spotonoracle.com:636
kzlg DB-LDAP init SSL succeeded.
kzlg bind success
kzlg AD user name: user1@SPOTONORACLE.COM
kzlg default naming ctx: dc=spotonoracle,dc=com
kzlg search -s base -b dc=spotonoracle,dc=com
kzlg search filter: objectclass=*
kzlg AD lockout_duration: 18000000000
kzlg AD max_pwd_age: 36288000000000
kzlg search_ext -s sub -b dc=spotonoracle,dc=com
kzlg search filter: (&(objectclass=user)(userPrincipalName=user1@SPOTONORACLE.COM))
kzlg number of entries: 1
kzlg found user entry: CN=user1,OU=users,OU=oracle,DC=spotonoracle,DC=com
kzlg search -s base -b
kzlg search filter: objectclass=*
kzlg get AD current time: 20181019155231.0Z
kzlg found user entry normalized: cn=user1,ou=users,ou=oracle,dc=spotonoracle,dc=com
kzlg search_ext -s sub -b dc=spotonoracle,dc=com
kzlg search filter: (&(objectclass=group)(member:1.2.840.113556.1.4.1941:=cn=user1,ou=users,ou=oracle,dc=spotonoracle,dc=com))
kzlg number of entries: 1
kzlg search_ext -s sub -b dc=spotonoracle,dc=com
kzlg search filter: (&(objectclass=group)(objectSid=S-1-5-21-4282430696-1338935355-568305779-513))
kzlg number of entries: 1
kzlg doing LDAP unbind 	

Failed TLS authentication

kzlg found dn in wallet
kzlg found pwd in wallet
kzlg found usr in wallet
kzlg found domain SPOTONORACLE; dc=spotonoracle,dc=com; 1 dirsrv
kzlg ldap_open win2012dc1.spotonoracle.com:636
kzlg DB-LDAP init SSL succeeded.
kzlg bind success
kzlg AD user name: cn=user1,ou=users,ou=oracle,dc=spotonoracle,dc=com
kzlg default naming ctx: dc=spotonoracle,dc=com
kzlg search -s base -b dc=spotonoracle,dc=com
kzlg search filter: objectclass=*
kzlg AD lockout_duration: 18000000000
kzlg AD max_pwd_age: 36288000000000
kzlg search_ext -s sub -b dc=spotonoracle,dc=com
kzlg search filter: (&(objectclass=user)(distinguishedName=cn=user1,ou=users,ou=oracle,dc=spotonoracle,dc=com))
KZLG_ERR: failed the search err=28304.
kzlg number of entries: 0
KZLG_ERR: LDAPERR=28304, OER=28304
KZLG_ERR: error=28304
kzlg doing LDAP unbind
kzlg found dn in wallet
kzlg found pwd in wallet
kzlg found usr in wallet 

Successful TLS authentication

kzlg found dn in wallet
kzlg found pwd in wallet
kzlg found usr in wallet
kzlg found domain SPOTONORACLE; dc=spotonoracle,dc=com; 1 dirsrv
kzlg ldap_open win2012dc1.spotonoracle.com:636
kzlg DB-LDAP init SSL succeeded.
kzlg bind success
kzlg AD user name: cn=user1,ou=users,ou=oracle,dc=spotonoracle,dc=com
kzlg default naming ctx: dc=spotonoracle,dc=com
kzlg search -s base -b dc=spotonoracle,dc=com
kzlg search filter: objectclass=*
kzlg AD lockout_duration: 18000000000
kzlg AD max_pwd_age: 36288000000000
kzlg search_ext -s sub -b dc=spotonoracle,dc=com
kzlg search filter: (&(objectclass=user)(distinguishedName=cn=user1,ou=users,ou=oracle,dc=spotonoracle,dc=com))
kzlg number of entries: 1
kzlg found user entry: CN=user1,OU=users,OU=oracle,DC=spotonoracle,DC=com
kzlg search -s base -b
kzlg search filter: objectclass=*
kzlg get AD current time: 20181019155506.0Z
kzlg found user entry normalized: cn=user1,ou=users,ou=oracle,dc=spotonoracle,dc=com
kzlg search_ext -s sub -b dc=spotonoracle,dc=com
kzlg search filter: (&(objectclass=group)(member:1.2.840.113556.1.4.1941:=cn=user1,ou=users,ou=oracle,dc=spotonoracle,dc=com))
kzlg number of entries: 1
kzlg search_ext -s sub -b dc=spotonoracle,dc=com
kzlg search filter: (&(objectclass=group)(objectSid=S-1-5-21-4282430696-1338935355-568305779-513))
kzlg number of entries: 1
kzlg doing LDAP unbind 

Thanks to this I could resolve the last road block. CMU with TLS/Kerberos is fully functioning.

SQLcl – enabling TLS for JDBC thin driver

This is a quick follow-up from my previous post. This time we’re going to add TLS capabilities to SQLcl. I’m going to reuse the java security file we created earlier to override the security provider’s list (/opt/sqldeveloper/sqldeveloper/bin/sqldev.java.security).

The thing with SQLcl is there is no configuration file like in SQL Developer. No problem, all we did was adding JVM options anyway, except for adding “oraclepki.jar” to the CLASSPATH. Luckily, SQLcl ships with a recent version so we don’t need that.
To add the JVM options we can use either JAVA_TOOL_OPTIONS or _JAVA_OPTIONS. It can’t get simpler than that, can it?

$ export JAVA_TOOL_OPTIONS='-Doracle.net.wallet_location="(SOURCE=(METHOD=FILE)(METHOD_DATA=(DIRECTORY=/home/btr/tns/wallet)))" -Doracle.net.authentication_services="(TCPS)" -Doracle.net.ssl_server_dn_match=false -Djava.security.properties=/opt/sqldeveloper/sqldeveloper/bin/sqldev.java.security'
 

Connect using username/password but use TLS for in-flight traffic encryption:

$ sql system/********@"(DESCRIPTION=(ADDRESS=(HOST=ol7122rac-scan.localdomain)(PROTOCOL=TCPS)(PORT=1523))(CONNECT_DATA=(SERVICE_NAME=pdbrac1.localdomain)))"
Picked up JAVA_TOOL_OPTIONS: -Doracle.net.wallet_location="(SOURCE=(METHOD=FILE)(METHOD_DATA=(DIRECTORY=/home/btr/tns/wallet)))" -Doracle.net.authentication_services="(TCPS)" -Doracle.net.ssl_server_dn_match=false -Djava.security.properties=/opt/sqldeveloper/sqldeveloper/bin/sqldev.java.security

SQLcl: Release 18.1.1 Production on Tue May 29 20:51:26 2018

Copyright (c) 1982, 2018, Oracle.  All rights reserved.

Last Successful login time: Tue May 29 2018 20:51:26 -04:00

Connected to:
Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production


SQL> 

Or connecting using TLS authentication:

$ sql /@"(DESCRIPTION=(ADDRESS=(HOST=ol7122rac-scan.localdomain)(PROTOCOL=TCPS)(PORT=1523))(CONNECT_DATA=(SERVICE_NAME=pdbrac1.localdomain)))"
Picked up JAVA_TOOL_OPTIONS: -Doracle.net.wallet_location="(SOURCE=(METHOD=FILE)(METHOD_DATA=(DIRECTORY=/home/btr/tns/wallet)))" -Doracle.net.authentication_services="(TCPS)" -Doracle.net.ssl_server_dn_match=false -Djava.security.properties=/opt/sqldeveloper/sqldeveloper/bin/sqldev.java.security

SQLcl: Release 18.1.1 Production on Tue May 29 20:57:30 2018

Copyright (c) 1982, 2018, Oracle.  All rights reserved.

Connected to:
Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production


SQL> 

SQL Developer – enabling TLS for JDBC thin driver

I don’t think this is documented anywhere (and no hits on google) but it shows once again how awsome SQL Developer is. 🙂
Instead of using “jdbc:oracle:oci” for TLS enabled connections we’re going to set SQL Developer up to do just that with “jdbc:oracle:thin”. In that regard, SQL Developer is no more special than any Java application.
To keep it short I’m sticking to the basics, maybe I’ll add some variations into the comments later.
First, we need to add “OraclePKIProvider” to the JRE security provider’s list. The provider’s list is defined in “$JAVA_HOME/jre/lib/security/java.security”.
I’m going to append to it by creating a new file that I only source from SQL Developer. In my case, the next free slot is 10. Depending on your OS and JRE version the list may slightly vary.
(also make sure that the java.security file is appendable: “security.overridePropertiesFile=true”)

$ vi /opt/sqldeveloper/sqldeveloper/bin/sqldev.java.security
security.provider.10=oracle.security.pki.OraclePKIProvider

Next, we add a few options to the sqldeveloper.conf file:

$ vi /opt/sqldeveloper/sqldeveloper/bin/sqldeveloper.conf
# add this to end of the config file

# specify the path to your SSO wallet file
AddVMOption -Doracle.net.wallet_location="(SOURCE=(METHOD=FILE)(METHOD_DATA=(DIRECTORY=/home/btr/tns/wallet)))"
# this is required if you want to use TLS authentication next to in-flight traffic encryption
AddVMOption -Doracle.net.authentication_services="(TCPS)"
# optionally, if you want the client to verify the server certificate
AddVMOption -Doracle.net.ssl_server_dn_match=true
# append/override JRE security config to add Oracle provider
AddVMOption -Djava.security.properties=/opt/sqldeveloper/sqldeveloper/bin/sqldev.java.security
# even SQLDev 18.1 ships with a very old version that is not compatible - just download the latest one from OTN
AddJavaLibFile /home/btr/orasec/jlib/oraclepki.jar

That’s it, we’re done.

Now you can create connections using TNS, LDAP, or Advanced. Basic/EZCONNECT does not support to specify the protocol, so it can’t be used for TLS.

Here we only use TLS for in-flight traffic encryption:

And here, we make use of TLS user authentication:

Secure Postgres connection from SQL Developer

Searching the web it seems most people use “org.postgresql.ssl.NonValidatingFactory” to connect to Postgres from SQL Developer.

This doesn’t seem to be a good idea as the documentation states it is not secure:
Provide a SSLSocketFactory that allows SSL connections to be made without validating the server's certificate. This is more convenient for some applications, but is less secure as it allows "man in the middle" attacks.

Looking into the jar file under “/org/postgres/ssl” you’ll also find SingleCertValidatingFactory. Now that’s much better:
Provides a SSLSocketFactory that authenticates the remote server against an explicit pre-shared SSL certificate. This is more secure than using the NonValidatingFactory as it prevents "man in the middle" attacks. It is also more secure than relying on a central CA signing your server's certificate as it pins the server's certificate.

Let’s get to it…

The boring part: load the Postgres JDBC driver
Go to “Tools” -> “Preferences…” -> “Third Party JDBC Drivers” and add the jar file

The tricky part: getting the connect string right (it’s actually not that hard)
Syntax:

<hostname>/<database>?ssl=true&sslfactory=org.postgresql.ssl.SingleCertValidatingFactory&sslfactoryarg=file:<path-to-cert-file>&

For example:

abc.xzy.us-east-1.rds.amazonaws.com/postgres?ssl=true&sslfactory=org.postgresql.ssl.SingleCertValidatingFactory&sslfactoryarg=file:/home/btr/certs/amazon-rds.crt&

Enter this entire string in the “Hostname” field.

The exciting part: playing with your newly connected Postgres database 🙂

Keep it secure!

Oracle 12.2 full database encryption (TDE)

Starting with Oracle 12.2 it is possible to encrypt all Tablespaces including SYSTEM, SYSAUX, TEMP, and UNDO. Off the top of my head I can think of a couple of reasons why encrypting SYSTEM and SYSAUX might be desired:

  • Histograms in SYSAUX might contain sensitive data
  • Application tables/indexes in SYSTEM or SYSAUX Tablespace
  • Hard-coded secrets in PL/SQL objects (yes, I’ve seen it all!)

Strangely enough, the 12.2 TDE FAQ discourages encryption of “internal objects”:

I believe this to be a leftover from previous releases which hasn’t been cleaned up, yet. Because it is supported and the Advanced Security Guide provides plenty of information about it.

Splitting hairs, “Fully Encrypt Databases” is not entirely correct. There is currently no supported way to encrypt PDB$SEED Tablespaces…and I’ve tried for the fun of it:-)
Forcing the PDB$SEED open read-write with “alter pluggable database PDB$SEED open force” and then open the keystore works. I could even create a new encryption key, but it won’t let me activate it:

SQL> show con_name
CON_NAME 
------------------------------
PDB$SEED

SQL> administer key management create encryption key using tag 'PDBSEED_MK_1' force keystore identified by "Strng-4-Scrty" with backup using 'bck1';
keystore altered.

SQL> column key_id new_value new_key_id
SQL> select key_id from v$encryption_keys where con_id = 0 and tag = 'PDBSEED_MK_1';
KEY_ID                                                                        
------------------------------------------------------------------------------
AVVfIyuTLk9sv2EyAw9dJ18AAAAAAAAAAAAAAAAAAAAAAAAAAAAA

SQL> administer key management use encryption key '&new_key_id' identified by "Strng-4-Scrty" with backup using 'bck2';
*
ERROR at line 1:
ORA-46656: operation not valid for container database

I know it doesn’t make a lot of sense as for instance, cloning from a encrypted PDB$SEED would be kind of a Catch-22.

Anyway, back to encrypting Oracle’s “internal” Tablespaces. It’s pretty straight forward. It’s the same as you would setup the database for any TDE encryption.
Add ENCRYPITON_WALLET_LOCATION to your sqlnet.ora

ENCRYPTION_WALLET_LOCATION =
  (SOURCE =
    (METHOD = FILE)
    (METHOD_DATA =
      (DIRECTORY = /u01/app/oracle/admin/$ORACLE_SID/wallet/tde)
    )
  )

Setup a keystore in the CDB$ROOT and generate a new key to use:

alter session set container=cdb$root;

-- create PKCS#12 keystore (ewallet.p12)
administer key management
  create keystore '/u01/app/oracle/admin/DEV1/wallet/tde'
  identified by "Strng-4-Scrty"
;

-- open the keystore
administer key management
  set keystore open
  identified by "Strng-4-Scrty"
  container = current
;

-- create a new encryption key
administer key management
  create encryption key using tag 'DEV1_ROOT_MK_1'
  identified by "Strng-4-Scrty"
  with backup using 'Before-Create-DEV1_ROOT_MK_1'
  container = current
;

-- activate the key for use in this container
--   note: newly created keys have "con_id = 0", as soon as they are activated they get assigned to the respective PDB
column key_id new_value new_key_id
select key_id from v$encryption_keys where con_id = 0 and tag = 'DEV1_ROOT_MK_1';
administer key management
  use encryption key '&new_key_id'
  using tag 'DEV1_ROOT_MK_1'
  identified by "Strng-4-Scrty"
  with backup using 'Before-Activate-DEV1_ROOT_MK_1'
;

-- we also want a local auto-login keystore (cwallet.sso)
--   otherwise manual keystore password input is required during database startup
administer key management
  create local auto_login keystore from keystore '/u01/app/oracle/admin/DEV1/wallet/tde'
  identified by "Strng-4-Scrty"
;

Now, that we have the key setup it’s as simple as an ALTER TABLESPACE statement. We used the default algorithm AES128 when generating the encryption key so we specify the same here.

alter tablespace system
  encryption online using 'AES128' encrypt
  file_name_convert = ('system', 'system-enc')
;
alter tablespace sysaux
  encryption online using 'AES128' encrypt
  file_name_convert = ('sysaux', 'sysaux-enc')
;

Be careful, if the specified algorithm is different from the one of the key the statement will still succeed. You’ll see following warning in the alert.log:

An algorithm different from the database key is specified for SYSTEM tablespace. The algorithm has been overridden to AES128 per V$DATABASE_KEY_INFO view.

Let’s move to my PDB.

alter session set container=pdbdev1;

-- assuming auto-login keystore is now open
-- otherwise open password keystore

administer key management
  create encryption key using tag 'PDBDEV1_MK_1'
  force keystore
  identified by "Strng-4-Scrty"
  with backup using 'Before-Create-PDBDEV1_MK_1'
;

column key_id new_value new_key_id
select key_id from v$encryption_keys where con_id = 0 and tag = 'PDBDEV1_MK_1';
administer key management
  use encryption key '&new_key_id'
  using tag 'PDBDEV1_MK_1'
  force keystore
  identified by "Strng-4-Scrty"
  with backup using 'Before-Activate-PDBDEV1_MK_1'
;

To encrypt the SYSTEM/SYSAUX Tablespace on our PDB you can use the same statements from above:

alter tablespace system
  encryption online using 'AES128' encrypt
  file_name_convert = ('system', 'system-enc')
;
alter tablespace sysaux
  encryption online using 'AES128' encrypt
  file_name_convert = ('sysaux', 'sysaux-enc')
;

According to the documentation, undo, temp, and redo data originating from objects in encrypted tablespaces will also be encrypted.

There’s still reason to encrypt temporary Tablespaces in full:

  • Temporary LOBs
  • Global temporary tables

Encrypting existing TEMP Tablespaces is not supported, we just create a new one and drop the old.

create temporary tablespace tempts_enc
  tempfile '/u02/oradata/DEV1/tempts_enc-001.dbf' size 512M autoextend off
  encryption using 'AES128' encrypt
;
alter database default temporary tablespace tempts_enc;
drop tablespace tempts including contents and datafiles;

Once I have encrypted SYSTEM, SYSAUX, TEMP, and all application Tablespaces there should be no need to encrypt the UNDO Tablespace, right? All undo data is then based on objects from encrypted Tablespaces and should therefore also be encrypted in the UNDO stream. This is to be further investigated. If you have any input, please leave a comment.

One final note: if you want to understand the performance impact of TDE I highly recommend to use SLOB.

Defeating ORA-12696 Double Encryption Turned On (tricking the ENVS)

This post is about configuring SQL*Net to support native SQL*Net encription and SSL authentication for different users at the same time. Imagine your security offices comes along and demands that every connection to the database must be encrypted and it must be enforced by the database. The application server (e.g. Weblogic) already authenticates itself to the database using SSL certificates, so the network traffic is already encrypted. All other connects use username/password authentication and must be encrypted by native SQL*Net encryption. The database server enforces encryption by setting SQLNET.ENCRYPTION_SERVER = REQUIRED. From a security point of view this is paramount.

Going ahead and configure everything according docs and white papers I bet you’ll end up with “ORA-12696 Double Encryption Turned On, login disallowed”.

[side note]
If SQLNET.ENCRYPTION_SERVER is set to REQUIRED then the client gets “ORA-28865: SSL connection closed”, but once you start digging in the SQL*Net trace file of the server process you’ll find ORA-12696.
Funnily, if SQLNET.ENCRYPTION_SERVER is set to REQUESTED then the client actually gets ORA-12696 reported.
[/side note]

The basic problem is, that there is only one $TNS_ADMIN location with one sqlnet.ora containing all the configuration. Ultimately, we need to find a way to use multiple $TNS_ADMIN locations and separate the sqlnet.ora configurations for the different listening end points/services.
One solution would be to fire up two listeners with different environment settings. Note that the server processes spawned by the listener inherit the properties from it, that being permissions and environment variables.
Following I’m going to show a solution with one single listener using a little known feature: specifying ENVS parameter in the static listener SID list.

But first let’s have a look a the current configuration which is giving us some trouble. There’s a listener configuration with two listening end points (one for TCP and one for TCPS) and a mixed configuration in sqlnet.ora.
My $TNS_ADMIN is pointing to /u01/app/oracle/network/admin which is the default location set in the oracle’s user profile and is used when starting up the listener.

$ cat /u01/app/oracle/network/admin/listener.ora

LISTENER =
  (DESCRIPTION_LIST =
    (DESCRIPTION =
      (ADDRESS = (PROTOCOL = TCP)(HOST = oel6ora12dev1.localdomain)(PORT = 2483))
    )
    (DESCRIPTION =
      (ADDRESS = (PROTOCOL = TCPS)(HOST = oel6ora12dev1.localdomain)(PORT = 2484))
    )
  )
SSL_CLIENT_AUTHENTICATION=FALSE
WALLET_LOCATION = (SOURCE = (METHOD = FILE) (METHOD_DATA = (DIRECTORY=/u01/app/oracle/admin/DEV1/wallet/auth)))

$ cat /u01/app/oracle/network/admin/sqlnet.ora

SQLNET.ENCRYPTION_SERVER=requested
SQLNET.ENCRYPTION_TYPES_SERVER=(aes256)
SQLNET.AUTHENTICATION_SERVICES=(BEQ,TCPS)
SSL_CLIENT_AUTHENTICATION=TRUE
SSL_CIPHER_SUITES=(SSL_RSA_WITH_AES_256_CBC_SHA)
WALLET_LOCATION = (SOURCE = (METHOD=FILE)(METHOD_DATA = (DIRECTORY = /u01/app/oracle/admin/DEV1/wallet/auth)))

If we try to connect from a client using SSL authentication it errors with “ORA-12696 Double Encryption Turned On, login disallowed”. Connecting using username/password works just fine and the network traffic is properly encrypted.

To be able to have different SQL*Net configurations we first need to create a separate directory structure. In this case I just create it next to the current network/admin directory and separate the configuration into two sqlnet.ora files:
$ mkdir /u01/app/oracle/network/adminssl
$ vi /u01/app/oracle/network/admin/sqlnet.ora

SQLNET.ENCRYPTION_SERVER=requested
SQLNET.ENCRYPTION_TYPES_SERVER=(aes256)

$ vi /u01/app/oracle/network/adminssl/sqlnet.ora

SQLNET.AUTHENTICATION_SERVICES=(BEQ,TCPS)
SSL_CLIENT_AUTHENTICATION=TRUE
SSL_CIPHER_SUITES=(SSL_RSA_WITH_AES_256_CBC_SHA)
WALLET_LOCATION = (SOURCE = (METHOD=FILE)(METHOD_DATA = (DIRECTORY = /u01/app/oracle/admin/DEV1/wallet/auth)))

Now, I need to tell the listener to set the proper environment when spawning server processes for incomming connection requests. This is done by specifying the TNS_ADMIN location in the ENVS parameter:

SID_LIST_LISTENER =
  (SID_LIST =
    (SID_DESC =
      (GLOBAL_DBNAME = DEV1_SSL.localdomain)
      (ORACLE_HOME = /u01/app/oracle/product/ora12101)
      (SID_NAME = DEV1)
      (ENVS="TNS_ADMIN=/u01/app/oracle/network/adminssl")
    )
  )

Now, whenever a connection is established via service DEV1_SSL.localdomain the listener exports the environment varialbe TNS_ADMIN before spawning the server process. The server process then looks up the environment variable and reads the sqlnet.ora file in /u01/app/oracle/network/adminssl.
As you can see, next to using different listening end points I also present an addtitional service on the listener.
Depending on which service and end point the client uses it gets either native SQL*Net encryption or SSL authentication including SSL encryption.

On the client side I use either one of the following TNS entries, depending on which authentication method the user uses.

DEV1_SSL.LOCALDOMAIN =
  (DESCRIPTION =
    (ADDRESS_LIST=
      (ADDRESS = (PROTOCOL = TCPS)(HOST = oel6ora12dev1.localdomain)(PORT = 2484))
    )
    (CONNECT_DATA =
      (SERVICE_NAME = DEV1_SSL.localdomain)
      (SECURITY=(SSL_SERVER_CERT_DN="CN=DEV1.localdomain"))
    )
  )

DEV1_NSE.LOCALDOMAIN =
  (DESCRIPTION =
    (ADDRESS_LIST=
      (ADDRESS = (PROTOCOL = TCP)(HOST = oel6ora12dev1.localdomain)(PORT = 2483))
    )
    (CONNECT_DATA =
      (SERVICE_NAME = DEV1.localdomain)
    )
  )

$ sqlplus /@DEV1_SSL
or
$ sqlplus system/manager@DEV1_NSE

Setting different values for TNS_ADMIN at the time of process spawning is useful in other situations, too. For instance, this technique can be applied if you have one listener for multiple databases and each database has its wallet.

In Grid Infrastructure environment there is also the possibility to set environment variables in Oracle Clusterware which is probably more comfortable to handle:

srvctl setenv database -d  -t TNS_ADMIN=

Foot note: All test have been run on Oracle 12.1.0.1