A test framework for testing SSL/TLS client certificate validation.
Note: TLSPretense is still undergoing a lot of polishing. It is currently usable, but features may change and documentation may be missing. As such, please bear with us over next few months as we find time to work on the tools, and feel free to file a bug with details.
TLSPretense provides a test framework for testing SSL certificate validation. It generates a set of certificates containing specific flaws, and it presents the certificates to a client that has been configured to trust a CA used by TLSPretense. The test framework then configures its system's firewall to redirect and intercept network traffic so that the test runner can present its certificate to the client. To speed up testing, the test runner starts the next test as soon as the current test finishes.
The test framework must be run on a Unix-like OS that contains a supported firewall, but the program being tested can run on any device whose network traffic can be routed through the system hosting the test framework. Currently, it supports netfilter (Linux), ipfw (Mac OS X 10.6, *BSD), and PF on Mac OS X Lion.
It also has an implementation for a newer version of PF, although this is untested.
TLSPretense requires the TLS client software to be configured to trust a CA that TLPretense controls. That way "good" certificates created by TLSPretense will be accepted by the client.
Once the system hosting the test runner has been configured to be a gateway for the network traffic of thest client being tested, it will add a firewall rule to redirect network traffic to a test listener. The test listener checks to see whether the client is trying to connect to a predefined host. If the client is connecting to the desired host, then the test listener presents a test certificate chain to the client. The test runner then determines whether the test passes or fails based on whether the client completes the TLS handshake or not.
The test harness was designed to anticipate working with a client that may connect to more than one host. The config.yml file specifies a hostname that should be used for the actual test — all other intercepted SSL connections are essentially ignored (although they currently have their certificate re-signed by the goodca in order to make interception easier).
A Unix-like system that uses a supported firewall/routing implementation. TLSPretense currently supports:
Netfilter on Linux
IPFW on MacOSX 10.6 and earlier, and *BSD
PFRdr on MacOSX 10.7 (and probably also 10.8)
Ruby 1.9.x (Developed with 1.9.3). Check your version with:
ruby --version
Some systems will install Ruby 1.9.x with a suffix, like `ruby1.9`. Ruby must also be built against a version of OpenSSL that supports the SNI TLS extension. You can check for this if you run the following Ruby script (on some systems, Ruby 1.9.x will be installed as ruby1.9, and commands like gem will also have the 1.9 suffix):
ruby -ropenssl -e 'puts OpenSSL::SSL::SSLSocket.public_instance_methods.include? :hostname='
Ruby 1.8.7 will mostly work, but Ruby 1.8’s OpenSSL wrapper library does not support the ability for clients to use the SNI TLS extension, which is needed to grab the correct remote certificate for proxying miscellaneous connections. Use Ruby 1.8.x at your own risk.
The SSL client/HTTPS user agent has to trust the CA used by TLSPretense. You can either generate a new goodca and install it in the client’s trust store, or you can use an existing test CA with TLSPretense to generate the test certificates.
Install with rubygems:
umask 0022 ; sudo gem install tlspretense
Create a new project:
tlspretense init myproject cd myproject
And edit config.yml to suit your needs. If you want to create a new test CA (not necessary if you want to use the default or your own):
tlspretense ca
Generate certificates for the test cases:
tlspretense certs
You will also need to setup the host’s networking stack and firewall to support TLSPretense. More details can be found in TLSPretense Setup and in the system-specific guides.
Finally, run all of the configured test cases:
sudo tlspretense run
Or just certain tests (in the order specified):
sudo tlspretense run unknownca wrongcname
The Server Name Indication (SNI) TLS extension does not have full support in Ruby 1.8.7.
Protocols that explicitly call STARTTLS to enable SSL/TLS (eg, SMTP and IMAP) are not yet supported. They would require protocol-specific support. The version of these protocols where they are wrapped in SSL should be testable though.
It currently uses the goodca to re-sign certificates from hostnames that do not match the configured test hostname.
The existing PFDivert rule implementation does not work on Mac OS X 10.7 (and probably not on 10.8 either). OpenBSD 5.0(?) and FreeBSD 9 can make use of this functionality though.
Convert SSLClient’s initial connection to use a non-blocking connect.
Change the pre-flight in SSLSmartProxy to disable the accepted server socket until the pre-flight finishes. (the SSLClient within SSLTransparentProxy probably also should do this until the client connects)
Fix the subjectAltName null byte test. It requires manual construction of the ASN1 encoding due to the null byte causing a truncation somewhere in OpenSSL or Ruby’s OpenSSL.
Add wildcard tests
cert hostname: *.%PARENTHOSTNAME%
bad hostname: *.other.com
tld hostname: *.%TLDHOSTNAME% (reject)
do we want to test more complicated wildcards?
Decide how to deal with a wildcard cert at the original destination. If we are testing foo.somehost.com, and the client connects to other hostnames like bar.somehost.com, and they all use a *.somehost.com certificate, then TLSPretense gets confused.
Add extension criticality tests (need an extension oid that nobody knows how to verify)
If an extension is marked critical and the verifier doesn’t know how to verify, then it should fail. If it is not marked critical, then it is allowed to skip the extension.
Advanced: Add name constraints tests.
success: dnsName of leaf matches exactly the dnsName permitted constraint nameConstraints=permitted;dnsName:%HOSTNAME%
reject: constraint is a different hostname nameConstraints=permitted;dnsName:some.other.com
success: dnsName of leaf is a subdomain in addition to dnsName constraint constraint = parent domain of hostname (need to ensure hostname has enough labels) nameConstraints=permitted;dnsName:%PARENTHOSTNAME% do it this way vs trying a subdomain of the original hostname to
reject: constraint is a slightly different hostname nameConstraints=permitted;dnsName:a%HOSTNAME%
success: dirname matches the default subject’s DN
reject: dirname does not match the default subject’s DN
URI constraints
Document how to run SSLTest from MacOSX
Document how to run SSLTest from a Linux VM on Windows (eg, with VMWare)
Document how to deal with certificate pinning and other things that may make testing certificate validation logic difficult.
Command line option to specify where to write the results to
Add more result output formats
(X)HTML
CSV
LaTeX?
XML?
SQLite?
truly unique serial numbers for a given CA. Alternatively, we could use the first cert’s serial as a starting point and increment from there.
Build certs and chains of certs for each test so that something like s_server could use them.
Make initialization interactive. It should prompt the user to choose or confirm configuration details like the interception/firewalling method to use, the network device to listen on, the default hostname, etc. It could then auto-generate the necessary certificates as well.
Config file validator
Add an API for interacting with an external test controller. This could be a little web service, although that lacks real-time responses. A TCP/unix socket interface that sends/receives JSON messages (or something simpler) might be better. The client would tell TLSPretense which test to start, and then TLSPretense would reply with a result when it completes.
Check out the latest master to make sure the feature hasn’t been implemented or the bug hasn’t been fixed yet.
Check out the issue tracker to make sure someone already hasn’t requested it and/or contributed it.
Fork the project.
Start a feature/bugfix branch.
Commit and push until you are happy with your contribution.
Make sure to add tests for it. This is important so I don’t break it in a future version unintentionally.
Please try not to mess with the Rakefile, version, or history. If you want to have your own version, or is otherwise necessary, that is fine, but please isolate to its own commit so I can cherry-pick around it.
William (B.J.) Snow Orvis (iSEC Partners)
Copyright © 2012 iSEC Partners
See LICENSE.txt for further details.