CryptoLyzer: A comprehensive cryptographic settings analyzer

CryptoLyzer is a multiprotocol cryptographic settings analyzer with SSL/TLS, SSH, and HTTP header analysis ability. The main purpose of the tool is to tell you what kind of cryptographic related settings are enabled on a client or server.

If you are not interested in the principles behind the project, but the practice, feel free to skip the next section and jump to the Practice section.

Principles

Rationale

There are many notable open-source projects (SSLyze, CipherScan, testssl.sh, tls-scan, …) and several SaaS solutions (CryptCheck, CypherCraft, Hardenize, ImmuniWeb, Mozilla Observatory, SSL Labs, …) to do a security setting analysis, especially when we are talking about TLS, which is the most common and popular cryptographic protocol. However, most of these tools heavily depend on one or more versions of one or more cryptographic protocol libraries, like GnuTLS, OpenSSL, or wolfSSL. But why is this such a problem?

The minor problem is the dependency easily stucks them in SSL/TLS/DTLS as other cryptographic protocols (eg: IPSec VPN, Kerberos, OpenVPN, SSH, …) can not be implemented directly by these libraries. Supporting them by the analyzer application needs extra effort. Anyway, most of the analysis of the cryptographic setting does not require any cryptography because before the parties could agree on the cryptographic algorithms they use plain text.

The major problem is the fact that analysis should test special and corner cases of the protocol that are intentionally triggered. It is hard to do that with a cryptographic protocol library, which was designed for production not for penetration testing or settings analysis. During an analysis, connections are tried to be established with hardly supported, experimental, obsoleted, or even deprecated mechanisms or algorithms to identify which ones are supported by the given client or server implementation. These mechanisms and algorithms may or may not be supported by the latest or any version of any cryptographic protocol implementations.

That is why most of the existing tools require special build(s) of the dependent library where all the protocol versions and algorithms of the good old days are reenabled to make the chance to set up these libraries to offer them as clients or as servers. But what if we want to test an algorithm or a mechanism that has never been implemented by the dependent cryptographic library?

It is not just a theory. A special fork of OpenSSL, maintained by Pluralsight author Peter Mosmans, aims to have as many ciphers as possible. This fork is used and recommended by Mozilla Cipherscan, however, it can offer less than two hundred cipher suites, but there are more than three hundred in the different RFCs according to Cipher Suite Info. The majority of them are weak or insecure, which makes it particularly important to be part of the analysis. In addition, it is also true that there are cipher suites that are not on the Cipher Suite Info list, for instance, Russian standard (GOST) cipher suites. These are rarely used cipher suites, but there is an OpenSSL engine that implements them, so they should be checked.

The situation is similar in the case of other parts of the TLS protocol, like extensions. It is unlikely that a cryptographic library will support each extension, but some of them are possibly implemented by some of the libraries. For instance, multiple certificate status request extension and Diffie-Hellman key exchange in TLS 1.3 are not implemented by OpenSSL 1.x but implemented by GnuTLS and WolfSSL. Another example can be the X.509 certificate-based authentication in SSH protocol, which is supported by Tectia SSH and has an open-source implementation but is not supported by the default OpenSSH. Even though they are not so common a comprehensive settings analyzer tool should run checks against them.

Summing up the main reason for establishing a greenfield project instead of contributing to existing one is the opinion, that an analysis is mostly testing when we trigger special and corner cases, hardly supported, experimental, obsoleted, or even deprecated mechanisms and algorithms, so a comprehensive analyzer should be implemented independently from current cryptographic protocol implementations as much as possible.

Goals

CrypytoLyzer focuses on comprehensiveness, multi-protocol ability, and library independence. To do so cryptographic protocols clients are implemented focusing only on the necessary parts (usually the handshake). This kind of client can check a server against rarely or privately used, deprecated, or completely insecure algorithms and mechanisms.

• Zero Trust
  • should not require any privilege during the analysis
  • Should not require any credentials (except it is strictly necessary)
  • should not provide any information for 3rd party
• Completeness
  • should be able to support as many cryptographic protocols (DTLS, IPSec, OpenVPN, TLS, SSL, SSH, …) as possible
  • should be able to handle special/corner cases of the cryptographic protocols
• Usability
  • should work both in public and private networks
  • should be able to give a human-readable summary (Markdown)
  • should be able to give machine-readable detailed output (JSON)
  • should be customizable to meet special needs (Python library)

The existing solutions are focusing on only one cryptographic protocol (TLS or SSH) despite the fact, that all the cryptographic protocols have the same building blocks (authentication, key exchange, symmetric ciphers, integrity), so they can be analyzed in the same (or almost the same) way, only the information which the analysis is based on can be acquired differently.

Features

The goals above are just goals, not the current state of the development, particularly since the actual version number is 0.8.0 at the time of writing. The supported cryptographic protocol families, for now, are the SSL/TLS and SSH. You can read comparative analysis with either open source or proprietary (SaaS) solutions below. I wanted to stay as objective as an author can with the competitors, but forgive me if I have not been managed to.

Strengths

Transport Layer Security (TLS)

Specialties

Cryptolyzer supports both the early and deprecated Secure Socket Layer (SSL) 2.0 and each Transport Layer Security version up to the TLS 1.3 version including draft versions. Some features although be checked hardly or difficultly using the most popular SSL/TLS implementations (eg: GnuTLS, LibreSSL, OpenSSL, wolfSSL, …) or just not in the scope of them. The checks are the specialties of Cryptolyzer such as supporting each cipher suite discussed on ciphersuite.info (and more) including the Russian governmental standard (GOST) cipher suites, Diffie-Hellman parameter checks, JA3 tag generation, …

	Crypto Lyzer	Qualys SSL Labs	Immuni Web	Hard enize	Moz. Obs.	testssl .sh	SSL-yze
TLS 1.3 Draft Versions	✅	❌	❌	✅	❌	❌	✅	❌
TLS 1.3 DH Parameters	✅	❌	❌	❌	❌	❌	✅	❌
TLS GOST Ciphers	✅	❌	❌	❌	❌	❌	❌	❌
TLS DH Param Prime Check	✅	✅	❌	❌	❌	❌	❌	❌
TLS DHE Param Reuse Check	✅	✅	❌	❌	❌	❌	❌	❌
TLS ECDH Param Reuse Check	0.9.x	✅	❌	❌	❌	❌	❌	❌
SSH DH Param Check	✅	❌	❌	❌	❌	❌	❌	❌
JA3 tag Generation	✅	❌	❌	❌	❌	❌	❌	❌
JA3 tag Decode	✅	❌	❌	❌	❌	❌	❌	❌
	89%	33%	0%	11%	0%	0%	22%	0%

Opportunistic TLS

Several application-layer protocols have an extension to upgrade a plain text connection to an encrypted one (TLS) without using a separate port referred to as opportunistic TLS (STARTTLS). This mechanism is supported by the analyzer in the case of most protocols.

	Crypto Lyzer	Qualys SSL Labs	Immuni Web	Hard enize	Moz. Obs.	testssl .sh	SSL-yze
FTP	✅	❌	❌	✅	❌	✅	✅
IMAP	✅	❌	❌	❌	❌	✅	✅
LDAP	✅	❌	❌	❌	❌	❌	✅
LMTP	✅	❌	❌	❌	❌	✅	❌
MySQL	0.9.x	❌	❌	❌	❌	✅	❌
NNTP	✅	❌	❌	❌	❌	✅	❌
POP3	✅	❌	❌	❌	❌	✅	✅
PostgreSQL	✅	❌	❌	❌	❌	✅	✅
RDP	✅	❌	❌	❌	❌	❌	✅
Sieve	✅	❌	❌	❌	❌	✅	❌
SMTP	✅	❌	✅	✅	❌	✅	✅
XMPP (Jabber)	✅	❌	❌	❌	❌	✅	✅
	92%	0%	8%	17%	0%	83%	67%

Extensions

There are several extensions for the original TLS specifications which have both security and compatibility considerations. A high number of extension types can be analyzed by CryptoLyzer compared to most of the competitors.

	Crypto Lyzer	Qualys SSL Labs	Immuni Web	Hard enize	Moz. Obs.	testssl .sh	SSL-yze
Fallback SCSV	✅	✅	✅	❌	❌	✅	✅
TLS Clock Skew	✅	❌	❌	❌	❌	✅	✅
Compression	✅	✅	✅	❌	❌	❌	✅
Secure Renegotiation	✅	✅	✅	❌	❌	✅	✅
Insecure Renegotiation	0.8.x	✅	✅	❌	❌	✅	✅
Session Caching	✅	✅	❌	❌	❌	✅	✅
Session Ticketing	✅	✅	❌	❌	❌	✅	✅
Elliptic Curves	✅	✅	✅	❌	❌	✅	✅
EC Point Format	✅	❌	✅	❌	❌	❌	❌
App. Layer Proto. Negotiation	✅	✅	❌	❌	❌	✅	❌
Next Protocol Negotiation	✅	✅	❌	❌	❌	✅	❌
Extended Master Secret	✅	❌	❌	❌	❌	✅	❌
Encrypt-then-mac	✅	❌	❌	❌	❌	✅	❌
GREASE check	0.9.x	✅	❌	❌	❌	❌	❌
Handshake Length Intolerance	0.9.x	✅	❌	❌	❌	❌	❌
	80%	73%	40%	0%	0%	73%	53%

Secure Shell (SSH)

In addition to TLS, there are some other cryptographic protocols, such as SSH. The cryptographic protocols have the same building blocks (authentication, key exchange, symmetric ciphers, integrity), so they can be analyzed in the same manner and it can be done with the same tool using CryptoLyzer without compromise.

	Crypto Lyzer	Moz. Obs.	Operous SSH Check	Rebex SSH Check	SSH Conf. Auditor	ssh-audit	ssh-scan
Algorithms	✅	✅	✅	✅	✅	✅	✅
Host Keys	✅	❌	✅	❌	✅	✅	✅
Host Certificates	0.9.x	❌	❌	❌	❌	❌	❌
X.509 Certificates	0.10.x	❌	❌	❌	❌	❌	❌
DH Param Check	✅	❌	❌	❌	❌	❌	❌
Banner Analysis	✅	❌	✅	❌	❌	❌	✅
	67%	17%	50%	17%	33%	33%	50%

HTTP Headers

Security

There are security mechanisms in the application layer too, not only in the transport layer. The HTTP protocol is a good example of that as there are encryption (HSTS), authentication (Expect-CT, Expect-Staple), content integrity (subresource integrity, content security policy) related headers which are parsed by CryptoLyzer in detail.

	Crypto Lyzer	Qualys SSL Labs	Immuni Web	Hard enize	Moz. Obs.	testssl .sh	SSL-yze
Content Security Policy	0.9.x	❌	❌	❌	✅	❌	❌
CSP Report-To	0.9.x	❌	❌	❌	❌	❌	❌
Expect Certificate Transparency	✅	❌	❌	✅	✅	❌	✅
Expect OCSP Staple	✅	❌	❌	❌	❌	❌	❌
HTTP Strict Transport Security	✅	✅	✅	✅	✅	✅	✅
HTTP Public Key Pinning	0.10.x	✅	❌	❌	✅	✅	❌
Referrer Policy	✅	❌	❌	❌	✅	❌	❌
Subresource Integrity	0.8.x	❌	❌	✅	✅	❌	❌
X Content Type Options	✅	❌	❌	✅	✅	❌	❌
X Frame Options	✅	❌	❌	✅	✅	❌	❌
X XSS Protection	0.8.x	❌	❌	✅	✅	❌	❌
	55%	18%	9%	55%	82%	18%	18%

Generic

Several HTTP headers are not closely related to security. Although, some among them, especially caching headers (Age, Cache-Control, Date, ETag, Expires, Last-Modified), can have significance when obsolescence of security-related information (eg: CRL) is indicated by caching headers.

	Crypto Lyzer	Qualys SSL Labs	Immuni Web	Hard enize	Moz. Obs.	testssl .sh	SSL-yze
Content-Type	✅	❌	❌	❌	✅	❌	❌
Server	✅	❌	❌	❌	✅	✅	❌
Application banner	0.9.x	❌	❌	❌	❌	✅	❌
Age	✅	❌	❌	❌	❌	❌	❌
Cache-Control	✅	❌	❌	❌	✅	❌	❌
Date	✅	❌	❌	❌	✅	✅	❌
ETag	✅	❌	❌	❌	✅	❌	❌
Expires	✅	❌	❌	❌	✅	❌	❌
Last-Modified	✅	❌	❌	❌	✅	❌	❌
Pragma	✅	❌	❌	❌	✅	❌	❌
Proxy banner	0.9.x	❌	❌	❌	❌	✅	❌
Set-Cookie	0.10.x	❌	❌	✅	✅	✅	❌
	75%	0%	0	8%	75%	42%	0%

Weaknesses

Transport Layer Security (TLS)

X.509 Extensions

There are some X.509 related mechanisms in TLS that can have a serious impact on security in particular the revocation checking which is a weak point of the public key infrastructure (PKI). Cryptolyzer lagging behind competitors in this field.

	Crypto Lyzer	Qualys SSL Labs	Immuni Web	Hard enize	Moz. Obs.	testssl .sh	SSL-yze
OCSP Stapling	✅	✅	✅	✅	✅	✅	✅
OCSP Must Staple	0.8.x	✅	✅	✅	❌	✅	✅
Certificate Transparency	0.8.x	✅	✅	✅	❌	✅	✅
Multiple Trusted Root CA Stores	0.9.x	✅	✅	✅	✅	❌	✅
Revocation Status	0.10.x	✅	❌	✅	❌	✅	❌
	20%	100%	80%	100%	40%	80%	80%

Generic

Usability is another field where Cryptolyzer should evolve. Comprehensiveness is not enough, weaknesses, vulnerabilities should be highlighted to make the analysis results more easily understandable.

	Crypto Lyzer	Qualys SSL Labs	Immuni Web	Hard enize	Moz. Obs.	testssl .sh	SSL-yze
Client Emulation	❌	✅	❌	❌	✅	✅	❌
Weakness Check	❌	✅	✅	✅	✅	✅	❌
Vulnerability Check	❌	✅	✅	❌	❌	✅	✅
Highlights in Output	❌	✅	✅	✅	✅	✅	❌
	0%	100%	75%	50%	75%	100%	25%

Opportunities

DNS Records

Nowadays information related to security settings is stored in DNS records especially, but not exclusively in the case of the SMTP protocol. These records can be analyzed by CryptoLyzer and the necessary implementations are planned to be done soon.

	Crypto Lyzer	Qualys SSL Labs	Immuni Web	Hard enize	Moz. Obs.	testssl .sh	SSL-yze
DNS CAA	❌	✅	✅	✅	✅	✅	❌
MTA-STS	❌	❌	❌	✅	❌	❌	❌
DANE	❌	❌	❌	✅	❌	❌	❌
SPF	❌	❌	❌	✅	❌	❌	❌
DMARC	❌	❌	❌	✅	❌	❌	❌
TLS-RPT	❌	❌	❌	✅	❌	❌	❌
	0%	17%	17%	100%	17%	17%	0%

Other protocols

Some cryptographic protocols use the variant of the TLS protocol. Some protocols wrap the original TLS protocol such as OpenVPN, QUIC, and there is a modified version of TLS, according to the need of UDP, called DTLS. Their support would be implemented relatively easily in CryptoLyzer in the future.

Practice

Installation

CryptoLyzer can be easily installed from The Python Package Index (PyPi)

$ pip install cryptolyzer

or via Docker from Docker Hub Container Image Library (docker hub)

$ docker pull coroner/cryptolyzer

and there are prebuilt packages for DEB (Debian, Ubuntu, …) and RPM (Fedora, RHEL, SUSE) based systems.

Usage

The command-line interface is slightly similar to the OpenSSL command-line tool. On each level of subcommands, comprehensive help can be given using the –help option. The analysis subject can be given multiple times as an argument in URL format. Most of the time scheme and port part of the URL has default values according to the analyzer (eg: tls, ssh).

$ cryptolyze tls all www.example.com

$ cryptolyze tls1_2 ciphers https://www.example.com:443

$ cryptolyze ssh all www.example.com

$ cryptolyze ssh2 ciphers ssh://www.example.com:22

$ cryptolyze http headers http://www.example.com/path?query#fragment

or can be used via Docker

$ docker run --rm coroner/cryptolyzer tls all www.example.com

$ docker run --rm coroner/cryptolyzer tls1_2 ciphers https://www.example.com:443

$ docker run --rm coroner/cryptolyzer ssh all www.example.com

$ docker run --rm coroner/cryptolyzer ssh2 ciphers ssh://www.example.com:22

$ docker run --rm coroner/cryptolyzer http headers http://www.example.com/path?query#fragment

CryptoLyzer: A comprehensive cryptographic settings analyzer is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License