Scanning .nz for HTTPS support

As part of our efforts to understand the .nz namespace better, we started at the beginning of 2017 to check domains for the presence of a secure website using HTTPS, and collect information about certificates, protocol features and other valuable information in the process.

The collection process is straightforward:

  • Extract the list of active .nz domains in the register
  • For each domain, verify if there is an A record for the host www.domain. If the resolution process fails, the domain won't be included.
  • Test each domain using sslyze. We have a script that will test for different versions of SSL and TLS, protocol features and will collect information about the certificate chain on the site.
  • Once the collection is completed, produce aggregated counters and make them available on IDP, our Internet Data Portal.

We started in January 2017 and so far the process has completed five times, giving us some valuable datapoints.

HTTPS support

Let's start with the overall picture of how much of the .nz namespace has a secure website.

On our first collection, we didn't register how many domains failed the test due to incorrect DNS response, it was added afterwards. Despite that detail, there is a lot to notice here. Around 14% of the domains fail the DNS test, and around 0.7% have invalid certificates, for example, where the name in the certificate doesn't match the website name. The positive news is HTTPS support grew from 44% to 47% during this year.

Protocol Support

HTTPS relies on cryptographic protocols to ensure privacy and authenticity, such as SSL and TLS. A given webserver can support multiple crypto protocols at the same time. SSL v2.0 was released in February 1995, SSL v3.0 released in 1996, TLS v1.0 was defined in January 1999 as a replacement for SSL v3.0, TLS v1.1 was published in April 2006, TLS v1.2 was published in August 2008 and TLS v1.3 is currently being drafted in the IETF and we don't test for it.

As SSL v2.0 was deprecated and prohibited in 2011 in RFC 6176, and SSL v3.0 was deprecated in June 2015 by RFC 7568, there SHOULD NOT be any domains supporting it. Sites with those crypto protocols activated are a risk given the number of known exploitable vulnerabilities against SSL. If you are an administrator of a secure website, you can use the excellent Qualys SSL Site tester to verify for weaknesses.

Certificate Public Keys

Crypto protocols use public key cryptography to provide privacy and authenticity. To achieve this, SSL and TLS rely on certificates issued by Certificate Authorities (CA), that authenticate the identity of the website you are visiting, and contain a cryptographic key to encrypt traffic.

Cryptographic keys have two main properties: the algorithm used to generated them, where we can have RSA and ECC, and the key size measured in bits.

As part of the testing, we collect what kind of cryptographic keys the sites with HTTPS enabled are using.

You can see 3 different key sizes for RSA keys: 1024, 2048 and 4096. A 1024-bit RSA key is considered weak and should not be used. The fact keys of that size were visible at the beginning of this year but now have disappeared shows a healthy attitude towards security maintenance. The transition from 2048-bit keys to 4096-keys observed since August 2017 also is a great indication of the hygiene of the .nz namespace.

What about the 256 id-ecPublicKey cases? Those are sites using the new Eliptic Curve DSA cryptography, which uses smaller key sizes. There are also a few cases using 384-bit ECDSA keys.

To make the visualisation easy to read, we omit values with less than 1% of domains, but we can find some oddities such as sites with unusual key sizes. For example, there are still a few cases with weak 512-bit RSA keys, or 3000-bit keys, or 1536-bit keys (1536 is half-way between 1024 and 2048).

Certificate Signature Algorithms

Each SSL certificate contains a digital signature, a hash of the certificate content, signed with the issuing Certificate Authority private key. This digital signature allows the verification of the integrity of the certificate and enables browsers to verify the validity of the certificate.

There are a few hashing algorithms used for signatures, such as MD5, SHA-1 family and SHA-256 family, including SHA-256, SHA-384 and SHA-512.

Great news right? Most of the sites use the strong SHA-256 hashes, with RSA or ECDSA. As SHA-1 is subject to collision attacks, it's feasible to generate fraudulent certificates and trick users into using the wrong website. Firefox now warns users visiting secure websites using certificates signed with SHA-1, and NIST has recommended moving away from SHA-1 since 2014. So despite the fraction of sites with weak hashes moving from 7.1% to 4.5%, we still have reasons to be concerned.

As with the previous plot, for clarity we didn't include signature algorithms with less than 1% of the domains. There are a few domains using MD5 as hashing algorithms in their certificates, considering MD5 was deprecated back in 2013, that's certainly not good news.

Certificate Authorities

As we mentioned before, certificates are issued by Certificate Authorities, and browsers are configured to trust a set of CAs. An administrator can relatively easily set up their own CA and create certificates, but those won't be verifiable by a browser.

The plot below shows the top 10 issuing Certificate Authorities observed in the .nz namespace. For simplicity, we group the long tail of CAs including self-issued certificates into the 'Other' category.

The figure shows the biggest players in the certificate industry as observed in New Zealand: UserTrust Network, GoDaddy, GeoTrust, Comodo and Let's Encrypt. The interesting bit is the evolution across time: Let's Encrypt grew from 16.29% to 22.84% of the domains in 9 months, and that growth came at the cost of more traditional CAs such as GoDaddy, GeoTrust and Comodo. The other CA with a massive growth is UserTrust Network, that nearly tripled their market share since January 2017.

Let's Encrypt provides an interesting story: started back in December 2015 to issue certificates for free, with a validity of 90 days and a fully automated process. Before that, all CAs were for profit organisations, charging in some cases considerable fees to issue a certificate. Let's Encrypt is supported by the Internet Security Research Group (ISRG), which includes Facebook, the Mozilla Foundation, Google Chrome, EFF and many others.

Wrap up

We started this collection with the motivation of finding out more about trust and security in the .nz namespace and we've learned a lot in the process. The collection will carry on every other month and a future blog post could cover more details about the protocols, such as negotiation features, certificate chain length and intermediate CAs.