Banner image of Pros and cons of online assessment tools for web server security

Pros and cons of online assessment tools for web server security

Maybe you already use the Observatory by Mozilla, Webbkoll, SSL Labs, or other online assessment tools to get the first expression of a website’s security. All of these tools are easy to use; however, their possibilities to assess the security of a website and especially of a web server are limited. We show you the pros and cons of several tools in this article.

Always stay in the loop!
Subscribe to our RSS/Atom feeds.

General note

First of all, no tool provides a holistic view of the security level of a server. Many vital aspects can only be tested when you can directly access the web server and check its configuration files. For example, most tools don’t check if the installed web server (Apache, nginx, etc.), PHP packages, database software, or operating system (Debian, Ubuntu, etc.) are up-to-date. Checking these aspects is crucial, though.

Furthermore, a server-side web application firewall may block some or all online assessment tools tests—the blocking results in partially or entirely wrong output. For instance, some of the tools state that we requested to be excluded while others show that HTTP response headers are missing (both is wrong).

If you are a server administrator, you should conduct internal testing. Use tools like Nmap, OpenSSL, SSLyze, Lynis, etc. to directly assess your system’s security. Do not solely rely on the following web tools!

Online assessment tools

You can use all of the following online assessment tools in your web browser without installing additional software on your device.

Notice
Kindly note that the features of these tools may change over time. Due to this, some of our findings can be outdated. We update this page from time to time.

Observatory by Mozilla – overview

The Observatory by Mozilla has the following features:

  • It checks the Content Security Policy (CSP).
  • It checks security-relevant HTTP response headers (e.g., HSTS).
  • It checks cookie configuration, HTTPS redirection, referrer policy, and so on.
  • It checks TLS configuration, CAA, AEAD, and PFS.
  • It checks for OCSP stapling and cipher preference.
  • It shows other online assessment tools for further assessment.

There are three different “observatories” on this website:

HTTP Observatory

  • Websites can get up to 135 points. A rating of more than 100 points means that you see a green A+. The problem: You gain 10 bonus points for secure cookies (HttpOnly, Secure, SameSite) and Subresource Integrity (SRI). You don’t get these points if there are no cookies at all, and there is no reason for implementing SRI.
  • X-Frame-Options are assessed by checking the CSP. If undefined in the CSP, the Observatory looks for a header. So, do you need an extra header at all if you set this in your CSP? Yes, because some assessment tools and web browsers only look for the header. Make sure that frame-ancestors (CSP) and your X-Frame-Options header have the same configuration.
  • Websites gain 5 bonus points for being “Preloaded via the HTTP Strict Transport Security (HSTS) preloading process.” Nowadays, mainstream web browsers are shipped with a built-in list containing websites which have to be loaded via HTTPS. Administrators can add their website to the list (HSTS preload list, link below). However, after a website is added to the list, it can take a long time until the Observatory recognizes this. It is more reliable to check the HSTS preload list for your website directly.

TLS Observatory

  • It doesn’t recognize TLS 1.3 at the moment.
  • The TLS Observatory doesn’t identify modern cipher suites with ChaCha20-Poly1305. They don’t appear on the list. It only recognizes “CHACHA20-POLY1305-OLD.”
  • There is additional (more technical) information when you click on Scan Summary → Certificate Explainer.

SSH Observatory

You can use the SSH Observatory by manually starting the assessment. Please note that the Observatory must be allowed to connect to port 22 of the web server to check the SSH configuration.

Hardenize – another overview

Hardenize is another tool to get a first impression and very similar to the Observatory by Mozilla. However, it also checks the mail server, if available. Its main features are:

  • It checks DNS records, DNSSEC configuration, and CAA.
  • It checks for TLS (including TLS 1.3) configuration, HTTPS redirection, cookies, etc.
  • It checks security features like MTA-STS, TLS-RPT, DANE, SPF, and DMARC of the mail server.
  • It checks for Certificate Transparency and cipher preference.
  • It checks for an Expect-CT header and submits a test report to the provided report-uri.
  • It checks any IPv4 and IPv6 addresses of the domain.

You have to keep in mind:

  • You should click on every single check because there are sometimes additional hints for better configuration or warnings due to lax configuration.
  • The CSP check recommends setting either block-all-mixed-content or upgrade-insecure-requests. We think that this is unnecessary if you don’t embed external content, and your website is preloaded (HSTS).
  • If there is no server-side cipher preference, Hardenize states, “Servers should always enforce their own cipher suite preference, as that is the only approach that guarantees that the best possible suite is selected.” We think this isn’t necessary as long as the server only allows strong TLS cipher suites.
  • If the server doesn’t set the X-Frame-Options header, Hardenize shows it as “Feature not implemented or disabled.” Hardenize doesn’t recognize that you can set the same behavior with the frame-ancestors directive of the Content Security Policy (this is encouraged nowadays).
  • It shows that Subresource Integrity (SRI) is required even if files are provided by the same web server (e.g., you check xyz.cz that embeds resources of xyz.sk, but both domains are hosted on the same web server).
  • It recommends the legacy X-Xss-Protection header that isn’t supported by most current web browsers.
  • Only cipher suites supporting PFS and AEAD are shown in green color. If you are an administrator, ensure that you only enable “green” cipher suites.

Qualys SSL Labs – certificate details

SSL Labs mainly checks certificates and TLS configuration of a web server. Its main features are:

  • Certificate:
    • It evaluates Extended Validation, Certificate Transparency, OCSP Must-Staple, and CAA.
    • It checks the revocation status of certificates.
  • TLS protocols:
    • It supports SSL (2, 3) and all TLS versions (1.0–1.3).
  • Cipher suites:
    • It checks for PFS and AEAD. Cipher suites using CBC are considered “weak.”
  • General information:
    • It shows if the server exposes its signature (however, this isn’t evaluated).
    • It checks whether a website is on the HSTS preload list.
    • It checks for several vulnerabilities like POODLE, GOLDENDOODLE, 0-Length OpenSSL, and Sleeping POODLE.
  • SSL Labs checks all IPv4 and IPv6 addresses which belong to the domain name.

High-Tech Bridge – detailed security configuration

High-Tech Bridge’s “ImmuniWeb” offers two different tests for free:

ImmuniWeb SSL Security Test

  • This test evaluates certificates and ciphers suites, including TLS 1.3.
  • It visualizes the certificate chain(s).
  • It evaluates Extended Validation, Certificate Transparency, and OCSP Must-Staple.
  • It shows the supported elliptic curves (including X25519).
  • It checks for DNS CAA, Expect-CT, TLS 1.3 Early Data, cipher suite preference, HTTPS redirect, HSTS, and more.

You have to keep in mind:

  • Missing support for TLS 1.3 is considered as “misconfiguration or weakness.”
  • If the server has no cipher suites preference, it is considered “misconfiguration or weakness.” ImmuniWeb states “The server does not prefer cipher suites. We advise to enable this feature in order to enforce usage of the best cipher suites selected.” This advice is only correct if the server offers legacy cipher suites (cipher suites without support for PFS/AEAD) additionally to modern ciphers suites (supporting AEAD and PFS). The goal is to prevent client-side downgrade attacks. If a web server only offers modern cipher suites, it isn’t recommended to set server preference anymore.
  • This test checks for compliance with PCI DSS, HIPAA, and NIST. HIPAA and NIST guidelines don’t accept ECDSA certificates signed by RSA intermediates and require you to disable cipher suites using ChaCha20-Poly1305. If you don’t have to comply with these standards, ignore the checks.
  • Compared with most of the other tools in this article, the rating of cipher suites is somewhat wishy-washy. For example, TLS 1.0 and TLS 1.1 are still considered as “good protocol compatibility, allowing users with older browsers to access your website.” Our recommendation is to disable all TLS versions before 1.2.
  • The test still mentions HPKP (HTTP Public Key Pinning). HPKP is outdated, and most web browsers don’t support it anymore. So, don’t implement it.

ImmuniWeb Website Security Test

  • This test evaluates enabled HTTP methods, ALPN (part of HTTP/2), and more.
  • It also checks if the web server exposes its server signature and recognizes additional software used on the web server (jQuery, Bootstrap, etc.). This scan can also detect WAFs (web application firewalls).
  • It checks for PCI DSS compliance (irrelevant for many private websites).
  • It evaluates the CSP and shows tips for better configuration.
  • It evaluates HSTS, Expect-CT, and other response headers and shows misconfiguration.
  • It analyzes cookies (HttpOnly, Secure, SameSite).
  • It shows third-party content and the corresponding TLS security level of these external connections (even if the content is on the same web server).

You have to keep in mind:

  • If you run a web application firewall, the test for different HTTP types (like TRACE, POST, DELETE, …) gets confused and shows every HTTP type as enabled. This result is wrong.
  • NPN (Next Protocol Negotiation) was a draft for the SPDY protocol that isn’t in use nowadays. When you use HTTP/2, you should enable ALPN and disable NPN. Most web browsers already dropped support for NPN.
  • Testing websites with this scanner is noisy and probably triggers web application firewalls. As mentioned above, web application firewalls can alter the testing results.

CryptCheck – detailed cipher suites

CryptCheck is like a magnifier for TLS and cipher suites. Its features are:

  • It checks algorithms used for key exchange, authentication, encryption, and MAC in detail.
  • It checks for PFS and HSTS and shows supported TLS protocols.
  • It rates the strength of keys.
  • It checks all IPv4 and IPv6 addresses belonging to the domain name.

You have to keep in mind:

  • It doesn’t recognize TLS 1.3 or AEAD at the moment.
  • We didn’t find any explanation for the colors in use by CryptCheck. Our guess is:
    • Blue values represent a modern and secure configuration.
    • Green values represent a secure configuration.
    • Yellow values represent a configuration that may be insecure in some instances.
    • Red values represent an insecure configuration.
    • Black values represent an insecure configuration that can be easily exploited.
    • Gray values: Good question …
  • CryptCheck uses the HTTP method HEAD to check for HSTS headers. If you have disabled this in your server configuration, HSTS and HSTS_LONG won’t be recognized by CryptCheck. In this case, CryptCheck incorrectly shows that HSTS is missing, resulting in a lower score for the website.
  • If you configured IPv6 (AAAA record) and your web server is only available via IPv4, CryptCheck won’t recognize HSTS and HSTS_LONG.
  • CryptCheck seems to recognize only one certificate. If you deployed two (e.g., RSA and ECDSA), you would only see one of them.
  • As an administrator, only set values that are shown in blue or green color. If there are some yellow values, search for any known vulnerabilities.
  • You have to disable AES-128 if you want to get a 100% rating. We don’t see any reasons to do so.
  • CryptCheck can also check the SSH configuration of a server. Of course, you have to allow CryptCheck to connect with port 22 of your web server.

securityheaders.com – security headers only

securityheaders.com rates the same response headers of a website as shown before: CSP, referrer policy, HSTS, X-Xss-Protection, X-Frame-Options, X-Content-Type-Options, Expect-CT, Expect-Staple, Permissions-Policy, Network Error Logging (NEL), and Report-To. It also shows all headers send by the web server (like the Observatory by Mozilla).

Since July 2019, securityheaders.com doesn’t recommend using “X-Xss-Protection” anymore since major web browsers don’t support this header by default, or they are about to remove support for it. Furthermore, one should primarily use the CSP directive “frame-ancestors” instead of setting an “X-Frame-Options” header.

Additional checks are:

  • It checks whether the web server exposes its software manufacturer and version information.
  • It checks for the Expect-CT header. This header can be used to log errors if Certificate Transparency is deployed.
  • It checks for the Expect-Staple header. This header can be used to log errors if OCSP Must-Staple is deployed.
  • It checks for the upcoming Permissions Policy. Permissions Policies allow admins to enable or disable certain client features, but most web browsers don’t support them.
  • It checks for a NEL header. The Network Error Logging (NEL) header defines a mechanism enabling web applications to declare a reporting policy that can be used by the user-agent to report network errors for a given origin. The logged network errors even include problems that occurred before a connection was successfully established.
  • It checks for a Report-To header. The Report-To header is defined in the Reporting API. It aims to provide a best-effort report delivery system that executes out-of-band with website activity and introduces new report types.

CSP Evaluator – assess CSPs

Google’s CSP Evaluator evaluates the Content Security Policy of a website. The evaluator supports all CSP versions (1–3).

At the moment, it recommends setting the require-trusted-types-for directive. This directive is experimental. Trusted Types is an upcoming client API developed by Google.

Webbkoll – connections and cookies

Some people consider Webbkoll as the ultimate privacy tool. They enter a URL, click on check, wait several seconds, and are happy if there are many zeros and green check marks. However, Webbkoll has several drawbacks that we already discussed in another article: “Limits of Webbkoll.” Its main features are:

  • It presents you information about HTTPS configuration, HSTS, CSP, Referrer-Policy, SRI, localStorage, and other security-relevant HTTP response headers.
  • It lists third-party requests (including reporting directives of some HTTP response headers) and the approximate server location.
  • It shows GDPR-relevant information.

Please note that the results are only valid for the link you entered. Other pages of the same website can differ (see three examples). Webbkoll partially uses the code of the Observatory by Mozilla for its checks.

PrivacyScore – assessment mix

PrivacyScore is developed by German universities and lists four categories:

  • NoTrack: Webbkoll covers most of these tests. Additionally, PrivacyScore checks whether the web server or mail server is located in the EU.
  • EncWeb: These tests include evaluating response headers, PFS, TLS protocol versions, cipher preference, and several attacks on TLS.
  • Attacks: This category mainly checks for response headers like securityheaders.com.
  • EncMail: These checks are mostly the same as in the EncWeb category, but the mail server is checked.

You have to keep in mind:

  • It doesn’t recognize TLS 1.3 or AEAD at the moment.
  • Like the Observatory by Mozilla, websites recently added to the HSTS preload list aren’t recognized for a long time.
  • Even web servers with strict configuration can’t get 100% ratings. For example, we are vulnerable to the BREACH attack (according to this site) due to activated gzip compression. Most other websites are vulnerable to Lucky 13 due to cipher suites that support CBC instead of AEAD.
  • Testing websites with this scanner is noisy and probably triggers web application firewalls. As mentioned above, web application firewalls can alter the testing results. If your web application firewall repeatedly blocks PrivacyScore from scanning your website, it incorrectly shows, “This site is excluded from further scans. The owner of this website has asked us to not perform further scans. For reasons of transparency we display the results of the last scan before we received the request of the owner. The owner may have implemented changes in the meantime, which are not reflected in the results below.” In this case, you can’t scan the website anymore.

Web Cookies Scanner – information about cookies

The Web Cookies Scanner shows the number of third-party domains contacted by the given web page and the number of persistent and session cookies set by the given page.

Additionally, it rates the overall “Privacy Impact Score,” a “simple indicator of website’s usage of potentially privacy intrusive technologies such as third-party or permanent cookies, CANVAS trackers etc.” There is also a check of TLS protection and a check for security-related HTTP headers (X-Frame-Options, HSTS, X-Content-Type-Options, X-XSS-Protection, CSP).

Please note:

  • Session cookies and first-party cookies aren’t assessed and aren’t part of the Privacy Impact Score.
  • As always, it only scans the given web page. So there can be other pages on a web server that set cookies.
  • The scanner doesn’t recognize TLS on our and some other tested websites (error “Certificate path cannot be verified to a known root certificate”). This error is likely related to our ECDSA certificate. We assume that it only supports RSA certificates.
  • The scanner doesn’t recognize that we set the legacy header X-Frame-Options in our CSP (via frame-ancestors).
  • The scanner recommends the legacy header X-XSS-Protection.
  • The scanner recommends Subresource Integrity for files that are hosted on the same web server. For us, this recommendation doesn’t make sense.

Internet.nl website test – a mix of general and security tests

The “website test” of internet.nl checks for modern internet standards like an IPv6 address, DNSSEC, HTTPS (including TLS 1.3), and security-related HTTP response headers.

Please note:

  • The scanner recommends disabling all TLS/SSL versions except TLS 1.2 and TLS 1.3. It also recommends TLS cipher suites that are considered secure nowadays.
  • The scanner notes that HTTP compression could be a security risk. As mentioned above, most websites are vulnerable to the BREACH attack. However, disabling HTTP compression results in a noticeable performance loss. Due to this, most web servers keep HTTP compression enabled.
  • The scanner recommends disabling 0-RTT (Zero Round Trip Time Resumption). 0-RTT is a new performance feature of TLS 1.3 that accelerates HTTPS requests. Under certain conditions, TLS becomes vulnerable to replay attacks if 0-RTT is enabled. If this vulnerability is present, an attacker could copy a client request and resend it to the server, pretending to be another user. There are several mitigations available. Kindly note that InfoSec Handbook doesn’t host any user-specific data. Due to this, hypothetically successful replay attacks don’t expose any secrets or user-specific data.
  • The scanner doesn’t recognize that we set the legacy header X-Frame-Options in our CSP (via frame-ancestors).
  • The scanner recommends the legacy header X-XSS-Protection.

Follow us on Mastodon:
@infosechandbook

Summary

Hopefully, you got a first impression of what most tools can test and what they can’t. Many tools evaluate response headers, TLS protocols, and cipher suites. Additionally, some tools give extensive information about their rating and tips on how admins can improve their security configuration. You should evaluate websites regularly since tests are added, modified, or deprecated from time to time.

As mentioned above, keep in mind that no web tool can provide a holistic view of a server’s security level. Even scanning a website with every tool mentioned above only gives minimal insight into security.

Changelog

  • Sep 14, 2020: securityheaders.com now detects the Permissions Policy. Updated the Hardenize section. Added Trusted Types to the CSP Evaluator.
  • May 28, 2020: The Feature Policy is renamed to Permissions Policy.
  • Jan 1, 2020: Added internet.nl.
  • Oct 19, 2019: Added webcookies.org.
  • Jul 27, 2019: Updated the securityheaders.com section, addressing their update from 2019-7-22.
  • Jun 30, 2019: Updated High-Tech Bridge products according to their website. Added information regarding an incorrect statement on PrivacyScore if a WAF repeatedly blocks it.
  • May 15, 2019: Updated the SSL Labs section according to their announcement in “Zombie POODLE and GOLDENDOODLE Vulnerabilities” (April 2019).
  • Dec 11, 2018: Updated this article, including Webbkoll’s update from 2018-11-30.
  • Nov 17, 2018: Updated nearly all feature lists.
  • Nov 1, 2018: Updated securityheaders.com’s feature list.
  • May 26, 2018: Updated High-Tech Bridge products according to their website.
  • May 23, 2018: Removed HPKP in Webbkoll section since Webbkoll removed its recommendation due to this article.

Read also