Apache security on mkirby.org

I use the default Apache httpd packages from Fedora.   There are several changes that I make to the default configuration.

I have a logformat that is easily consumable with Splunk by using key=value pairs.  I added several values to be logged in addition to the default logs.  The x-forwarded-for, via, and clientip values help me determine the origin of the request if it came through a proxy.
LogFormat “request-line=\”%r\” remote-host=%h remote-ip=%a proxyfor=%{PROXYFOR}i x-forwarded-for=%{X-Forwarded-For}i via=%{VIA}i clientip=%{ClientIP}i canonical-port=%p response-size=%B request-time-taken=%D request-protocol=%H keepalive=%k request-method=%m process-id=%P query-string=%q handler=%R http-status-code=%s  http-request-time=\”%t\” request-time-taken=%T remote-user=%u http-request-url=%U servername-canonical=%v servername=%V http-connection-status=%X http-bytes-received=%I http-bytes-sent=%O http-referer=\”%{Referer}i\” http-user-agent=\”%{User-Agent}i\” ssl_protocol=\”%{SSL_PROTOCOL}x\” ssl_cipher=\”%{SSL_CIPHER}x\”” extended

My site uses vhosts with a default “null” vhost.  This null vhost is the first listed in the configuration.  Apache will choose the first vhost listed when it cannot match a servername to a valid vhost.  This is the vhost that is chosen when a browser hits the site by IP rather than hostname.  I do the same for my default https vhost.  For https, I use a self-signed SSL certificate that does not reveal any of the vhosts I host.  The certificate simply uses a commonname of NULL.  My null vhost is configured to use the directory /var/www/null, which is owned by root and chmod’d 000.  Furthermore, any request will be redirected with a rewrite rule back to the browser’s localhost.  The reason for this is to annoy scanners and hackers.  Most of the scanners out in the wild that are run by script kiddies will probe by IP address rather than server or domain names.  Those scanners won’t find this blog or anything else I host.  I don’t know if the redirect causes the scanners to scan themselves, but I like to hope so.  Here is the rewrite rule that I use:
I append the “YOU_GOT_HACKED=LOL” to the redirect.  If the request came from a compromised webserver that is being used by a hacker to scan (usually to hide their tracks), then hopefully the scanner will follow the redirect and the admin of the compromised webserver will see this message in the logs.

For my other vhosts and directories, I have a similar redirect that is done when the requested file, directory, or symlink does not exist.  Here is the config:
        RewriteCond %{REQUEST_FILENAME} !-f
        RewriteCond %{REQUEST_FILENAME} !-d
        RewriteCond %{REQUEST_FILENAME} !-l
        RewriteCond %{REQUEST_FILENAME} !/wp-admin
        RewriteRule .*{REQUEST_URI}?YOU_GOT_HACKED=LOL [R,L,QSA]


By default, the Fedora httpd package sets up a user called apache with /sbin/nologin as the shell.  Back when I compiled Apache from source, I would make sure that the Apache user had an invalid shell and an invalid home directory.  I also make sure that the Apache user does not have write access to anything that is not necessary for the site to run.

The SSL certificates for mkirby.org are only readable by root.  When Apache starts up, the certificates are ready before it forks and runs as the Apache user.

I disable weak SSL ciphers and only allow for SSLv3 and TLSv1-1.2.  My SSL configuration has earned me an “A” rating from Qualys SSL Labs.  You can scan my site with Qualys by clicking here.  The scan will take about a minute to run.
Below is the ciphersuites I setup.  Notice that I enabled ephemeral Diffie-Hellman.  This allows for forward secrecy.  In addition to the RSA asymmetric key exchange, it performs another on-the-fly Diffie-Hellman key exchange before performing the symmetric key exchange.  This means that if my private SSL key were stolen, the hacker would still be unable to decrypt my https traffic.  I also have SSLHonorCipherOrder enabled so that the Apache server will attempt to use the strongest ciphers first and negotiate the most secure cipher supported by the browser.  As of October 2013, all browsers fail to support ECDHE (Elliptial-Curve Diffie-Hellman Ephemeral), but do support DHE (Diffie-Hellman Ephemeral).  My server will be able to support the ECDHE when browser vendors decide to support it.
SSLProtocol -ALL +SSLv3 +TLSv1 +TLSv1.1 +TLSV1.2

I also disable SSL renegotiation to mitigate known Man-In-The-Middle attacks.  The SSLv3 and TLSv1.0 protocols have a known MITM attack when the server supports SSL renegotiation.  I disabled SSL renegotiation with a simple setting: SSLInsecureRenegotiation off
A cache management feature for Apache makes use of an entity tag (ETag) header. When this option is enabled and a request is made for a document relating to a file, an ETag response header is returned containing various file attributes for caching purposes. ETag information allows subsequent file requests to contain specific information, such as the file’s inode number.  I modified the etag setting to not disclose the inode number with this setting:  FileETag MTime Size

I enabled SSL Stapling and configured it with the following settings:
     SSLUseStapling on
     SSLStaplingResponderTimeout 5
     SSLStaplingReturnResponderErrors off
     SSLStaplingCache “shmcb:/var/run/httpd/ocsp(128000)”
SSL Stapling allows the webserver to cache it’s cert validity by connecting to the CA’s revocation list server and providing the results to the browser. In the past, the browser was supposed to connect to the revocation list servers for each cert it came across, which added latency.
Since I block everything egress by default, I had to make an exception for the apache user to connect to the CA revocation site.

I also set a X-Frame-Options header to “SAMEORIGIN” for clickjacking protection.  This means that any xframes on a remote that reference my site will be rejected as the xframe did not originate from my site.
There are three available options for this header: deny – no rendering within a frame, sameorigin – no rendering if origin mismatch, allow-from: URL – allow rendering frame if loaded from URL.
More details are available at https://www.owasp.org/index.php/List_of_useful_HTTP_headers
     Header always append X-Frame-Options SAMEORIGIN
     Header always append Frame-Options SAMEORIGIN

I also set the X-XSS-Protection header to “1; mode=block” to mitigate against XSS.
This header enables a Cross-site scripting (XSS) filter built into most recent web browsers. It’s usually enabled by default anyway, so role of this headers is to re-enable for this particular website if it was disabled by the user.
More details are available at https://www.owasp.org/index.php/List_of_useful_HTTP_headers
     Header always append X-XSS-Protection “1; mode=block”

I also set the  X-Content-Type-Options header to “nosniff”.
The only defined value, “nosniff”, prevents Internet Explorer and Google Chrome from MIME-sniffing a response away from the declared content-type. This also applies to Google Chrome, when downloading extensions. This reduces exposure to drive-by download attacks and sites serving user uploaded content that, by clever naming, could be treated by MSIE as executable or dynamic HTML files.
More details are available at https://www.owasp.org/index.php/List_of_useful_HTTP_headers
Header always append X-Content-Type-Options nosniff

I disabled the TRACE method by adding “TraceEnable off” in the default directory directive.  The TRACE/TRACK methods are only used for debugging and should never be allowed on an external-facing webserver.

I also disabled htaccess overrides with “AllowOverride None“.  This is added per directory stanza in the configs.  This disables the use of .htaccess which means that only the root-owned Apache config files can determine the access rules for each directory.  The .htaccess override method can be exploited to gain access to restricted areas.  This prohibits its use.

Another change I make per-directory is to remove “Indexes” and “MultiViews” for the Options directive.  By default, /icons allows for indexes.  I don’t have any directories on my site that should be browsable by index or multi-language.  Almost every directory stanza I have contains the “Options None” directive.
By default, an alias to the directory where the Apache manual is set.  I removed this alias as there is no need for visitors to view Apache documentation on my site.

There are a few dozen modules that are included in the default configuration of Apache.  I have disabled the modules that I don’t use.  I recommend that anyone running an Apache server start their security lock-down by disabling the following modules:  mod_userdir mod_autoindex mod_dav mod_dav_fs mod_speling mod_proxy* mod_asis mod_cern_meta.

To hide the Apache version, I set “ServerSignature Off” and “ServerTokens Prod“.
The ServerSignature will disable webserver version advertisement on error pages.
The ServerTokens will change the “Server” banner so that it only states “Apache” rather than the version and some module version information.

I have server-status and server-info enabled, however I have changed the names and they are only available internally with this setting:
<Location /secret-status>
Order deny,allow
Deny from all
Allow from

I limit the HTTP methods to only allow GET POST OPTIONS and HEAD
<Location />

I hide all of the default error pages with ErrorDocument.  This is very important for 500 error pages as they may contain error information that could be useful to a hacker.  I take it a step further and setup a simple “error” string for anything that could go wrong.
ErrorDocument 400 “error”
ErrorDocument 401 “error”
ErrorDocument 402 “error”
ErrorDocument 403 “error”
ErrorDocument 404 “error”
ErrorDocument 405 “error”
ErrorDocument 406 “error”
ErrorDocument 407 “error”
ErrorDocument 408 “error”
ErrorDocument 409 “error”
ErrorDocument 410 “error”
ErrorDocument 411 “error”
ErrorDocument 412 “error”
ErrorDocument 413 “error”
ErrorDocument 414 “error”
ErrorDocument 415 “error”
ErrorDocument 500 “error”
ErrorDocument 501 “error”
ErrorDocument 502 “error”
ErrorDocument 503 “error”
ErrorDocument 504 “error”
ErrorDocument 505 “error”
ErrorDocument 506 “error”

When I create users in Apache, I use bcrypt with a value of 15 for the computing time.
htpasswd -B -C 15 -c /etc/httpd/.htpasswd username

I often use self-signed ssl certs on my internal servers. I don’t use the documented settings in OpenSSL. I add a few extra security features, such as aes256 cipher, sha512 message digest, and 4096 bits. I also tell OpenSSL to specifically use /dev/random rather than relying on it’s internal rng. I set the expiration date to 10 years unless I want to have my cert signed by a 3rd party CA.
Here is how I generate my certs:
$ openssl genrsa -rand /dev/random -aes256 -out server.key.pw 4096
$ openssl rsa -in server.key.pw -out server.key
$ openssl req -rand /dev/random -new -sha512 -key server.key -out server.csr
$ openssl genrsa -rand /dev/random -aes256 -out ca.key.pw 4096

If I want a 3rd party CA to sign my cert, I stop here and submit it. Otherwise, continue to build the CA and sign the server cert
$ openssl genrsa -rand /dev/random -aes256 -out ca.key.pw 4096
$ openssl rsa -in ca.key.pw -out ca.key
$ openssl req -rand /dev/random -new -sha512 -x509 -days 3650 -key ca.key -out ca.crt
$ cat /dev/null > ca.db.index
$ echo 01 > ca.db.serial
$ cat >ca.config <<EOT
[ ca ]
default_ca = CA_own
[ CA_own ]
dir = .
certs = \$dir
new_certs_dir = \$dir
database = \$dir/ca.db.index
serial = \$dir/ca.db.serial
RANDFILE = /dev/random
certificate = \$dir/ca.crt
private_key = \$dir/ca.key
default_days = 3650
default_crl_days = 30
default_md = sha512
preserve = no
policy = policy_anything
[ policy_anything ]
countryName = optional
stateOrProvinceName = optional
localityName = optional
organizationName = optional
organizationalUnitName = optional
commonName = supplied
emailAddress = optional
$ openssl ca -config ca.config -out server.crt -infiles server.csr