How to Install and Configure ModSecurity with Nginx and Apache on a Linux VPS: Complete Guide — CLIQHOST

Web attacks are growing in sophistication every year: SQL injection, XSS, CSRF, path traversal — the threat landscape is vast. A Web Application Firewall (WAF) is an essential defense layer for any server running web applications. ModSecurity is one of the most powerful and mature open-source WAFs available, compatible with both Nginx and Apache.

In this complete guide, you'll learn how to install and configure ModSecurity on a CLIQHOST NVMe VPS running Ubuntu 22.04, together with the OWASP Core Rule Set (CRS) — the industry-standard ruleset for web application protection.

What Is ModSecurity and Why Do You Need It?

ModSecurity is a WAF module that integrates directly into your web server and inspects every HTTP request before it reaches your application. It can:

Block malicious requests in real time
Log attack attempts for forensic analysis
Detect anomalies in HTTP traffic patterns
Filter dangerous payloads (SQL injection, XSS, etc.)

Whether you run an e-commerce store, a corporate site, or a SaaS application on a CLIQHOST SSD VPS, ModSecurity provides a robust, battle-tested shield against the most common attack vectors.

Prerequisites

Before you begin, make sure you have:

A VPS running Ubuntu 22.04 (or Debian 11/12)
Root access or a user with sudo privileges
Nginx or Apache installed and running
An active SSH session

If you don't have a server yet, you can quickly spin up a Linux VPS at CLIQHOST with Ubuntu pre-installed.

Part 1: Installing ModSecurity with Apache

Step 1 — Update the System and Install ModSecurity

sudo apt update && sudo apt upgrade -y
sudo apt install libapache2-mod-security2 -y

Enable the module:

sudo a2enmod security2
sudo systemctl restart apache2

Verify the module is active:

apache2ctl -M | grep security
# Expected output: security2_module (shared)

Step 2 — Configure ModSecurity in Detection Mode

Copy the recommended configuration file:

sudo cp /etc/modsecurity/modsecurity.conf-recommended /etc/modsecurity/modsecurity.conf

Open the file and change the engine mode:

sudo nano /etc/modsecurity/modsecurity.conf

Find:

SecRuleEngine DetectionOnly

Replace with:

SecRuleEngine On

Best Practice: Start in DetectionOnly mode first, monitor the logs for false positives, then switch to On once you're confident in your setup.

Step 3 — Install the OWASP Core Rule Set (CRS)

sudo apt install git -y
sudo git clone https://github.com/coreruleset/coreruleset.git /etc/modsecurity/crs
sudo cp /etc/modsecurity/crs/crs-setup.conf.example /etc/modsecurity/crs/crs-setup.conf

Edit the Apache ModSecurity configuration:

sudo nano /etc/apache2/mods-enabled/security2.conf

Add or update the Include directives:

<IfModule security2_module>
    SecDataDir /tmp/
    Include /etc/modsecurity/modsecurity.conf
    Include /etc/modsecurity/crs/crs-setup.conf
    Include /etc/modsecurity/crs/rules/*.conf
</IfModule>

Restart Apache:

sudo systemctl restart apache2

Part 2: Installing ModSecurity with Nginx

For Nginx, ModSecurity works as a dynamic module (libmodsecurity3 + the ngx_http_modsecurity_module connector).

Step 1 — Install Build Dependencies

sudo apt install -y build-essential libpcre3 libpcre3-dev \
    libssl-dev libtool autoconf libxml2-dev libgeoip-dev \
    libcurl4-openssl-dev libpcre++-dev libyajl-dev git

Step 2 — Compile libmodsecurity3

cd /opt
sudo git clone --depth 1 -b v3/master https://github.com/SpiderLabs/ModSecurity
cd ModSecurity
sudo git submodule init
sudo git submodule update
sudo ./build.sh
sudo ./configure
sudo make
sudo make install

Compilation typically takes 5–15 minutes depending on server resources. On a CLIQHOST NVMe VPS, the fast disk I/O speeds up the build noticeably.

Step 3 — Download the Nginx Connector for ModSecurity

Check your Nginx version:

nginx -v
# Example: nginx version: nginx/1.24.0

Download the matching Nginx source:

cd /opt
NGINX_VERSION=$(nginx -v 2>&1 | sed 's/.*nginx\///' | tr -d '\n')
wget http://nginx.org/download/nginx-${NGINX_VERSION}.tar.gz
tar -xvzf nginx-${NGINX_VERSION}.tar.gz

Clone the connector:

sudo git clone --depth 1 https://github.com/SpiderLabs/ModSecurity-nginx.git

Compile only the dynamic module:

cd nginx-${NGINX_VERSION}
sudo ./configure --with-compat --add-dynamic-module=../ModSecurity-nginx
sudo make modules
sudo cp objs/ngx_http_modsecurity_module.so /etc/nginx/modules/

Step 4 — Load the Module in Nginx

Add at the very top of /etc/nginx/nginx.conf:

load_module modules/ngx_http_modsecurity_module.so;

Set up the ModSecurity configuration directory:

sudo mkdir /etc/nginx/modsecurity
sudo cp /opt/ModSecurity/modsecurity.conf-recommended /etc/nginx/modsecurity/modsecurity.conf
sudo cp /opt/ModSecurity/unicode.mapping /etc/nginx/modsecurity/

Enable the rule engine:

sudo sed -i 's/SecRuleEngine DetectionOnly/SecRuleEngine On/' /etc/nginx/modsecurity/modsecurity.conf

Step 5 — Install OWASP CRS for Nginx

sudo git clone https://github.com/coreruleset/coreruleset.git /etc/nginx/modsecurity/crs
sudo cp /etc/nginx/modsecurity/crs/crs-setup.conf.example /etc/nginx/modsecurity/crs/crs-setup.conf

Create the main include file:

sudo nano /etc/nginx/modsecurity/main.conf

Content:

Include /etc/nginx/modsecurity/modsecurity.conf
Include /etc/nginx/modsecurity/crs/crs-setup.conf
Include /etc/nginx/modsecurity/crs/rules/*.conf

Step 6 — Enable ModSecurity in Your Nginx Virtual Host

sudo nano /etc/nginx/sites-available/example.com

Add inside the server block:

server {
    listen 80;
    server_name example.com www.example.com;

    modsecurity on;
    modsecurity_rules_file /etc/nginx/modsecurity/main.conf;

    location / {
        proxy_pass http://127.0.0.1:8080;
    }
}

Test and reload Nginx:

sudo nginx -t
sudo systemctl reload nginx

Testing ModSecurity

The easiest way to verify ModSecurity is working is to simulate a SQL injection attack:

curl -I "http://example.com/?id=1%20UNION%20SELECT%201,2,3--"

If ModSecurity is active, the response will be 403 Forbidden.

Check the logs for confirmation:

# Apache
sudo tail -f /var/log/apache2/error.log | grep ModSecurity

# Nginx
sudo tail -f /var/log/nginx/error.log | grep ModSecurity

A typical blocked-attack log entry looks like this:

ModSecurity: Access denied with code 403 (phase 2).
Matched "Operator `DetectSQLi' with parameter `' at REQUEST_URI.
[id "942100"] [msg "SQL Injection Attack Detected via libinjection"]

Handling False Positives

OWASP CRS can occasionally block legitimate requests. The recommended approach:

Disabling a Specific Rule (Apache)

<LocationMatch "/wp-admin">
    SecRuleRemoveById 942100
</LocationMatch>

Disabling a Specific Rule (Nginx)

Create an exceptions file:

sudo nano /etc/nginx/modsecurity/exceptions.conf

Content:

SecRuleRemoveById 942100

Include it in main.conf before the CRS rules.

Tuning the CRS Paranoia Level

In crs-setup.conf, adjust paranoia_level:

SecAction \
  "id:900000,\
   phase:1,\
   nolog,\
   pass,\
   t:none,\
   setvar:tx.paranoia_level=1"

Level 1 is the most permissive; level 4 is the strictest. Start with level 1–2 and increase gradually.

Automatic OWASP CRS Updates

Keep your ruleset current with a weekly cron job:

sudo crontab -e

Add:

0 3 * * 1 cd /etc/nginx/modsecurity/crs && git pull && systemctl reload nginx

Advanced Logging

Configure detailed audit logging in modsecurity.conf:

SecAuditEngine RelevantOnly
SecAuditLog /var/log/modsecurity/audit.log
SecAuditLogParts ABIJDEFHZ
SecAuditLogType Serial

Create the log directory:

sudo mkdir /var/log/modsecurity
sudo chown www-data:www-data /var/log/modsecurity

For a complete security stack, you can integrate ModSecurity logs with Fail2Ban to automatically ban repeat offenders. Find more server security guides on the CLIQHOST blog.

ModSecurity with Managed Hosting

If you'd rather delegate the setup and ongoing maintenance, CLIQHOST server management services include WAF installation, configuration, and tuning tailored to your specific application.

For larger projects requiring dedicated resources and top-tier performance, a CLIQHOST managed dedicated server gives you both the horsepower and the hands-on technical support.

Don't forget to pair ModSecurity with a valid SSL certificate from CLIQHOST — encrypted traffic + WAF inspection is the gold standard for web security.

Quick Reference — Key Steps

Step	Apache	Nginx
1	`apt install libapache2-mod-security2`	Compile `libmodsecurity3`
2	`a2enmod security2`	Compile Nginx dynamic module
3	Copy `modsecurity.conf`	Copy `modsecurity.conf`
4	Clone OWASP CRS	Clone OWASP CRS
5	`SecRuleEngine On`	`SecRuleEngine On`
6	Restart Apache	`modsecurity on` in vhost + reload

Conclusion

ModSecurity with the OWASP Core Rule Set is a powerful, free, and production-proven WAF solution for any Linux web server. Whether you run Apache or Nginx, setup takes under an hour and adds a meaningful layer of protection against the most prevalent web attacks.

Pairing ModSecurity with a fast CLIQHOST NVMe VPS and a properly configured SSL certificate gives your web infrastructure a solid, security-first foundation.

Need help getting your server secured and optimized? Contact the CLIQHOST team — we're here to help.

How to Install and Configure ModSecurity with Nginx and Apache on a Linux VPS: Complete Guide — CLIQHOST