Configuring NGINX to log HTTP POST data on Linux / RHEL

Configuring NGINX to log HTTP POST data on linux /RHEL

Posted byBachir Chihani on May 23, 2016May 23, 2016

NGINX is a powerful web server that can easily handle high volumesof HTTP traffic. Each time NGINX handles a connection, a log entry is generated to store some information thisconnection likeremote IP address, response size and status code, etc. The complete set of logged information with more details can be found here .

In some cases, you may be more interested in storing the body of requests, specifically POST requests. Lucky, the NGINX ecosystem is rich, and includesquite a few handy modules . One suchmodule is the Echo module ,which provides useful functionality for things like: echo , time , and sleep commands.

In our use case, to log a request body, what we need is to use the echo_read_request_body command and the $request_body variable (contains the request body of the Echo module). However, this module is not distributed with NGINX by default, and to be able to use it we have to build NGINX from itssource code by building it withthe source code of the Echo module included.

The following steps detail how to build NGINX so thatthe Echo module is included (here is the complete build bash file ):

Download the source codes for NGINX and Echo using the following commands:

curl -L -O 'https://github.com/openresty/echo-nginx-module/archive/v0.58.tar.gz' tar -xzvf v0.58.tar.gz && rm v0.58.tar.gz mv echo-nginx-module-0.58 /tmp/echo-nginx-module curl -O 'http://nginx.org/download/nginx-1.9.7.tar.gz' tar -xzvf nginx-1.9.7.tar.gz && rm nginx-1.9.7.tar.gz

Create an nginx user, which we willusetorun the NGINXprocess:

groupadd nginx useradd -G nginx nginx

Install NGINXfrom source code:

cd nginx-1.9.7/ && ./configure \ --user=nginx \ --group=nginx \ --prefix=/etc/nginx \ --sbin-path=/usr/sbin/nginx \ --conf-path=/etc/nginx/nginx.conf \ --pid-path=/var/run/nginx.pid \ --lock-path=/var/run/nginx.lock \ --error-log-path=/var/log/nginx/error.log \ --http-log-path=/var/log/nginx/access.log \ --with-http_gzip_static_module \ --with-http_stub_status_module \ --with-http_ssl_module \ --with-pcre \ --with-file-aio \ --with-http_realip_module \ --add-module=/tmp/echo-nginx-module make -j2 make install

Now NGINX is ready to start, but before doing sowe need to modify the default NGINX configuration file under /etc/nginx/nginx.conf to enable logging of the HTTP requestbody.

NGINX writes quite a bitof information about incoming requests to the location indicated in the access_log directive. What is and is not logged in each log event can be customized in the log_format directive. Further, the Echo module stores the request body in the request_body variable by calling the echo_read_request_body method. We have to modify these directivesas follows (here is the complete nginx.conf ):

http { ... log_format custom '$request_body'; access_log /var/log/nginx/access.log custom; ... server { location / { echo_read_request_body; ... } } }

If you are in a high-throughput environment, you may need to increase the allowed limit of linux open files, for user nginx in order to be able to handle thousands of requests per second. You can do so by using the following commands to append this setting to your configurations (you can also edit the files directly):

echo "fs.file-max = 1073741824" >> /etc/sysctl.conf echo "nginx soft nofile 40960" >> /etc/security/limits.conf echo "nginx hard nofile 81920" >> /etc/security/limits.conf

Now, we are ready to start NGINX and log incoming HTTP requests’ body. Run NGINX:

nginx

I hope this helps give an understanding of how to control some of the NGINX logging functionality, and how to build, install, and utilize the Echo module. The complete script and details can be found in this gist .

JoinRed Hat Developers, a developer program for you to learn, share, and code faster and get access to Red Hat software for your development. The developer program and software are both free!

Take advantage of your Red Hat Developers membership anddownload RHEL today at no cost.