Logrotate for a Web Server: overview and examples

It’s always important to keep your server logs around for as long as it makes business sense. These may be required for auditing system access, discovering abuses, or identifying the root causes of problems, among other reasons.

The challenge, though, is that depending on the service being provided and the amount of traffic received, your logs are capable of growing to gargantuan sizes, consuming every last bit of disk space available.

Logrotate allows us to better manage our logs to prevent them from consuming too much disk space.

Installation

Logrotate is installed by default on most recent distributions, so there is nothing to be done other than using it.

Create a Log Rotate Configuration

Logrotate configuration files are stored under /etc/logrotate.d

/var/www/mysite/log/*.log {
        su www-data www-data
        daily
        missingok
        rotate 8
        compress
        notifempty
        create 0640 www-data www-data
        sharedscripts
        postrotate
            sudo service apache2 reload
        endscript
    }

Test Your Configuration

As with anything, before you roll your configuration into production you will want to test it to ensure everything works.

Logrotate includes a feature that allows us to run a configuration, in debug mode, without it doing any work. If there are errors discovered in the syntax or some other issues, you will be notified.

sudo logrotate -d /etc/logrotate.d/mysite.conf

Depending on how long your web server has been running, the output may vary. Below is an example for a new web server instance.

empty log files are not rotated, old logs are removed
    considering log /var/www/mysite/log/access_log
      log does not need rotating
    considering log /var/www/mysite/log/error_log
      log does not need rotating
    not running postrotate script, since no logs were rotated

None of the logs were rotated since I ran the test on a server that was newly built. However, if the logs had more content and were aged a little more, we would see a message indicating our logs were rotated. For now, we can take this as confirmation that our configuration file syntax is correct.

Run your configuration

Whether you are ready to run it for the first time or execute a one-off, you can run your configuration file with the following command.

sudo logrotate --force /etc/logrotate.d/mysite.conf

Hey, I'm getting an error. What am I doing wrong?

Before you start panicking, do consider the following most common mistakes:

1. Ensure no clashing instructions exist in your configuration file. For instance, setting both size and daily at the same time will make the script fail.
2. Ensure the file in /etc/logrotate.d/mysite.conf is owned by the root user. To do so, use sudo chown root:root /etc/logrotate.d/mysite.conf.
3. Ensure the file in /etc/logrotate.d/mysite.conf has at least 644 permissions. Again, you can change it with sudo chmod 644 /etc/logrotate.d/mysite.conf.
4. If you save your log files (as I do) in the web server folder — so I can later SSH on to them — ensure your files are managed with the user permissions of your destination folder. For example, my log files are owned by www-data:www-data, so I add su www-data www-data to /etc/logrotate.d/mysite.conf.