How to setup log rotate in Linux to avoid running out of the disk space 👋

Quick summary ↬  Setting up log rotate and monitoring the files is mandatory to avoid running out of the disk space. Log rotate is a linux service that helps us to safeguard from running out of the disk space.Logs from web servers and many different software applications are the reason why we run out of the disk space. When we deploy software applications having an observability layer is crucial to understand what is happening in the application but at an expense of disk storage. In this article we explore the ways log rotate can help us is rotating the logs we won’t run out of the disk space.

Description

Most of the linux services generate logs unless we configured it in a way that it won’t generate logs forever. That is not an ideal solution with any of the production systems. Without rotation there is a high chance the system will run out of space. The command to check the size of the files in Linux is ls -lh. All the linux service logs are stored in the /var/log/ folder space. Now let’s see how log rotate can help to rotate the logs.

What does the logrotate do?

Logrotate helps us to manage logs and not occupy a huge disk space by user-defined configs. It will just store all the historical logs for a prescribed time period and then delete it. All the old archive logs will be deleted based on the config. We could modify the config at any time based on our prediction on how quick logs may get accumulated from a Linux service. Since most of the logs are stored in the /var/log/ we could use the command du -h /var/log/ | sort -hr to see the size of the whole directory and also individual files.

Any Linux machine that hosts a web server will generate lots of logs because of the connection requests it recieves from the outer world. Log rotate has many functionalities that helps us to manage these logs in an efficient manner.

Installing Logrotate

Many Linux distros comes with an installed version of Logrotate and services that generated lots of log files. First thing to do is to check whether logrotate is available in the Linux version or not using the below command: which logrotate .

If there is no logrotate available in the system then install it using the below command in any version of Ubuntu:

sudo apt-get install logrotate

Use Yum incase of CentOS, brew incase of MAC OS, etc.

How Logrotate works?

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

/var/log/nginx/*.log {
        weekly
        missingok
        rotate 52
        compress
        delaycompress
        notifempty
        create 0640 www-data adm
        sharedscripts
        prerotate
                if [ -d /etc/logrotate.d/httpd-prerotate ]; then 
                        run-parts /etc/logrotate.d/httpd-prerotate; 
                fi 
        endscript
        postrotate
                invoke-rc.d nginx rotate >/dev/null 2>&1
        endscript
}

In the above example, the first line tells the logrotate service which log file to look for and apply this configuration. * represents all the files with the extension log. Now let us closely look into all the configuration parameters and find what functionality it provides us.

Rotation interval

The rotation interval defines how often logrotate wants to rotate the logs. Logrotate has pre-defined set of values: daily,weekly, monthly,yearly. If the rotation interval is not set then the logs are rotated whenever logrotate runs if the time interval is also not in the configuration.

We could set the time interval by creating a file in /etc/cron.hourly. Not much detail here as this is a never ending topic to dig in and learn.