3 Ways to Watch Logs in Real Time in Linux

You know how to view files in Linux. You use cat command or probably less command for this purpose.

That's good for files that has static content. But log files are dynamic and their content change with time. To monitor logs, you need to watch the log file as its content changes.

How do you see the content of log files in real time? Tail is the most popular command for this purpose but there are some other tools as well. I'll show them to you in this tutorial.

Method 1: Watch log files with the tail command

The tail command is so popular for viewing log files in real life that sysadmins use the term 'tail the log file'.

The tail command is essentially used for showing the lines of a file from the end and hence the term 'tail'.

You can use the -f option to follow the tail of a file, which means that it will keep on showing the new lines added to the file continuously.

tail -f location_of_log_file

To stop the tailing of the log file, use Ctrl+C terminal shortcut.

Tail and grep

Alright! So the tail command solves a problem by showing the file changes in real life. But watching the log file continuously when there are so many rapid changes happening in real time is not very helpful.

You'll often be looking for a particular term when monitoring the log file. Finding that in the flood of incoming new lines is close to impossible.

To make things easier, combine the tail and grep command like this:

tail -f log_file | grep search_term

This is good, right? Let's make it a bit better.

I have often found that just the lines with searched terms don't reveal the necessary details. This is why I use grep command to show a few lines before and after the searched term with option -C.

tail -f log_file | grep -C 3 search_term

Now, you'll see the lines matching the search term along with 3 lines before and after it. This will give a better perspective on what's happening.

Want to make it even better? You can grep on multiple search term and even make it a case insensitive search:

tail -f log_file | grep -C 3 -i - E 'search_term_1|search_term_2'

Tailing the file with log rotation

If you are working on an enterprise server, chances are that logs are rotated. This means that after the current log file reaches a certain size, it is renamed and zipped.

That creates a problem if you are tailing a log file in real time. By default, the tail command works on the file descriptor. If the current log file is rotated, tail command will now be pointing to an archived log file which will not be recording any change now.

The solution is to follow a log file by its name. This way, even when log rotation takes place, the tail will be pointing to the current log file (because its name never changes).

tail --follow=name log_file | grep -C 3 -i - E 'search_term_1|search_term_2'

This is a lot better now. The next time you tail a log file, use it this way to monitor it more effectively.

Tail is nice for monitoring a log file in real time but what if you have to analyze more than one log files at the same time? The answer lies in the next section.

Watching multiple log files with tail

This should work in Linux systems. You can monitor multiple log files at the same time with the tail command. Just provide the path of the file in this manner:

tail -f log_file_1 -f log_file_2

You'll see that it starts showing the real time changes along with the file name before it so that you can distinguish between different log sources.

There is a slightly better way to view multiple log files at once using a utility called multitail.

Method 2: Monitor multiple log files at once with multitail

Multitail, as the name suggests, is used to display multiple files at once.

What's the big deal? The tail command can also do the same, right?

But Multitail has some advantage over the conventional tail command. It shows the files in split views and you can even show different files in different rows and columns.

Remember, tail shows everything in the same view and that becomes difficult to follow. Multitail overcomes this difficulty by providing split view like the screen command.

You can provide several files to it but I think more than 3 files would be difficult to follow at a time.

multitail log_file_1 log_file_2

By default, multitail works the same as tail -f. It shows the last 100 lines and then goes in the real time view. By default, it splits the view in rows.

You can press b to open a selector window and select log file of your choice to view it and scroll through it for further and deeper analysis.

Press q to exit from all kind of views in multitail.

You can split the views in columns like this:

multitail -s 2 log_file_1 log_file_2

There is a mandatory space between -s and the number of columns.

Multitail is capable of doing some other things but I won't go into those details in this tutorial.

So far, you have seen two ways of monitoring log files. There is another but less conventional way of seeing file changes in real life and that is using the less command

Method 3: View log file changes in real time with less command

The less command is more for reading text files without cluttering the screen. It can also be used for reading files with real time changes.

The option +F allows less to follow the changes made to a text file.

less +F log_file

It opens the log files with changes being written to it displayed in real time.

Press Ctrl+c to interrupt and q to exit the view.

This method allows you to have a quick view of log changes without cluttering the screen, unlike the tail command.

Conclusion

This method of monitoring log files in Linux works for the traditional text based log files. For the system logs, syslogs are still there but many Linux distributions have switched to journal logs and to view and analyze the journal logs, you have to use journalctl commands.

If you have full access to the Linux system, you may want to use the Rust-based CLI tool tailspin that highlights the log file elements.

Apart from that, there are other sophisticated tools like Graylog for log analysis on a deeper level with dashboards and graphs. More on that in some other article.

I hope you learned a couple of new things from this seemingly easy topic on real time log monitoring in Linux. Your feedback is welcome.