Elastic Beats — how to optimize Beats self-logging

Most of the Elastic Stack users use the Beats as data agents to ship data from different sources.

This is a sample list of Beats inputs that I use in my work:
Metricbeat — collect metrics (servers, databases, etc.)
Filebeat — collect logs or any text data from the files
Packetbeat — collect captured network traffic
Heartbeat — collect keepalive signals from applications
This is not a complete list of beats inputs Elastic provides, but these are the most popular ones.

The Beats installation is pretty straightforward. The one issue that remains unclear: what we are supposed to do with their own logs (and an even better question: are we supposed to do something with them or can we leave them as is)?

In the default configuration, each beat writes the logs to the /var/log/messages, and part of them is self-monitoring logs.
IMHO This is not so user-friendly for reading and sometimes can fill up the system disk with unnecessary data.

In my work, I use a lot of Beats, and as part of the installation process, I consolidate the configuration, that defined the log’s destination to the dedicated folder and disables the self-metrics.
This configuration helps me to clean the /var/log/message file and provides easier access to beats specific logs while troubleshooting
The complete logging guide can be found here:
https://www.elastic.co/guide/en/beats/metricbeat/current/configuration-logging.html
This is the user guide for the Metricbeat, but you can use it for all Beats
This is the configuration, that you’ll need to insert into the Beat configuration file.

For example for the Metricbeat you’ll need to update the metricbeat.yml file, with the following configuration:

#Disable self-metrics logs
logging.metrics.enabled: false
#Log level
logging.level: info
#Write logs to files
logging.to_files: true
#Log files configuration
logging.files:
  #Path to the log folder
  path: <LOGS-FOLDER-PATH>
  #Log file name prefix
  name: metricbeat
  #Number of files
  keepfiles: 7
  #Permissions for logs files
  permissions: 0644
#Don't write all logging output to standard error output
logging.to_stderr: false
#Don't write logs to the syslog output
logging.to_syslog: false

After the service starts you'll see the new files: /my_logs/metricbeat-<DATE>.ndjson
This file will contain the Metricbeat logs.
For other Beats, you’ll need to update their configuration files: filebeat.yml, packetbeat.yml, etc. There is only one difference in the configuration: logging.file.name, which is supposed to be like Beat’s name.
Hope this will help you to optimize and simplify your Beats usage.

That's All Folks:)
Have a good Beats usage!