Archiving Logs


Using the log archiving service provided by cluster management, the logs collected by cluster can be quickly archived to the specified storage in a specific format. When needed, the archived logs can be restored to the environment for query and analysis just like ordinary logs.

Prerequisite

  • The user must have the authority to use cluster management (administrator, cluster administrator, or namespace administrator).
  • Organization and namespace name are selected.
  • The storage required for log archiving has been initialized.

Note

The log archive service is under the cluster management module. You need to switch to Cluster Management in the upper left corner of the page menu to access it.

../../../_images/cluster_log_archive.png

Creating Repository

The steps to create the repository are as below:

  1. Select Log > Log Archiving to view all the archived Log files from the left navigation bar.

  2. Under the Repository tab, click New Repository, enter the repository name, and select the repository type.

  3. According to the selected repository type, complete the repository address information configuration.

    ../../../_images/creating_log_repo1.png

  4. Click Confirm button to complete the repository creation.

Creating Archiving Policy

After completed create the repository, you can create an archiving policy. The steps to create the archiving policy are as below:

  1. Under the Archiving Policy tab, click New Policy to complete the archiving policy configuration in a new window:

    • Policy Name : Enter the Archiving Policy Name.
    • File prefix : Enter the prefix of the generated archive file.
    • Repository : Select the name of the created Repository.
    • Index : Log index, by default all indexes in the cluster will be archived; If all indexes are turned off, you can select the indexes that you want to archive. The index types are described below.
    • Storage Time : The Storage Time of the archive file in the system. After the expiration, the archive will be automatically deleted by the system.
    • Frequency : Select the frequency at which to run the log archive task.
    ../../../_images/log_archiving_policy1.png

  2. Click Confirm button to submit the policy. The system triggers the archiving task according to the policy.

Index Type Description

Index Type Description
kublog-app-* Application type log, can be queried in the Log Center after restoring the archived content.
kublog-serverless-* Serverless type log, can be queried in the Log Center after restoring the archived content.
kublog-spark-* After archiving, cannot be searched in the Log Center, but you can search through the interface.
kublog-kube-eventer-* After archiving, cannot be searched in the Log Center, but you can search through the interface.
kublog-job-* After archiving, cannot be searched in the Log Center, but you can search through the interface.
kublog-kube-system-* After archiving, cannot be searched in the Log Center, but you can search through the interface.

Manage Archived Files

When both the repository and the archive policy are in effect, the system will generate archive files according to the rules defined in the policy. You can view the generated results in the archive file list.


../../../_images/archived_log_files1.gif


The generated archive file contains the log index at the archive time node, and you can perform operations on it, such as actively deleting or restoring the archive file before the file storage time expires.

Note

Log archive, restore, and delete operations will be executed serially in the background after being triggered. If operations such as log archiving and restoration are performed at the same time, the performance of the log service will be affected. Please operate with caution.

Manage Archive Restoring

Switch to the Archive Restoring tab to view the restoration status of the archive file. You can filter the information by archive file name, database name, time range, and recovery status. You can also delete the restored archived files from the cluster.

../../../_images/archive_restoration1.gif