Implementing log rotate in Go

#log #go #logs

Tools

lumberjeack is the primary tool we will be using
(optional)zerolog package providing logger interface

Implementation with log package

Normal log package is okay if you're looking for something simple yet effective.

We create multiwriter io so that we can see output in the stdout and at the same time have the writer write to .log file.

As you can see we created the file writer with the lumberjack package where we defined a few variables.

Filename - takes the full path. It uses -lumberjack.log in os.TempDir() if you do not set it.
MaxSize - maximum size in megabytes of the log file before it gets rotated
MaxBackups - maximum number of old log files to retain
MaxAge - maximum number of days to retain old log files based on the timestamp encoded in their filename
Compress - determines if we compress the rotated log files with gzip

package logger

import (
    "fmt"
    "io"
    "log"
    "os"
    "path/filepath"

    "gopkg.in/natefinch/lumberjack.v2"
)

func New(logPath string) (*log.Logger, error) {
    if err := os.MkdirAll(filepath.Dir(logPath), 0755); err != nil {
        return nil, fmt.Errorf("create log directory: %w", err)
    }

    fileWriter := &lumberjack.Logger{
        Filename:   logPath,
        MaxSize:    10,   // megabytes
        MaxBackups: 5,    // number of backups to keep
        MaxAge:     30,   // days
        Compress:   true, // compress old logs
    }

    // Create a multi-writer that writes to both stdout and the file
    var writers []io.Writer
    writers = append(writers, os.Stdout)
    writers = append(writers, fileWriter)

    multiWriter := io.MultiWriter(writers...)

    logger := log.New(multiWriter, "", log.LstdFlags|log.Lshortfile)

    return logger, nil
}

Implementation with zerolog package

Same implementation but more powerful - here we can defined if we're debugging the application and change the level that zerolog is sending the logs through. In this example we have base level of info and debug level (if active) of trace.

package logger

import (
    "fmt"
    "os"
    "path/filepath"

    "github.com/rs/zerolog"
    "gopkg.in/natefinch/lumberjack.v2"
)

func New(isDebug bool, logPath string) (*zerolog.Logger, error) {
    logLevel := zerolog.InfoLevel
    if isDebug {
        logLevel = zerolog.TraceLevel
    }

    if err := os.MkdirAll(filepath.Dir(logPath), 0755); err != nil {
        return nil, fmt.Errorf("create log directory: %w", err)
    }

    fileWriter := &lumberjack.Logger{
        Filename:   logPath,
        MaxSize:    10,   // megabytes
        MaxBackups: 5,    // number of backups to keep
        MaxAge:     30,   // days
        Compress:   true, // compress old logs
    }

    zerolog.SetGlobalLevel(logLevel)
    multi := zerolog.MultiLevelWriter(zerolog.ConsoleWriter{Out: os.Stdout}, fileWriter)
    logger := zerolog.New(multi).With().Timestamp().Logger()

    return &logger, nil
}