4
0
mirror of https://github.com/cwinfo/matterbridge.git synced 2025-07-05 14:04:02 +00:00

Update mattermost library (#2152)

* Update mattermost library

* Fix linting
This commit is contained in:
Wim
2024-05-24 23:08:09 +02:00
committed by GitHub
parent 65d78e38af
commit d16645c952
1003 changed files with 89451 additions and 114025 deletions

View File

@ -1,4 +0,0 @@
language: go
sudo: false
go:
- 1.x

View File

@ -1,23 +1,27 @@
![Logr_Logo](https://user-images.githubusercontent.com/7295363/200433587-ae9df127-9427-4753-a0a0-85723a216e0e.png)
> A fully asynchronous, contextual logger for Go.
# logr
[![GoDoc](https://godoc.org/github.com/mattermost/logr?status.svg)](http://godoc.org/github.com/mattermost/logr)
[![Report Card](https://goreportcard.com/badge/github.com/mattermost/logr)](https://goreportcard.com/report/github.com/mattermost/logr)
Logr is a fully asynchronous, contextual logger for Go.
It is very much inspired by [Logrus](https://github.com/sirupsen/logrus) but addresses two issues:
Logr is inspired by [Logrus](https://github.com/sirupsen/logrus) and [Zap](https://github.com/uber-go/zap) but addresses a number of issues:
1. Logr is fully asynchronous, meaning that all formatting and writing is done in the background. Latency sensitive applications benefit from not waiting for logging to complete.
2. Logr provides custom filters which provide more flexibility than Trace, Debug, Info... levels. If you need to temporarily increase verbosity of logging while tracking down a problem you can avoid the fire-hose that typically comes from Debug or Trace by using custom filters.
3. Logr generates much less allocations than Logrus, and is close to Zap in allocations.
## Concepts
<!-- markdownlint-disable MD033 -->
| entity | description |
| ------ | ----------- |
| Logr | Engine instance typically instantiated once; used to configure logging.<br>```lgr,_ := logr.New()```|
| Logger | Provides contextual logging via fields; lightweight, can be created once and accessed globally or create on demand.<br>```logger := lgr.NewLogger()```<br>```logger2 := logger.WithField("user", "Sam")```|
| Logger | Provides contextual logging via fields; lightweight, can be created once and accessed globally, or created on demand.<br>```logger := lgr.NewLogger()```<br>```logger2 := logger.With(logr.String("user", "Sam"))```|
| Target | A destination for log items such as console, file, database or just about anything that can be written to. Each target has its own filter/level and formatter, and any number of targets can be added to a Logr. Targets for file, syslog and any io.Writer are built-in and it is easy to create your own. You can also use any [Logrus hooks](https://github.com/sirupsen/logrus/wiki/Hooks) via a simple [adapter](https://github.com/wiggin77/logrus4logr).|
| Filter | Determines which logging calls get written versus filtered out. Also determines which logging calls generate a stack trace.<br>```filter := &logr.StdFilter{Lvl: logr.Warn, Stacktrace: logr.Fatal}```|
| Formatter | Formats the output. Logr includes built-in formatters for JSON and plain text with delimiters. It is easy to create your own formatters or you can also use any [Logrus formatters](https://github.com/sirupsen/logrus#formatters) via a simple [adapter](https://github.com/wiggin77/logrus4logr).<br>```formatter := &format.Plain{Delim: " \| "}```|
@ -39,7 +43,7 @@ lgr.AddTarget(t)
// One or more Loggers can be created, shared, used concurrently,
// or created on demand.
logger := lgr.NewLogger().WithField("user", "Sarah")
logger := lgr.NewLogger().With("user", "Sarah")
// Now we can log to the target(s).
logger.Debug("login attempt")
@ -53,22 +57,22 @@ lgr.Shutdown()
Fields allow for contextual logging, meaning information can be added to log statements without changing the statements themselves. Information can be shared across multiple logging statements thus allowing log analysis tools to group them.
Fields are added via Loggers:
Fields can be added to a Logger via `Logger.With` or included with each log record:
```go
lgr,_ := logr.New()
// ... add targets ...
logger := lgr.NewLogger().WithFields(logr.Fields{
"user": user,
"role": role})
logger.Info("login attempt")
logger := lgr.NewLogger().With(
logr.Any("user": user),
logr.String("role", role)
)
logger.Info("login attempt", logr.Int("attempt_count", count))
// ... later ...
logger.Info("login successful")
logger.Info("login", logr.String("result", result))
```
`Logger.WithFields` can be used to create additional Loggers that add more fields.
Logr fields are inspired by and work the same as [Logrus fields](https://github.com/sirupsen/logrus#fields).
Logr fields are inspired by and work the same as [Zap fields](https://pkg.go.dev/go.uber.org/zap#Field).
## Filters
@ -97,21 +101,21 @@ Logr also supports custom filters (logr.CustomFilter) which allow fine grained i
formatter := &formatters.Plain{Delim: " | "}
tgr := targets.NewWriterTarget(filter, formatter, os.StdOut, 1000)
lgr.AddTarget(tgr)
logger := lgr.NewLogger().WithFields(logr.Fields{"user": "Bob", "role": "admin"})
logger := lgr.NewLogger().With(logr.String("user": "Bob"), logr.String("role": "admin"))
logger.Log(LoginLevel, "this item will get logged")
logger.Debug("won't be logged since Debug wasn't added to custom filter")
```
Both filter types allow you to determine which levels require a stack trace to be output. Note that generating stack traces cannot happen fully asynchronously and thus add latency to the calling goroutine.
Both filter types allow you to determine which levels force a stack trace to be output. Note that generating stack traces cannot happen fully asynchronously and thus add some latency to the calling goroutine.
## Targets
There are built-in targets for outputting to syslog, file, or any `io.Writer`. More will be added.
There are built-in targets for outputting to syslog, file, TCP, or any `io.Writer`. More will be added.
You can use any [Logrus hooks](https://github.com/sirupsen/logrus/wiki/Hooks) via a simple [adapter](https://github.com/wiggin77/logrus4logr).
You can create your own target by implementing the [Target](./target.go) interface.
You can create your own target by implementing the simple [Target](./target.go) interface.
Example target that outputs to `io.Writer`:
@ -130,7 +134,7 @@ func (w *Writer) Init() error {
return nil
}
// Write will always be called by a single goroutine, so no locking needed.
// Write will always be called by a single internal Logr goroutine, so no locking needed.
func (w *Writer) Write(p []byte, rec *logr.LogRec) (int, error) {
return w.out.Write(buf.Bytes())
}
@ -153,9 +157,18 @@ You can create your own formatter by implementing the [Formatter](./formatter.go
Format(rec *LogRec, stacktrace bool, buf *bytes.Buffer) (*bytes.Buffer, error)
```
## Handlers
## Configuration options
When creating the Logr instance, you can add several handlers that get called when exceptional events occur:
When creating the Logr instance, you can set configuration options. For example:
```go
lgr, err := logr.New(
logr.MaxQueueSize(1000),
logr.StackFilter("mypackage1", "mypackage2"),
)
```
Some options are documented below. See [options.go](./options.go) for all available configuration options.
### ```Logr.OnLoggerError(err error)```
@ -168,7 +181,7 @@ It may be tempting to log this error, however there is a danger that logging thi
Called on an attempt to add a log record to a full Logr queue. This generally means the Logr maximum queue size is too small, or at least one target is very slow. Logr maximum queue size can be changed before adding any targets via:
```go
lgr := logr.Logr{MaxQueueSize: 10000}
lgr, err := logr.New(logr.MaxQueueSize(2000))
```
Returning true will drop the log record. False will block until the log record can be added, which creates a natural throttle at the expense of latency for the calling goroutine. The default is to block.
@ -186,3 +199,7 @@ OnExit and OnPanic are called when the Logger.FatalXXX and Logger.PanicXXX funct
In both cases the default behavior is to shut down gracefully, draining all targets, and calling `os.Exit` or `panic` respectively.
When adding your own handlers, be sure to call `Logr.Shutdown` before exiting the application to avoid losing log records.
### ```Logr.StackFilter(pkg ...string)```
StackFilter sets a list of package names to exclude from the top of stack traces. The `Logr` packages are automatically filtered.

View File

@ -157,7 +157,7 @@ func newTarget(targetType string, options json.RawMessage, factory TargetFactory
return t, nil
}
}
return nil, fmt.Errorf("target type '%s' is unrecogized", targetType)
return nil, fmt.Errorf("target type '%s' is unrecognized", targetType)
}
func newFormatter(format string, options json.RawMessage, factory FormatterFactory) (logr.Formatter, error) {
@ -205,5 +205,5 @@ func newFormatter(format string, options json.RawMessage, factory FormatterFacto
return f, nil
}
}
return nil, fmt.Errorf("format '%s' is unrecogized", format)
return nil, fmt.Errorf("format '%s' is unrecognized", format)
}

View File

@ -31,4 +31,7 @@ const (
// DefaultMaxPooledBuffer is the maximum size a pooled buffer can be.
// Buffers that grow beyond this size are garbage collected.
DefaultMaxPooledBuffer = 1024 * 1024
// DefaultMaxFieldLength is the maximum size of a String or fmt.Stringer field can be.
DefaultMaxFieldLength = -1
)

View File

@ -187,16 +187,22 @@ func (f Field) ValueString(w io.Writer, shouldQuote func(s string) bool) error {
break arr
}
}
if _, err = w.Write(Comma); err != nil {
break arr
if i != a.Len()-1 {
if _, err = w.Write(Comma); err != nil {
break arr
}
}
}
case MapType:
a := reflect.ValueOf(f.Interface)
iter := a.MapRange()
// Already advance to first element
if !iter.Next() {
return nil
}
it:
for iter.Next() {
for {
if _, err = io.WriteString(w, iter.Key().String()); err != nil {
break it
}
@ -219,9 +225,15 @@ func (f Field) ValueString(w io.Writer, shouldQuote func(s string) bool) error {
break it
}
}
if !iter.Next() {
break it
}
if _, err = w.Write(Comma); err != nil {
break it
}
}
case UnknownType:
@ -269,19 +281,19 @@ func fieldForAny(key string, val interface{}) Field {
}
return Bool(key, *v)
case float64:
return Float64(key, v)
return Float(key, v)
case *float64:
if v == nil {
return nilField(key)
}
return Float64(key, *v)
return Float(key, *v)
case float32:
return Float32(key, v)
return Float(key, v)
case *float32:
if v == nil {
return nilField(key)
}
return Float32(key, *v)
return Float(key, *v)
case int:
return Int(key, v)
case *int:
@ -290,33 +302,33 @@ func fieldForAny(key string, val interface{}) Field {
}
return Int(key, *v)
case int64:
return Int64(key, v)
return Int(key, v)
case *int64:
if v == nil {
return nilField(key)
}
return Int64(key, *v)
return Int(key, *v)
case int32:
return Int32(key, v)
return Int(key, v)
case *int32:
if v == nil {
return nilField(key)
}
return Int32(key, *v)
return Int(key, *v)
case int16:
return Int32(key, int32(v))
return Int(key, int32(v))
case *int16:
if v == nil {
return nilField(key)
}
return Int32(key, int32(*v))
return Int(key, int32(*v))
case int8:
return Int32(key, int32(v))
return Int(key, int32(v))
case *int8:
if v == nil {
return nilField(key)
}
return Int32(key, int32(*v))
return Int(key, int32(*v))
case string:
return String(key, v)
case *string:
@ -332,33 +344,33 @@ func fieldForAny(key string, val interface{}) Field {
}
return Uint(key, *v)
case uint64:
return Uint64(key, v)
return Uint(key, v)
case *uint64:
if v == nil {
return nilField(key)
}
return Uint64(key, *v)
return Uint(key, *v)
case uint32:
return Uint32(key, v)
return Uint(key, v)
case *uint32:
if v == nil {
return nilField(key)
}
return Uint32(key, *v)
return Uint(key, *v)
case uint16:
return Uint32(key, uint32(v))
return Uint(key, uint32(v))
case *uint16:
if v == nil {
return nilField(key)
}
return Uint32(key, uint32(*v))
return Uint(key, uint32(*v))
case uint8:
return Uint32(key, uint32(v))
return Uint(key, uint32(v))
case *uint8:
if v == nil {
return nilField(key)
}
return Uint32(key, uint32(*v))
return Uint(key, uint32(*v))
case []byte:
if v == nil {
return nilField(key)

View File

@ -9,53 +9,70 @@ import (
// For best performance when passing a struct (or struct pointer),
// implement `logr.LogWriter` on the struct, otherwise reflection
// will be used to generate a string representation.
func Any(key string, val interface{}) Field {
func Any(key string, val any) Field {
return fieldForAny(key, val)
}
// Int64 constructs a field containing a key and Int64 value.
//
// Deprecated: Use [logr.Int] instead.
func Int64(key string, val int64) Field {
return Field{Key: key, Type: Int64Type, Integer: val}
}
// Int32 constructs a field containing a key and Int32 value.
//
// Deprecated: Use [logr.Int] instead.
func Int32(key string, val int32) Field {
return Field{Key: key, Type: Int32Type, Integer: int64(val)}
}
// Int constructs a field containing a key and Int value.
func Int(key string, val int) Field {
// Int constructs a field containing a key and int value.
func Int[T ~int | ~int8 | ~int16 | ~int32 | ~int64](key string, val T) Field {
return Field{Key: key, Type: IntType, Integer: int64(val)}
}
// Uint64 constructs a field containing a key and Uint64 value.
//
// Deprecated: Use [logr.Uint] instead.
func Uint64(key string, val uint64) Field {
return Field{Key: key, Type: Uint64Type, Integer: int64(val)}
}
// Uint32 constructs a field containing a key and Uint32 value.
//
// Deprecated: Use [logr.Uint] instead
func Uint32(key string, val uint32) Field {
return Field{Key: key, Type: Uint32Type, Integer: int64(val)}
}
// Uint constructs a field containing a key and Uint value.
func Uint(key string, val uint) Field {
// Uint constructs a field containing a key and uint value.
func Uint[T ~uint | ~uint8 | ~uint16 | ~uint32 | ~uint64 | ~uintptr](key string, val T) Field {
return Field{Key: key, Type: UintType, Integer: int64(val)}
}
// Float64 constructs a field containing a key and Float64 value.
//
// Deprecated: Use [logr.Float] instead
func Float64(key string, val float64) Field {
return Field{Key: key, Type: Float64Type, Float: val}
}
// Float32 constructs a field containing a key and Float32 value.
//
// Deprecated: Use [logr.Float] instead
func Float32(key string, val float32) Field {
return Field{Key: key, Type: Float32Type, Float: float64(val)}
}
// Float32 constructs a field containing a key and float value.
func Float[T ~float32 | ~float64](key string, val T) Field {
return Field{Key: key, Type: Float32Type, Float: float64(val)}
}
// String constructs a field containing a key and String value.
func String(key string, val string) Field {
return Field{Key: key, Type: StringType, String: val}
func String[T ~string | ~[]byte](key string, val T) Field {
return Field{Key: key, Type: StringType, String: string(val)}
}
// Stringer constructs a field containing a key and a `fmt.Stringer` value.
@ -75,7 +92,7 @@ func NamedErr(key string, err error) Field {
}
// Bool constructs a field containing a key and bool value.
func Bool(key string, val bool) Field {
func Bool[T ~bool](key string, val T) Field {
var b int64
if val {
b = 1
@ -100,11 +117,11 @@ func Millis(key string, val int64) Field {
}
// Array constructs a field containing a key and array value.
func Array(key string, val interface{}) Field {
func Array[S ~[]E, E any](key string, val S) Field {
return Field{Key: key, Type: ArrayType, Interface: val}
}
// Map constructs a field containing a key and map value.
func Map(key string, val interface{}) Field {
func Map[M ~map[K]V, K comparable, V any](key string, val M) Field {
return Field{Key: key, Type: MapType, Interface: val}
}

View File

@ -2,6 +2,7 @@ package logr
import (
"bytes"
"fmt"
"io"
"runtime"
"strconv"
@ -27,6 +28,31 @@ const (
TimestampMillisFormat = "Jan _2 15:04:05.000"
)
// LimitByteSlice discards the bytes from a slice that exceeds the limit
func LimitByteSlice(b []byte, limit int) []byte {
if limit > 0 && limit < len(b) {
lb := make([]byte, limit, limit+3)
copy(lb, b[:limit])
return append(lb, []byte("...")...)
}
return b
}
// LimitString discards the runes from a slice that exceeds the limit
func LimitString(b string, limit int) string {
return string(LimitByteSlice([]byte(b), limit))
}
type LimitedStringer struct {
fmt.Stringer
Limit int
}
func (ls *LimitedStringer) String() string {
return LimitString(ls.Stringer.String(), ls.Limit)
}
type Writer struct {
io.Writer
}

View File

@ -79,7 +79,7 @@ type gelfRecord struct {
func (gr gelfRecord) MarshalJSONObject(enc *gojay.Encoder) {
enc.AddStringKey(GelfVersionKey, GelfVersion)
enc.AddStringKey(GelfHostKey, gr.getHostname())
enc.AddStringKey(GelfShortKey, gr.Msg())
enc.AddStringKey(GelfShortKey, gr.safeMsg("-")) // Gelf requires a non-empty `short_message`
if gr.level.Stacktrace {
frames := gr.StackFrames()
@ -131,6 +131,15 @@ func (gr gelfRecord) IsNil() bool {
return gr.LogRec == nil
}
// safeMsg returns the log record Message field or an alternate string when msg is empty.
func (gr gelfRecord) safeMsg(alt string) string {
s := gr.Msg()
if s == "" {
s = alt
}
return s
}
func (g *Gelf) getHostname() string {
if g.Hostname != "" {
return g.Hostname

View File

@ -42,6 +42,7 @@ func New(opts ...Option) (*Logr, error) {
shutdownTimeout: DefaultShutdownTimeout,
flushTimeout: DefaultFlushTimeout,
maxPooledBuffer: DefaultMaxPooledBuffer,
maxFieldLen: DefaultMaxFieldLength,
}
lgr := &Logr{options: options}
@ -246,6 +247,29 @@ func (lgr *Logr) SetMetricsCollector(collector MetricsCollector, updateFreqMilli
// this function either blocks or the log record is dropped, depending on
// the result of calling `OnQueueFull`.
func (lgr *Logr) enqueue(rec *LogRec) {
// check if a limit has been configured
if limit := lgr.options.maxFieldLen; limit > 0 {
// we limit the message
rec.msg = LimitString(rec.msg, limit)
// then we range over fields to apply the limit
for i := range rec.fields {
switch rec.fields[i].Type {
case StringType:
rec.fields[i].String = LimitString(rec.fields[i].String, limit)
case StringerType:
if v, ok := rec.fields[i].Interface.(fmt.Stringer); ok {
rec.fields[i].Interface = &LimitedStringer{
Stringer: v,
Limit: limit,
}
}
default:
// no limits for other field types
}
}
}
select {
case lgr.in <- rec:
default:

View File

@ -23,6 +23,7 @@ type options struct {
metricsCollector MetricsCollector
metricsUpdateFreqMillis int64
stackFilter map[string]struct{}
maxFieldLen int
}
// MaxQueueSize is the maximum number of log records that can be queued.
@ -76,7 +77,7 @@ func OnTargetQueueFull(f func(target Target, rec *LogRec, maxQueueSize int) bool
}
// OnExit, when not nil, is called when a FatalXXX style log API is called.
// When nil, then the default behavior is to cleanly shut down this Logr and
// When nil, the default behavior is to cleanly shut down this Logr and
// call `os.Exit(code)`.
func OnExit(f func(code int)) Option {
return func(l *Logr) error {
@ -86,7 +87,7 @@ func OnExit(f func(code int)) Option {
}
// OnPanic, when not nil, is called when a PanicXXX style log API is called.
// When nil, then the default behavior is to cleanly shut down this Logr and
// When nil, the default behavior is to cleanly shut down this Logr and
// call `panic(err)`.
func OnPanic(f func(err interface{})) Option {
return func(l *Logr) error {
@ -190,3 +191,16 @@ func StackFilter(pkg ...string) Option {
return nil
}
}
// MaxFieldLen is the maximum number of characters for a field.
// If exceeded, remaining bytes will be discarded.
// Defaults to DefaultMaxFieldLength.
func MaxFieldLen(size int) Option {
return func(l *Logr) error {
if size < 0 {
return errors.New("size cannot be less than zero")
}
l.options.maxFieldLen = size
return nil
}
}

View File

@ -1,3 +1,4 @@
//go:build !windows && !nacl && !plan9
// +build !windows,!nacl,!plan9
package targets
@ -19,7 +20,7 @@ type Syslog struct {
// SyslogOptions provides parameters for dialing a syslog daemon.
type SyslogOptions struct {
IP string `json:"ip,omitempty"` // deprecated
IP string `json:"ip,omitempty"` // deprecated (use Host instead)
Host string `json:"host"`
Port int `json:"port"`
TLS bool `json:"tls"`
@ -55,6 +56,11 @@ func (s *Syslog) Init() error {
network := "tcp"
var config *tls.Config
host := s.params.Host
if host == "" {
host = s.params.IP
}
if s.params.TLS {
network = "tcp+tls"
config = &tls.Config{InsecureSkipVerify: s.params.Insecure}
@ -66,7 +72,7 @@ func (s *Syslog) Init() error {
config.RootCAs = pool
}
}
raddr := fmt.Sprintf("%s:%d", s.params.IP, s.params.Port)
raddr := fmt.Sprintf("%s:%d", host, s.params.Port)
if raddr == ":0" {
// If no IP:port provided then connect to local syslog.
raddr = ""