Release v1.11.1

Currently numbers are stripped not just after the color control code (\x03) but also after other formatting such as bold (\x02) and italic (\x1D), which is both unnecessary and leads to missing text from irc. This fixes that by only stripping numbers after the color control code.

.github/ISSUE_TEMPLATE.md

@@ -1,22 +1,36 @@
-If you have a configuration problem, please first try to create a basic configuration following the instructions on [the wiki](https://github.com/42wim/matterbridge/wiki/How-to-create-your-config) before filing an issue.
+<!-- This is a bug report template. By following the instructions below and
+filling out the sections with your information, you will help the us to get all
+the necessary data to fix your issue.
 
-Please answer the following questions. 
+You can also preview your report before submitting it.
 
-### Which version of matterbridge are you using?
-run ```matterbridge -version```
+Text between <!-- and --​> marks will be invisible in the report.
+-->
 
-### If you're having problems with mattermost please specify mattermost version. 
+<!-- If you have a configuration problem, please first try to create a basic configuration following the instructions on [the wiki](https://github.com/42wim/matterbridge/wiki/How-to-create-your-config) before filing an issue. -->
 
 
+### Environment
+<!-- run `matterbridge -version` -->
+<!-- If you're having problems with mattermost also specify the mattermost version. -->
+Version:
+
+<!-- What operating system are you using ? (be as specific as possible) -->
+Operating system:
+
+<!-- If you compiled matterbridge yourself:
+       * Specify the output of `go version` 
+       * Specify the output of `git rev-parse HEAD` -->
+
 ### Please describe the expected behavior.
 
 
 ### Please describe the actual behavior. 
-#### Use logs from running ```matterbridge -debug``` if possible.
+<!-- Use logs from running `matterbridge -debug` if possible. -->
 
 
 ### Any steps to reproduce the behavior?
 
 
 ### Please add your configuration file 
-#### (be sure to exclude or anonymize private data (tokens/passwords))
+<!-- (be sure to exclude or anonymize private data (tokens/passwords)) -->

.github/ISSUE_TEMPLATE/Bug_report.md

@@ -0,0 +1,26 @@
+---
+name: Bug report
+about: Create a report to help us improve. (Check the FAQ on the wiki first)
+
+---
+
+**Describe the bug**
+A clear and concise description of what the bug is.
+
+**To Reproduce**
+Steps to reproduce the behavior:
+
+**Expected behavior**
+A clear and concise description of what you expected to happen.
+
+**Screenshots/debug logs**
+If applicable, add screenshots to help explain your problem.
+Use logs from running `matterbridge -debug` if possible.
+
+**Environment (please complete the following information):**
+ - OS: [e.g. linux]
+ - Matterbridge version: output of  `matterbridge -version`
+ - If self compiled: output of `git rev-parse HEAD`
+
+**Additional context**
+Please add your configuration file  (be sure to exclude or anonymize private data (tokens/passwords))

.github/ISSUE_TEMPLATE/Feature_request.md

@@ -0,0 +1,17 @@
+---
+name: Feature request
+about: Suggest an idea for this project
+
+---
+
+**Is your feature request related to a problem? Please describe.**
+A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
+
+**Describe the solution you'd like**
+A clear and concise description of what you want to happen.
+
+**Describe alternatives you've considered**
+A clear and concise description of any alternative solutions or features you've considered.
+
+**Additional context**
+Add any other context or screenshots about the feature request here.

.travis.yml

@@ -1,7 +1,7 @@
 language: go
 go:
     #- 1.7.x
-    - 1.8.x
+    - 1.10.x
       # - tip
 
 # we have everything vendored
@@ -34,15 +34,17 @@ before_script:
 # flunk the build and immediately stop. It's sorta like having
 # set -e enabled in bash. 
 script:
-  - test -z $(gofmt -s -l $GO_FILES)  # Fail if a .go file hasn't been formatted with gofmt
+ #- test -z $(gofmt -s -l $GO_FILES)  # Fail if a .go file hasn't been formatted with gofmt
   - go test -v -race $PKGS            # Run all the tests with the race detector enabled
-  - go vet $PKGS                      # go vet is the official Go static analyzer
+ #  - go vet $PKGS                      # go vet is the official Go static analyzer
   - megacheck $PKGS                   # "go vet on steroids" + linter
   - /bin/bash ci/bintray.sh
   #- golint -set_exit_status $PKGS     # one last linter
 
 deploy:
   provider: bintray
+  edge:
+    branch: v1.8.47
   file: ci/deploy.json
   user: 42wim
   key:

README.md

@@ -1,17 +1,21 @@
 # matterbridge
 Click on one of the badges below to join the chat   
 
-[![Gitter](https://img.shields.io/gitter/room/nwjs/nw.js.svg?colorB=42f4242)](https://gitter.im/42wim/matterbridge) [![Join the IRC chat at https://webchat.freenode.net/?channels=matterbridgechat](https://img.shields.io/badge/IRC-matterbridgechat-green.svg?colorB=42f4242)](https://webchat.freenode.net/?channels=matterbridgechat) [![Discord](https://img.shields.io/badge/discord-matterbridge-green.svg?colorB=42f4242)](https://discord.gg/AkKPtrQ) [![Matrix](https://img.shields.io/badge/matrix-matterbridge-green.svg?colorB=42f4242)](https://riot.im/app/#/room/#matterbridge:matrix.org) [![Slack](https://img.shields.io/badge/slack-matterbridgechat-green.svg?colorB=42f4242)](https://join.slack.com/matterbridgechat/shared_invite/MjEwODMxNjU1NDMwLTE0OTk2MTU3NTMtMzZkZmRiNDZhOA) [![Mattermost](https://img.shields.io/badge/mattermost-matterbridge-green.svg?colorB=42f4242)](https://framateam.org/signup_user_complete/?id=tfqm33ggop8x3qgu4boeieta6e) ![Xmpp](https://img.shields.io/badge/xmpp-matterbridge@muc.im.koderoot.net-green.svg?colorB=42f4242)
+[![Gitter](https://img.shields.io/gitter/room/nwjs/nw.js.svg?colorB=42f4242)](https://gitter.im/42wim/matterbridge) [![Join the IRC chat at https://webchat.freenode.net/?channels=matterbridgechat](https://img.shields.io/badge/IRC-matterbridgechat-green.svg?colorB=42f4242)](https://webchat.freenode.net/?channels=matterbridgechat) [![Discord](https://img.shields.io/badge/discord-matterbridge-green.svg?colorB=42f4242)](https://discord.gg/AkKPtrQ) [![Matrix](https://img.shields.io/badge/matrix-matterbridge-green.svg?colorB=42f4242)](https://riot.im/app/#/room/#matterbridge:matrix.org) [![Slack](https://img.shields.io/badge/slack-matterbridgechat-green.svg?colorB=42f4242)](https://join.slack.com/matterbridgechat/shared_invite/MjEwODMxNjU1NDMwLTE0OTk2MTU3NTMtMzZkZmRiNDZhOA) [![Mattermost](https://img.shields.io/badge/mattermost-matterbridge-green.svg?colorB=42f4242)](https://framateam.org/signup_user_complete/?id=tfqm33ggop8x3qgu4boeieta6e) [![Xmpp](https://img.shields.io/badge/xmpp-matterbridge@conference.jabber.de-green.svg?colorB=42f4242)](https://inverse.chat) [![Twitch](https://img.shields.io/badge/twitch-matterbridge-green.svg?colorB=42f4242)](https://www.twitch.tv/matterbridge) [![Zulip](https://img.shields.io/badge/zulip-matterbridge-green.svg?colorB=42f4242)](https://matterbridge.zulipchat.com/register/)
 
 [![Download stable](https://img.shields.io/github/release/42wim/matterbridge.svg?label=download%20stable)](https://github.com/42wim/matterbridge/releases/latest) [![Download dev](https://img.shields.io/bintray/v/42wim/nightly/Matterbridge.svg?label=download%20dev&colorB=007ec6)](https://bintray.com/42wim/nightly/Matterbridge/_latestVersion)
 
-![matterbridge.gif](https://s15.postimg.org/qpjhp6y3f/matterbridge.gif)
+![matterbridge.gif](https://github.com/42wim/matterbridge/blob/master/img/matterbridge.gif)
 
-Simple bridge between Mattermost, IRC, XMPP, Gitter, Slack, Discord, Telegram, Rocket.Chat, Hipchat(via xmpp), Matrix and Steam.
-Has a REST API.
+Simple bridge between IRC, XMPP, Gitter, Mattermost, Slack, Discord, Telegram, Rocket.Chat, Hipchat(via xmpp), Matrix, Steam, ssh-chat and Zulip
+Has a REST API.   
+Minecraft server chat support via [MatterLink](https://github.com/elytra/MatterLink)
+
+**Mattermost isn't required to run matterbridge. It bridges between any supported protocol.**   
+(The name matterbridge is a remnant when it was only bridging mattermost)
 
 # Table of Contents
- * [Features](#features)
+ * [Features](https://github.com/42wim/matterbridge/wiki/Features)
  * [Requirements](#requirements)
  * [Screenshots](https://github.com/42wim/matterbridge/wiki/)
  * [Installing](#installing)
@@ -27,16 +31,25 @@ Has a REST API.
  * [Thanks](#thanks)
 
 # Features
-* Relays public channel messages between multiple mattermost, IRC, XMPP, Gitter, Slack, Discord, Telegram, Rocket.Chat, Hipchat (via xmpp), Matrix and Steam. 
-  Pick and mix.
-* Matterbridge can also work with private groups on your mattermost/slack.
-* Allow for bridging the same bridges, which means you can eg bridge between multiple mattermosts.
-* The bridge is now a gateway which has support multiple in and out bridges. (and supports multiple gateways).
-* REST API to read/post messages to bridges (WIP).
+* [Support bridging between any protocols](https://github.com/42wim/matterbridge/wiki/Features#support-bridging-between-any-protocols)
+* [Support multiple gateways(bridges) for your protocols](https://github.com/42wim/matterbridge/wiki/Features#support-multiple-gatewaysbridges-for-your-protocols)
+* [Message edits and deletes](https://github.com/42wim/matterbridge/wiki/Features#message-edits-and-deletes)
+* [Attachment / files handling](https://github.com/42wim/matterbridge/wiki/Features#attachment--files-handling)
+* [Username and avatar spoofing](https://github.com/42wim/matterbridge/wiki/Features#username-and-avatar-spoofing)
+* [Private groups](https://github.com/42wim/matterbridge/wiki/Features#private-groups)
+* [API](https://github.com/42wim/matterbridge/wiki/Features#api)
+
+## API
+The API is very basic at the moment and rather undocumented.
+
+Used by at least 2 projects. Feel free to make a PR to add your project to this list.
+
+* [MatterLink](https://github.com/elytra/MatterLink) (Matterbridge link for Minecraft Server chat)
+* [pyCord](https://github.com/NikkyAI/pyCord) (crossplatform chatbot)
 
 # Requirements
 Accounts to one of the supported bridges
-* [Mattermost](https://github.com/mattermost/platform/) 3.5.x - 3.10.x, 4.0.x
+* [Mattermost](https://github.com/mattermost/platform/) 3.8.x - 3.10.x, 4.x, 5.x
 * [IRC](http://www.mirc.com/servers.html)
 * [XMPP](https://jabber.org)
 * [Gitter](https://gitter.im)
@@ -47,17 +60,22 @@ Accounts to one of the supported bridges
 * [Rocket.chat](https://rocket.chat)
 * [Matrix](https://matrix.org)
 * [Steam](https://store.steampowered.com/)
+* [Twitch](https://twitch.tv)
+* [Ssh-chat](https://github.com/shazow/ssh-chat)
+* [Zulip](https://zulipchat.com)
 
 # Screenshots
 See https://github.com/42wim/matterbridge/wiki
 
 # Installing
 ## Binaries
-* Latest stable release [v1.0.1](https://github.com/42wim/matterbridge/releases/latest)
+* Latest stable release [v1.11.1](https://github.com/42wim/matterbridge/releases/latest)
 * Development releases (follows master) can be downloaded [here](https://dl.bintray.com/42wim/nightly/)  
 
 ## Building
-Go 1.7+ is required. Make sure you have [Go](https://golang.org/doc/install) properly installed, including setting up your [GOPATH] (https://golang.org/doc/code.html#GOPATH)
+Go 1.8+ is required. Make sure you have [Go](https://golang.org/doc/install) properly installed, including setting up your [GOPATH](https://golang.org/doc/code.html#GOPATH).
+
+After Go is setup, download matterbridge to your $GOPATH directory. 
 
 ```
 cd $GOPATH
@@ -169,15 +187,19 @@ Want to tip ?
 * btc: 1N7cKHj5SfqBHBzDJ6kad4BzeqUBBS2zhs
 
 # Thanks
+[![Digitalocean](https://snag.gy/3LVifX.jpg)](https://www.digitalocean.com/) for sponsoring demo/testing droplets.
+
 Matterbridge wouldn't exist without these libraries:
 * discord - https://github.com/bwmarrin/discordgo
 * echo - https://github.com/labstack/echo
 * gitter - https://github.com/sromku/go-gitter
 * gops - https://github.com/google/gops
-* irc - https://github.com/thoj/go-ircevent
+* gozulipbot - https://github.com/ifo/gozulipbot
+* irc - https://github.com/lrstanley/girc
 * mattermost - https://github.com/mattermost/platform
 * matrix - https://github.com/matrix-org/gomatrix
 * slack - https://github.com/nlopes/slack
 * steam - https://github.com/Philipp15b/go-steam
 * telegram - https://github.com/go-telegram-bot-api/telegram-bot-api
 * xmpp - https://github.com/mattn/go-xmpp
+* zulip - https://github.com/ifo/gozulipbot

bridge/api/api.go

@@ -1,21 +1,22 @@
 package api
 
 import (
+	"encoding/json"
+	"net/http"
+	"sync"
+	"time"
+
+	"github.com/42wim/matterbridge/bridge"
 	"github.com/42wim/matterbridge/bridge/config"
-	log "github.com/Sirupsen/logrus"
 	"github.com/labstack/echo"
 	"github.com/labstack/echo/middleware"
 	"github.com/zfjagann/golang-ring"
-	"net/http"
-	"sync"
 )
 
 type Api struct {
-	Config   *config.Protocol
-	Remote   chan config.Message
-	Account  string
 	Messages ring.Ring
 	sync.RWMutex
+	*bridge.Config
 }
 
 type ApiMessage struct {
@@ -26,30 +27,27 @@ type ApiMessage struct {
 	Gateway  string `json:"gateway"`
 }
 
-var flog *log.Entry
-var protocol = "api"
-
-func init() {
-	flog = log.WithFields(log.Fields{"module": protocol})
-}
-
-func New(cfg config.Protocol, account string, c chan config.Message) *Api {
-	b := &Api{}
+func New(cfg *bridge.Config) bridge.Bridger {
+	b := &Api{Config: cfg}
 	e := echo.New()
+	e.HideBanner = true
+	e.HidePort = true
 	b.Messages = ring.Ring{}
-	b.Messages.SetCapacity(cfg.Buffer)
-	b.Config = &cfg
-	b.Account = account
-	b.Remote = c
-	if b.Config.Token != "" {
+	b.Messages.SetCapacity(b.GetInt("Buffer"))
+	if b.GetString("Token") != "" {
 		e.Use(middleware.KeyAuth(func(key string, c echo.Context) (bool, error) {
-			return key == b.Config.Token, nil
+			return key == b.GetString("Token"), nil
 		}))
 	}
 	e.GET("/api/messages", b.handleMessages)
+	e.GET("/api/stream", b.handleStream)
 	e.POST("/api/message", b.handlePostMessage)
 	go func() {
-		flog.Fatal(e.Start(cfg.BindAddress))
+		if b.GetString("BindAddress") == "" {
+			b.Log.Fatalf("No BindAddress configured.")
+		}
+		b.Log.Infof("Listening on %s", b.GetString("BindAddress"))
+		b.Log.Fatal(e.Start(b.GetString("BindAddress")))
 	}()
 	return b
 }
@@ -66,29 +64,30 @@ func (b *Api) JoinChannel(channel config.ChannelInfo) error {
 
 }
 
-func (b *Api) Send(msg config.Message) error {
+func (b *Api) Send(msg config.Message) (string, error) {
 	b.Lock()
 	defer b.Unlock()
+	// ignore delete messages
+	if msg.Event == config.EVENT_MSG_DELETE {
+		return "", nil
+	}
 	b.Messages.Enqueue(&msg)
-	return nil
+	return "", nil
 }
 
 func (b *Api) handlePostMessage(c echo.Context) error {
-	message := &ApiMessage{}
-	if err := c.Bind(message); err != nil {
+	message := config.Message{}
+	if err := c.Bind(&message); err != nil {
 		return err
 	}
-	flog.Debugf("Sending message from %s on %s to gateway", message.Username, "api")
-	b.Remote <- config.Message{
-		Text:     message.Text,
-		Username: message.Username,
-		UserID:   message.UserID,
-		Channel:  "api",
-		Avatar:   message.Avatar,
-		Account:  b.Account,
-		Gateway:  message.Gateway,
-		Protocol: "api",
-	}
+	// these values are fixed
+	message.Channel = "api"
+	message.Protocol = "api"
+	message.Account = b.Account
+	message.ID = ""
+	message.Timestamp = time.Now()
+	b.Log.Debugf("Sending message from %s on %s to gateway", message.Username, "api")
+	b.Remote <- message
 	return c.JSON(http.StatusOK, message)
 }
 
@@ -99,3 +98,24 @@ func (b *Api) handleMessages(c echo.Context) error {
 	b.Messages = ring.Ring{}
 	return nil
 }
+
+func (b *Api) handleStream(c echo.Context) error {
+	c.Response().Header().Set(echo.HeaderContentType, echo.MIMEApplicationJSON)
+	c.Response().WriteHeader(http.StatusOK)
+	closeNotifier := c.Response().CloseNotify()
+	for {
+		select {
+		case <-closeNotifier:
+			return nil
+		default:
+			msg := b.Messages.Dequeue()
+			if msg != nil {
+				if err := json.NewEncoder(c.Response()).Encode(msg); err != nil {
+					return err
+				}
+				c.Response().Flush()
+			}
+			time.Sleep(200 * time.Millisecond)
+		}
+	}
+}

bridge/bridge.go

@@ -1,41 +1,42 @@
 package bridge
 
 import (
-	"github.com/42wim/matterbridge/bridge/api"
 	"github.com/42wim/matterbridge/bridge/config"
-	"github.com/42wim/matterbridge/bridge/discord"
-	"github.com/42wim/matterbridge/bridge/gitter"
-	"github.com/42wim/matterbridge/bridge/irc"
-	"github.com/42wim/matterbridge/bridge/matrix"
-	"github.com/42wim/matterbridge/bridge/mattermost"
-	"github.com/42wim/matterbridge/bridge/rocketchat"
-	"github.com/42wim/matterbridge/bridge/slack"
-	"github.com/42wim/matterbridge/bridge/steam"
-	"github.com/42wim/matterbridge/bridge/telegram"
-	"github.com/42wim/matterbridge/bridge/xmpp"
-	log "github.com/Sirupsen/logrus"
+	log "github.com/sirupsen/logrus"
 
 	"strings"
 )
 
 type Bridger interface {
-	Send(msg config.Message) error
+	Send(msg config.Message) (string, error)
 	Connect() error
 	JoinChannel(channel config.ChannelInfo) error
 	Disconnect() error
 }
 
 type Bridge struct {
-	Config config.Protocol
 	Bridger
 	Name     string
 	Account  string
 	Protocol string
 	Channels map[string]config.ChannelInfo
 	Joined   map[string]bool
+	Log      *log.Entry
+	Config   *config.Config
+	General  *config.Protocol
 }
 
-func New(cfg *config.Config, bridge *config.Bridge, c chan config.Message) *Bridge {
+type Config struct {
+	//	General *config.Protocol
+	Remote chan config.Message
+	Log    *log.Entry
+	*Bridge
+}
+
+// Factory is the factory function to create a bridge
+type Factory func(*Config) Bridger
+
+func New(bridge *config.Bridge) *Bridge {
 	b := new(Bridge)
 	b.Channels = make(map[string]config.ChannelInfo)
 	accInfo := strings.Split(bridge.Account, ".")
@@ -45,44 +46,6 @@ func New(cfg *config.Config, bridge *config.Bridge, c chan config.Message) *Brid
 	b.Protocol = protocol
 	b.Account = bridge.Account
 	b.Joined = make(map[string]bool)
-
-	// override config from environment
-	config.OverrideCfgFromEnv(cfg, protocol, name)
-	switch protocol {
-	case "mattermost":
-		b.Config = cfg.Mattermost[name]
-		b.Bridger = bmattermost.New(cfg.Mattermost[name], bridge.Account, c)
-	case "irc":
-		b.Config = cfg.IRC[name]
-		b.Bridger = birc.New(cfg.IRC[name], bridge.Account, c)
-	case "gitter":
-		b.Config = cfg.Gitter[name]
-		b.Bridger = bgitter.New(cfg.Gitter[name], bridge.Account, c)
-	case "slack":
-		b.Config = cfg.Slack[name]
-		b.Bridger = bslack.New(cfg.Slack[name], bridge.Account, c)
-	case "xmpp":
-		b.Config = cfg.Xmpp[name]
-		b.Bridger = bxmpp.New(cfg.Xmpp[name], bridge.Account, c)
-	case "discord":
-		b.Config = cfg.Discord[name]
-		b.Bridger = bdiscord.New(cfg.Discord[name], bridge.Account, c)
-	case "telegram":
-		b.Config = cfg.Telegram[name]
-		b.Bridger = btelegram.New(cfg.Telegram[name], bridge.Account, c)
-	case "rocketchat":
-		b.Config = cfg.Rocketchat[name]
-		b.Bridger = brocketchat.New(cfg.Rocketchat[name], bridge.Account, c)
-	case "matrix":
-		b.Config = cfg.Matrix[name]
-		b.Bridger = bmatrix.New(cfg.Matrix[name], bridge.Account, c)
-	case "steam":
-		b.Config = cfg.Steam[name]
-		b.Bridger = bsteam.New(cfg.Steam[name], bridge.Account, c)
-	case "api":
-		b.Config = cfg.Api[name]
-		b.Bridger = api.New(cfg.Api[name], bridge.Account, c)
-	}
 	return b
 }
 
@@ -94,7 +57,7 @@ func (b *Bridge) JoinChannels() error {
 func (b *Bridge) joinChannels(channels map[string]config.ChannelInfo, exists map[string]bool) error {
 	for ID, channel := range channels {
 		if !exists[ID] {
-			log.Infof("%s: joining %s (%s)", b.Account, channel.Name, ID)
+			b.Log.Infof("%s: joining %s (ID: %s)", b.Account, channel.Name, ID)
 			err := b.JoinChannel(channel)
 			if err != nil {
 				return err
@@ -104,3 +67,38 @@ func (b *Bridge) joinChannels(channels map[string]config.ChannelInfo, exists map
 	}
 	return nil
 }
+
+func (b *Bridge) GetBool(key string) bool {
+	if b.Config.GetBool(b.Account + "." + key) {
+		return b.Config.GetBool(b.Account + "." + key)
+	}
+	return b.Config.GetBool("general." + key)
+}
+
+func (b *Bridge) GetInt(key string) int {
+	if b.Config.GetInt(b.Account+"."+key) != 0 {
+		return b.Config.GetInt(b.Account + "." + key)
+	}
+	return b.Config.GetInt("general." + key)
+}
+
+func (b *Bridge) GetString(key string) string {
+	if b.Config.GetString(b.Account+"."+key) != "" {
+		return b.Config.GetString(b.Account + "." + key)
+	}
+	return b.Config.GetString("general." + key)
+}
+
+func (b *Bridge) GetStringSlice(key string) []string {
+	if len(b.Config.GetStringSlice(b.Account+"."+key)) != 0 {
+		return b.Config.GetStringSlice(b.Account + "." + key)
+	}
+	return b.Config.GetStringSlice("general." + key)
+}
+
+func (b *Bridge) GetStringSlice2D(key string) [][]string {
+	if len(b.Config.GetStringSlice2D(b.Account+"."+key)) != 0 {
+		return b.Config.GetStringSlice2D(b.Account + "." + key)
+	}
+	return b.Config.GetStringSlice2D("general." + key)
+}

bridge/config/config.go

@@ -1,19 +1,27 @@
 package config
 
 import (
-	"github.com/BurntSushi/toml"
-	"log"
+	"bytes"
 	"os"
-	"reflect"
 	"strings"
+	"sync"
 	"time"
+
+	"github.com/fsnotify/fsnotify"
+	log "github.com/sirupsen/logrus"
+	"github.com/spf13/viper"
+	prefixed "github.com/x-cray/logrus-prefixed-formatter"
 )
 
 const (
-	EVENT_JOIN_LEAVE      = "join_leave"
-	EVENT_FAILURE         = "failure"
-	EVENT_REJOIN_CHANNELS = "rejoin_channels"
-	EVENT_USER_ACTION     = "user_action"
+	EVENT_JOIN_LEAVE        = "join_leave"
+	EVENT_TOPIC_CHANGE      = "topic_change"
+	EVENT_FAILURE           = "failure"
+	EVENT_FILE_FAILURE_SIZE = "file_failure_size"
+	EVENT_AVATAR_DOWNLOAD   = "avatar_download"
+	EVENT_REJOIN_CHANNELS   = "rejoin_channels"
+	EVENT_USER_ACTION       = "user_action"
+	EVENT_MSG_DELETE        = "msg_delete"
 )
 
 type Message struct {
@@ -27,6 +35,18 @@ type Message struct {
 	Protocol  string    `json:"protocol"`
 	Gateway   string    `json:"gateway"`
 	Timestamp time.Time `json:"timestamp"`
+	ID        string    `json:"id"`
+	Extra     map[string][]interface{}
+}
+
+type FileInfo struct {
+	Name    string
+	Data    *[]byte
+	Comment string
+	URL     string
+	Size    int64
+	Avatar  bool
+	SHA     string
 }
 
 type ChannelInfo struct {
@@ -42,50 +62,71 @@ type Protocol struct {
 	AuthCode               string // steam
 	BindAddress            string // mattermost, slack // DEPRECATED
 	Buffer                 int    // api
+	Charset                string // irc
+	ColorNicks             bool   // only irc for now
+	Debug                  bool   // general
+	DebugLevel             int    // only for irc now
 	EditSuffix             string // mattermost, slack, discord, telegram, gitter
 	EditDisable            bool   // mattermost, slack, discord, telegram, gitter
 	IconURL                string // mattermost, slack
 	IgnoreNicks            string // all protocols
 	IgnoreMessages         string // all protocols
 	Jid                    string // xmpp
+	Label                  string // all protocols
 	Login                  string // mattermost, matrix
-	Muc                    string // xmpp
-	Name                   string // all protocols
-	Nick                   string // all protocols
-	NickFormatter          string // mattermost, slack
-	NickServNick           string // IRC
-	NickServPassword       string // IRC
-	NicksPerRow            int    // mattermost, slack
-	NoHomeServerSuffix     bool   // matrix
-	NoTLS                  bool   // mattermost
-	Password               string // IRC,mattermost,XMPP,matrix
-	PrefixMessagesWithNick bool   // mattemost, slack
-	Protocol               string //all protocols
-	MessageQueue           int    // IRC, size of message queue for flood control
-	MessageDelay           int    // IRC, time in millisecond to wait between messages
-	MessageLength          int    // IRC, max length of a message allowed
-	MessageFormat          string // telegram
-	RemoteNickFormat       string // all protocols
-	Server                 string // IRC,mattermost,XMPP,discord
-	ShowJoinPart           bool   // all protocols
-	ShowEmbeds             bool   // discord
-	SkipTLSVerify          bool   // IRC, mattermost
-	Team                   string // mattermost
-	Token                  string // gitter, slack, discord, api
-	URL                    string // mattermost, slack // DEPRECATED
-	UseAPI                 bool   // mattermost, slack
-	UseSASL                bool   // IRC
-	UseTLS                 bool   // IRC
-	UseFirstName           bool   // telegram
-	UseUserName            bool   // discord
-	UseInsecureURL         bool   // telegram
-	WebhookBindAddress     string // mattermost, slack
-	WebhookURL             string // mattermost, slack
-	WebhookUse             string // mattermost, slack, discord
+	MediaDownloadBlackList []string
+	MediaDownloadPath      string // Basically MediaServerUpload, but instead of uploading it, just write it to a file on the same server.
+	MediaDownloadSize      int    // all protocols
+	MediaServerDownload    string
+	MediaServerUpload      string
+	MessageDelay           int        // IRC, time in millisecond to wait between messages
+	MessageFormat          string     // telegram
+	MessageLength          int        // IRC, max length of a message allowed
+	MessageQueue           int        // IRC, size of message queue for flood control
+	MessageSplit           bool       // IRC, split long messages with newlines on MessageLength instead of clipping
+	Muc                    string     // xmpp
+	Name                   string     // all protocols
+	Nick                   string     // all protocols
+	NickFormatter          string     // mattermost, slack
+	NickServNick           string     // IRC
+	NickServUsername       string     // IRC
+	NickServPassword       string     // IRC
+	NicksPerRow            int        // mattermost, slack
+	NoHomeServerSuffix     bool       // matrix
+	NoSendJoinPart         bool       // all protocols
+	NoTLS                  bool       // mattermost
+	Password               string     // IRC,mattermost,XMPP,matrix
+	PrefixMessagesWithNick bool       // mattemost, slack
+	Protocol               string     // all protocols
+	QuoteDisable           bool       // telegram
+	QuoteFormat            string     // telegram
+	RejoinDelay            int        // IRC
+	ReplaceMessages        [][]string // all protocols
+	ReplaceNicks           [][]string // all protocols
+	RemoteNickFormat       string     // all protocols
+	Server                 string     // IRC,mattermost,XMPP,discord
+	ShowJoinPart           bool       // all protocols
+	ShowTopicChange        bool       // slack
+	ShowEmbeds             bool       // discord
+	SkipTLSVerify          bool       // IRC, mattermost
+	StripNick              bool       // all protocols
+	Team                   string     // mattermost
+	Token                  string     // gitter, slack, discord, api
+	Topic                  string     // zulip
+	URL                    string     // mattermost, slack // DEPRECATED
+	UseAPI                 bool       // mattermost, slack
+	UseSASL                bool       // IRC
+	UseTLS                 bool       // IRC
+	UseFirstName           bool       // telegram
+	UseUserName            bool       // discord
+	UseInsecureURL         bool       // telegram
+	WebhookBindAddress     string     // mattermost, slack
+	WebhookURL             string     // mattermost, slack
+	WebhookUse             string     // mattermost, slack, discord
 }
 
 type ChannelOptions struct {
-	Key        string // irc
+	Key        string // irc, xmpp
 	WebhookURL string // discord
 }
 
@@ -111,9 +152,9 @@ type SameChannelGateway struct {
 	Accounts []string
 }
 
-type Config struct {
+type ConfigValues struct {
 	Api                map[string]Protocol
-	IRC                map[string]Protocol
+	Irc                map[string]Protocol
 	Mattermost         map[string]Protocol
 	Matrix             map[string]Protocol
 	Slack              map[string]Protocol
@@ -123,80 +164,118 @@ type Config struct {
 	Discord            map[string]Protocol
 	Telegram           map[string]Protocol
 	Rocketchat         map[string]Protocol
+	Sshchat            map[string]Protocol
+	Zulip              map[string]Protocol
 	General            Protocol
 	Gateway            []Gateway
 	SameChannelGateway []SameChannelGateway
 }
 
+type Config struct {
+	v *viper.Viper
+	*ConfigValues
+	sync.RWMutex
+}
+
 func NewConfig(cfgfile string) *Config {
-	var cfg Config
-	if _, err := toml.DecodeFile(cfgfile, &cfg); err != nil {
+	log.SetFormatter(&prefixed.TextFormatter{PrefixPadding: 13, DisableColors: true, FullTimestamp: false})
+	flog := log.WithFields(log.Fields{"prefix": "config"})
+	var cfg ConfigValues
+	viper.SetConfigType("toml")
+	viper.SetConfigFile(cfgfile)
+	viper.SetEnvPrefix("matterbridge")
+	viper.AddConfigPath(".")
+	viper.SetEnvKeyReplacer(strings.NewReplacer(".", "_"))
+	viper.AutomaticEnv()
+	f, err := os.Open(cfgfile)
+	if err != nil {
 		log.Fatal(err)
 	}
-	fail := false
-	for k, v := range cfg.Mattermost {
-		res := Deprecated(v, "mattermost."+k)
-		if res {
-			fail = res
-		}
+	err = viper.ReadConfig(f)
+	if err != nil {
+		log.Fatal(err)
 	}
-	for k, v := range cfg.Slack {
-		res := Deprecated(v, "slack."+k)
-		if res {
-			fail = res
-		}
+	err = viper.Unmarshal(&cfg)
+	if err != nil {
+		log.Fatal("blah", err)
 	}
-	for k, v := range cfg.Rocketchat {
-		res := Deprecated(v, "rocketchat."+k)
-		if res {
-			fail = res
-		}
+	mycfg := new(Config)
+	mycfg.v = viper.GetViper()
+	if cfg.General.MediaDownloadSize == 0 {
+		cfg.General.MediaDownloadSize = 1000000
 	}
-	if fail {
-		log.Fatalf("Fix your config. Please see changelog for more information")
-	}
-	return &cfg
+	viper.WatchConfig()
+	viper.OnConfigChange(func(e fsnotify.Event) {
+		flog.Println("Config file changed:", e.Name)
+	})
+
+	mycfg.ConfigValues = &cfg
+	return mycfg
 }
 
-func OverrideCfgFromEnv(cfg *Config, protocol string, account string) {
-	var protoCfg Protocol
-	val := reflect.ValueOf(cfg).Elem()
-	// loop over the Config struct
-	for i := 0; i < val.NumField(); i++ {
-		typeField := val.Type().Field(i)
-		// look for the protocol map (both lowercase)
-		if strings.ToLower(typeField.Name) == protocol {
-			// get the Protocol struct from the map
-			data := val.Field(i).MapIndex(reflect.ValueOf(account))
-			protoCfg = data.Interface().(Protocol)
-			protoStruct := reflect.ValueOf(&protoCfg).Elem()
-			// loop over the found protocol struct
-			for i := 0; i < protoStruct.NumField(); i++ {
-				typeField := protoStruct.Type().Field(i)
-				// build our environment key (eg MATTERBRIDGE_MATTERMOST_WORK_LOGIN)
-				key := "matterbridge_" + protocol + "_" + account + "_" + typeField.Name
-				key = strings.ToUpper(key)
-				// search the environment
-				res := os.Getenv(key)
-				// if it exists and the current field is a string
-				// then update the current field
-				if res != "" {
-					fieldVal := protoStruct.Field(i)
-					if fieldVal.Kind() == reflect.String {
-						log.Printf("config: overriding %s from env with %s\n", key, res)
-						fieldVal.Set(reflect.ValueOf(res))
-					}
-				}
+func NewConfigFromString(input []byte) *Config {
+	var cfg ConfigValues
+	viper.SetConfigType("toml")
+	err := viper.ReadConfig(bytes.NewBuffer(input))
+	if err != nil {
+		log.Fatal(err)
+	}
+	err = viper.Unmarshal(&cfg)
+	if err != nil {
+		log.Fatal(err)
+	}
+	mycfg := new(Config)
+	mycfg.v = viper.GetViper()
+	mycfg.ConfigValues = &cfg
+	return mycfg
+}
+
+func (c *Config) GetBool(key string) bool {
+	c.RLock()
+	defer c.RUnlock()
+	//	log.Debugf("getting bool %s = %#v", key, c.v.GetBool(key))
+	return c.v.GetBool(key)
+}
+
+func (c *Config) GetInt(key string) int {
+	c.RLock()
+	defer c.RUnlock()
+	//	log.Debugf("getting int %s = %d", key, c.v.GetInt(key))
+	return c.v.GetInt(key)
+}
+
+func (c *Config) GetString(key string) string {
+	c.RLock()
+	defer c.RUnlock()
+	//	log.Debugf("getting String %s = %s", key, c.v.GetString(key))
+	return c.v.GetString(key)
+}
+
+func (c *Config) GetStringSlice(key string) []string {
+	c.RLock()
+	defer c.RUnlock()
+	// log.Debugf("getting StringSlice %s = %#v", key, c.v.GetStringSlice(key))
+	return c.v.GetStringSlice(key)
+}
+
+func (c *Config) GetStringSlice2D(key string) [][]string {
+	c.RLock()
+	defer c.RUnlock()
+	result := [][]string{}
+	if res, ok := c.v.Get(key).([]interface{}); ok {
+		for _, entry := range res {
+			result2 := []string{}
+			for _, entry2 := range entry.([]interface{}) {
+				result2 = append(result2, entry2.(string))
 			}
-			// update the map with the modified Protocol (cfg.Protocol[account] = Protocol)
-			val.Field(i).SetMapIndex(reflect.ValueOf(account), reflect.ValueOf(protoCfg))
-			break
+			result = append(result, result2)
 		}
+		return result
 	}
+	return result
 }
 
-func GetIconURL(msg *Message, cfg *Protocol) string {
-	iconURL := cfg.IconURL
+func GetIconURL(msg *Message, iconURL string) string {
 	info := strings.Split(msg.Account, ".")
 	protocol := info[0]
 	name := info[1]
@@ -205,17 +284,3 @@ func GetIconURL(msg *Message, cfg *Protocol) string {
 	iconURL = strings.Replace(iconURL, "{PROTOCOL}", protocol, -1)
 	return iconURL
 }
-
-func Deprecated(cfg Protocol, account string) bool {
-	if cfg.BindAddress != "" {
-		log.Printf("ERROR: %s BindAddress is deprecated, you need to change it to WebhookBindAddress.", account)
-	} else if cfg.URL != "" {
-		log.Printf("ERROR: %s URL is deprecated, you need to change it to WebhookURL.", account)
-	} else if cfg.UseAPI {
-		log.Printf("ERROR: %s UseAPI is deprecated, it's enabled by default, please remove it from your config file.", account)
-	} else {
-		return false
-	}
-	return true
-	//log.Fatalf("ERROR: Fix your config: %s", account)
-}

bridge/discord/discord.go

@@ -1,19 +1,22 @@
 package bdiscord
 
 import (
-	"github.com/42wim/matterbridge/bridge/config"
-	log "github.com/Sirupsen/logrus"
-	"github.com/bwmarrin/discordgo"
+	"bytes"
+	"fmt"
 	"regexp"
 	"strings"
 	"sync"
+
+	"github.com/42wim/matterbridge/bridge"
+	"github.com/42wim/matterbridge/bridge/config"
+	"github.com/42wim/matterbridge/bridge/helper"
+	"github.com/bwmarrin/discordgo"
 )
 
-type bdiscord struct {
+const MessageLength = 1950
+
+type Bdiscord struct {
 	c              *discordgo.Session
-	Config         *config.Protocol
-	Remote         chan config.Message
-	Account        string
 	Channels       []*discordgo.Channel
 	Nick           string
 	UseChannelID   bool
@@ -23,83 +26,74 @@ type bdiscord struct {
 	webhookToken   string
 	channelInfoMap map[string]*config.ChannelInfo
 	sync.RWMutex
+	*bridge.Config
 }
 
-var flog *log.Entry
-var protocol = "discord"
-
-func init() {
-	flog = log.WithFields(log.Fields{"module": protocol})
-}
-
-func New(cfg config.Protocol, account string, c chan config.Message) *bdiscord {
-	b := &bdiscord{}
-	b.Config = &cfg
-	b.Remote = c
-	b.Account = account
+func New(cfg *bridge.Config) bridge.Bridger {
+	b := &Bdiscord{Config: cfg}
 	b.userMemberMap = make(map[string]*discordgo.Member)
 	b.channelInfoMap = make(map[string]*config.ChannelInfo)
-	if b.Config.WebhookURL != "" {
-		flog.Debug("Configuring Discord Incoming Webhook")
-		b.webhookID, b.webhookToken = b.splitURL(b.Config.WebhookURL)
+	if b.GetString("WebhookURL") != "" {
+		b.Log.Debug("Configuring Discord Incoming Webhook")
+		b.webhookID, b.webhookToken = b.splitURL(b.GetString("WebhookURL"))
 	}
 	return b
 }
 
-func (b *bdiscord) Connect() error {
+func (b *Bdiscord) Connect() error {
 	var err error
-	flog.Info("Connecting")
-	if b.Config.WebhookURL == "" {
-		flog.Info("Connecting using token")
+	var token string
+	b.Log.Info("Connecting")
+	if b.GetString("WebhookURL") == "" {
+		b.Log.Info("Connecting using token")
 	} else {
-		flog.Info("Connecting using webhookurl (for posting) and token")
+		b.Log.Info("Connecting using webhookurl (for posting) and token")
 	}
-	if !strings.HasPrefix(b.Config.Token, "Bot ") {
-		b.Config.Token = "Bot " + b.Config.Token
+	if !strings.HasPrefix(b.GetString("Token"), "Bot ") {
+		token = "Bot " + b.GetString("Token")
 	}
-	b.c, err = discordgo.New(b.Config.Token)
+	b.c, err = discordgo.New(token)
 	if err != nil {
-		flog.Debugf("%#v", err)
 		return err
 	}
-	flog.Info("Connection succeeded")
+	b.Log.Info("Connection succeeded")
 	b.c.AddHandler(b.messageCreate)
 	b.c.AddHandler(b.memberUpdate)
 	b.c.AddHandler(b.messageUpdate)
+	b.c.AddHandler(b.messageDelete)
 	err = b.c.Open()
 	if err != nil {
-		flog.Debugf("%#v", err)
 		return err
 	}
 	guilds, err := b.c.UserGuilds(100, "", "")
 	if err != nil {
-		flog.Debugf("%#v", err)
 		return err
 	}
 	userinfo, err := b.c.User("@me")
 	if err != nil {
-		flog.Debugf("%#v", err)
 		return err
 	}
 	b.Nick = userinfo.Username
 	for _, guild := range guilds {
-		if guild.Name == b.Config.Server {
+		if guild.Name == b.GetString("Server") {
 			b.Channels, err = b.c.GuildChannels(guild.ID)
 			b.guildID = guild.ID
 			if err != nil {
-				flog.Debugf("%#v", err)
 				return err
 			}
 		}
 	}
+	for _, channel := range b.Channels {
+		b.Log.Debugf("found channel %#v", channel)
+	}
 	return nil
 }
 
-func (b *bdiscord) Disconnect() error {
-	return nil
+func (b *Bdiscord) Disconnect() error {
+	return b.c.Close()
 }
 
-func (b *bdiscord) JoinChannel(channel config.ChannelInfo) error {
+func (b *Bdiscord) JoinChannel(channel config.ChannelInfo) error {
 	b.channelInfoMap[channel.ID] = &channel
 	idcheck := strings.Split(channel.Name, "ID:")
 	if len(idcheck) > 1 {
@@ -108,31 +102,50 @@ func (b *bdiscord) JoinChannel(channel config.ChannelInfo) error {
 	return nil
 }
 
-func (b *bdiscord) Send(msg config.Message) error {
-	flog.Debugf("Receiving %#v", msg)
+func (b *Bdiscord) Send(msg config.Message) (string, error) {
+	b.Log.Debugf("=> Receiving %#v", msg)
+
 	channelID := b.getChannelID(msg.Channel)
 	if channelID == "" {
-		flog.Errorf("Could not find channelID for %v", msg.Channel)
-		return nil
+		return "", fmt.Errorf("Could not find channelID for %v", msg.Channel)
 	}
+
+	// Make a action /me of the message
 	if msg.Event == config.EVENT_USER_ACTION {
 		msg.Text = "_" + msg.Text + "_"
 	}
 
+	// use initial webhook
 	wID := b.webhookID
 	wToken := b.webhookToken
+
+	// check if have a channel specific webhook
 	if ci, ok := b.channelInfoMap[msg.Channel+b.Account]; ok {
 		if ci.Options.WebhookURL != "" {
 			wID, wToken = b.splitURL(ci.Options.WebhookURL)
 		}
 	}
 
-	if wID == "" {
-		flog.Debugf("Broadcasting using token (API)")
-		b.c.ChannelMessageSend(channelID, msg.Username+msg.Text)
-	} else {
-		flog.Debugf("Broadcasting using Webhook")
-		b.c.WebhookExecute(
+	// Use webhook to send the message
+	if wID != "" {
+		// skip events
+		if msg.Event != "" && msg.Event != config.EVENT_JOIN_LEAVE && msg.Event != config.EVENT_TOPIC_CHANGE {
+			return "", nil
+		}
+		b.Log.Debugf("Broadcasting using Webhook")
+		for _, f := range msg.Extra["file"] {
+			fi := f.(config.FileInfo)
+			if fi.URL != "" {
+				msg.Text += " " + fi.URL
+			}
+		}
+		// skip empty messages
+		if msg.Text == "" {
+			return "", nil
+		}
+
+		msg.Text = helper.ClipMessage(msg.Text, MessageLength)
+		err := b.c.WebhookExecute(
 			wID,
 			wToken,
 			true,
@@ -141,23 +154,73 @@ func (b *bdiscord) Send(msg config.Message) error {
 				Username:  msg.Username,
 				AvatarURL: msg.Avatar,
 			})
+		return "", err
 	}
-	return nil
+
+	b.Log.Debugf("Broadcasting using token (API)")
+
+	// Delete message
+	if msg.Event == config.EVENT_MSG_DELETE {
+		if msg.ID == "" {
+			return "", nil
+		}
+		err := b.c.ChannelMessageDelete(channelID, msg.ID)
+		return "", err
+	}
+
+	// Upload a file if it exists
+	if msg.Extra != nil {
+		for _, rmsg := range helper.HandleExtra(&msg, b.General) {
+			rmsg.Text = helper.ClipMessage(rmsg.Text, MessageLength)
+			b.c.ChannelMessageSend(channelID, rmsg.Username+rmsg.Text)
+		}
+		// check if we have files to upload (from slack, telegram or mattermost)
+		if len(msg.Extra["file"]) > 0 {
+			return b.handleUploadFile(&msg, channelID)
+		}
+	}
+
+	msg.Text = helper.ClipMessage(msg.Text, MessageLength)
+	// Edit message
+	if msg.ID != "" {
+		_, err := b.c.ChannelMessageEdit(channelID, msg.ID, msg.Username+msg.Text)
+		return msg.ID, err
+	}
+
+	// Post normal message
+	res, err := b.c.ChannelMessageSend(channelID, msg.Username+msg.Text)
+	if err != nil {
+		return "", err
+	}
+	return res.ID, err
 }
 
-func (b *bdiscord) messageUpdate(s *discordgo.Session, m *discordgo.MessageUpdate) {
-	if b.Config.EditDisable {
+func (b *Bdiscord) messageDelete(s *discordgo.Session, m *discordgo.MessageDelete) {
+	rmsg := config.Message{Account: b.Account, ID: m.ID, Event: config.EVENT_MSG_DELETE, Text: config.EVENT_MSG_DELETE}
+	rmsg.Channel = b.getChannelName(m.ChannelID)
+	if b.UseChannelID {
+		rmsg.Channel = "ID:" + m.ChannelID
+	}
+	b.Log.Debugf("<= Sending message from %s to gateway", b.Account)
+	b.Log.Debugf("<= Message is %#v", rmsg)
+	b.Remote <- rmsg
+}
+
+func (b *Bdiscord) messageUpdate(s *discordgo.Session, m *discordgo.MessageUpdate) {
+	if b.GetBool("EditDisable") {
 		return
 	}
 	// only when message is actually edited
 	if m.Message.EditedTimestamp != "" {
-		flog.Debugf("Sending edit message")
-		m.Content = m.Content + b.Config.EditSuffix
+		b.Log.Debugf("Sending edit message")
+		m.Content = m.Content + b.GetString("EditSuffix")
 		b.messageCreate(s, (*discordgo.MessageCreate)(m))
 	}
 }
 
-func (b *bdiscord) messageCreate(s *discordgo.Session, m *discordgo.MessageCreate) {
+func (b *Bdiscord) messageCreate(s *discordgo.Session, m *discordgo.MessageCreate) {
+	var err error
+
 	// not relay our own messages
 	if m.Author.Username == b.Nick {
 		return
@@ -167,68 +230,73 @@ func (b *bdiscord) messageCreate(s *discordgo.Session, m *discordgo.MessageCreat
 		return
 	}
 
+	// add the url of the attachments to content
 	if len(m.Attachments) > 0 {
 		for _, attach := range m.Attachments {
 			m.Content = m.Content + "\n" + attach.URL
 		}
 	}
 
-	var text string
+	rmsg := config.Message{Account: b.Account, Avatar: "https://cdn.discordapp.com/avatars/" + m.Author.ID + "/" + m.Author.Avatar + ".jpg", UserID: m.Author.ID, ID: m.ID}
+
 	if m.Content != "" {
-		flog.Debugf("Receiving message %#v", m.Message)
-		if len(m.MentionRoles) > 0 {
-			m.Message.Content = b.replaceRoleMentions(m.Message.Content)
-		}
+		b.Log.Debugf("== Receiving event %#v", m.Message)
 		m.Message.Content = b.stripCustomoji(m.Message.Content)
 		m.Message.Content = b.replaceChannelMentions(m.Message.Content)
-		text = m.ContentWithMentionsReplaced()
+		rmsg.Text, err = m.ContentWithMoreMentionsReplaced(b.c)
+		if err != nil {
+			b.Log.Errorf("ContentWithMoreMentionsReplaced failed: %s", err)
+			rmsg.Text = m.ContentWithMentionsReplaced()
+		}
 	}
 
-	rmsg := config.Message{Account: b.Account, Avatar: "https://cdn.discordapp.com/avatars/" + m.Author.ID + "/" + m.Author.Avatar + ".jpg",
-		UserID: m.Author.ID}
-
+	// set channel name
 	rmsg.Channel = b.getChannelName(m.ChannelID)
 	if b.UseChannelID {
 		rmsg.Channel = "ID:" + m.ChannelID
 	}
 
-	if !b.Config.UseUserName {
+	// set username
+	if !b.GetBool("UseUserName") {
 		rmsg.Username = b.getNick(m.Author)
 	} else {
 		rmsg.Username = m.Author.Username
 	}
 
-	if b.Config.ShowEmbeds && m.Message.Embeds != nil {
+	// if we have embedded content add it to text
+	if b.GetBool("ShowEmbeds") && m.Message.Embeds != nil {
 		for _, embed := range m.Message.Embeds {
-			text = text + "embed: " + embed.Title + " - " + embed.Description + " - " + embed.URL + "\n"
+			rmsg.Text = rmsg.Text + "embed: " + embed.Title + " - " + embed.Description + " - " + embed.URL + "\n"
 		}
 	}
 
 	// no empty messages
-	if text == "" {
+	if rmsg.Text == "" {
 		return
 	}
 
-	text, ok := b.replaceAction(text)
+	// do we have a /me action
+	var ok bool
+	rmsg.Text, ok = b.replaceAction(rmsg.Text)
 	if ok {
 		rmsg.Event = config.EVENT_USER_ACTION
 	}
 
-	rmsg.Text = text
-	flog.Debugf("Sending message from %s on %s to gateway", m.Author.Username, b.Account)
+	b.Log.Debugf("<= Sending message from %s on %s to gateway", m.Author.Username, b.Account)
+	b.Log.Debugf("<= Message is %#v", rmsg)
 	b.Remote <- rmsg
 }
 
-func (b *bdiscord) memberUpdate(s *discordgo.Session, m *discordgo.GuildMemberUpdate) {
+func (b *Bdiscord) memberUpdate(s *discordgo.Session, m *discordgo.GuildMemberUpdate) {
 	b.Lock()
 	if _, ok := b.userMemberMap[m.Member.User.ID]; ok {
-		flog.Debugf("%s: memberupdate: user %s (nick %s) changes nick to %s", b.Account, m.Member.User.Username, b.userMemberMap[m.Member.User.ID].Nick, m.Member.Nick)
+		b.Log.Debugf("%s: memberupdate: user %s (nick %s) changes nick to %s", b.Account, m.Member.User.Username, b.userMemberMap[m.Member.User.ID].Nick, m.Member.Nick)
 	}
 	b.userMemberMap[m.Member.User.ID] = m.Member
 	b.Unlock()
 }
 
-func (b *bdiscord) getNick(user *discordgo.User) string {
+func (b *Bdiscord) getNick(user *discordgo.User) string {
 	var err error
 	b.Lock()
 	defer b.Unlock()
@@ -255,7 +323,7 @@ func (b *bdiscord) getNick(user *discordgo.User) string {
 	return user.Username
 }
 
-func (b *bdiscord) getChannelID(name string) string {
+func (b *Bdiscord) getChannelID(name string) string {
 	idcheck := strings.Split(name, "ID:")
 	if len(idcheck) > 1 {
 		return idcheck[1]
@@ -268,7 +336,7 @@ func (b *bdiscord) getChannelID(name string) string {
 	return ""
 }
 
-func (b *bdiscord) getChannelName(id string) string {
+func (b *Bdiscord) getChannelName(id string) string {
 	for _, channel := range b.Channels {
 		if channel.ID == id {
 			return channel.Name
@@ -277,19 +345,7 @@ func (b *bdiscord) getChannelName(id string) string {
 	return ""
 }
 
-func (b *bdiscord) replaceRoleMentions(text string) string {
-	roles, err := b.c.GuildRoles(b.guildID)
-	if err != nil {
-		flog.Debugf("%#v", string(err.(*discordgo.RESTError).ResponseBody))
-		return text
-	}
-	for _, role := range roles {
-		text = strings.Replace(text, "<@&"+role.ID+">", "@"+role.Name, -1)
-	}
-	return text
-}
-
-func (b *bdiscord) replaceChannelMentions(text string) string {
+func (b *Bdiscord) replaceChannelMentions(text string) string {
 	var err error
 	re := regexp.MustCompile("<#[0-9]+>")
 	text = re.ReplaceAllStringFunc(text, func(m string) string {
@@ -308,28 +364,31 @@ func (b *bdiscord) replaceChannelMentions(text string) string {
 	return text
 }
 
-func (b *bdiscord) replaceAction(text string) (string, bool) {
+func (b *Bdiscord) replaceAction(text string) (string, bool) {
 	if strings.HasPrefix(text, "_") && strings.HasSuffix(text, "_") {
 		return strings.Replace(text, "_", "", -1), true
 	}
 	return text, false
 }
 
-func (b *bdiscord) stripCustomoji(text string) string {
+func (b *Bdiscord) stripCustomoji(text string) string {
 	// <:doge:302803592035958784>
 	re := regexp.MustCompile("<(:.*?:)[0-9]+>")
 	return re.ReplaceAllString(text, `$1`)
 }
 
 // splitURL splits a webhookURL and returns the id and token
-func (b *bdiscord) splitURL(url string) (string, string) {
+func (b *Bdiscord) splitURL(url string) (string, string) {
 	webhookURLSplit := strings.Split(url, "/")
+	if len(webhookURLSplit) != 7 {
+		b.Log.Fatalf("%s is no correct discord WebhookURL", url)
+	}
 	return webhookURLSplit[len(webhookURLSplit)-2], webhookURLSplit[len(webhookURLSplit)-1]
 }
 
 // useWebhook returns true if we have a webhook defined somewhere
-func (b *bdiscord) useWebhook() bool {
-	if b.Config.WebhookURL != "" {
+func (b *Bdiscord) useWebhook() bool {
+	if b.GetString("WebhookURL") != "" {
 		return true
 	}
 	for _, channel := range b.channelInfoMap {
@@ -341,9 +400,9 @@ func (b *bdiscord) useWebhook() bool {
 }
 
 // isWebhookID returns true if the specified id is used in a defined webhook
-func (b *bdiscord) isWebhookID(id string) bool {
-	if b.Config.WebhookURL != "" {
-		wID, _ := b.splitURL(b.Config.WebhookURL)
+func (b *Bdiscord) isWebhookID(id string) bool {
+	if b.GetString("WebhookURL") != "" {
+		wID, _ := b.splitURL(b.GetString("WebhookURL"))
 		if wID == id {
 			return true
 		}
@@ -358,3 +417,18 @@ func (b *bdiscord) isWebhookID(id string) bool {
 	}
 	return false
 }
+
+// handleUploadFile handles native upload of files
+func (b *Bdiscord) handleUploadFile(msg *config.Message, channelID string) (string, error) {
+	var err error
+	for _, f := range msg.Extra["file"] {
+		fi := f.(config.FileInfo)
+		files := []*discordgo.File{}
+		files = append(files, &discordgo.File{fi.Name, "", bytes.NewReader(*fi.Data)})
+		_, err = b.c.ChannelMessageSendComplex(channelID, &discordgo.MessageSend{Content: msg.Username + fi.Comment, Files: files})
+		if err != nil {
+			return "", fmt.Errorf("file upload failed: %#v", err)
+		}
+	}
+	return "", nil
+}

bridge/gitter/gitter.go

@@ -2,47 +2,39 @@ package bgitter
 
 import (
 	"fmt"
-	"github.com/42wim/matterbridge/bridge/config"
-	log "github.com/Sirupsen/logrus"
-	"github.com/sromku/go-gitter"
 	"strings"
+
+	"github.com/42wim/go-gitter"
+	"github.com/42wim/matterbridge/bridge"
+	"github.com/42wim/matterbridge/bridge/config"
+	"github.com/42wim/matterbridge/bridge/helper"
 )
 
 type Bgitter struct {
-	c       *gitter.Gitter
-	Config  *config.Protocol
-	Remote  chan config.Message
-	Account string
-	Users   []gitter.User
-	Rooms   []gitter.Room
+	c     *gitter.Gitter
+	User  *gitter.User
+	Users []gitter.User
+	Rooms []gitter.Room
+	*bridge.Config
 }
 
-var flog *log.Entry
-var protocol = "gitter"
-
-func init() {
-	flog = log.WithFields(log.Fields{"module": protocol})
-}
-
-func New(cfg config.Protocol, account string, c chan config.Message) *Bgitter {
-	b := &Bgitter{}
-	b.Config = &cfg
-	b.Remote = c
-	b.Account = account
-	return b
+func New(cfg *bridge.Config) bridge.Bridger {
+	return &Bgitter{Config: cfg}
 }
 
 func (b *Bgitter) Connect() error {
 	var err error
-	flog.Info("Connecting")
-	b.c = gitter.New(b.Config.Token)
-	_, err = b.c.GetUser()
+	b.Log.Info("Connecting")
+	b.c = gitter.New(b.GetString("Token"))
+	b.User, err = b.c.GetUser()
 	if err != nil {
-		flog.Debugf("%#v", err)
 		return err
 	}
-	flog.Info("Connection succeeded")
-	b.Rooms, _ = b.c.GetRooms()
+	b.Rooms, err = b.c.GetRooms()
+	if err != nil {
+		return err
+	}
+	b.Log.Info("Connection succeeded")
 	return nil
 }
 
@@ -78,34 +70,74 @@ func (b *Bgitter) JoinChannel(channel config.ChannelInfo) error {
 		for event := range stream.Event {
 			switch ev := event.Data.(type) {
 			case *gitter.MessageReceived:
-				// check for ZWSP to see if it's not an echo
-				if !strings.HasSuffix(ev.Message.Text, "​") {
-					flog.Debugf("Sending message from %s on %s to gateway", ev.Message.From.Username, b.Account)
+				// ignore message sent from ourselves
+				if ev.Message.From.ID != b.User.ID {
+					b.Log.Debugf("<= Sending message from %s on %s to gateway", ev.Message.From.Username, b.Account)
 					rmsg := config.Message{Username: ev.Message.From.Username, Text: ev.Message.Text, Channel: room,
-						Account: b.Account, Avatar: b.getAvatar(ev.Message.From.Username), UserID: ev.Message.From.ID}
+						Account: b.Account, Avatar: b.getAvatar(ev.Message.From.Username), UserID: ev.Message.From.ID,
+						ID: ev.Message.ID}
 					if strings.HasPrefix(ev.Message.Text, "@"+ev.Message.From.Username) {
 						rmsg.Event = config.EVENT_USER_ACTION
 						rmsg.Text = strings.Replace(rmsg.Text, "@"+ev.Message.From.Username+" ", "", -1)
 					}
+					b.Log.Debugf("<= Message is %#v", rmsg)
 					b.Remote <- rmsg
 				}
 			case *gitter.GitterConnectionClosed:
-				flog.Errorf("connection with gitter closed for room %s", room)
+				b.Log.Errorf("connection with gitter closed for room %s", room)
 			}
 		}
-	}(stream, room.Name)
+	}(stream, room.URI)
 	return nil
 }
 
-func (b *Bgitter) Send(msg config.Message) error {
-	flog.Debugf("Receiving %#v", msg)
+func (b *Bgitter) Send(msg config.Message) (string, error) {
+	b.Log.Debugf("=> Receiving %#v", msg)
 	roomID := b.getRoomID(msg.Channel)
 	if roomID == "" {
-		flog.Errorf("Could not find roomID for %v", msg.Channel)
-		return nil
+		b.Log.Errorf("Could not find roomID for %v", msg.Channel)
+		return "", nil
 	}
-	// add ZWSP because gitter echoes our own messages
-	return b.c.SendMessage(roomID, msg.Username+msg.Text+" ​")
+
+	// Delete message
+	if msg.Event == config.EVENT_MSG_DELETE {
+		if msg.ID == "" {
+			return "", nil
+		}
+		// gitter has no delete message api so we edit message to ""
+		_, err := b.c.UpdateMessage(roomID, msg.ID, "")
+		if err != nil {
+			return "", err
+		}
+		return "", nil
+	}
+
+	// Upload a file (in gitter case send the upload URL because gitter has no native upload support)
+	if msg.Extra != nil {
+		for _, rmsg := range helper.HandleExtra(&msg, b.General) {
+			b.c.SendMessage(roomID, rmsg.Username+rmsg.Text)
+		}
+		if len(msg.Extra["file"]) > 0 {
+			return b.handleUploadFile(&msg, roomID)
+		}
+	}
+
+	// Edit message
+	if msg.ID != "" {
+		b.Log.Debugf("updating message with id %s", msg.ID)
+		_, err := b.c.UpdateMessage(roomID, msg.ID, msg.Username+msg.Text)
+		if err != nil {
+			return "", err
+		}
+		return "", nil
+	}
+
+	// Post normal message
+	resp, err := b.c.SendMessage(roomID, msg.Username+msg.Text)
+	if err != nil {
+		return "", err
+	}
+	return resp.ID, nil
 }
 
 func (b *Bgitter) getRoomID(channel string) string {
@@ -128,3 +160,23 @@ func (b *Bgitter) getAvatar(user string) string {
 	}
 	return avatar
 }
+
+func (b *Bgitter) handleUploadFile(msg *config.Message, roomID string) (string, error) {
+	for _, f := range msg.Extra["file"] {
+		fi := f.(config.FileInfo)
+		if fi.Comment != "" {
+			msg.Text += fi.Comment + ": "
+		}
+		if fi.URL != "" {
+			msg.Text = fi.URL
+			if fi.Comment != "" {
+				msg.Text = fi.Comment + ": " + fi.URL
+			}
+		}
+		_, err := b.c.SendMessage(roomID, msg.Username+msg.Text)
+		if err != nil {
+			return "", err
+		}
+	}
+	return "", nil
+}

bridge/helper/helper.go

@@ -0,0 +1,130 @@
+package helper
+
+import (
+	"bytes"
+	"fmt"
+	"io"
+	"net/http"
+	"regexp"
+	"strings"
+	"time"
+	"unicode/utf8"
+
+	"github.com/42wim/matterbridge/bridge/config"
+	log "github.com/sirupsen/logrus"
+)
+
+func DownloadFile(url string) (*[]byte, error) {
+	return DownloadFileAuth(url, "")
+}
+
+func DownloadFileAuth(url string, auth string) (*[]byte, error) {
+	var buf bytes.Buffer
+	client := &http.Client{
+		Timeout: time.Second * 5,
+	}
+	req, err := http.NewRequest("GET", url, nil)
+	if auth != "" {
+		req.Header.Add("Authorization", auth)
+	}
+	if err != nil {
+		return nil, err
+	}
+	resp, err := client.Do(req)
+	if err != nil {
+		return nil, err
+	}
+	defer resp.Body.Close()
+	io.Copy(&buf, resp.Body)
+	data := buf.Bytes()
+	return &data, nil
+}
+
+func SplitStringLength(input string, length int) string {
+	a := []rune(input)
+	str := ""
+	for i, r := range a {
+		str = str + string(r)
+		if i > 0 && (i+1)%length == 0 {
+			str += "\n"
+		}
+	}
+	return str
+}
+
+// handle all the stuff we put into extra
+func HandleExtra(msg *config.Message, general *config.Protocol) []config.Message {
+	extra := msg.Extra
+	rmsg := []config.Message{}
+	if len(extra[config.EVENT_FILE_FAILURE_SIZE]) > 0 {
+		for _, f := range extra[config.EVENT_FILE_FAILURE_SIZE] {
+			fi := f.(config.FileInfo)
+			text := fmt.Sprintf("file %s too big to download (%#v > allowed size: %#v)", fi.Name, fi.Size, general.MediaDownloadSize)
+			rmsg = append(rmsg, config.Message{Text: text, Username: "<system> ", Channel: msg.Channel, Account: msg.Account})
+		}
+		return rmsg
+	}
+	return rmsg
+}
+
+func GetAvatar(av map[string]string, userid string, general *config.Protocol) string {
+	if sha, ok := av[userid]; ok {
+		return general.MediaServerDownload + "/" + sha + "/" + userid + ".png"
+	}
+	return ""
+}
+
+func HandleDownloadSize(flog *log.Entry, msg *config.Message, name string, size int64, general *config.Protocol) error {
+	// check blacklist here
+	for _, entry := range general.MediaDownloadBlackList {
+		if entry != "" {
+			re, err := regexp.Compile(entry)
+			if err != nil {
+				flog.Errorf("incorrect regexp %s for %s", entry, msg.Account)
+				continue
+			}
+			if re.MatchString(name) {
+				return fmt.Errorf("Matching blacklist %s. Not downloading %s", entry, name)
+			}
+		}
+	}
+	flog.Debugf("Trying to download %#v with size %#v", name, size)
+	if int(size) > general.MediaDownloadSize {
+		msg.Event = config.EVENT_FILE_FAILURE_SIZE
+		msg.Extra[msg.Event] = append(msg.Extra[msg.Event], config.FileInfo{Name: name, Comment: msg.Text, Size: size})
+		return fmt.Errorf("File %#v to large to download (%#v). MediaDownloadSize is %#v", name, size, general.MediaDownloadSize)
+	}
+	return nil
+}
+
+func HandleDownloadData(flog *log.Entry, msg *config.Message, name, comment, url string, data *[]byte, general *config.Protocol) {
+	var avatar bool
+	flog.Debugf("Download OK %#v %#v", name, len(*data))
+	if msg.Event == config.EVENT_AVATAR_DOWNLOAD {
+		avatar = true
+	}
+	msg.Extra["file"] = append(msg.Extra["file"], config.FileInfo{Name: name, Data: data, URL: url, Comment: comment, Avatar: avatar})
+}
+
+func RemoveEmptyNewLines(msg string) string {
+	lines := ""
+	for _, line := range strings.Split(msg, "\n") {
+		if line != "" {
+			lines += line + "\n"
+		}
+	}
+	lines = strings.TrimRight(lines, "\n")
+	return lines
+}
+
+func ClipMessage(text string, length int) string {
+	// clip too long messages
+	if len(text) > length {
+		text = text[:length-len(" *message clipped*")]
+		if r, size := utf8.DecodeLastRuneInString(text); r == utf8.RuneError {
+			text = text[:len(text)-size]
+		}
+		text += " *message clipped*"
+	}
+	return text
+}

bridge/irc/irc.go

@@ -1,59 +1,62 @@
 package birc
 
 import (
+	"bytes"
 	"crypto/tls"
 	"fmt"
-	"github.com/42wim/go-ircevent"
-	"github.com/42wim/matterbridge/bridge/config"
-	log "github.com/Sirupsen/logrus"
-	"github.com/paulrosania/go-charset/charset"
-	_ "github.com/paulrosania/go-charset/data"
-	"github.com/saintfish/chardet"
-	ircm "github.com/sorcix/irc"
+	"hash/crc32"
 	"io"
 	"io/ioutil"
+	"net"
 	"regexp"
 	"sort"
 	"strconv"
 	"strings"
 	"time"
+	"unicode/utf8"
+
+	"github.com/42wim/matterbridge/bridge"
+	"github.com/42wim/matterbridge/bridge/config"
+	"github.com/42wim/matterbridge/bridge/helper"
+	"github.com/dfordsoft/golib/ic"
+	"github.com/lrstanley/girc"
+	"github.com/paulrosania/go-charset/charset"
+	_ "github.com/paulrosania/go-charset/data"
+	"github.com/saintfish/chardet"
 )
 
 type Birc struct {
-	i               *irc.Connection
-	Nick            string
-	names           map[string][]string
-	Config          *config.Protocol
-	Remote          chan config.Message
-	connected       chan struct{}
-	Local           chan config.Message // local queue for flood control
-	Account         string
-	FirstConnection bool
+	i                                         *girc.Client
+	Nick                                      string
+	names                                     map[string][]string
+	connected                                 chan struct{}
+	Local                                     chan config.Message // local queue for flood control
+	FirstConnection                           bool
+	MessageDelay, MessageQueue, MessageLength int
+
+	*bridge.Config
 }
 
-var flog *log.Entry
-var protocol = "irc"
-
-func init() {
-	flog = log.WithFields(log.Fields{"module": protocol})
-}
-
-func New(cfg config.Protocol, account string, c chan config.Message) *Birc {
+func New(cfg *bridge.Config) bridge.Bridger {
 	b := &Birc{}
-	b.Config = &cfg
-	b.Nick = b.Config.Nick
-	b.Remote = c
+	b.Config = cfg
+	b.Nick = b.GetString("Nick")
 	b.names = make(map[string][]string)
-	b.Account = account
 	b.connected = make(chan struct{})
-	if b.Config.MessageDelay == 0 {
-		b.Config.MessageDelay = 1300
+	if b.GetInt("MessageDelay") == 0 {
+		b.MessageDelay = 1300
+	} else {
+		b.MessageDelay = b.GetInt("MessageDelay")
 	}
-	if b.Config.MessageQueue == 0 {
-		b.Config.MessageQueue = 30
+	if b.GetInt("MessageQueue") == 0 {
+		b.MessageQueue = 30
+	} else {
+		b.MessageQueue = b.GetInt("MessageQueue")
 	}
-	if b.Config.MessageLength == 0 {
-		b.Config.MessageLength = 400
+	if b.GetInt("MessageLength") == 0 {
+		b.MessageLength = 400
+	} else {
+		b.MessageLength = b.GetInt("MessageLength")
 	}
 	b.FirstConnection = true
 	return b
@@ -62,107 +65,207 @@ func New(cfg config.Protocol, account string, c chan config.Message) *Birc {
 func (b *Birc) Command(msg *config.Message) string {
 	switch msg.Text {
 	case "!users":
-		b.i.AddCallback(ircm.RPL_NAMREPLY, b.storeNames)
-		b.i.AddCallback(ircm.RPL_ENDOFNAMES, b.endNames)
-		b.i.SendRaw("NAMES " + msg.Channel)
+		b.i.Handlers.Add(girc.RPL_NAMREPLY, b.storeNames)
+		b.i.Handlers.Add(girc.RPL_ENDOFNAMES, b.endNames)
+		b.i.Cmd.SendRaw("NAMES " + msg.Channel)
 	}
 	return ""
 }
 
 func (b *Birc) Connect() error {
-	b.Local = make(chan config.Message, b.Config.MessageQueue+10)
-	flog.Infof("Connecting %s", b.Config.Server)
-	i := irc.IRC(b.Config.Nick, b.Config.Nick)
-	if log.GetLevel() == log.DebugLevel {
-		i.Debug = true
-	}
-	i.UseTLS = b.Config.UseTLS
-	i.UseSASL = b.Config.UseSASL
-	i.SASLLogin = b.Config.NickServNick
-	i.SASLPassword = b.Config.NickServPassword
-	i.TLSConfig = &tls.Config{InsecureSkipVerify: b.Config.SkipTLSVerify}
-	i.KeepAlive = time.Minute
-	i.PingFreq = time.Minute
-	if b.Config.Password != "" {
-		i.Password = b.Config.Password
-	}
-	i.AddCallback(ircm.RPL_WELCOME, b.handleNewConnection)
-	err := i.Connect(b.Config.Server)
+	b.Local = make(chan config.Message, b.MessageQueue+10)
+	b.Log.Infof("Connecting %s", b.GetString("Server"))
+	server, portstr, err := net.SplitHostPort(b.GetString("Server"))
 	if err != nil {
 		return err
 	}
+	port, err := strconv.Atoi(portstr)
+	if err != nil {
+		return err
+	}
+	// fix strict user handling of girc
+	user := b.GetString("Nick")
+	for !girc.IsValidUser(user) {
+		if len(user) == 1 {
+			user = "matterbridge"
+			break
+		}
+		user = user[1:]
+	}
+
+	i := girc.New(girc.Config{
+		Server:     server,
+		ServerPass: b.GetString("Password"),
+		Port:       port,
+		Nick:       b.GetString("Nick"),
+		User:       user,
+		Name:       b.GetString("Nick"),
+		SSL:        b.GetBool("UseTLS"),
+		TLSConfig:  &tls.Config{InsecureSkipVerify: b.GetBool("SkipTLSVerify"), ServerName: server},
+		PingDelay:  time.Minute,
+	})
+
+	if b.GetBool("UseSASL") {
+		i.Config.SASL = &girc.SASLPlain{b.GetString("NickServNick"), b.GetString("NickServPassword")}
+	}
+
+	i.Handlers.Add(girc.RPL_WELCOME, b.handleNewConnection)
+	i.Handlers.Add(girc.RPL_ENDOFMOTD, b.handleOtherAuth)
+	i.Handlers.Add(girc.ALL_EVENTS, b.handleOther)
+	go func() {
+		for {
+			if err := i.Connect(); err != nil {
+				b.Log.Errorf("disconnect: error: %s", err)
+			} else {
+				b.Log.Info("disconnect: client requested quit")
+			}
+
+			b.Log.Info("reconnecting in 30 seconds...")
+			time.Sleep(30 * time.Second)
+			i.Handlers.Clear(girc.RPL_WELCOME)
+			i.Handlers.Add(girc.RPL_WELCOME, func(client *girc.Client, event girc.Event) {
+				b.Remote <- config.Message{Username: "system", Text: "rejoin", Channel: "", Account: b.Account, Event: config.EVENT_REJOIN_CHANNELS}
+				// set our correct nick on reconnect if necessary
+				b.Nick = event.Source.Name
+			})
+		}
+	}()
 	b.i = i
 	select {
 	case <-b.connected:
-		flog.Info("Connection succeeded")
+		b.Log.Info("Connection succeeded")
 	case <-time.After(time.Second * 30):
 		return fmt.Errorf("connection timed out")
 	}
-	i.Debug = false
-	// clear on reconnects
-	i.ClearCallback(ircm.RPL_WELCOME)
-	i.AddCallback(ircm.RPL_WELCOME, func(event *irc.Event) {
-		b.Remote <- config.Message{Username: "system", Text: "rejoin", Channel: "", Account: b.Account, Event: config.EVENT_REJOIN_CHANNELS}
-		// set our correct nick on reconnect if necessary
-		b.Nick = event.Nick
-	})
-	go i.Loop()
+	//i.Debug = false
+	if b.GetInt("DebugLevel") == 0 {
+		i.Handlers.Clear(girc.ALL_EVENTS)
+	}
 	go b.doSend()
 	return nil
 }
 
 func (b *Birc) Disconnect() error {
-	//b.i.Disconnect()
+	b.i.Close()
 	close(b.Local)
 	return nil
 }
 
 func (b *Birc) JoinChannel(channel config.ChannelInfo) error {
 	if channel.Options.Key != "" {
-		flog.Debugf("using key %s for channel %s", channel.Options.Key, channel.Name)
-		b.i.Join(channel.Name + " " + channel.Options.Key)
+		b.Log.Debugf("using key %s for channel %s", channel.Options.Key, channel.Name)
+		b.i.Cmd.JoinKey(channel.Name, channel.Options.Key)
 	} else {
-		b.i.Join(channel.Name)
+		b.i.Cmd.Join(channel.Name)
 	}
 	return nil
 }
 
-func (b *Birc) Send(msg config.Message) error {
-	flog.Debugf("Receiving %#v", msg)
+func (b *Birc) Send(msg config.Message) (string, error) {
+	// ignore delete messages
+	if msg.Event == config.EVENT_MSG_DELETE {
+		return "", nil
+	}
+
+	b.Log.Debugf("=> Receiving %#v", msg)
+
+	// we can be in between reconnects #385
+	if !b.i.IsConnected() {
+		b.Log.Error("Not connected to server, dropping message")
+	}
+
+	// Execute a command
 	if strings.HasPrefix(msg.Text, "!") {
 		b.Command(&msg)
 	}
-	for _, text := range strings.Split(msg.Text, "\n") {
-		if len(text) > b.Config.MessageLength {
-			text = text[:b.Config.MessageLength] + " <message clipped>"
+
+	// convert to specified charset
+	if b.GetString("Charset") != "" {
+		switch b.GetString("Charset") {
+		case "gbk", "gb18030", "gb2312", "big5", "euc-kr", "euc-jp", "shift-jis", "iso-2022-jp":
+			msg.Text = ic.ConvertString("utf-8", b.GetString("Charset"), msg.Text)
+		default:
+			buf := new(bytes.Buffer)
+			w, err := charset.NewWriter(b.GetString("Charset"), buf)
+			if err != nil {
+				b.Log.Errorf("charset from utf-8 conversion failed: %s", err)
+				return "", err
+			}
+			fmt.Fprint(w, msg.Text)
+			w.Close()
+			msg.Text = buf.String()
 		}
-		if len(b.Local) < b.Config.MessageQueue {
-			if len(b.Local) == b.Config.MessageQueue-1 {
+	}
+
+	// Handle files
+	if msg.Extra != nil {
+		for _, rmsg := range helper.HandleExtra(&msg, b.General) {
+			b.Local <- rmsg
+		}
+		if len(msg.Extra["file"]) > 0 {
+			for _, f := range msg.Extra["file"] {
+				fi := f.(config.FileInfo)
+				if fi.Comment != "" {
+					msg.Text += fi.Comment + ": "
+				}
+				if fi.URL != "" {
+					msg.Text = fi.URL
+					if fi.Comment != "" {
+						msg.Text = fi.Comment + ": " + fi.URL
+					}
+				}
+				b.Local <- config.Message{Text: msg.Text, Username: msg.Username, Channel: msg.Channel, Event: msg.Event}
+			}
+			return "", nil
+		}
+	}
+
+	// split long messages on messageLength, to avoid clipped messages #281
+	if b.GetBool("MessageSplit") {
+		msg.Text = helper.SplitStringLength(msg.Text, b.MessageLength)
+	}
+	for _, text := range strings.Split(msg.Text, "\n") {
+		if len(text) > b.MessageLength {
+			text = text[:b.MessageLength-len(" <message clipped>")]
+			if r, size := utf8.DecodeLastRuneInString(text); r == utf8.RuneError {
+				text = text[:len(text)-size]
+			}
+			text += " <message clipped>"
+		}
+		if len(b.Local) < b.MessageQueue {
+			if len(b.Local) == b.MessageQueue-1 {
 				text = text + " <message clipped>"
 			}
 			b.Local <- config.Message{Text: text, Username: msg.Username, Channel: msg.Channel, Event: msg.Event}
 		} else {
-			flog.Debugf("flooding, dropping message (queue at %d)", len(b.Local))
+			b.Log.Debugf("flooding, dropping message (queue at %d)", len(b.Local))
 		}
 	}
-	return nil
+	return "", nil
 }
 
 func (b *Birc) doSend() {
-	rate := time.Millisecond * time.Duration(b.Config.MessageDelay)
+	rate := time.Millisecond * time.Duration(b.MessageDelay)
 	throttle := time.NewTicker(rate)
 	for msg := range b.Local {
 		<-throttle.C
+		username := msg.Username
+		if b.GetBool("Colornicks") {
+			checksum := crc32.ChecksumIEEE([]byte(msg.Username))
+			colorCode := checksum%14 + 2 // quick fix - prevent white or black color codes
+			username = fmt.Sprintf("\x03%02d%s\x0F", colorCode, msg.Username)
+		}
 		if msg.Event == config.EVENT_USER_ACTION {
-			b.i.Action(msg.Channel, msg.Username+msg.Text)
+			b.i.Cmd.Action(msg.Channel, username+msg.Text)
 		} else {
-			b.i.Privmsg(msg.Channel, msg.Username+msg.Text)
+			b.Log.Debugf("Sending to channel %s", msg.Channel)
+			b.i.Cmd.Message(msg.Channel, username+msg.Text)
 		}
 	}
 }
 
-func (b *Birc) endNames(event *irc.Event) {
-	channel := event.Arguments[1]
+func (b *Birc) endNames(client *girc.Client, event girc.Event) {
+	channel := event.Params[1]
 	sort.Strings(b.names[channel])
 	maxNamesPerPost := (300 / b.nicksPerRow()) * b.nicksPerRow()
 	continued := false
@@ -175,160 +278,188 @@ func (b *Birc) endNames(event *irc.Event) {
 	b.Remote <- config.Message{Username: b.Nick, Text: b.formatnicks(b.names[channel], continued),
 		Channel: channel, Account: b.Account}
 	b.names[channel] = nil
-	b.i.ClearCallback(ircm.RPL_NAMREPLY)
-	b.i.ClearCallback(ircm.RPL_ENDOFNAMES)
+	b.i.Handlers.Clear(girc.RPL_NAMREPLY)
+	b.i.Handlers.Clear(girc.RPL_ENDOFNAMES)
 }
 
-func (b *Birc) handleNewConnection(event *irc.Event) {
-	flog.Debug("Registering callbacks")
+func (b *Birc) handleNewConnection(client *girc.Client, event girc.Event) {
+	b.Log.Debug("Registering callbacks")
 	i := b.i
-	b.Nick = event.Arguments[0]
-	i.AddCallback("PRIVMSG", b.handlePrivMsg)
-	i.AddCallback("CTCP_ACTION", b.handlePrivMsg)
-	i.AddCallback(ircm.RPL_TOPICWHOTIME, b.handleTopicWhoTime)
-	i.AddCallback(ircm.NOTICE, b.handleNotice)
-	//i.AddCallback(ircm.RPL_MYINFO, func(e *irc.Event) { flog.Infof("%s: %s", e.Code, strings.Join(e.Arguments[1:], " ")) })
-	i.AddCallback("PING", func(e *irc.Event) {
-		i.SendRaw("PONG :" + e.Message())
-		flog.Debugf("PING/PONG")
-	})
-	i.AddCallback("JOIN", b.handleJoinPart)
-	i.AddCallback("PART", b.handleJoinPart)
-	i.AddCallback("QUIT", b.handleJoinPart)
-	i.AddCallback("KICK", b.handleJoinPart)
-	i.AddCallback("*", b.handleOther)
+	b.Nick = event.Params[0]
+
+	i.Handlers.Add(girc.RPL_ENDOFMOTD, b.handleOtherAuth)
+	i.Handlers.Add("PRIVMSG", b.handlePrivMsg)
+	i.Handlers.Add("CTCP_ACTION", b.handlePrivMsg)
+	i.Handlers.Add(girc.RPL_TOPICWHOTIME, b.handleTopicWhoTime)
+	i.Handlers.Add(girc.NOTICE, b.handleNotice)
+	i.Handlers.Add("JOIN", b.handleJoinPart)
+	i.Handlers.Add("PART", b.handleJoinPart)
+	i.Handlers.Add("QUIT", b.handleJoinPart)
+	i.Handlers.Add("KICK", b.handleJoinPart)
 	// we are now fully connected
 	b.connected <- struct{}{}
 }
 
-func (b *Birc) handleJoinPart(event *irc.Event) {
-	channel := event.Arguments[0]
-	if event.Code == "KICK" {
-		flog.Infof("Got kicked from %s by %s", channel, event.Nick)
+func (b *Birc) handleJoinPart(client *girc.Client, event girc.Event) {
+	if len(event.Params) == 0 {
+		b.Log.Debugf("handleJoinPart: empty Params? %#v", event)
+		return
+	}
+	channel := strings.ToLower(event.Params[0])
+	if event.Command == "KICK" {
+		b.Log.Infof("Got kicked from %s by %s", channel, event.Source.Name)
+		time.Sleep(time.Duration(b.GetInt("RejoinDelay")) * time.Second)
 		b.Remote <- config.Message{Username: "system", Text: "rejoin", Channel: channel, Account: b.Account, Event: config.EVENT_REJOIN_CHANNELS}
 		return
 	}
-	if event.Code == "QUIT" {
-		if event.Nick == b.Nick && strings.Contains(event.Raw, "Ping timeout") {
-			flog.Infof("%s reconnecting ..", b.Account)
+	if event.Command == "QUIT" {
+		if event.Source.Name == b.Nick && strings.Contains(event.Trailing, "Ping timeout") {
+			b.Log.Infof("%s reconnecting ..", b.Account)
 			b.Remote <- config.Message{Username: "system", Text: "reconnect", Channel: channel, Account: b.Account, Event: config.EVENT_FAILURE}
 			return
 		}
 	}
-	if event.Nick != b.Nick {
-		flog.Debugf("Sending JOIN_LEAVE event from %s to gateway", b.Account)
-		b.Remote <- config.Message{Username: "system", Text: event.Nick + " " + strings.ToLower(event.Code) + "s", Channel: channel, Account: b.Account, Event: config.EVENT_JOIN_LEAVE}
+	if event.Source.Name != b.Nick {
+		if b.GetBool("nosendjoinpart") {
+			return
+		}
+		b.Log.Debugf("<= Sending JOIN_LEAVE event from %s to gateway", b.Account)
+		msg := config.Message{Username: "system", Text: event.Source.Name + " " + strings.ToLower(event.Command) + "s", Channel: channel, Account: b.Account, Event: config.EVENT_JOIN_LEAVE}
+		b.Log.Debugf("<= Message is %#v", msg)
+		b.Remote <- msg
 		return
 	}
-	flog.Debugf("handle %#v", event)
+	b.Log.Debugf("handle %#v", event)
 }
 
-func (b *Birc) handleNotice(event *irc.Event) {
-	if strings.Contains(event.Message(), "This nickname is registered") && event.Nick == b.Config.NickServNick {
-		b.i.Privmsg(b.Config.NickServNick, "IDENTIFY "+b.Config.NickServPassword)
+func (b *Birc) handleNotice(client *girc.Client, event girc.Event) {
+	if strings.Contains(event.String(), "This nickname is registered") && event.Source.Name == b.GetString("NickServNick") {
+		b.i.Cmd.Message(b.GetString("NickServNick"), "IDENTIFY "+b.GetString("NickServPassword"))
 	} else {
-		b.handlePrivMsg(event)
+		b.handlePrivMsg(client, event)
 	}
 }
 
-func (b *Birc) handleOther(event *irc.Event) {
-	switch event.Code {
+func (b *Birc) handleOther(client *girc.Client, event girc.Event) {
+	if b.GetInt("DebugLevel") == 1 {
+		if event.Command != "CLIENT_STATE_UPDATED" &&
+			event.Command != "CLIENT_GENERAL_UPDATED" {
+			b.Log.Debugf("%#v", event.String())
+		}
+		return
+	}
+	switch event.Command {
 	case "372", "375", "376", "250", "251", "252", "253", "254", "255", "265", "266", "002", "003", "004", "005":
 		return
 	}
-	flog.Debugf("%#v", event.Raw)
+	b.Log.Debugf("%#v", event.String())
 }
 
-func (b *Birc) handlePrivMsg(event *irc.Event) {
+func (b *Birc) handleOtherAuth(client *girc.Client, event girc.Event) {
+	if strings.EqualFold(b.GetString("NickServNick"), "Q@CServe.quakenet.org") {
+		b.Log.Debugf("Authenticating %s against %s", b.GetString("NickServUsername"), b.GetString("NickServNick"))
+		b.i.Cmd.Message(b.GetString("NickServNick"), "AUTH "+b.GetString("NickServUsername")+" "+b.GetString("NickServPassword"))
+	}
+}
+
+func (b *Birc) skipPrivMsg(event girc.Event) bool {
+	// Our nick can be changed
 	b.Nick = b.i.GetNick()
+
 	// freenode doesn't send 001 as first reply
-	if event.Code == "NOTICE" {
-		return
+	if event.Command == "NOTICE" {
+		return true
 	}
 	// don't forward queries to the bot
-	if event.Arguments[0] == b.Nick {
-		return
+	if event.Params[0] == b.Nick {
+		return true
 	}
 	// don't forward message from ourself
-	if event.Nick == b.Nick {
+	if event.Source.Name == b.Nick {
+		return true
+	}
+	return false
+}
+
+func (b *Birc) handlePrivMsg(client *girc.Client, event girc.Event) {
+	if b.skipPrivMsg(event) {
 		return
 	}
-	rmsg := config.Message{Username: event.Nick, Channel: event.Arguments[0], Account: b.Account, UserID: event.User + "@" + event.Host}
-	flog.Debugf("handlePrivMsg() %s %s %#v", event.Nick, event.Message(), event)
-	msg := ""
-	if event.Code == "CTCP_ACTION" {
-		//	msg = event.Nick + " "
+	rmsg := config.Message{Username: event.Source.Name, Channel: strings.ToLower(event.Params[0]), Account: b.Account, UserID: event.Source.Ident + "@" + event.Source.Host}
+	b.Log.Debugf("== Receiving PRIVMSG: %s %s %#v", event.Source.Name, event.Trailing, event)
+
+	// set action event
+	if event.IsAction() {
 		rmsg.Event = config.EVENT_USER_ACTION
 	}
-	msg += event.Message()
+
+	// strip action, we made an event if it was an action
+	rmsg.Text += event.StripAction()
+
 	// strip IRC colors
-	re := regexp.MustCompile(`[[:cntrl:]](\d+,|)\d+`)
-	msg = re.ReplaceAllString(msg, "")
+	re := regexp.MustCompile(`\x03(?:\d{1,2}(?:,\d{1,2})?)?|[[:cntrl:]]`)
+	rmsg.Text = re.ReplaceAllString(rmsg.Text, "")
 
-	// detect what were sending so that we convert it to utf-8
-	detector := chardet.NewTextDetector()
-	result, err := detector.DetectBest([]byte(msg))
-	if err != nil {
-		flog.Infof("detection failed for msg: %#v", msg)
-		return
-	}
-	flog.Debugf("detected %s confidence %#v", result.Charset, result.Confidence)
+	// start detecting the charset
 	var r io.Reader
-	r, err = charset.NewReader(result.Charset, strings.NewReader(msg))
-	// if we're not sure, just pick ISO-8859-1
-	if result.Confidence < 80 {
-		r, err = charset.NewReader("ISO-8859-1", strings.NewReader(msg))
+	var err error
+	mycharset := b.GetString("Charset")
+	if mycharset == "" {
+		// detect what were sending so that we convert it to utf-8
+		detector := chardet.NewTextDetector()
+		result, err := detector.DetectBest([]byte(rmsg.Text))
+		if err != nil {
+			b.Log.Infof("detection failed for rmsg.Text: %#v", rmsg.Text)
+			return
+		}
+		b.Log.Debugf("detected %s confidence %#v", result.Charset, result.Confidence)
+		mycharset = result.Charset
+		// if we're not sure, just pick ISO-8859-1
+		if result.Confidence < 80 {
+			mycharset = "ISO-8859-1"
+		}
 	}
-	if err != nil {
-		flog.Errorf("charset to utf-8 conversion failed: %s", err)
-		return
+	switch mycharset {
+	case "gbk", "gb18030", "gb2312", "big5", "euc-kr", "euc-jp", "shift-jis", "iso-2022-jp":
+		rmsg.Text = ic.ConvertString("utf-8", b.GetString("Charset"), rmsg.Text)
+	default:
+		r, err = charset.NewReader(mycharset, strings.NewReader(rmsg.Text))
+		if err != nil {
+			b.Log.Errorf("charset to utf-8 conversion failed: %s", err)
+			return
+		}
+		output, _ := ioutil.ReadAll(r)
+		rmsg.Text = string(output)
 	}
-	output, _ := ioutil.ReadAll(r)
-	msg = string(output)
 
-	flog.Debugf("Sending message from %s on %s to gateway", event.Arguments[0], b.Account)
-	rmsg.Text = msg
+	b.Log.Debugf("<= Sending message from %s on %s to gateway", event.Params[0], b.Account)
 	b.Remote <- rmsg
 }
 
-func (b *Birc) handleTopicWhoTime(event *irc.Event) {
-	parts := strings.Split(event.Arguments[2], "!")
-	t, err := strconv.ParseInt(event.Arguments[3], 10, 64)
+func (b *Birc) handleTopicWhoTime(client *girc.Client, event girc.Event) {
+	parts := strings.Split(event.Params[2], "!")
+	t, err := strconv.ParseInt(event.Params[3], 10, 64)
 	if err != nil {
-		flog.Errorf("Invalid time stamp: %s", event.Arguments[3])
+		b.Log.Errorf("Invalid time stamp: %s", event.Params[3])
 	}
 	user := parts[0]
 	if len(parts) > 1 {
 		user += " [" + parts[1] + "]"
 	}
-	flog.Debugf("%s: Topic set by %s [%s]", event.Code, user, time.Unix(t, 0))
+	b.Log.Debugf("%s: Topic set by %s [%s]", event.Command, user, time.Unix(t, 0))
 }
 
 func (b *Birc) nicksPerRow() int {
 	return 4
-	/*
-		if b.Config.Mattermost.NicksPerRow < 1 {
-			return 4
-		}
-		return b.Config.Mattermost.NicksPerRow
-	*/
 }
 
-func (b *Birc) storeNames(event *irc.Event) {
-	channel := event.Arguments[2]
+func (b *Birc) storeNames(client *girc.Client, event girc.Event) {
+	channel := event.Params[2]
 	b.names[channel] = append(
 		b.names[channel],
-		strings.Split(strings.TrimSpace(event.Message()), " ")...)
+		strings.Split(strings.TrimSpace(event.Trailing), " ")...)
 }
 
 func (b *Birc) formatnicks(nicks []string, continued bool) string {
 	return plainformatter(nicks, b.nicksPerRow())
-	/*
-		switch b.Config.Mattermost.NickFormatter {
-		case "table":
-			return tableformatter(nicks, b.nicksPerRow(), continued)
-		default:
-			return plainformatter(nicks, b.nicksPerRow())
-		}
-	*/
 }

bridge/matrix/matrix.go

@@ -1,60 +1,51 @@
 package bmatrix
 
 import (
+	"bytes"
+	"fmt"
+	"mime"
 	"regexp"
+	"strings"
 	"sync"
 
+	"github.com/42wim/matterbridge/bridge"
 	"github.com/42wim/matterbridge/bridge/config"
-	log "github.com/Sirupsen/logrus"
-	matrix "github.com/matrix-org/gomatrix"
+	"github.com/42wim/matterbridge/bridge/helper"
+	matrix "github.com/matterbridge/gomatrix"
 )
 
 type Bmatrix struct {
 	mc      *matrix.Client
-	Config  *config.Protocol
-	Remote  chan config.Message
-	Account string
 	UserID  string
 	RoomMap map[string]string
 	sync.RWMutex
+	*bridge.Config
 }
 
-var flog *log.Entry
-var protocol = "matrix"
-
-func init() {
-	flog = log.WithFields(log.Fields{"module": protocol})
-}
-
-func New(cfg config.Protocol, account string, c chan config.Message) *Bmatrix {
-	b := &Bmatrix{}
+func New(cfg *bridge.Config) bridge.Bridger {
+	b := &Bmatrix{Config: cfg}
 	b.RoomMap = make(map[string]string)
-	b.Config = &cfg
-	b.Account = account
-	b.Remote = c
 	return b
 }
 
 func (b *Bmatrix) Connect() error {
 	var err error
-	flog.Infof("Connecting %s", b.Config.Server)
-	b.mc, err = matrix.NewClient(b.Config.Server, "", "")
+	b.Log.Infof("Connecting %s", b.GetString("Server"))
+	b.mc, err = matrix.NewClient(b.GetString("Server"), "", "")
 	if err != nil {
-		flog.Debugf("%#v", err)
 		return err
 	}
 	resp, err := b.mc.Login(&matrix.ReqLogin{
 		Type:     "m.login.password",
-		User:     b.Config.Login,
-		Password: b.Config.Password,
+		User:     b.GetString("Login"),
+		Password: b.GetString("Password"),
 	})
 	if err != nil {
-		flog.Debugf("%#v", err)
 		return err
 	}
 	b.mc.SetCredentials(resp.UserID, resp.AccessToken)
 	b.UserID = resp.UserID
-	flog.Info("Connection succeeded")
+	b.Log.Info("Connection succeeded")
 	go b.handlematrix()
 	return nil
 }
@@ -74,17 +65,54 @@ func (b *Bmatrix) JoinChannel(channel config.ChannelInfo) error {
 	return err
 }
 
-func (b *Bmatrix) Send(msg config.Message) error {
-	flog.Debugf("Receiving %#v", msg)
+func (b *Bmatrix) Send(msg config.Message) (string, error) {
+	b.Log.Debugf("=> Receiving %#v", msg)
+
 	channel := b.getRoomID(msg.Channel)
-	flog.Debugf("Sending to channel %s", channel)
+	b.Log.Debugf("Channel %s maps to channel id %s", msg.Channel, channel)
+
+	// Make a action /me of the message
 	if msg.Event == config.EVENT_USER_ACTION {
-		b.mc.SendMessageEvent(channel, "m.room.message",
+		resp, err := b.mc.SendMessageEvent(channel, "m.room.message",
 			matrix.TextMessage{"m.emote", msg.Username + msg.Text})
-		return nil
+		if err != nil {
+			return "", err
+		}
+		return resp.EventID, err
 	}
-	b.mc.SendText(channel, msg.Username+msg.Text)
-	return nil
+
+	// Delete message
+	if msg.Event == config.EVENT_MSG_DELETE {
+		if msg.ID == "" {
+			return "", nil
+		}
+		resp, err := b.mc.RedactEvent(channel, msg.ID, &matrix.ReqRedact{})
+		if err != nil {
+			return "", err
+		}
+		return resp.EventID, err
+	}
+
+	// Upload a file if it exists
+	if msg.Extra != nil {
+		for _, rmsg := range helper.HandleExtra(&msg, b.General) {
+			b.mc.SendText(channel, rmsg.Username+rmsg.Text)
+		}
+		// check if we have files to upload (from slack, telegram or mattermost)
+		if len(msg.Extra["file"]) > 0 {
+			return b.handleUploadFile(&msg, channel)
+		}
+	}
+
+	// Edit message if we have an ID
+	// matrix has no editing support
+
+	// Post normal message
+	resp, err := b.mc.SendText(channel, msg.Username+msg.Text)
+	if err != nil {
+		return "", err
+	}
+	return resp.EventID, err
 }
 
 func (b *Bmatrix) getRoomID(channel string) string {
@@ -97,37 +125,188 @@ func (b *Bmatrix) getRoomID(channel string) string {
 	}
 	return ""
 }
+
 func (b *Bmatrix) handlematrix() error {
 	syncer := b.mc.Syncer.(*matrix.DefaultSyncer)
-	syncer.OnEventType("m.room.message", func(ev *matrix.Event) {
-		if (ev.Content["msgtype"].(string) == "m.text" || ev.Content["msgtype"].(string) == "m.emote") && ev.Sender != b.UserID {
-			b.RLock()
-			channel, ok := b.RoomMap[ev.RoomID]
-			b.RUnlock()
-			if !ok {
-				flog.Debugf("Unknown room %s", ev.RoomID)
-				return
-			}
-			username := ev.Sender[1:]
-			if b.Config.NoHomeServerSuffix {
-				re := regexp.MustCompile("(.*?):.*")
-				username = re.ReplaceAllString(username, `$1`)
-			}
-			rmsg := config.Message{Username: username, Text: ev.Content["body"].(string), Channel: channel, Account: b.Account, UserID: ev.Sender}
-			if ev.Content["msgtype"].(string) == "m.emote" {
-				rmsg.Event = config.EVENT_USER_ACTION
-			}
-			flog.Debugf("Sending message from %s on %s to gateway", ev.Sender, b.Account)
-			b.Remote <- rmsg
-		}
-		flog.Debugf("Received: %#v", ev)
-	})
+	syncer.OnEventType("m.room.redaction", b.handleEvent)
+	syncer.OnEventType("m.room.message", b.handleEvent)
 	go func() {
 		for {
 			if err := b.mc.Sync(); err != nil {
-				flog.Println("Sync() returned ", err)
+				b.Log.Println("Sync() returned ", err)
 			}
 		}
 	}()
 	return nil
 }
+
+func (b *Bmatrix) handleEvent(ev *matrix.Event) {
+	b.Log.Debugf("== Receiving event: %#v", ev)
+	if ev.Sender != b.UserID {
+		b.RLock()
+		channel, ok := b.RoomMap[ev.RoomID]
+		b.RUnlock()
+		if !ok {
+			b.Log.Debugf("Unknown room %s", ev.RoomID)
+			return
+		}
+
+		// TODO download avatar
+
+		// Create our message
+		rmsg := config.Message{Username: ev.Sender[1:], Channel: channel, Account: b.Account, UserID: ev.Sender, ID: ev.ID}
+
+		// Text must be a string
+		if rmsg.Text, ok = ev.Content["body"].(string); !ok {
+			b.Log.Errorf("Content[body] wasn't a %T ?", rmsg.Text)
+			return
+		}
+
+		// Remove homeserver suffix if configured
+		if b.GetBool("NoHomeServerSuffix") {
+			re := regexp.MustCompile("(.*?):.*")
+			rmsg.Username = re.ReplaceAllString(rmsg.Username, `$1`)
+		}
+
+		// Delete event
+		if ev.Type == "m.room.redaction" {
+			rmsg.Event = config.EVENT_MSG_DELETE
+			rmsg.ID = ev.Redacts
+			rmsg.Text = config.EVENT_MSG_DELETE
+			b.Remote <- rmsg
+			return
+		}
+
+		// Do we have a /me action
+		if ev.Content["msgtype"].(string) == "m.emote" {
+			rmsg.Event = config.EVENT_USER_ACTION
+		}
+
+		// Do we have attachments
+		if b.containsAttachment(ev.Content) {
+			err := b.handleDownloadFile(&rmsg, ev.Content)
+			if err != nil {
+				b.Log.Errorf("download failed: %#v", err)
+			}
+		}
+
+		b.Log.Debugf("<= Sending message from %s on %s to gateway", ev.Sender, b.Account)
+		b.Remote <- rmsg
+	}
+}
+
+// handleDownloadFile handles file download
+func (b *Bmatrix) handleDownloadFile(rmsg *config.Message, content map[string]interface{}) error {
+	var (
+		ok                        bool
+		url, name, msgtype, mtype string
+		info                      map[string]interface{}
+		size                      float64
+	)
+
+	rmsg.Extra = make(map[string][]interface{})
+	if url, ok = content["url"].(string); !ok {
+		return fmt.Errorf("url isn't a %T", url)
+	}
+	url = strings.Replace(url, "mxc://", b.GetString("Server")+"/_matrix/media/v1/download/", -1)
+
+	if info, ok = content["info"].(map[string]interface{}); !ok {
+		return fmt.Errorf("info isn't a %T", info)
+	}
+	if size, ok = info["size"].(float64); !ok {
+		return fmt.Errorf("size isn't a %T", size)
+	}
+	if name, ok = content["body"].(string); !ok {
+		return fmt.Errorf("name isn't a %T", name)
+	}
+	if msgtype, ok = content["msgtype"].(string); !ok {
+		return fmt.Errorf("msgtype isn't a %T", msgtype)
+	}
+	if mtype, ok = info["mimetype"].(string); !ok {
+		return fmt.Errorf("mtype isn't a %T", mtype)
+	}
+
+	// check if we have an image uploaded without extension
+	if !strings.Contains(name, ".") {
+		if msgtype == "m.image" {
+			mext, _ := mime.ExtensionsByType(mtype)
+			if len(mext) > 0 {
+				name = name + mext[0]
+			}
+		} else {
+			// just a default .png extension if we don't have mime info
+			name = name + ".png"
+		}
+	}
+
+	// check if the size is ok
+	err := helper.HandleDownloadSize(b.Log, rmsg, name, int64(size), b.General)
+	if err != nil {
+		return err
+	}
+	// actually download the file
+	data, err := helper.DownloadFile(url)
+	if err != nil {
+		return fmt.Errorf("download %s failed %#v", url, err)
+	}
+	// add the downloaded data to the message
+	helper.HandleDownloadData(b.Log, rmsg, name, "", url, data, b.General)
+	return nil
+}
+
+// handleUploadFile handles native upload of files
+func (b *Bmatrix) handleUploadFile(msg *config.Message, channel string) (string, error) {
+	for _, f := range msg.Extra["file"] {
+		fi := f.(config.FileInfo)
+		content := bytes.NewReader(*fi.Data)
+		sp := strings.Split(fi.Name, ".")
+		mtype := mime.TypeByExtension("." + sp[len(sp)-1])
+		if strings.Contains(mtype, "image") ||
+			strings.Contains(mtype, "video") {
+			if fi.Comment != "" {
+				_, err := b.mc.SendText(channel, msg.Username+fi.Comment)
+				if err != nil {
+					b.Log.Errorf("file comment failed: %#v", err)
+				}
+			}
+			b.Log.Debugf("uploading file: %s %s", fi.Name, mtype)
+			res, err := b.mc.UploadToContentRepo(content, mtype, int64(len(*fi.Data)))
+			if err != nil {
+				b.Log.Errorf("file upload failed: %#v", err)
+				continue
+			}
+			if strings.Contains(mtype, "video") {
+				b.Log.Debugf("sendVideo %s", res.ContentURI)
+				_, err = b.mc.SendVideo(channel, fi.Name, res.ContentURI)
+				if err != nil {
+					b.Log.Errorf("sendVideo failed: %#v", err)
+				}
+			}
+			if strings.Contains(mtype, "image") {
+				b.Log.Debugf("sendImage %s", res.ContentURI)
+				_, err = b.mc.SendImage(channel, fi.Name, res.ContentURI)
+				if err != nil {
+					b.Log.Errorf("sendImage failed: %#v", err)
+				}
+			}
+			b.Log.Debugf("result: %#v", res)
+		}
+	}
+	return "", nil
+}
+
+// skipMessages returns true if this message should not be handled
+func (b *Bmatrix) containsAttachment(content map[string]interface{}) bool {
+	// Skip empty messages
+	if content["msgtype"] == nil {
+		return false
+	}
+
+	// Only allow image,video or file msgtypes
+	if !(content["msgtype"].(string) == "m.image" ||
+		content["msgtype"].(string) == "m.video" ||
+		content["msgtype"].(string) == "m.file") {
+		return false
+	}
+	return true
+}

bridge/mattermost/mattermost.go

@@ -2,51 +2,29 @@ package bmattermost
 
 import (
 	"errors"
+	"fmt"
+	"strings"
+
+	"github.com/42wim/matterbridge/bridge"
 	"github.com/42wim/matterbridge/bridge/config"
+	"github.com/42wim/matterbridge/bridge/helper"
 	"github.com/42wim/matterbridge/matterclient"
 	"github.com/42wim/matterbridge/matterhook"
-	log "github.com/Sirupsen/logrus"
-	"strings"
+	"github.com/rs/xid"
 )
 
-type MMhook struct {
-	mh *matterhook.Client
-}
-
-type MMapi struct {
-	mc    *matterclient.MMClient
-	mmMap map[string]string
-}
-
-type MMMessage struct {
-	Text     string
-	Channel  string
-	Username string
-	UserID   string
-}
-
 type Bmattermost struct {
-	MMhook
-	MMapi
-	Config  *config.Protocol
-	Remote  chan config.Message
-	TeamId  string
-	Account string
+	mh     *matterhook.Client
+	mc     *matterclient.MMClient
+	uuid   string
+	TeamID string
+	*bridge.Config
+	avatarMap map[string]string
 }
 
-var flog *log.Entry
-var protocol = "mattermost"
-
-func init() {
-	flog = log.WithFields(log.Fields{"module": protocol})
-}
-
-func New(cfg config.Protocol, account string, c chan config.Message) *Bmattermost {
-	b := &Bmattermost{}
-	b.Config = &cfg
-	b.Remote = c
-	b.Account = account
-	b.mmMap = make(map[string]string)
+func New(cfg *bridge.Config) bridge.Bridger {
+	b := &Bmattermost{Config: cfg, avatarMap: make(map[string]string)}
+	b.uuid = xid.New().String()
 	return b
 }
 
@@ -55,34 +33,47 @@ func (b *Bmattermost) Command(cmd string) string {
 }
 
 func (b *Bmattermost) Connect() error {
-	if b.Config.WebhookBindAddress != "" {
-		if b.Config.WebhookURL != "" {
-			flog.Info("Connecting using webhookurl (sending) and webhookbindaddress (receiving)")
-			b.mh = matterhook.New(b.Config.WebhookURL,
-				matterhook.Config{InsecureSkipVerify: b.Config.SkipTLSVerify,
-					BindAddress: b.Config.WebhookBindAddress})
-		} else if b.Config.Login != "" {
-			flog.Info("Connecting using login/password (sending)")
+	if b.GetString("WebhookBindAddress") != "" {
+		if b.GetString("WebhookURL") != "" {
+			b.Log.Info("Connecting using webhookurl (sending) and webhookbindaddress (receiving)")
+			b.mh = matterhook.New(b.GetString("WebhookURL"),
+				matterhook.Config{InsecureSkipVerify: b.GetBool("SkipTLSVerify"),
+					BindAddress: b.GetString("WebhookBindAddress")})
+		} else if b.GetString("Token") != "" {
+			b.Log.Info("Connecting using token (sending)")
+			err := b.apiLogin()
+			if err != nil {
+				return err
+			}
+		} else if b.GetString("Login") != "" {
+			b.Log.Info("Connecting using login/password (sending)")
 			err := b.apiLogin()
 			if err != nil {
 				return err
 			}
 		} else {
-			flog.Info("Connecting using webhookbindaddress (receiving)")
-			b.mh = matterhook.New(b.Config.WebhookURL,
-				matterhook.Config{InsecureSkipVerify: b.Config.SkipTLSVerify,
-					BindAddress: b.Config.WebhookBindAddress})
+			b.Log.Info("Connecting using webhookbindaddress (receiving)")
+			b.mh = matterhook.New(b.GetString("WebhookURL"),
+				matterhook.Config{InsecureSkipVerify: b.GetBool("SkipTLSVerify"),
+					BindAddress: b.GetString("WebhookBindAddress")})
 		}
 		go b.handleMatter()
 		return nil
 	}
-	if b.Config.WebhookURL != "" {
-		flog.Info("Connecting using webhookurl (sending)")
-		b.mh = matterhook.New(b.Config.WebhookURL,
-			matterhook.Config{InsecureSkipVerify: b.Config.SkipTLSVerify,
+	if b.GetString("WebhookURL") != "" {
+		b.Log.Info("Connecting using webhookurl (sending)")
+		b.mh = matterhook.New(b.GetString("WebhookURL"),
+			matterhook.Config{InsecureSkipVerify: b.GetBool("SkipTLSVerify"),
 				DisableServer: true})
-		if b.Config.Login != "" {
-			flog.Info("Connecting using login/password (receiving)")
+		if b.GetString("Token") != "" {
+			b.Log.Info("Connecting using token (receiving)")
+			err := b.apiLogin()
+			if err != nil {
+				return err
+			}
+			go b.handleMatter()
+		} else if b.GetString("Login") != "" {
+			b.Log.Info("Connecting using login/password (receiving)")
 			err := b.apiLogin()
 			if err != nil {
 				return err
@@ -90,16 +81,23 @@ func (b *Bmattermost) Connect() error {
 			go b.handleMatter()
 		}
 		return nil
-	} else if b.Config.Login != "" {
-		flog.Info("Connecting using login/password (sending and receiving)")
+	} else if b.GetString("Token") != "" {
+		b.Log.Info("Connecting using token (sending and receiving)")
+		err := b.apiLogin()
+		if err != nil {
+			return err
+		}
+		go b.handleMatter()
+	} else if b.GetString("Login") != "" {
+		b.Log.Info("Connecting using login/password (sending and receiving)")
 		err := b.apiLogin()
 		if err != nil {
 			return err
 		}
 		go b.handleMatter()
 	}
-	if b.Config.WebhookBindAddress == "" && b.Config.WebhookURL == "" && b.Config.Login == "" {
-		return errors.New("No connection method found. See that you have WebhookBindAddress, WebhookURL or Login/Password/Server/Team configured.")
+	if b.GetString("WebhookBindAddress") == "" && b.GetString("WebhookURL") == "" && b.GetString("Login") == "" && b.GetString("Token") == "" {
+		return errors.New("no connection method found. See that you have WebhookBindAddress, WebhookURL or Token/Login/Password/Server/Team configured")
 	}
 	return nil
 }
@@ -110,136 +108,358 @@ func (b *Bmattermost) Disconnect() error {
 
 func (b *Bmattermost) JoinChannel(channel config.ChannelInfo) error {
 	// we can only join channels using the API
-	if b.Config.WebhookURL == "" && b.Config.WebhookBindAddress == "" {
-		return b.mc.JoinChannel(b.mc.GetChannelId(channel.Name, ""))
+	if b.GetString("WebhookURL") == "" && b.GetString("WebhookBindAddress") == "" {
+		id := b.mc.GetChannelId(channel.Name, "")
+		if id == "" {
+			return fmt.Errorf("Could not find channel ID for channel %s", channel.Name)
+		}
+		return b.mc.JoinChannel(id)
 	}
 	return nil
 }
 
-func (b *Bmattermost) Send(msg config.Message) error {
-	flog.Debugf("Receiving %#v", msg)
+func (b *Bmattermost) Send(msg config.Message) (string, error) {
+	b.Log.Debugf("=> Receiving %#v", msg)
+
+	// Make a action /me of the message
 	if msg.Event == config.EVENT_USER_ACTION {
 		msg.Text = "*" + msg.Text + "*"
 	}
-	nick := msg.Username
-	message := msg.Text
-	channel := msg.Channel
 
-	if b.Config.PrefixMessagesWithNick {
-		message = nick + message
+	// map the file SHA to our user (caches the avatar)
+	if msg.Event == config.EVENT_AVATAR_DOWNLOAD {
+		return b.cacheAvatar(&msg)
 	}
-	if b.Config.WebhookURL != "" {
-		matterMessage := matterhook.OMessage{IconURL: b.Config.IconURL}
-		matterMessage.IconURL = msg.Avatar
-		matterMessage.Channel = channel
-		matterMessage.UserName = nick
-		matterMessage.Type = ""
-		matterMessage.Text = message
-		err := b.mh.Send(matterMessage)
-		if err != nil {
-			flog.Info(err)
-			return err
+
+	// Use webhook to send the message
+	if b.GetString("WebhookURL") != "" {
+		return b.sendWebhook(msg)
+	}
+
+	// Delete message
+	if msg.Event == config.EVENT_MSG_DELETE {
+		if msg.ID == "" {
+			return "", nil
 		}
-		return nil
+		return msg.ID, b.mc.DeleteMessage(msg.ID)
 	}
-	b.mc.PostMessage(b.mc.GetChannelId(channel, ""), message)
-	return nil
+
+	// Upload a file if it exists
+	if msg.Extra != nil {
+		for _, rmsg := range helper.HandleExtra(&msg, b.General) {
+			b.mc.PostMessage(b.mc.GetChannelId(rmsg.Channel, ""), rmsg.Username+rmsg.Text)
+		}
+		if len(msg.Extra["file"]) > 0 {
+			return b.handleUploadFile(&msg)
+		}
+	}
+
+	// Prepend nick if configured
+	if b.GetBool("PrefixMessagesWithNick") {
+		msg.Text = msg.Username + msg.Text
+	}
+
+	// Edit message if we have an ID
+	if msg.ID != "" {
+		return b.mc.EditMessage(msg.ID, msg.Text)
+	}
+
+	// Post normal message
+	return b.mc.PostMessage(b.mc.GetChannelId(msg.Channel, ""), msg.Text)
 }
 
 func (b *Bmattermost) handleMatter() {
-	mchan := make(chan *MMMessage)
-	if b.Config.WebhookBindAddress != "" {
-		flog.Debugf("Choosing webhooks based receiving")
-		go b.handleMatterHook(mchan)
+	messages := make(chan *config.Message)
+	if b.GetString("WebhookBindAddress") != "" {
+		b.Log.Debugf("Choosing webhooks based receiving")
+		go b.handleMatterHook(messages)
 	} else {
-		flog.Debugf("Choosing login/password based receiving")
-		go b.handleMatterClient(mchan)
-	}
-	for message := range mchan {
-		rmsg := config.Message{Username: message.Username, Channel: message.Channel, Account: b.Account, UserID: message.UserID}
-		text, ok := b.replaceAction(message.Text)
-		if ok {
-			rmsg.Event = config.EVENT_USER_ACTION
+		if b.GetString("Token") != "" {
+			b.Log.Debugf("Choosing token based receiving")
+		} else {
+			b.Log.Debugf("Choosing login/password based receiving")
 		}
-		rmsg.Text = text
-		flog.Debugf("Sending message from %s on %s to gateway", message.Username, b.Account)
-		b.Remote <- rmsg
+		go b.handleMatterClient(messages)
+	}
+	var ok bool
+	for message := range messages {
+		message.Avatar = helper.GetAvatar(b.avatarMap, message.UserID, b.General)
+		message.Account = b.Account
+		if nick := b.mc.GetNickName(message.UserID); nick != "" {
+			message.Username = nick
+		}
+		message.Text, ok = b.replaceAction(message.Text)
+		if ok {
+			message.Event = config.EVENT_USER_ACTION
+		}
+		b.Log.Debugf("<= Sending message from %s on %s to gateway", message.Username, b.Account)
+		b.Log.Debugf("<= Message is %#v", message)
+		b.Remote <- *message
 	}
 }
 
-func (b *Bmattermost) handleMatterClient(mchan chan *MMMessage) {
+func (b *Bmattermost) handleMatterClient(messages chan *config.Message) {
 	for message := range b.mc.MessageChan {
-		flog.Debugf("%#v", message.Raw.Data)
-		if message.Type == "system_join_leave" ||
-			message.Type == "system_join_channel" ||
-			message.Type == "system_leave_channel" {
-			flog.Debugf("Sending JOIN_LEAVE event from %s to gateway", b.Account)
-			b.Remote <- config.Message{Username: "system", Text: message.Text, Channel: message.Channel, Account: b.Account, Event: config.EVENT_JOIN_LEAVE}
+		b.Log.Debugf("%#v", message.Raw.Data)
+
+		if b.skipMessage(message) {
+			b.Log.Debugf("Skipped message: %#v", message)
 			continue
 		}
-		if (message.Raw.Event == "post_edited") && b.Config.EditDisable {
-			continue
+
+		// only download avatars if we have a place to upload them (configured mediaserver)
+		if b.General.MediaServerUpload != "" || b.General.MediaDownloadPath != "" {
+			b.handleDownloadAvatar(message.UserID, message.Channel)
 		}
-		// do not post our own messages back to irc
-		// only listen to message from our team
-		if (message.Raw.Event == "posted" || message.Raw.Event == "post_edited") &&
-			b.mc.User.Username != message.Username && message.Raw.Data["team_id"].(string) == b.TeamId {
-			// if the message has reactions don't repost it (for now, until we can correlate reaction with message)
-			if message.Post.HasReactions {
-				continue
+
+		b.Log.Debugf("== Receiving event %#v", message)
+
+		rmsg := &config.Message{Username: message.Username, UserID: message.UserID, Channel: message.Channel, Text: message.Text, ID: message.Post.Id, Extra: make(map[string][]interface{})}
+
+		// handle mattermost post properties (override username and attachments)
+		props := message.Post.Props
+		if props != nil {
+			if _, ok := props["override_username"].(string); ok {
+				rmsg.Username = props["override_username"].(string)
 			}
-			flog.Debugf("Receiving from matterclient %#v", message)
-			m := &MMMessage{}
-			m.UserID = message.UserID
-			m.Username = message.Username
-			m.Channel = message.Channel
-			m.Text = message.Text
-			if message.Raw.Event == "post_edited" && !b.Config.EditDisable {
-				m.Text = message.Text + b.Config.EditSuffix
-			}
-			if len(message.Post.FileIds) > 0 {
-				for _, link := range b.mc.GetFileLinks(message.Post.FileIds) {
-					m.Text = m.Text + "\n" + link
+			if _, ok := props["attachments"].([]interface{}); ok {
+				rmsg.Extra["attachments"] = props["attachments"].([]interface{})
+				if rmsg.Text == "" {
+					for _, attachment := range rmsg.Extra["attachments"] {
+						attach := attachment.(map[string]interface{})
+						if attach["text"].(string) != "" {
+							rmsg.Text += attach["text"].(string)
+							continue
+						}
+						if attach["fallback"].(string) != "" {
+							rmsg.Text += attach["fallback"].(string)
+						}
+					}
 				}
 			}
-			mchan <- m
 		}
+
+		// create a text for bridges that don't support native editing
+		if message.Raw.Event == "post_edited" && !b.GetBool("EditDisable") {
+			rmsg.Text = message.Text + b.GetString("EditSuffix")
+		}
+
+		if message.Raw.Event == "post_deleted" {
+			rmsg.Event = config.EVENT_MSG_DELETE
+		}
+
+		if len(message.Post.FileIds) > 0 {
+			for _, id := range message.Post.FileIds {
+				err := b.handleDownloadFile(rmsg, id)
+				if err != nil {
+					b.Log.Errorf("download failed: %s", err)
+				}
+			}
+		}
+		messages <- rmsg
 	}
 }
 
-func (b *Bmattermost) handleMatterHook(mchan chan *MMMessage) {
+func (b *Bmattermost) handleMatterHook(messages chan *config.Message) {
 	for {
 		message := b.mh.Receive()
-		flog.Debugf("Receiving from matterhook %#v", message)
-		m := &MMMessage{}
-		m.UserID = message.UserID
-		m.Username = message.UserName
-		m.Text = message.Text
-		m.Channel = message.ChannelName
-		mchan <- m
+		b.Log.Debugf("Receiving from matterhook %#v", message)
+		messages <- &config.Message{UserID: message.UserID, Username: message.UserName, Text: message.Text, Channel: message.ChannelName}
 	}
 }
 
 func (b *Bmattermost) apiLogin() error {
-	b.mc = matterclient.New(b.Config.Login, b.Config.Password,
-		b.Config.Team, b.Config.Server)
-	b.mc.SkipTLSVerify = b.Config.SkipTLSVerify
-	b.mc.NoTLS = b.Config.NoTLS
-	flog.Infof("Connecting %s (team: %s) on %s", b.Config.Login, b.Config.Team, b.Config.Server)
+	password := b.GetString("Password")
+	if b.GetString("Token") != "" {
+		password = "MMAUTHTOKEN=" + b.GetString("Token")
+	}
+
+	b.mc = matterclient.New(b.GetString("Login"), password, b.GetString("Team"), b.GetString("Server"))
+	if b.GetBool("debug") {
+		b.mc.SetLogLevel("debug")
+	}
+	b.mc.SkipTLSVerify = b.GetBool("SkipTLSVerify")
+	b.mc.NoTLS = b.GetBool("NoTLS")
+	b.Log.Infof("Connecting %s (team: %s) on %s", b.GetString("Login"), b.GetString("Team"), b.GetString("Server"))
 	err := b.mc.Login()
 	if err != nil {
 		return err
 	}
-	flog.Info("Connection succeeded")
-	b.TeamId = b.mc.GetTeamId()
+	b.Log.Info("Connection succeeded")
+	b.TeamID = b.mc.GetTeamId()
 	go b.mc.WsReceiver()
 	go b.mc.StatusLoop()
 	return nil
 }
 
+// replaceAction replace the message with the correct action (/me) code
 func (b *Bmattermost) replaceAction(text string) (string, bool) {
 	if strings.HasPrefix(text, "*") && strings.HasSuffix(text, "*") {
 		return strings.Replace(text, "*", "", -1), true
 	}
 	return text, false
 }
+
+func (b *Bmattermost) cacheAvatar(msg *config.Message) (string, error) {
+	fi := msg.Extra["file"][0].(config.FileInfo)
+	/* if we have a sha we have successfully uploaded the file to the media server,
+	so we can now cache the sha */
+	if fi.SHA != "" {
+		b.Log.Debugf("Added %s to %s in avatarMap", fi.SHA, msg.UserID)
+		b.avatarMap[msg.UserID] = fi.SHA
+	}
+	return "", nil
+}
+
+// handleDownloadAvatar downloads the avatar of userid from channel
+// sends a EVENT_AVATAR_DOWNLOAD message to the gateway if successful.
+// logs an error message if it fails
+func (b *Bmattermost) handleDownloadAvatar(userid string, channel string) {
+	rmsg := config.Message{Username: "system", Text: "avatar", Channel: channel, Account: b.Account, UserID: userid, Event: config.EVENT_AVATAR_DOWNLOAD, Extra: make(map[string][]interface{})}
+	if _, ok := b.avatarMap[userid]; !ok {
+		data, resp := b.mc.Client.GetProfileImage(userid, "")
+		if resp.Error != nil {
+			b.Log.Errorf("ProfileImage download failed for %#v %s", userid, resp.Error)
+			return
+		}
+		err := helper.HandleDownloadSize(b.Log, &rmsg, userid+".png", int64(len(data)), b.General)
+		if err != nil {
+			b.Log.Error(err)
+			return
+		}
+		helper.HandleDownloadData(b.Log, &rmsg, userid+".png", rmsg.Text, "", &data, b.General)
+		b.Remote <- rmsg
+	}
+}
+
+// handleDownloadFile handles file download
+func (b *Bmattermost) handleDownloadFile(rmsg *config.Message, id string) error {
+	url, _ := b.mc.Client.GetFileLink(id)
+	finfo, resp := b.mc.Client.GetFileInfo(id)
+	if resp.Error != nil {
+		return resp.Error
+	}
+	err := helper.HandleDownloadSize(b.Log, rmsg, finfo.Name, finfo.Size, b.General)
+	if err != nil {
+		return err
+	}
+	data, resp := b.mc.Client.DownloadFile(id, true)
+	if resp.Error != nil {
+		return resp.Error
+	}
+	helper.HandleDownloadData(b.Log, rmsg, finfo.Name, rmsg.Text, url, &data, b.General)
+	return nil
+}
+
+// handleUploadFile handles native upload of files
+func (b *Bmattermost) handleUploadFile(msg *config.Message) (string, error) {
+	var err error
+	var res, id string
+	channelID := b.mc.GetChannelId(msg.Channel, "")
+	for _, f := range msg.Extra["file"] {
+		fi := f.(config.FileInfo)
+		id, err = b.mc.UploadFile(*fi.Data, channelID, fi.Name)
+		if err != nil {
+			return "", err
+		}
+		msg.Text = fi.Comment
+		if b.GetBool("PrefixMessagesWithNick") {
+			msg.Text = msg.Username + msg.Text
+		}
+		res, err = b.mc.PostMessageWithFiles(channelID, msg.Text, []string{id})
+	}
+	return res, err
+}
+
+// sendWebhook uses the configured WebhookURL to send the message
+func (b *Bmattermost) sendWebhook(msg config.Message) (string, error) {
+	// skip events
+	if msg.Event != "" {
+		return "", nil
+	}
+
+	if b.GetBool("PrefixMessagesWithNick") {
+		msg.Text = msg.Username + msg.Text
+	}
+	if msg.Extra != nil {
+		// this sends a message only if we received a config.EVENT_FILE_FAILURE_SIZE
+		for _, rmsg := range helper.HandleExtra(&msg, b.General) {
+			iconURL := config.GetIconURL(&rmsg, b.GetString("iconurl"))
+			matterMessage := matterhook.OMessage{IconURL: iconURL, Channel: rmsg.Channel, UserName: rmsg.Username, Text: rmsg.Text, Props: make(map[string]interface{})}
+			matterMessage.Props["matterbridge_"+b.uuid] = true
+			b.mh.Send(matterMessage)
+		}
+
+		// webhook doesn't support file uploads, so we add the url manually
+		if len(msg.Extra["file"]) > 0 {
+			for _, f := range msg.Extra["file"] {
+				fi := f.(config.FileInfo)
+				if fi.URL != "" {
+					msg.Text += fi.URL
+				}
+			}
+		}
+	}
+
+	iconURL := config.GetIconURL(&msg, b.GetString("iconurl"))
+	matterMessage := matterhook.OMessage{IconURL: iconURL, Channel: msg.Channel, UserName: msg.Username, Text: msg.Text, Props: make(map[string]interface{})}
+	if msg.Avatar != "" {
+		matterMessage.IconURL = msg.Avatar
+	}
+	matterMessage.Props["matterbridge_"+b.uuid] = true
+	err := b.mh.Send(matterMessage)
+	if err != nil {
+		b.Log.Info(err)
+		return "", err
+	}
+	return "", nil
+}
+
+// skipMessages returns true if this message should not be handled
+func (b *Bmattermost) skipMessage(message *matterclient.Message) bool {
+	// Handle join/leave
+	if message.Type == "system_join_leave" ||
+		message.Type == "system_join_channel" ||
+		message.Type == "system_leave_channel" {
+		if b.GetBool("nosendjoinpart") {
+			return true
+		}
+		b.Log.Debugf("Sending JOIN_LEAVE event from %s to gateway", b.Account)
+		b.Remote <- config.Message{Username: "system", Text: message.Text, Channel: message.Channel, Account: b.Account, Event: config.EVENT_JOIN_LEAVE}
+		return true
+	}
+
+	// Handle edited messages
+	if (message.Raw.Event == "post_edited") && b.GetBool("EditDisable") {
+		return true
+	}
+
+	// Ignore messages sent from matterbridge
+	if message.Post.Props != nil {
+		if _, ok := message.Post.Props["matterbridge_"+b.uuid].(bool); ok {
+			b.Log.Debugf("sent by matterbridge, ignoring")
+			return true
+		}
+	}
+
+	// Ignore messages sent from a user logged in as the bot
+	if b.mc.User.Username == message.Username {
+		return true
+	}
+
+	// if the message has reactions don't repost it (for now, until we can correlate reaction with message)
+	if message.Post.HasReactions {
+		return true
+	}
+
+	// ignore messages from other teams than ours
+	if message.Raw.Data["team_id"].(string) != b.TeamID {
+		return true
+	}
+
+	// only handle posted, edited or deleted events
+	if !(message.Raw.Event == "posted" || message.Raw.Event == "post_edited" || message.Raw.Event == "post_deleted") {
+		return true
+	}
+	return false
+}

bridge/rocketchat/rocketchat.go

@@ -1,10 +1,11 @@
 package brocketchat
 
 import (
+	"github.com/42wim/matterbridge/bridge"
 	"github.com/42wim/matterbridge/bridge/config"
+	"github.com/42wim/matterbridge/bridge/helper"
 	"github.com/42wim/matterbridge/hook/rockethook"
 	"github.com/42wim/matterbridge/matterhook"
-	log "github.com/Sirupsen/logrus"
 )
 
 type MMhook struct {
@@ -14,24 +15,11 @@ type MMhook struct {
 
 type Brocketchat struct {
 	MMhook
-	Config  *config.Protocol
-	Remote  chan config.Message
-	Account string
+	*bridge.Config
 }
 
-var flog *log.Entry
-var protocol = "rocketchat"
-
-func init() {
-	flog = log.WithFields(log.Fields{"module": protocol})
-}
-
-func New(cfg config.Protocol, account string, c chan config.Message) *Brocketchat {
-	b := &Brocketchat{}
-	b.Config = &cfg
-	b.Remote = c
-	b.Account = account
-	return b
+func New(cfg *bridge.Config) bridge.Bridger {
+	return &Brocketchat{Config: cfg}
 }
 
 func (b *Brocketchat) Command(cmd string) string {
@@ -39,11 +27,11 @@ func (b *Brocketchat) Command(cmd string) string {
 }
 
 func (b *Brocketchat) Connect() error {
-	flog.Info("Connecting webhooks")
-	b.mh = matterhook.New(b.Config.WebhookURL,
-		matterhook.Config{InsecureSkipVerify: b.Config.SkipTLSVerify,
+	b.Log.Info("Connecting webhooks")
+	b.mh = matterhook.New(b.GetString("WebhookURL"),
+		matterhook.Config{InsecureSkipVerify: b.GetBool("SkipTLSVerify"),
 			DisableServer: true})
-	b.rh = rockethook.New(b.Config.WebhookURL, rockethook.Config{BindAddress: b.Config.WebhookBindAddress})
+	b.rh = rockethook.New(b.GetString("WebhookURL"), rockethook.Config{BindAddress: b.GetString("WebhookBindAddress")})
 	go b.handleRocketHook()
 	return nil
 }
@@ -57,30 +45,51 @@ func (b *Brocketchat) JoinChannel(channel config.ChannelInfo) error {
 	return nil
 }
 
-func (b *Brocketchat) Send(msg config.Message) error {
-	flog.Debugf("Receiving %#v", msg)
-	matterMessage := matterhook.OMessage{IconURL: b.Config.IconURL}
+func (b *Brocketchat) Send(msg config.Message) (string, error) {
+	// ignore delete messages
+	if msg.Event == config.EVENT_MSG_DELETE {
+		return "", nil
+	}
+	b.Log.Debugf("=> Receiving %#v", msg)
+	if msg.Extra != nil {
+		for _, rmsg := range helper.HandleExtra(&msg, b.General) {
+			iconURL := config.GetIconURL(&rmsg, b.GetString("iconurl"))
+			matterMessage := matterhook.OMessage{IconURL: iconURL, Channel: rmsg.Channel, UserName: rmsg.Username, Text: rmsg.Text}
+			b.mh.Send(matterMessage)
+		}
+		if len(msg.Extra["file"]) > 0 {
+			for _, f := range msg.Extra["file"] {
+				fi := f.(config.FileInfo)
+				if fi.URL != "" {
+					msg.Text += fi.URL
+				}
+			}
+		}
+	}
+
+	iconURL := config.GetIconURL(&msg, b.GetString("iconurl"))
+	matterMessage := matterhook.OMessage{IconURL: iconURL}
 	matterMessage.Channel = msg.Channel
 	matterMessage.UserName = msg.Username
 	matterMessage.Type = ""
 	matterMessage.Text = msg.Text
 	err := b.mh.Send(matterMessage)
 	if err != nil {
-		flog.Info(err)
-		return err
+		b.Log.Info(err)
+		return "", err
 	}
-	return nil
+	return "", nil
 }
 
 func (b *Brocketchat) handleRocketHook() {
 	for {
 		message := b.rh.Receive()
-		flog.Debugf("Receiving from rockethook %#v", message)
+		b.Log.Debugf("Receiving from rockethook %#v", message)
 		// do not loop
-		if message.UserName == b.Config.Nick {
+		if message.UserName == b.GetString("Nick") {
 			continue
 		}
-		flog.Debugf("Sending message from %s on %s to gateway", message.UserName, b.Account)
+		b.Log.Debugf("<= Sending message from %s on %s to gateway", message.UserName, b.Account)
 		b.Remote <- config.Message{Text: message.Text, Username: message.UserName, Channel: message.ChannelName, Account: b.Account, UserID: message.UserID}
 	}
 }

bridge/slack/slack.go

@@ -1,52 +1,41 @@
 package bslack
 
 import (
+	"bytes"
 	"errors"
 	"fmt"
-	"github.com/42wim/matterbridge/bridge/config"
-	"github.com/42wim/matterbridge/matterhook"
-	log "github.com/Sirupsen/logrus"
-	"github.com/nlopes/slack"
 	"html"
 	"regexp"
 	"strings"
+	"sync"
 	"time"
+
+	"github.com/42wim/matterbridge/bridge"
+	"github.com/42wim/matterbridge/bridge/config"
+	"github.com/42wim/matterbridge/bridge/helper"
+	"github.com/42wim/matterbridge/matterhook"
+	"github.com/nlopes/slack"
+	"github.com/rs/xid"
 )
 
-type MMMessage struct {
-	Text     string
-	Channel  string
-	Username string
-	UserID   string
-	Raw      *slack.MessageEvent
-}
-
 type Bslack struct {
-	mh       *matterhook.Client
-	sc       *slack.Client
-	Config   *config.Protocol
-	rtm      *slack.RTM
-	Plus     bool
-	Remote   chan config.Message
-	Users    []slack.User
-	Account  string
-	si       *slack.Info
-	channels []slack.Channel
+	mh           *matterhook.Client
+	sc           *slack.Client
+	rtm          *slack.RTM
+	Users        []slack.User
+	Usergroups   []slack.UserGroup
+	si           *slack.Info
+	channels     []slack.Channel
+	UseChannelID bool
+	uuid         string
+	*bridge.Config
+	sync.RWMutex
 }
 
-var flog *log.Entry
-var protocol = "slack"
+const messageDeleted = "message_deleted"
 
-func init() {
-	flog = log.WithFields(log.Fields{"module": protocol})
-}
-
-func New(cfg config.Protocol, account string, c chan config.Message) *Bslack {
-	b := &Bslack{}
-	b.Config = &cfg
-	b.Remote = c
-	b.Account = account
-	return b
+func New(cfg *bridge.Config) bridge.Bridger {
+	return &Bslack{Config: cfg, uuid: xid.New().String()}
 }
 
 func (b *Bslack) Command(cmd string) string {
@@ -54,130 +43,191 @@ func (b *Bslack) Command(cmd string) string {
 }
 
 func (b *Bslack) Connect() error {
-	if b.Config.WebhookBindAddress != "" {
-		if b.Config.WebhookURL != "" {
-			flog.Info("Connecting using webhookurl (sending) and webhookbindaddress (receiving)")
-			b.mh = matterhook.New(b.Config.WebhookURL,
-				matterhook.Config{InsecureSkipVerify: b.Config.SkipTLSVerify,
-					BindAddress: b.Config.WebhookBindAddress})
-		} else if b.Config.Token != "" {
-			flog.Info("Connecting using token (sending)")
-			b.sc = slack.New(b.Config.Token)
+	b.RLock()
+	defer b.RUnlock()
+	if b.GetString("WebhookBindAddress") != "" {
+		if b.GetString("WebhookURL") != "" {
+			b.Log.Info("Connecting using webhookurl (sending) and webhookbindaddress (receiving)")
+			b.mh = matterhook.New(b.GetString("WebhookURL"),
+				matterhook.Config{InsecureSkipVerify: b.GetBool("SkipTLSVerify"),
+					BindAddress: b.GetString("WebhookBindAddress")})
+		} else if b.GetString("Token") != "" {
+			b.Log.Info("Connecting using token (sending)")
+			b.sc = slack.New(b.GetString("Token"))
 			b.rtm = b.sc.NewRTM()
 			go b.rtm.ManageConnection()
-			flog.Info("Connecting using webhookbindaddress (receiving)")
-			b.mh = matterhook.New(b.Config.WebhookURL,
-				matterhook.Config{InsecureSkipVerify: b.Config.SkipTLSVerify,
-					BindAddress: b.Config.WebhookBindAddress})
+			b.Log.Info("Connecting using webhookbindaddress (receiving)")
+			b.mh = matterhook.New(b.GetString("WebhookURL"),
+				matterhook.Config{InsecureSkipVerify: b.GetBool("SkipTLSVerify"),
+					BindAddress: b.GetString("WebhookBindAddress")})
 		} else {
-			flog.Info("Connecting using webhookbindaddress (receiving)")
-			b.mh = matterhook.New(b.Config.WebhookURL,
-				matterhook.Config{InsecureSkipVerify: b.Config.SkipTLSVerify,
-					BindAddress: b.Config.WebhookBindAddress})
+			b.Log.Info("Connecting using webhookbindaddress (receiving)")
+			b.mh = matterhook.New(b.GetString("WebhookURL"),
+				matterhook.Config{InsecureSkipVerify: b.GetBool("SkipTLSVerify"),
+					BindAddress: b.GetString("WebhookBindAddress")})
 		}
 		go b.handleSlack()
 		return nil
 	}
-	if b.Config.WebhookURL != "" {
-		flog.Info("Connecting using webhookurl (sending)")
-		b.mh = matterhook.New(b.Config.WebhookURL,
-			matterhook.Config{InsecureSkipVerify: b.Config.SkipTLSVerify,
+	if b.GetString("WebhookURL") != "" {
+		b.Log.Info("Connecting using webhookurl (sending)")
+		b.mh = matterhook.New(b.GetString("WebhookURL"),
+			matterhook.Config{InsecureSkipVerify: b.GetBool("SkipTLSVerify"),
 				DisableServer: true})
-		if b.Config.Token != "" {
-			flog.Info("Connecting using token (receiving)")
-			b.sc = slack.New(b.Config.Token)
+		if b.GetString("Token") != "" {
+			b.Log.Info("Connecting using token (receiving)")
+			b.sc = slack.New(b.GetString("Token"))
 			b.rtm = b.sc.NewRTM()
 			go b.rtm.ManageConnection()
 			go b.handleSlack()
 		}
-	} else if b.Config.Token != "" {
-		flog.Info("Connecting using token (sending and receiving)")
-		b.sc = slack.New(b.Config.Token)
+	} else if b.GetString("Token") != "" {
+		b.Log.Info("Connecting using token (sending and receiving)")
+		b.sc = slack.New(b.GetString("Token"))
 		b.rtm = b.sc.NewRTM()
 		go b.rtm.ManageConnection()
 		go b.handleSlack()
 	}
-	if b.Config.WebhookBindAddress == "" && b.Config.WebhookURL == "" && b.Config.Token == "" {
-		return errors.New("No connection method found. See that you have WebhookBindAddress, WebhookURL or Token configured.")
+	if b.GetString("WebhookBindAddress") == "" && b.GetString("WebhookURL") == "" && b.GetString("Token") == "" {
+		return errors.New("no connection method found. See that you have WebhookBindAddress, WebhookURL or Token configured")
 	}
 	return nil
 }
 
 func (b *Bslack) Disconnect() error {
-	return nil
-
+	return b.rtm.Disconnect()
 }
 
 func (b *Bslack) JoinChannel(channel config.ChannelInfo) error {
+	// use ID:channelid and resolve it to the actual name
+	idcheck := strings.Split(channel.Name, "ID:")
+	if len(idcheck) > 1 {
+		b.UseChannelID = true
+		ch, err := b.sc.GetChannelInfo(idcheck[1])
+		if err != nil {
+			return err
+		}
+		channel.Name = ch.Name
+		if err != nil {
+			return err
+		}
+	}
+
 	// we can only join channels using the API
-	if b.Config.WebhookURL == "" && b.Config.WebhookBindAddress == "" {
-		if strings.HasPrefix(b.Config.Token, "xoxb") {
+	if b.sc != nil {
+		if strings.HasPrefix(b.GetString("Token"), "xoxb") {
 			// TODO check if bot has already joined channel
 			return nil
 		}
 		_, err := b.sc.JoinChannel(channel.Name)
 		if err != nil {
-			if err.Error() != "name_taken" {
-				return err
+			switch err.Error() {
+			case "name_taken", "restricted_action":
+			case "default":
+				{
+					return err
+				}
 			}
 		}
 	}
 	return nil
 }
 
-func (b *Bslack) Send(msg config.Message) error {
-	flog.Debugf("Receiving %#v", msg)
+func (b *Bslack) Send(msg config.Message) (string, error) {
+	b.Log.Debugf("=> Receiving %#v", msg)
+
+	// Make a action /me of the message
 	if msg.Event == config.EVENT_USER_ACTION {
 		msg.Text = "_" + msg.Text + "_"
 	}
-	nick := msg.Username
-	message := msg.Text
-	channel := msg.Channel
-	if b.Config.PrefixMessagesWithNick {
-		message = nick + " " + message
+
+	// Use webhook to send the message
+	if b.GetString("WebhookURL") != "" {
+		return b.sendWebhook(msg)
 	}
-	if b.Config.WebhookURL != "" {
-		matterMessage := matterhook.OMessage{IconURL: b.Config.IconURL}
-		matterMessage.Channel = channel
-		matterMessage.UserName = nick
-		matterMessage.Type = ""
-		matterMessage.Text = message
-		err := b.mh.Send(matterMessage)
-		if err != nil {
-			flog.Info(err)
-			return err
+
+	channelID := b.getChannelID(msg.Channel)
+
+	// Delete message
+	if msg.Event == config.EVENT_MSG_DELETE {
+		// some protocols echo deletes, but with empty ID
+		if msg.ID == "" {
+			return "", nil
 		}
-		return nil
+		// we get a "slack <ID>", split it
+		ts := strings.Fields(msg.ID)
+		_, _, err := b.sc.DeleteMessage(channelID, ts[1])
+		if err != nil {
+			return msg.ID, err
+		}
+		return msg.ID, nil
 	}
-	schannel, err := b.getChannelByName(channel)
-	if err != nil {
-		return err
+
+	// Prepend nick if configured
+	if b.GetBool("PrefixMessagesWithNick") {
+		msg.Text = msg.Username + msg.Text
 	}
+
+	// Edit message if we have an ID
+	if msg.ID != "" {
+		ts := strings.Fields(msg.ID)
+		_, _, _, err := b.sc.UpdateMessage(channelID, ts[1], msg.Text)
+		if err != nil {
+			return msg.ID, err
+		}
+		return msg.ID, nil
+	}
+
+	// create slack new post parameters
 	np := slack.NewPostMessageParameters()
-	if b.Config.PrefixMessagesWithNick {
+	if b.GetBool("PrefixMessagesWithNick") {
 		np.AsUser = true
 	}
-	np.Username = nick
-	np.IconURL = config.GetIconURL(&msg, b.Config)
+	np.Username = msg.Username
+	np.LinkNames = 1 // replace mentions
+	np.IconURL = config.GetIconURL(&msg, b.GetString("iconurl"))
 	if msg.Avatar != "" {
 		np.IconURL = msg.Avatar
 	}
-	np.Attachments = append(np.Attachments, slack.Attachment{CallbackID: "matterbridge"})
-	b.sc.PostMessage(schannel.ID, message, np)
+	// add a callback ID so we can see we created it
+	np.Attachments = append(np.Attachments, slack.Attachment{CallbackID: "matterbridge_" + b.uuid})
+	// add file attachments
+	np.Attachments = append(np.Attachments, b.createAttach(msg.Extra)...)
+	// add slack attachments (from another slack bridge)
+	if len(msg.Extra["slack_attachment"]) > 0 {
+		for _, attach := range msg.Extra["slack_attachment"] {
+			np.Attachments = append(np.Attachments, attach.([]slack.Attachment)...)
+		}
+	}
 
-	/*
-	   newmsg := b.rtm.NewOutgoingMessage(message, schannel.ID)
-	   b.rtm.SendMessage(newmsg)
-	*/
+	// Upload a file if it exists
+	if msg.Extra != nil {
+		for _, rmsg := range helper.HandleExtra(&msg, b.General) {
+			b.sc.PostMessage(channelID, rmsg.Username+rmsg.Text, np)
+		}
+		// check if we have files to upload (from slack, telegram or mattermost)
+		if len(msg.Extra["file"]) > 0 {
+			b.handleUploadFile(&msg, channelID)
+		}
+	}
 
-	return nil
+	// Post normal message
+	_, id, err := b.sc.PostMessage(channelID, msg.Text, np)
+	if err != nil {
+		return "", err
+	}
+	return "slack " + id, nil
 }
 
-func (b *Bslack) getAvatar(user string) string {
+func (b *Bslack) Reload(cfg *bridge.Config) (string, error) {
+	return "", nil
+}
+
+func (b *Bslack) getAvatar(userid string) string {
 	var avatar string
 	if b.Users != nil {
 		for _, u := range b.Users {
-			if user == u.Name {
+			if userid == u.ID {
 				return u.Profile.Image48
 			}
 		}
@@ -185,6 +235,7 @@ func (b *Bslack) getAvatar(user string) string {
 	return avatar
 }
 
+/*
 func (b *Bslack) getChannelByName(name string) (*slack.Channel, error) {
 	if b.channels == nil {
 		return nil, fmt.Errorf("%s: channel %s not found (no channels found)", b.Account, name)
@@ -196,6 +247,7 @@ func (b *Bslack) getChannelByName(name string) (*slack.Channel, error) {
 	}
 	return nil, fmt.Errorf("%s: channel %s not found", b.Account, name)
 }
+*/
 
 func (b *Bslack) getChannelByID(ID string) (*slack.Channel, error) {
 	if b.channels == nil {
@@ -210,106 +262,61 @@ func (b *Bslack) getChannelByID(ID string) (*slack.Channel, error) {
 }
 
 func (b *Bslack) handleSlack() {
-	mchan := make(chan *MMMessage)
-	if b.Config.WebhookBindAddress != "" {
-		flog.Debugf("Choosing webhooks based receiving")
-		go b.handleMatterHook(mchan)
+	messages := make(chan *config.Message)
+	if b.GetString("WebhookBindAddress") != "" {
+		b.Log.Debugf("Choosing webhooks based receiving")
+		go b.handleMatterHook(messages)
 	} else {
-		flog.Debugf("Choosing token based receiving")
-		go b.handleSlackClient(mchan)
+		b.Log.Debugf("Choosing token based receiving")
+		go b.handleSlackClient(messages)
 	}
 	time.Sleep(time.Second)
-	flog.Debug("Start listening for Slack messages")
-	for message := range mchan {
-		// do not send messages from ourself
-		if b.Config.WebhookURL == "" && b.Config.WebhookBindAddress == "" && message.Username == b.si.User.Name {
-			continue
-		}
-		if message.Text == "" || message.Username == "" {
-			continue
-		}
-		texts := strings.Split(message.Text, "\n")
-		for _, text := range texts {
-			text = b.replaceURL(text)
-			text = html.UnescapeString(text)
-			flog.Debugf("Sending message from %s on %s to gateway", message.Username, b.Account)
-			msg := config.Message{Text: text, Username: message.Username, Channel: message.Channel, Account: b.Account, Avatar: b.getAvatar(message.Username), UserID: message.UserID}
-			if message.Raw.SubType == "me_message" {
-				msg.Event = config.EVENT_USER_ACTION
-			}
-			b.Remote <- msg
-		}
+	b.Log.Debug("Start listening for Slack messages")
+	for message := range messages {
+		b.Log.Debugf("<= Sending message from %s on %s to gateway", message.Username, b.Account)
+
+		// cleanup the message
+		message.Text = b.replaceMention(message.Text)
+		message.Text = b.replaceVariable(message.Text)
+		message.Text = b.replaceChannel(message.Text)
+		message.Text = b.replaceURL(message.Text)
+		message.Text = html.UnescapeString(message.Text)
+
+		// Add the avatar
+		message.Avatar = b.getAvatar(message.UserID)
+
+		b.Log.Debugf("<= Message is %#v", message)
+		b.Remote <- *message
 	}
 }
 
-func (b *Bslack) handleSlackClient(mchan chan *MMMessage) {
-	count := 0
+func (b *Bslack) handleSlackClient(messages chan *config.Message) {
 	for msg := range b.rtm.IncomingEvents {
+		if msg.Type != "user_typing" && msg.Type != "latency_report" {
+			b.Log.Debugf("== Receiving event %#v", msg.Data)
+		}
 		switch ev := msg.Data.(type) {
 		case *slack.MessageEvent:
-			// ignore first message
-			if count > 0 {
-				flog.Debugf("Receiving from slackclient %#v", ev)
-				if len(ev.Attachments) > 0 {
-					// skip messages we made ourselves
-					if ev.Attachments[0].CallbackID == "matterbridge" {
-						continue
-					}
-				}
-				if !b.Config.EditDisable && ev.SubMessage != nil && ev.SubMessage.ThreadTimestamp != ev.SubMessage.Timestamp {
-					flog.Debugf("SubMessage %#v", ev.SubMessage)
-					ev.User = ev.SubMessage.User
-					ev.Text = ev.SubMessage.Text + b.Config.EditSuffix
-				}
-				// use our own func because rtm.GetChannelInfo doesn't work for private channels
-				channel, err := b.getChannelByID(ev.Channel)
-				if err != nil {
-					continue
-				}
-				m := &MMMessage{}
-				if ev.BotID == "" {
-					user, err := b.rtm.GetUserInfo(ev.User)
-					if err != nil {
-						continue
-					}
-					m.UserID = user.ID
-					m.Username = user.Name
-				}
-				m.Channel = channel.Name
-				m.Text = ev.Text
-				if m.Text == "" {
-					for _, attach := range ev.Attachments {
-						if attach.Text != "" {
-							m.Text = attach.Text
-						} else {
-							m.Text = attach.Fallback
-						}
-					}
-				}
-				m.Raw = ev
-				m.Text = b.replaceMention(m.Text)
-				// when using webhookURL we can't check if it's our webhook or not for now
-				if ev.BotID != "" && b.Config.WebhookURL == "" {
-					bot, err := b.rtm.GetBotInfo(ev.BotID)
-					if err != nil {
-						continue
-					}
-					if bot.Name != "" {
-						m.Username = bot.Name
-						m.UserID = bot.ID
-					}
-				}
-				mchan <- m
+			if b.skipMessageEvent(ev) {
+				b.Log.Debugf("Skipped message: %#v", ev)
+				continue
 			}
-			count++
+			rmsg, err := b.handleMessageEvent(ev)
+			if err != nil {
+				b.Log.Errorf("%#v", err)
+				continue
+			}
+			messages <- rmsg
 		case *slack.OutgoingErrorEvent:
-			flog.Debugf("%#v", ev.Error())
+			b.Log.Debugf("%#v", ev.Error())
 		case *slack.ChannelJoinedEvent:
 			b.Users, _ = b.sc.GetUsers()
+			b.Usergroups, _ = b.sc.GetUserGroups()
 		case *slack.ConnectedEvent:
 			b.channels = ev.Info.Channels
 			b.si = ev.Info
 			b.Users, _ = b.sc.GetUsers()
+			b.Usergroups, _ = b.sc.GetUserGroups()
 			// add private channels
 			groups, _ := b.sc.GetGroups(true)
 			for _, g := range groups {
@@ -319,50 +326,389 @@ func (b *Bslack) handleSlackClient(mchan chan *MMMessage) {
 				b.channels = append(b.channels, *channel)
 			}
 		case *slack.InvalidAuthEvent:
-			flog.Fatalf("Invalid Token %#v", ev)
+			b.Log.Fatalf("Invalid Token %#v", ev)
+		case *slack.ConnectionErrorEvent:
+			b.Log.Errorf("Connection failed %#v %#v", ev.Error(), ev.ErrorObj)
 		default:
 		}
 	}
 }
 
-func (b *Bslack) handleMatterHook(mchan chan *MMMessage) {
+func (b *Bslack) handleMatterHook(messages chan *config.Message) {
 	for {
 		message := b.mh.Receive()
-		flog.Debugf("receiving from matterhook (slack) %#v", message)
-		m := &MMMessage{}
-		m.Username = message.UserName
-		m.Text = message.Text
-		m.Text = b.replaceMention(m.Text)
-		m.Channel = message.ChannelName
-		if m.Username == "slackbot" {
+		b.Log.Debugf("receiving from matterhook (slack) %#v", message)
+		if message.UserName == "slackbot" {
 			continue
 		}
-		mchan <- m
+		messages <- &config.Message{Username: message.UserName, Text: message.Text, Channel: message.ChannelName}
 	}
 }
 
 func (b *Bslack) userName(id string) string {
 	for _, u := range b.Users {
 		if u.ID == id {
+			if u.Profile.DisplayName != "" {
+				return u.Profile.DisplayName
+			}
 			return u.Name
 		}
 	}
 	return ""
 }
 
+/*
+func (b *Bslack) userGroupName(id string) string {
+	for _, u := range b.Usergroups {
+		if u.ID == id {
+			return u.Name
+		}
+	}
+	return ""
+}
+*/
+
+// @see https://api.slack.com/docs/message-formatting#linking_to_channels_and_users
 func (b *Bslack) replaceMention(text string) string {
-	results := regexp.MustCompile(`<@([a-zA-z0-9]+)>`).FindAllStringSubmatch(text, -1)
+	results := regexp.MustCompile(`<@([a-zA-Z0-9]+)>`).FindAllStringSubmatch(text, -1)
 	for _, r := range results {
 		text = strings.Replace(text, "<@"+r[1]+">", "@"+b.userName(r[1]), -1)
-
 	}
 	return text
 }
 
-func (b *Bslack) replaceURL(text string) string {
-	results := regexp.MustCompile(`<(.*?)\|.*?>`).FindAllStringSubmatch(text, -1)
+// @see https://api.slack.com/docs/message-formatting#linking_to_channels_and_users
+func (b *Bslack) replaceChannel(text string) string {
+	results := regexp.MustCompile(`<#[a-zA-Z0-9]+\|(.+?)>`).FindAllStringSubmatch(text, -1)
 	for _, r := range results {
-		text = strings.Replace(text, r[0], r[1], -1)
+		text = strings.Replace(text, r[0], "#"+r[1], -1)
 	}
 	return text
 }
+
+// @see https://api.slack.com/docs/message-formatting#variables
+func (b *Bslack) replaceVariable(text string) string {
+	results := regexp.MustCompile(`<!((?:subteam\^)?[a-zA-Z0-9]+)(?:\|@?(.+?))?>`).FindAllStringSubmatch(text, -1)
+	for _, r := range results {
+		if r[2] != "" {
+			text = strings.Replace(text, r[0], "@"+r[2], -1)
+		} else {
+			text = strings.Replace(text, r[0], "@"+r[1], -1)
+		}
+	}
+	return text
+}
+
+// @see https://api.slack.com/docs/message-formatting#linking_to_urls
+func (b *Bslack) replaceURL(text string) string {
+	results := regexp.MustCompile(`<(.*?)(\|.*?)?>`).FindAllStringSubmatch(text, -1)
+	for _, r := range results {
+		if len(strings.TrimSpace(r[2])) == 1 { // A display text separator was found, but the text was blank
+			text = strings.Replace(text, r[0], "", -1)
+		} else {
+			text = strings.Replace(text, r[0], r[1], -1)
+		}
+	}
+	return text
+}
+
+func (b *Bslack) createAttach(extra map[string][]interface{}) []slack.Attachment {
+	var attachs []slack.Attachment
+	for _, v := range extra["attachments"] {
+		entry := v.(map[string]interface{})
+		s := slack.Attachment{}
+		s.Fallback = entry["fallback"].(string)
+		s.Color = entry["color"].(string)
+		s.Pretext = entry["pretext"].(string)
+		s.AuthorName = entry["author_name"].(string)
+		s.AuthorLink = entry["author_link"].(string)
+		s.AuthorIcon = entry["author_icon"].(string)
+		s.Title = entry["title"].(string)
+		s.TitleLink = entry["title_link"].(string)
+		s.Text = entry["text"].(string)
+		s.ImageURL = entry["image_url"].(string)
+		s.ThumbURL = entry["thumb_url"].(string)
+		s.Footer = entry["footer"].(string)
+		s.FooterIcon = entry["footer_icon"].(string)
+		attachs = append(attachs, s)
+	}
+	return attachs
+}
+
+// handleDownloadFile handles file download
+func (b *Bslack) handleDownloadFile(rmsg *config.Message, file *slack.File) error {
+	// if we have a file attached, download it (in memory) and put a pointer to it in msg.Extra
+	// limit to 1MB for now
+	comment := ""
+	results := regexp.MustCompile(`.*?commented: (.*)`).FindAllStringSubmatch(rmsg.Text, -1)
+	if len(results) > 0 {
+		comment = results[0][1]
+	}
+
+	err := helper.HandleDownloadSize(b.Log, rmsg, file.Name, int64(file.Size), b.General)
+	if err != nil {
+		return err
+	}
+	// actually download the file
+	data, err := helper.DownloadFileAuth(file.URLPrivateDownload, "Bearer "+b.GetString("Token"))
+	if err != nil {
+		return fmt.Errorf("download %s failed %#v", file.URLPrivateDownload, err)
+	}
+	// add the downloaded data to the message
+	helper.HandleDownloadData(b.Log, rmsg, file.Name, comment, file.URLPrivateDownload, data, b.General)
+	return nil
+}
+
+// handleUploadFile handles native upload of files
+func (b *Bslack) handleUploadFile(msg *config.Message, channelID string) (string, error) {
+	var err error
+	for _, f := range msg.Extra["file"] {
+		fi := f.(config.FileInfo)
+		_, err = b.sc.UploadFile(slack.FileUploadParameters{
+			Reader:         bytes.NewReader(*fi.Data),
+			Filename:       fi.Name,
+			Channels:       []string{channelID},
+			InitialComment: fi.Comment,
+		})
+		if err != nil {
+			b.Log.Errorf("uploadfile %#v", err)
+		}
+	}
+	return "", nil
+}
+
+// handleMessageEvent handles the message events
+func (b *Bslack) handleMessageEvent(ev *slack.MessageEvent) (*config.Message, error) {
+	// update the userlist on a channel_join
+	if ev.SubType == "channel_join" {
+		b.Users, _ = b.sc.GetUsers()
+	}
+
+	// Edit message
+	if !b.GetBool("EditDisable") && ev.SubMessage != nil && ev.SubMessage.ThreadTimestamp != ev.SubMessage.Timestamp {
+		b.Log.Debugf("SubMessage %#v", ev.SubMessage)
+		ev.User = ev.SubMessage.User
+		ev.Text = ev.SubMessage.Text + b.GetString("EditSuffix")
+	}
+
+	// use our own func because rtm.GetChannelInfo doesn't work for private channels
+	channel, err := b.getChannelByID(ev.Channel)
+	if err != nil {
+		return nil, err
+	}
+
+	rmsg := config.Message{Text: ev.Text, Channel: channel.Name, Account: b.Account, ID: "slack " + ev.Timestamp, Extra: make(map[string][]interface{})}
+
+	if b.UseChannelID {
+		rmsg.Channel = "ID:" + channel.ID
+	}
+
+	// find the user id and name
+	if ev.User != "" && ev.SubType != messageDeleted && ev.SubType != "file_comment" {
+		user, err := b.rtm.GetUserInfo(ev.User)
+		if err != nil {
+			return nil, err
+		}
+		rmsg.UserID = user.ID
+		rmsg.Username = user.Name
+		if user.Profile.DisplayName != "" {
+			rmsg.Username = user.Profile.DisplayName
+		}
+	}
+
+	// See if we have some text in the attachments
+	if rmsg.Text == "" {
+		for _, attach := range ev.Attachments {
+			if attach.Text != "" {
+				if attach.Title != "" {
+					rmsg.Text = attach.Title + "\n"
+				}
+				rmsg.Text += attach.Text
+			} else {
+				rmsg.Text = attach.Fallback
+			}
+		}
+	}
+
+	// when using webhookURL we can't check if it's our webhook or not for now
+	if rmsg.Username == "" && ev.BotID != "" && b.GetString("WebhookURL") == "" {
+		bot, err := b.rtm.GetBotInfo(ev.BotID)
+		if err != nil {
+			return nil, err
+		}
+		if bot.Name != "" {
+			rmsg.Username = bot.Name
+			if ev.Username != "" {
+				rmsg.Username = ev.Username
+			}
+			rmsg.UserID = bot.ID
+		}
+
+		// fixes issues with matterircd users
+		if bot.Name == "Slack API Tester" {
+			user, err := b.rtm.GetUserInfo(ev.User)
+			if err != nil {
+				return nil, err
+			}
+			rmsg.UserID = user.ID
+			rmsg.Username = user.Name
+			if user.Profile.DisplayName != "" {
+				rmsg.Username = user.Profile.DisplayName
+			}
+		}
+	}
+
+	// file comments are set by the system (because there is no username given)
+	if ev.SubType == "file_comment" {
+		rmsg.Username = "system"
+	}
+
+	// do we have a /me action
+	if ev.SubType == "me_message" {
+		rmsg.Event = config.EVENT_USER_ACTION
+	}
+
+	// Handle join/leave
+	if ev.SubType == "channel_leave" || ev.SubType == "channel_join" {
+		rmsg.Username = "system"
+		rmsg.Event = config.EVENT_JOIN_LEAVE
+	}
+
+	// edited messages have a submessage, use this timestamp
+	if ev.SubMessage != nil {
+		rmsg.ID = "slack " + ev.SubMessage.Timestamp
+	}
+
+	// deleted message event
+	if ev.SubType == messageDeleted {
+		rmsg.Text = config.EVENT_MSG_DELETE
+		rmsg.Event = config.EVENT_MSG_DELETE
+		rmsg.ID = "slack " + ev.DeletedTimestamp
+	}
+
+	// topic change event
+	if ev.SubType == "channel_topic" || ev.SubType == "channel_purpose" {
+		rmsg.Event = config.EVENT_TOPIC_CHANGE
+	}
+
+	// Only deleted messages can have a empty username and text
+	if (rmsg.Text == "" || rmsg.Username == "") && ev.SubType != messageDeleted {
+		// this is probably a webhook we couldn't resolve
+		if ev.BotID != "" {
+			return nil, fmt.Errorf("probably an incoming webhook we couldn't resolve (maybe ourselves)")
+		}
+		return nil, fmt.Errorf("empty message and not a deleted message")
+	}
+
+	// save the attachments, so that we can send them to other slack (compatible) bridges
+	if len(ev.Attachments) > 0 {
+		rmsg.Extra["slack_attachment"] = append(rmsg.Extra["slack_attachment"], ev.Attachments)
+	}
+
+	// if we have a file attached, download it (in memory) and put a pointer to it in msg.Extra
+	if ev.File != nil {
+		err := b.handleDownloadFile(&rmsg, ev.File)
+		if err != nil {
+			b.Log.Errorf("download failed: %s", err)
+		}
+	}
+
+	return &rmsg, nil
+}
+
+// sendWebhook uses the configured WebhookURL to send the message
+func (b *Bslack) sendWebhook(msg config.Message) (string, error) {
+	// skip events
+	if msg.Event != "" {
+		return "", nil
+	}
+
+	if b.GetBool("PrefixMessagesWithNick") {
+		msg.Text = msg.Username + msg.Text
+	}
+
+	if msg.Extra != nil {
+		// this sends a message only if we received a config.EVENT_FILE_FAILURE_SIZE
+		for _, rmsg := range helper.HandleExtra(&msg, b.General) {
+			iconURL := config.GetIconURL(&rmsg, b.GetString("iconurl"))
+			matterMessage := matterhook.OMessage{IconURL: iconURL, Channel: msg.Channel, UserName: rmsg.Username, Text: rmsg.Text}
+			b.mh.Send(matterMessage)
+		}
+
+		// webhook doesn't support file uploads, so we add the url manually
+		if len(msg.Extra["file"]) > 0 {
+			for _, f := range msg.Extra["file"] {
+				fi := f.(config.FileInfo)
+				if fi.URL != "" {
+					msg.Text += " " + fi.URL
+				}
+			}
+		}
+	}
+
+	// if we have native slack_attachments add them
+	var attachs []slack.Attachment
+	if len(msg.Extra["slack_attachment"]) > 0 {
+		for _, attach := range msg.Extra["slack_attachment"] {
+			attachs = append(attachs, attach.([]slack.Attachment)...)
+		}
+	}
+
+	iconURL := config.GetIconURL(&msg, b.GetString("iconurl"))
+	matterMessage := matterhook.OMessage{IconURL: iconURL, Attachments: attachs, Channel: msg.Channel, UserName: msg.Username, Text: msg.Text}
+	if msg.Avatar != "" {
+		matterMessage.IconURL = msg.Avatar
+	}
+	err := b.mh.Send(matterMessage)
+	if err != nil {
+		b.Log.Error(err)
+		return "", err
+	}
+	return "", nil
+}
+
+// skipMessageEvent skips event that need to be skipped :-)
+func (b *Bslack) skipMessageEvent(ev *slack.MessageEvent) bool {
+	if ev.SubType == "channel_leave" || ev.SubType == "channel_join" {
+		return b.GetBool("nosendjoinpart")
+	}
+
+	// ignore pinned items
+	if ev.SubType == "pinned_item" || ev.SubType == "unpinned_item" {
+		return true
+	}
+
+	// do not send messages from ourself
+	if b.GetString("WebhookURL") == "" && b.GetString("WebhookBindAddress") == "" && ev.Username == b.si.User.Name {
+		return true
+	}
+
+	// skip messages we made ourselves
+	if len(ev.Attachments) > 0 {
+		if ev.Attachments[0].CallbackID == "matterbridge_"+b.uuid {
+			return true
+		}
+	}
+
+	if !b.GetBool("EditDisable") && ev.SubMessage != nil && ev.SubMessage.ThreadTimestamp != ev.SubMessage.Timestamp {
+		// it seems ev.SubMessage.Edited == nil when slack unfurls
+		// do not forward these messages #266
+		if ev.SubMessage.Edited == nil {
+			return true
+		}
+	}
+	return false
+}
+
+func (b *Bslack) getChannelID(name string) string {
+	idcheck := strings.Split(name, "ID:")
+	if len(idcheck) > 1 {
+		return idcheck[1]
+	}
+	for _, channel := range b.channels {
+		if channel.Name == name {
+			return channel.ID
+		}
+	}
+	return ""
+}

bridge/sshchat/sshchat.go

@@ -0,0 +1,141 @@
+package bsshchat
+
+import (
+	"bufio"
+	"io"
+	"strings"
+
+	"github.com/42wim/matterbridge/bridge"
+	"github.com/42wim/matterbridge/bridge/config"
+	"github.com/42wim/matterbridge/bridge/helper"
+	"github.com/shazow/ssh-chat/sshd"
+	log "github.com/sirupsen/logrus"
+)
+
+type Bsshchat struct {
+	r *bufio.Scanner
+	w io.WriteCloser
+	*bridge.Config
+}
+
+func New(cfg *bridge.Config) bridge.Bridger {
+	return &Bsshchat{Config: cfg}
+}
+
+func (b *Bsshchat) Connect() error {
+	var err error
+	b.Log.Infof("Connecting %s", b.GetString("Server"))
+	go func() {
+		err = sshd.ConnectShell(b.GetString("Server"), b.GetString("Nick"), func(r io.Reader, w io.WriteCloser) error {
+			b.r = bufio.NewScanner(r)
+			b.w = w
+			b.r.Scan()
+			w.Write([]byte("/theme mono\r\n"))
+			b.handleSshChat()
+			return nil
+		})
+	}()
+	if err != nil {
+		b.Log.Debugf("%#v", err)
+		return err
+	}
+	b.Log.Info("Connection succeeded")
+	return nil
+}
+
+func (b *Bsshchat) Disconnect() error {
+	return nil
+}
+
+func (b *Bsshchat) JoinChannel(channel config.ChannelInfo) error {
+	return nil
+}
+
+func (b *Bsshchat) Send(msg config.Message) (string, error) {
+	// ignore delete messages
+	if msg.Event == config.EVENT_MSG_DELETE {
+		return "", nil
+	}
+	b.Log.Debugf("=> Receiving %#v", msg)
+	if msg.Extra != nil {
+		for _, rmsg := range helper.HandleExtra(&msg, b.General) {
+			b.w.Write([]byte(rmsg.Username + rmsg.Text + "\r\n"))
+		}
+		if len(msg.Extra["file"]) > 0 {
+			for _, f := range msg.Extra["file"] {
+				fi := f.(config.FileInfo)
+				if fi.Comment != "" {
+					msg.Text += fi.Comment + ": "
+				}
+				if fi.URL != "" {
+					msg.Text = fi.URL
+					if fi.Comment != "" {
+						msg.Text = fi.Comment + ": " + fi.URL
+					}
+				}
+				b.w.Write([]byte(msg.Username + msg.Text))
+			}
+			return "", nil
+		}
+	}
+	b.w.Write([]byte(msg.Username + msg.Text + "\r\n"))
+	return "", nil
+}
+
+/*
+func (b *Bsshchat) sshchatKeepAlive() chan bool {
+	done := make(chan bool)
+	go func() {
+		ticker := time.NewTicker(90 * time.Second)
+		defer ticker.Stop()
+		for {
+			select {
+			case <-ticker.C:
+				b.Log.Debugf("PING")
+				err := b.xc.PingC2S("", "")
+				if err != nil {
+					b.Log.Debugf("PING failed %#v", err)
+				}
+			case <-done:
+				return
+			}
+		}
+	}()
+	return done
+}
+*/
+
+func stripPrompt(s string) string {
+	pos := strings.LastIndex(s, "\033[K")
+	if pos < 0 {
+		return s
+	}
+	return s[pos+3:]
+}
+
+func (b *Bsshchat) handleSshChat() error {
+	/*
+		done := b.sshchatKeepAlive()
+		defer close(done)
+	*/
+	wait := true
+	for {
+		if b.r.Scan() {
+			// ignore messages from ourselves
+			if !strings.Contains(b.r.Text(), "\033[K") {
+				continue
+			}
+			res := strings.Split(stripPrompt(b.r.Text()), ":")
+			if res[0] == "-> Set theme" {
+				wait = false
+				log.Debugf("mono found, allowing")
+				continue
+			}
+			if !wait {
+				b.Log.Debugf("<= Message %#v", res)
+				rmsg := config.Message{Username: res[0], Text: strings.Join(res[1:], ":"), Channel: "sshchat", Account: b.Account, UserID: "nick"}
+				b.Remote <- rmsg
+			}
+		}
+	}
+}

bridge/steam/steam.go

@@ -2,11 +2,13 @@ package bsteam
 
 import (
 	"fmt"
+
+	"github.com/42wim/matterbridge/bridge"
 	"github.com/42wim/matterbridge/bridge/config"
+	"github.com/42wim/matterbridge/bridge/helper"
 	"github.com/Philipp15b/go-steam"
 	"github.com/Philipp15b/go-steam/protocol/steamlang"
 	"github.com/Philipp15b/go-steam/steamid"
-	log "github.com/Sirupsen/logrus"
 	//"io/ioutil"
 	"strconv"
 	"sync"
@@ -16,38 +18,26 @@ import (
 type Bsteam struct {
 	c         *steam.Client
 	connected chan struct{}
-	Config    *config.Protocol
-	Remote    chan config.Message
-	Account   string
 	userMap   map[steamid.SteamId]string
 	sync.RWMutex
+	*bridge.Config
 }
 
-var flog *log.Entry
-var protocol = "steam"
-
-func init() {
-	flog = log.WithFields(log.Fields{"module": protocol})
-}
-
-func New(cfg config.Protocol, account string, c chan config.Message) *Bsteam {
-	b := &Bsteam{}
-	b.Config = &cfg
-	b.Remote = c
-	b.Account = account
+func New(cfg *bridge.Config) bridge.Bridger {
+	b := &Bsteam{Config: cfg}
 	b.userMap = make(map[steamid.SteamId]string)
 	b.connected = make(chan struct{})
 	return b
 }
 
 func (b *Bsteam) Connect() error {
-	flog.Info("Connecting")
+	b.Log.Info("Connecting")
 	b.c = steam.NewClient()
 	go b.handleEvents()
 	go b.c.Connect()
 	select {
 	case <-b.connected:
-		flog.Info("Connection succeeded")
+		b.Log.Info("Connection succeeded")
 	case <-time.After(time.Second * 30):
 		return fmt.Errorf("connection timed out")
 	}
@@ -69,13 +59,41 @@ func (b *Bsteam) JoinChannel(channel config.ChannelInfo) error {
 	return nil
 }
 
-func (b *Bsteam) Send(msg config.Message) error {
+func (b *Bsteam) Send(msg config.Message) (string, error) {
+	// ignore delete messages
+	if msg.Event == config.EVENT_MSG_DELETE {
+		return "", nil
+	}
 	id, err := steamid.NewId(msg.Channel)
 	if err != nil {
-		return err
+		return "", err
 	}
+
+	// Handle files
+	if msg.Extra != nil {
+		for _, rmsg := range helper.HandleExtra(&msg, b.General) {
+			b.c.Social.SendMessage(id, steamlang.EChatEntryType_ChatMsg, rmsg.Username+rmsg.Text)
+		}
+		if len(msg.Extra["file"]) > 0 {
+			for _, f := range msg.Extra["file"] {
+				fi := f.(config.FileInfo)
+				if fi.Comment != "" {
+					msg.Text += fi.Comment + ": "
+				}
+				if fi.URL != "" {
+					msg.Text = fi.URL
+					if fi.Comment != "" {
+						msg.Text = fi.Comment + ": " + fi.URL
+					}
+				}
+				b.c.Social.SendMessage(id, steamlang.EChatEntryType_ChatMsg, msg.Username+msg.Text)
+			}
+			return "", nil
+		}
+	}
+
 	b.c.Social.SendMessage(id, steamlang.EChatEntryType_ChatMsg, msg.Username+msg.Text)
-	return nil
+	return "", nil
 }
 
 func (b *Bsteam) getNick(id steamid.SteamId) string {
@@ -89,24 +107,29 @@ func (b *Bsteam) getNick(id steamid.SteamId) string {
 
 func (b *Bsteam) handleEvents() {
 	myLoginInfo := new(steam.LogOnDetails)
-	myLoginInfo.Username = b.Config.Login
-	myLoginInfo.Password = b.Config.Password
-	myLoginInfo.AuthCode = b.Config.AuthCode
+	myLoginInfo.Username = b.GetString("Login")
+	myLoginInfo.Password = b.GetString("Password")
+	myLoginInfo.AuthCode = b.GetString("AuthCode")
 	// Attempt to read existing auth hash to avoid steam guard.
 	// Maybe works
 	//myLoginInfo.SentryFileHash, _ = ioutil.ReadFile("sentry")
 	for event := range b.c.Events() {
-		//flog.Info(event)
+		//b.Log.Info(event)
 		switch e := event.(type) {
 		case *steam.ChatMsgEvent:
-			flog.Debugf("Receiving ChatMsgEvent: %#v", e)
-			flog.Debugf("Sending message from %s on %s to gateway", b.getNick(e.ChatterId), b.Account)
-			// for some reason we have to remove 0x18000000000000
-			channel := int64(e.ChatRoomId) - 0x18000000000000
+			b.Log.Debugf("Receiving ChatMsgEvent: %#v", e)
+			b.Log.Debugf("<= Sending message from %s on %s to gateway", b.getNick(e.ChatterId), b.Account)
+			var channel int64
+			if e.ChatRoomId == 0 {
+				channel = int64(e.ChatterId)
+			} else {
+				// for some reason we have to remove 0x18000000000000
+				channel = int64(e.ChatRoomId) - 0x18000000000000
+			}
 			msg := config.Message{Username: b.getNick(e.ChatterId), Text: e.Message, Channel: strconv.FormatInt(channel, 10), Account: b.Account, UserID: strconv.FormatInt(int64(e.ChatterId), 10)}
 			b.Remote <- msg
 		case *steam.PersonaStateEvent:
-			flog.Debugf("PersonaStateEvent: %#v\n", e)
+			b.Log.Debugf("PersonaStateEvent: %#v\n", e)
 			b.Lock()
 			b.userMap[e.FriendId] = e.Name
 			b.Unlock()
@@ -114,47 +137,47 @@ func (b *Bsteam) handleEvents() {
 			b.c.Auth.LogOn(myLoginInfo)
 		case *steam.MachineAuthUpdateEvent:
 			/*
-				flog.Info("authupdate", e)
-				flog.Info("hash", e.Hash)
+				b.Log.Info("authupdate", e)
+				b.Log.Info("hash", e.Hash)
 				ioutil.WriteFile("sentry", e.Hash, 0666)
 			*/
 		case *steam.LogOnFailedEvent:
-			flog.Info("Logon failed", e)
+			b.Log.Info("Logon failed", e)
 			switch e.Result {
 			case steamlang.EResult_AccountLogonDeniedNeedTwoFactorCode:
 				{
-					flog.Info("Steam guard isn't letting me in! Enter 2FA code:")
+					b.Log.Info("Steam guard isn't letting me in! Enter 2FA code:")
 					var code string
 					fmt.Scanf("%s", &code)
 					myLoginInfo.TwoFactorCode = code
 				}
 			case steamlang.EResult_AccountLogonDenied:
 				{
-					flog.Info("Steam guard isn't letting me in! Enter auth code:")
+					b.Log.Info("Steam guard isn't letting me in! Enter auth code:")
 					var code string
 					fmt.Scanf("%s", &code)
 					myLoginInfo.AuthCode = code
 				}
 			default:
-				log.Errorf("LogOnFailedEvent: %#v ", e.Result)
+				b.Log.Errorf("LogOnFailedEvent: %#v ", e.Result)
 				// TODO: Handle EResult_InvalidLoginAuthCode
 				return
 			}
 		case *steam.LoggedOnEvent:
-			flog.Debugf("LoggedOnEvent: %#v", e)
+			b.Log.Debugf("LoggedOnEvent: %#v", e)
 			b.connected <- struct{}{}
-			flog.Debugf("setting online")
+			b.Log.Debugf("setting online")
 			b.c.Social.SetPersonaState(steamlang.EPersonaState_Online)
 		case *steam.DisconnectedEvent:
-			flog.Info("Disconnected")
-			flog.Info("Attempting to reconnect...")
+			b.Log.Info("Disconnected")
+			b.Log.Info("Attempting to reconnect...")
 			b.c.Connect()
 		case steam.FatalErrorEvent:
-			flog.Error(e)
+			b.Log.Error(e)
 		case error:
-			flog.Error(e)
+			b.Log.Error(e)
 		default:
-			flog.Debugf("unknown event %#v", e)
+			b.Log.Debugf("unknown event %#v", e)
 		}
 	}
 }

bridge/telegram/html.go

@@ -2,15 +2,16 @@ package btelegram
 
 import (
 	"bytes"
-	"github.com/russross/blackfriday"
 	"html"
+
+	"github.com/russross/blackfriday"
 )
 
-type customHtml struct {
+type customHTML struct {
 	blackfriday.Renderer
 }
 
-func (options *customHtml) Paragraph(out *bytes.Buffer, text func() bool) {
+func (options *customHTML) Paragraph(out *bytes.Buffer, text func() bool) {
 	marker := out.Len()
 
 	if !text() {
@@ -20,32 +21,32 @@ func (options *customHtml) Paragraph(out *bytes.Buffer, text func() bool) {
 	out.WriteString("\n")
 }
 
-func (options *customHtml) BlockCode(out *bytes.Buffer, text []byte, lang string) {
+func (options *customHTML) BlockCode(out *bytes.Buffer, text []byte, lang string) {
 	out.WriteString("<pre>")
 
 	out.WriteString(html.EscapeString(string(text)))
 	out.WriteString("</pre>\n")
 }
 
-func (options *customHtml) Header(out *bytes.Buffer, text func() bool, level int, id string) {
+func (options *customHTML) Header(out *bytes.Buffer, text func() bool, level int, id string) {
 	options.Paragraph(out, text)
 }
 
-func (options *customHtml) HRule(out *bytes.Buffer) {
+func (options *customHTML) HRule(out *bytes.Buffer) {
 	out.WriteByte('\n')
 }
 
-func (options *customHtml) BlockQuote(out *bytes.Buffer, text []byte) {
+func (options *customHTML) BlockQuote(out *bytes.Buffer, text []byte) {
 	out.WriteString("> ")
 	out.Write(text)
 	out.WriteByte('\n')
 }
 
-func (options *customHtml) List(out *bytes.Buffer, text func() bool, flags int) {
+func (options *customHTML) List(out *bytes.Buffer, text func() bool, flags int) {
 	options.Paragraph(out, text)
 }
 
-func (options *customHtml) ListItem(out *bytes.Buffer, text []byte, flags int) {
+func (options *customHTML) ListItem(out *bytes.Buffer, text []byte, flags int) {
 	out.WriteString("- ")
 	out.Write(text)
 	out.WriteByte('\n')
@@ -53,7 +54,7 @@ func (options *customHtml) ListItem(out *bytes.Buffer, text []byte, flags int) {
 
 func makeHTML(input string) string {
 	return string(blackfriday.Markdown([]byte(input),
-		&customHtml{blackfriday.HtmlRenderer(blackfriday.HTML_USE_XHTML|blackfriday.HTML_SKIP_IMAGES, "", "")},
+		&customHTML{blackfriday.HtmlRenderer(blackfriday.HTML_USE_XHTML|blackfriday.HTML_SKIP_IMAGES, "", "")},
 		blackfriday.EXTENSION_NO_INTRA_EMPHASIS|
 			blackfriday.EXTENSION_FENCED_CODE|
 			blackfriday.EXTENSION_AUTOLINK|

bridge/telegram/telegram.go

@@ -1,137 +1,251 @@
 package btelegram
 
 import (
+	"html"
+	"regexp"
 	"strconv"
+	"strings"
 
+	"github.com/42wim/matterbridge/bridge"
 	"github.com/42wim/matterbridge/bridge/config"
-	log "github.com/Sirupsen/logrus"
+	"github.com/42wim/matterbridge/bridge/helper"
 	"github.com/go-telegram-bot-api/telegram-bot-api"
 )
 
 type Btelegram struct {
-	c       *tgbotapi.BotAPI
-	Config  *config.Protocol
-	Remote  chan config.Message
-	Account string
+	c *tgbotapi.BotAPI
+	*bridge.Config
+	avatarMap map[string]string // keep cache of userid and avatar sha
 }
 
-var flog *log.Entry
-var protocol = "telegram"
-
-func init() {
-	flog = log.WithFields(log.Fields{"module": protocol})
-}
-
-func New(cfg config.Protocol, account string, c chan config.Message) *Btelegram {
-	b := &Btelegram{}
-	b.Config = &cfg
-	b.Remote = c
-	b.Account = account
-	return b
+func New(cfg *bridge.Config) bridge.Bridger {
+	return &Btelegram{Config: cfg, avatarMap: make(map[string]string)}
 }
 
 func (b *Btelegram) Connect() error {
 	var err error
-	flog.Info("Connecting")
-	b.c, err = tgbotapi.NewBotAPI(b.Config.Token)
+	b.Log.Info("Connecting")
+	b.c, err = tgbotapi.NewBotAPI(b.GetString("Token"))
 	if err != nil {
-		flog.Debugf("%#v", err)
+		b.Log.Debugf("%#v", err)
 		return err
 	}
-	updates, err := b.c.GetUpdatesChan(tgbotapi.NewUpdate(0))
+	u := tgbotapi.NewUpdate(0)
+	u.Timeout = 60
+	updates, err := b.c.GetUpdatesChan(u)
 	if err != nil {
-		flog.Debugf("%#v", err)
+		b.Log.Debugf("%#v", err)
 		return err
 	}
-	flog.Info("Connection succeeded")
+	b.Log.Info("Connection succeeded")
 	go b.handleRecv(updates)
 	return nil
 }
 
 func (b *Btelegram) Disconnect() error {
 	return nil
-
 }
 
 func (b *Btelegram) JoinChannel(channel config.ChannelInfo) error {
 	return nil
 }
 
-func (b *Btelegram) Send(msg config.Message) error {
-	flog.Debugf("Receiving %#v", msg)
+func (b *Btelegram) Send(msg config.Message) (string, error) {
+	b.Log.Debugf("=> Receiving %#v", msg)
+
+	// get the chatid
 	chatid, err := strconv.ParseInt(msg.Channel, 10, 64)
 	if err != nil {
-		return err
+		return "", err
 	}
 
-	if b.Config.MessageFormat == "HTML" {
+	// map the file SHA to our user (caches the avatar)
+	if msg.Event == config.EVENT_AVATAR_DOWNLOAD {
+		return b.cacheAvatar(&msg)
+	}
+
+	if b.GetString("MessageFormat") == "HTML" {
 		msg.Text = makeHTML(msg.Text)
 	}
-	m := tgbotapi.NewMessage(chatid, msg.Username+msg.Text)
-	if b.Config.MessageFormat == "HTML" {
-		m.ParseMode = tgbotapi.ModeHTML
+
+	// Delete message
+	if msg.Event == config.EVENT_MSG_DELETE {
+		if msg.ID == "" {
+			return "", nil
+		}
+		msgid, err := strconv.Atoi(msg.ID)
+		if err != nil {
+			return "", err
+		}
+		_, err = b.c.DeleteMessage(tgbotapi.DeleteMessageConfig{ChatID: chatid, MessageID: msgid})
+		return "", err
 	}
-	_, err = b.c.Send(m)
-	return err
+
+	// Upload a file if it exists
+	if msg.Extra != nil {
+		for _, rmsg := range helper.HandleExtra(&msg, b.General) {
+			b.sendMessage(chatid, rmsg.Username, rmsg.Text)
+		}
+		// check if we have files to upload (from slack, telegram or mattermost)
+		if len(msg.Extra["file"]) > 0 {
+			b.handleUploadFile(&msg, chatid)
+		}
+	}
+
+	// edit the message if we have a msg ID
+	if msg.ID != "" {
+		msgid, err := strconv.Atoi(msg.ID)
+		if err != nil {
+			return "", err
+		}
+		if strings.ToLower(b.GetString("MessageFormat")) == "htmlnick" {
+			b.Log.Debug("Using mode HTML - nick only")
+			msg.Text = html.EscapeString(msg.Text)
+		}
+		m := tgbotapi.NewEditMessageText(chatid, msgid, msg.Username+msg.Text)
+		if b.GetString("MessageFormat") == "HTML" {
+			b.Log.Debug("Using mode HTML")
+			m.ParseMode = tgbotapi.ModeHTML
+		}
+		if b.GetString("MessageFormat") == "Markdown" {
+			b.Log.Debug("Using mode markdown")
+			m.ParseMode = tgbotapi.ModeMarkdown
+		}
+		if strings.ToLower(b.GetString("MessageFormat")) == "htmlnick" {
+			b.Log.Debug("Using mode HTML - nick only")
+			m.ParseMode = tgbotapi.ModeHTML
+		}
+		_, err = b.c.Send(m)
+		if err != nil {
+			return "", err
+		}
+		return "", nil
+	}
+
+	// Post normal message
+	return b.sendMessage(chatid, msg.Username, msg.Text)
 }
 
 func (b *Btelegram) handleRecv(updates <-chan tgbotapi.Update) {
 	for update := range updates {
-		flog.Debugf("Receiving from telegram: %#v", update.Message)
+		b.Log.Debugf("== Receiving event: %#v", update.Message)
+
+		if update.Message == nil && update.ChannelPost == nil && update.EditedMessage == nil && update.EditedChannelPost == nil {
+			b.Log.Error("Getting nil messages, this shouldn't happen.")
+			continue
+		}
+
 		var message *tgbotapi.Message
-		username := ""
-		channel := ""
-		text := ""
+
+		rmsg := config.Message{Account: b.Account, Extra: make(map[string][]interface{})}
+
 		// handle channels
 		if update.ChannelPost != nil {
 			message = update.ChannelPost
+			rmsg.Text = message.Text
 		}
-		if update.EditedChannelPost != nil && !b.Config.EditDisable {
+
+		// edited channel message
+		if update.EditedChannelPost != nil && !b.GetBool("EditDisable") {
 			message = update.EditedChannelPost
-			message.Text = message.Text + b.Config.EditSuffix
+			rmsg.Text = rmsg.Text + message.Text + b.GetString("EditSuffix")
 		}
+
 		// handle groups
 		if update.Message != nil {
 			message = update.Message
-		}
-		if update.EditedMessage != nil && !b.Config.EditDisable {
-			message = update.EditedMessage
-			message.Text = message.Text + b.Config.EditSuffix
-		}
-		if message.From != nil {
-			if b.Config.UseFirstName {
-				username = message.From.FirstName
-			}
-			if username == "" {
-				username = message.From.UserName
-				if username == "" {
-					username = message.From.FirstName
-				}
-			}
-			text = message.Text
-			channel = strconv.FormatInt(message.Chat.ID, 10)
+			rmsg.Text = message.Text
 		}
 
-		if username == "" {
-			username = "unknown"
+		// edited group message
+		if update.EditedMessage != nil && !b.GetBool("EditDisable") {
+			message = update.EditedMessage
+			rmsg.Text = rmsg.Text + message.Text + b.GetString("EditSuffix")
 		}
-		if message.Sticker != nil && b.Config.UseInsecureURL {
-			text = text + " " + b.getFileDirectURL(message.Sticker.FileID)
+
+		// set the ID's from the channel or group message
+		rmsg.ID = strconv.Itoa(message.MessageID)
+		rmsg.Channel = strconv.FormatInt(message.Chat.ID, 10)
+
+		// handle username
+		if message.From != nil {
+			rmsg.UserID = strconv.Itoa(message.From.ID)
+			if b.GetBool("UseFirstName") {
+				rmsg.Username = message.From.FirstName
+			}
+			if rmsg.Username == "" {
+				rmsg.Username = message.From.UserName
+				if rmsg.Username == "" {
+					rmsg.Username = message.From.FirstName
+				}
+			}
+			// only download avatars if we have a place to upload them (configured mediaserver)
+			if b.General.MediaServerUpload != "" {
+				b.handleDownloadAvatar(message.From.ID, rmsg.Channel)
+			}
 		}
-		if message.Video != nil && b.Config.UseInsecureURL {
-			text = text + " " + b.getFileDirectURL(message.Video.FileID)
+
+		// if we really didn't find a username, set it to unknown
+		if rmsg.Username == "" {
+			rmsg.Username = "unknown"
 		}
-		if message.Photo != nil && b.Config.UseInsecureURL {
-			photos := *message.Photo
-			// last photo is the biggest
-			text = text + " " + b.getFileDirectURL(photos[len(photos)-1].FileID)
+
+		// handle any downloads
+		err := b.handleDownload(message, &rmsg)
+		if err != nil {
+			b.Log.Errorf("download failed: %s", err)
 		}
-		if message.Document != nil && b.Config.UseInsecureURL {
-			text = text + " " + message.Document.FileName + " : " + b.getFileDirectURL(message.Document.FileID)
+
+		// handle forwarded messages
+		if message.ForwardFrom != nil {
+			usernameForward := ""
+			if b.GetBool("UseFirstName") {
+				usernameForward = message.ForwardFrom.FirstName
+			}
+			if usernameForward == "" {
+				usernameForward = message.ForwardFrom.UserName
+				if usernameForward == "" {
+					usernameForward = message.ForwardFrom.FirstName
+				}
+			}
+			if usernameForward == "" {
+				usernameForward = "unknown"
+			}
+			rmsg.Text = "Forwarded from " + usernameForward + ": " + rmsg.Text
 		}
-		if text != "" {
-			flog.Debugf("Sending message from %s on %s to gateway", username, b.Account)
-			b.Remote <- config.Message{Username: username, Text: text, Channel: channel, Account: b.Account, UserID: strconv.Itoa(message.From.ID)}
+
+		// quote the previous message
+		if message.ReplyToMessage != nil {
+			usernameReply := ""
+			if message.ReplyToMessage.From != nil {
+				if b.GetBool("UseFirstName") {
+					usernameReply = message.ReplyToMessage.From.FirstName
+				}
+				if usernameReply == "" {
+					usernameReply = message.ReplyToMessage.From.UserName
+					if usernameReply == "" {
+						usernameReply = message.ReplyToMessage.From.FirstName
+					}
+				}
+			}
+			if usernameReply == "" {
+				usernameReply = "unknown"
+			}
+			if !b.GetBool("QuoteDisable") {
+				rmsg.Text = b.handleQuote(rmsg.Text, usernameReply, message.ReplyToMessage.Text)
+			}
+		}
+
+		if rmsg.Text != "" || len(rmsg.Extra) > 0 {
+			rmsg.Text = helper.RemoveEmptyNewLines(rmsg.Text)
+			// channels don't have (always?) user information. see #410
+			if message.From != nil {
+				rmsg.Avatar = helper.GetAvatar(b.avatarMap, strconv.Itoa(message.From.ID), b.General)
+			}
+
+			b.Log.Debugf("<= Sending message from %s on %s to gateway", rmsg.Username, b.Account)
+			b.Log.Debugf("<= Message is %#v", rmsg)
+			b.Remote <- rmsg
 		}
 	}
 }
@@ -143,3 +257,185 @@ func (b *Btelegram) getFileDirectURL(id string) string {
 	}
 	return res
 }
+
+// handleDownloadAvatar downloads the avatar of userid from channel
+// sends a EVENT_AVATAR_DOWNLOAD message to the gateway if successful.
+// logs an error message if it fails
+func (b *Btelegram) handleDownloadAvatar(userid int, channel string) {
+	rmsg := config.Message{Username: "system", Text: "avatar", Channel: channel, Account: b.Account, UserID: strconv.Itoa(userid), Event: config.EVENT_AVATAR_DOWNLOAD, Extra: make(map[string][]interface{})}
+	if _, ok := b.avatarMap[strconv.Itoa(userid)]; !ok {
+		photos, err := b.c.GetUserProfilePhotos(tgbotapi.UserProfilePhotosConfig{UserID: userid, Limit: 1})
+		if err != nil {
+			b.Log.Errorf("Userprofile download failed for %#v %s", userid, err)
+		}
+
+		if len(photos.Photos) > 0 {
+			photo := photos.Photos[0][0]
+			url := b.getFileDirectURL(photo.FileID)
+			name := strconv.Itoa(userid) + ".png"
+			b.Log.Debugf("trying to download %#v fileid %#v with size %#v", name, photo.FileID, photo.FileSize)
+
+			err := helper.HandleDownloadSize(b.Log, &rmsg, name, int64(photo.FileSize), b.General)
+			if err != nil {
+				b.Log.Error(err)
+				return
+			}
+			data, err := helper.DownloadFile(url)
+			if err != nil {
+				b.Log.Errorf("download %s failed %#v", url, err)
+				return
+			}
+			helper.HandleDownloadData(b.Log, &rmsg, name, rmsg.Text, "", data, b.General)
+			b.Remote <- rmsg
+		}
+	}
+}
+
+// handleDownloadFile handles file download
+func (b *Btelegram) handleDownload(message *tgbotapi.Message, rmsg *config.Message) error {
+	size := 0
+	var url, name, text string
+
+	if message.Sticker != nil {
+		v := message.Sticker
+		size = v.FileSize
+		url = b.getFileDirectURL(v.FileID)
+		urlPart := strings.Split(url, "/")
+		name = urlPart[len(urlPart)-1]
+		if !strings.HasSuffix(name, ".webp") {
+			name = name + ".webp"
+		}
+		text = " " + url
+	}
+	if message.Video != nil {
+		v := message.Video
+		size = v.FileSize
+		url = b.getFileDirectURL(v.FileID)
+		urlPart := strings.Split(url, "/")
+		name = urlPart[len(urlPart)-1]
+		text = " " + url
+	}
+	if message.Photo != nil {
+		photos := *message.Photo
+		size = photos[len(photos)-1].FileSize
+		url = b.getFileDirectURL(photos[len(photos)-1].FileID)
+		urlPart := strings.Split(url, "/")
+		name = urlPart[len(urlPart)-1]
+		text = " " + url
+	}
+	if message.Document != nil {
+		v := message.Document
+		size = v.FileSize
+		url = b.getFileDirectURL(v.FileID)
+		name = v.FileName
+		text = " " + v.FileName + " : " + url
+	}
+	if message.Voice != nil {
+		v := message.Voice
+		size = v.FileSize
+		url = b.getFileDirectURL(v.FileID)
+		urlPart := strings.Split(url, "/")
+		name = urlPart[len(urlPart)-1]
+		text = " " + url
+		if !strings.HasSuffix(name, ".ogg") {
+			name = name + ".ogg"
+		}
+	}
+	if message.Audio != nil {
+		v := message.Audio
+		size = v.FileSize
+		url = b.getFileDirectURL(v.FileID)
+		urlPart := strings.Split(url, "/")
+		name = urlPart[len(urlPart)-1]
+		text = " " + url
+	}
+	// if name is empty we didn't match a thing to download
+	if name == "" {
+		return nil
+	}
+	// use the URL instead of native upload
+	if b.GetBool("UseInsecureURL") {
+		b.Log.Debugf("Setting message text to :%s", text)
+		rmsg.Text = rmsg.Text + text
+		return nil
+	}
+	// if we have a file attached, download it (in memory) and put a pointer to it in msg.Extra
+	err := helper.HandleDownloadSize(b.Log, rmsg, name, int64(size), b.General)
+	if err != nil {
+		return err
+	}
+	data, err := helper.DownloadFile(url)
+	if err != nil {
+		return err
+	}
+	helper.HandleDownloadData(b.Log, rmsg, name, message.Caption, "", data, b.General)
+	return nil
+}
+
+// handleUploadFile handles native upload of files
+func (b *Btelegram) handleUploadFile(msg *config.Message, chatid int64) (string, error) {
+	var c tgbotapi.Chattable
+	for _, f := range msg.Extra["file"] {
+		fi := f.(config.FileInfo)
+		file := tgbotapi.FileBytes{fi.Name, *fi.Data}
+		re := regexp.MustCompile(".(jpg|png)$")
+		if re.MatchString(fi.Name) {
+			c = tgbotapi.NewPhotoUpload(chatid, file)
+		} else {
+			c = tgbotapi.NewDocumentUpload(chatid, file)
+		}
+		_, err := b.c.Send(c)
+		if err != nil {
+			b.Log.Errorf("file upload failed: %#v", err)
+		}
+		if fi.Comment != "" {
+			b.sendMessage(chatid, msg.Username, fi.Comment)
+		}
+	}
+	return "", nil
+}
+
+func (b *Btelegram) sendMessage(chatid int64, username, text string) (string, error) {
+	m := tgbotapi.NewMessage(chatid, "")
+	m.Text = username + text
+	if b.GetString("MessageFormat") == "HTML" {
+		b.Log.Debug("Using mode HTML")
+		m.ParseMode = tgbotapi.ModeHTML
+	}
+	if b.GetString("MessageFormat") == "Markdown" {
+		b.Log.Debug("Using mode markdown")
+		m.ParseMode = tgbotapi.ModeMarkdown
+	}
+	if strings.ToLower(b.GetString("MessageFormat")) == "htmlnick" {
+		b.Log.Debug("Using mode HTML - nick only")
+		m.Text = username + html.EscapeString(text)
+		m.ParseMode = tgbotapi.ModeHTML
+	}
+	res, err := b.c.Send(m)
+	if err != nil {
+		return "", err
+	}
+	return strconv.Itoa(res.MessageID), nil
+}
+
+func (b *Btelegram) cacheAvatar(msg *config.Message) (string, error) {
+	fi := msg.Extra["file"][0].(config.FileInfo)
+	/* if we have a sha we have successfully uploaded the file to the media server,
+	so we can now cache the sha */
+	if fi.SHA != "" {
+		b.Log.Debugf("Added %s to %s in avatarMap", fi.SHA, msg.UserID)
+		b.avatarMap[msg.UserID] = fi.SHA
+	}
+	return "", nil
+}
+
+func (b *Btelegram) handleQuote(message, quoteNick, quoteMessage string) string {
+	format := b.GetString("quoteformat")
+	if format == "" {
+		format = "{MESSAGE} (re @{QUOTENICK}: {QUOTEMESSAGE})"
+	}
+	format = strings.Replace(format, "{MESSAGE}", message, -1)
+	format = strings.Replace(format, "{QUOTENICK}", quoteNick, -1)
+	format = strings.Replace(format, "{QUOTEMESSAGE}", quoteMessage, -1)
+	return format
+}

bridge/xmpp/xmpp.go

@@ -2,48 +2,38 @@ package bxmpp
 
 import (
 	"crypto/tls"
-	"github.com/42wim/matterbridge/bridge/config"
-	log "github.com/Sirupsen/logrus"
-	"github.com/jpillora/backoff"
-	"github.com/mattn/go-xmpp"
-
 	"strings"
 	"time"
+
+	"github.com/42wim/matterbridge/bridge"
+	"github.com/42wim/matterbridge/bridge/config"
+	"github.com/42wim/matterbridge/bridge/helper"
+	"github.com/jpillora/backoff"
+	"github.com/matterbridge/go-xmpp"
+	"github.com/rs/xid"
 )
 
 type Bxmpp struct {
 	xc      *xmpp.Client
 	xmppMap map[string]string
-	Config  *config.Protocol
-	Remote  chan config.Message
-	Account string
+	*bridge.Config
 }
 
-var flog *log.Entry
-var protocol = "xmpp"
-
-func init() {
-	flog = log.WithFields(log.Fields{"module": protocol})
-}
-
-func New(cfg config.Protocol, account string, c chan config.Message) *Bxmpp {
-	b := &Bxmpp{}
+func New(cfg *bridge.Config) bridge.Bridger {
+	b := &Bxmpp{Config: cfg}
 	b.xmppMap = make(map[string]string)
-	b.Config = &cfg
-	b.Account = account
-	b.Remote = c
 	return b
 }
 
 func (b *Bxmpp) Connect() error {
 	var err error
-	flog.Infof("Connecting %s", b.Config.Server)
+	b.Log.Infof("Connecting %s", b.GetString("Server"))
 	b.xc, err = b.createXMPP()
 	if err != nil {
-		flog.Debugf("%#v", err)
+		b.Log.Debugf("%#v", err)
 		return err
 	}
-	flog.Info("Connection succeeded")
+	b.Log.Info("Connection succeeded")
 	go func() {
 		initial := true
 		bf := &backoff.Backoff{
@@ -53,16 +43,16 @@ func (b *Bxmpp) Connect() error {
 		}
 		for {
 			if initial {
-				b.handleXmpp()
+				b.handleXMPP()
 				initial = false
 			}
 			d := bf.Duration()
-			flog.Infof("Disconnected. Reconnecting in %s", d)
+			b.Log.Infof("Disconnected. Reconnecting in %s", d)
 			time.Sleep(d)
 			b.xc, err = b.createXMPP()
 			if err == nil {
 				b.Remote <- config.Message{Username: "system", Text: "rejoin", Channel: "", Account: b.Account, Event: config.EVENT_REJOIN_CHANNELS}
-				b.handleXmpp()
+				b.handleXMPP()
 				bf.Reset()
 			}
 		}
@@ -75,36 +65,65 @@ func (b *Bxmpp) Disconnect() error {
 }
 
 func (b *Bxmpp) JoinChannel(channel config.ChannelInfo) error {
-	b.xc.JoinMUCNoHistory(channel.Name+"@"+b.Config.Muc, b.Config.Nick)
+	if channel.Options.Key != "" {
+		b.Log.Debugf("using key %s for channel %s", channel.Options.Key, channel.Name)
+		b.xc.JoinProtectedMUC(channel.Name+"@"+b.GetString("Muc"), b.GetString("Nick"), channel.Options.Key, xmpp.NoHistory, 0, nil)
+	} else {
+		b.xc.JoinMUCNoHistory(channel.Name+"@"+b.GetString("Muc"), b.GetString("Nick"))
+	}
 	return nil
 }
 
-func (b *Bxmpp) Send(msg config.Message) error {
-	flog.Debugf("Receiving %#v", msg)
-	b.xc.Send(xmpp.Chat{Type: "groupchat", Remote: msg.Channel + "@" + b.Config.Muc, Text: msg.Username + msg.Text})
-	return nil
+func (b *Bxmpp) Send(msg config.Message) (string, error) {
+	var msgid = ""
+	var msgreplaceid = ""
+	// ignore delete messages
+	if msg.Event == config.EVENT_MSG_DELETE {
+		return "", nil
+	}
+	b.Log.Debugf("=> Receiving %#v", msg)
+
+	// Upload a file (in xmpp case send the upload URL because xmpp has no native upload support)
+	if msg.Extra != nil {
+		for _, rmsg := range helper.HandleExtra(&msg, b.General) {
+			b.xc.Send(xmpp.Chat{Type: "groupchat", Remote: rmsg.Channel + "@" + b.GetString("Muc"), Text: rmsg.Username + rmsg.Text})
+		}
+		if len(msg.Extra["file"]) > 0 {
+			return b.handleUploadFile(&msg)
+		}
+	}
+
+	msgid = xid.New().String()
+	if msg.ID != "" {
+		msgid = msg.ID
+		msgreplaceid = msg.ID
+	}
+	// Post normal message
+	_, err := b.xc.Send(xmpp.Chat{Type: "groupchat", Remote: msg.Channel + "@" + b.GetString("Muc"), Text: msg.Username + msg.Text, ID: msgid, ReplaceID: msgreplaceid})
+	if err != nil {
+		return "", err
+	}
+	return msgid, nil
 }
 
 func (b *Bxmpp) createXMPP() (*xmpp.Client, error) {
 	tc := new(tls.Config)
-	tc.InsecureSkipVerify = b.Config.SkipTLSVerify
-	tc.ServerName = strings.Split(b.Config.Server, ":")[0]
+	tc.InsecureSkipVerify = b.GetBool("SkipTLSVerify")
+	tc.ServerName = strings.Split(b.GetString("Server"), ":")[0]
 	options := xmpp.Options{
-		Host:      b.Config.Server,
-		User:      b.Config.Jid,
-		Password:  b.Config.Password,
-		NoTLS:     true,
-		StartTLS:  true,
-		TLSConfig: tc,
-
-		//StartTLS:      false,
-		Debug:                        true,
+		Host:                         b.GetString("Server"),
+		User:                         b.GetString("Jid"),
+		Password:                     b.GetString("Password"),
+		NoTLS:                        true,
+		StartTLS:                     true,
+		TLSConfig:                    tc,
+		Debug:                        b.GetBool("debug"),
+		Logger:                       b.Log.Writer(),
 		Session:                      true,
 		Status:                       "",
 		StatusMessage:                "",
 		Resource:                     "",
 		InsecureAllowUnencryptedAuth: false,
-		//InsecureAllowUnencryptedAuth: true,
 	}
 	var err error
 	b.xc, err = options.NewClient()
@@ -119,10 +138,10 @@ func (b *Bxmpp) xmppKeepAlive() chan bool {
 		for {
 			select {
 			case <-ticker.C:
-				flog.Debugf("PING")
+				b.Log.Debugf("PING")
 				err := b.xc.PingC2S("", "")
 				if err != nil {
-					flog.Debugf("PING failed %#v", err)
+					b.Log.Debugf("PING failed %#v", err)
 				}
 			case <-done:
 				return
@@ -132,11 +151,11 @@ func (b *Bxmpp) xmppKeepAlive() chan bool {
 	return done
 }
 
-func (b *Bxmpp) handleXmpp() error {
+func (b *Bxmpp) handleXMPP() error {
 	var ok bool
+	var msgid string
 	done := b.xmppKeepAlive()
 	defer close(done)
-	nodelay := time.Time{}
 	for {
 		m, err := b.xc.Recv()
 		if err != nil {
@@ -144,25 +163,26 @@ func (b *Bxmpp) handleXmpp() error {
 		}
 		switch v := m.(type) {
 		case xmpp.Chat:
-			var channel, nick string
 			if v.Type == "groupchat" {
-				s := strings.Split(v.Remote, "@")
-				if len(s) >= 2 {
-					channel = s[0]
+				b.Log.Debugf("== Receiving %#v", v)
+				// skip invalid messages
+				if b.skipMessage(v) {
+					continue
 				}
-				s = strings.Split(s[1], "/")
-				if len(s) == 2 {
-					nick = s[1]
+				msgid = v.ID
+				if v.ReplaceID != "" {
+					msgid = v.ReplaceID
 				}
-				if nick != b.Config.Nick && v.Stamp == nodelay && v.Text != "" {
-					rmsg := config.Message{Username: nick, Text: v.Text, Channel: channel, Account: b.Account, UserID: v.Remote}
-					rmsg.Text, ok = b.replaceAction(rmsg.Text)
-					if ok {
-						rmsg.Event = config.EVENT_USER_ACTION
-					}
-					flog.Debugf("Sending message from %s on %s to gateway", nick, b.Account)
-					b.Remote <- rmsg
+				rmsg := config.Message{Username: b.parseNick(v.Remote), Text: v.Text, Channel: b.parseChannel(v.Remote), Account: b.Account, UserID: v.Remote, ID: msgid}
+
+				// check if we have an action event
+				rmsg.Text, ok = b.replaceAction(rmsg.Text)
+				if ok {
+					rmsg.Event = config.EVENT_USER_ACTION
 				}
+				b.Log.Debugf("<= Sending message from %s on %s to gateway", rmsg.Username, b.Account)
+				b.Log.Debugf("<= Message is %#v", rmsg)
+				b.Remote <- rmsg
 			}
 		case xmpp.Presence:
 			// do nothing
@@ -176,3 +196,71 @@ func (b *Bxmpp) replaceAction(text string) (string, bool) {
 	}
 	return text, false
 }
+
+// handleUploadFile handles native upload of files
+func (b *Bxmpp) handleUploadFile(msg *config.Message) (string, error) {
+	var urldesc = ""
+
+	for _, f := range msg.Extra["file"] {
+		fi := f.(config.FileInfo)
+		if fi.Comment != "" {
+			msg.Text += fi.Comment + ": "
+		}
+		if fi.URL != "" {
+			msg.Text = fi.URL
+			if fi.Comment != "" {
+				msg.Text = fi.Comment + ": " + fi.URL
+				urldesc = fi.Comment
+			}
+		}
+		_, err := b.xc.Send(xmpp.Chat{Type: "groupchat", Remote: msg.Channel + "@" + b.GetString("Muc"), Text: msg.Username + msg.Text})
+		if err != nil {
+			return "", err
+		}
+		if fi.URL != "" {
+			b.xc.SendOOB(xmpp.Chat{Type: "groupchat", Remote: msg.Channel + "@" + b.GetString("Muc"), Ooburl: fi.URL, Oobdesc: urldesc})
+		}
+	}
+	return "", nil
+}
+
+func (b *Bxmpp) parseNick(remote string) string {
+	s := strings.Split(remote, "@")
+	if len(s) > 0 {
+		s = strings.Split(s[1], "/")
+		if len(s) == 2 {
+			return s[1] // nick
+		}
+	}
+	return ""
+}
+
+func (b *Bxmpp) parseChannel(remote string) string {
+	s := strings.Split(remote, "@")
+	if len(s) >= 2 {
+		return s[0] // channel
+	}
+	return ""
+}
+
+// skipMessage skips messages that need to be skipped
+func (b *Bxmpp) skipMessage(message xmpp.Chat) bool {
+	// skip messages from ourselves
+	if b.parseNick(message.Remote) == b.GetString("Nick") {
+		return true
+	}
+
+	// skip empty messages
+	if message.Text == "" {
+		return true
+	}
+
+	// skip subject messages
+	if strings.Contains(message.Text, "</subject>") {
+		return true
+	}
+
+	// skip delayed messages
+	t := time.Time{}
+	return message.Stamp != t
+}

bridge/zulip/zulip.go

@@ -0,0 +1,170 @@
+package bzulip
+
+import (
+	"encoding/json"
+	"io/ioutil"
+	"strconv"
+	"time"
+
+	"github.com/42wim/matterbridge/bridge"
+	"github.com/42wim/matterbridge/bridge/config"
+	"github.com/42wim/matterbridge/bridge/helper"
+	gzb "github.com/matterbridge/gozulipbot"
+)
+
+type Bzulip struct {
+	q       *gzb.Queue
+	bot     *gzb.Bot
+	streams map[int]string
+	*bridge.Config
+}
+
+func New(cfg *bridge.Config) bridge.Bridger {
+	return &Bzulip{Config: cfg, streams: make(map[int]string)}
+}
+
+func (b *Bzulip) Connect() error {
+	bot := gzb.Bot{APIKey: b.GetString("token"), APIURL: b.GetString("server") + "/api/v1/", Email: b.GetString("login")}
+	bot.Init()
+	q, err := bot.RegisterAll()
+	b.q = q
+	b.bot = &bot
+	if err != nil {
+		b.Log.Errorf("Connect() %#v", err)
+		return err
+	}
+	// init stream
+	b.getChannel(0)
+	b.Log.Info("Connection succeeded")
+	go b.handleQueue()
+	return nil
+}
+
+func (b *Bzulip) Disconnect() error {
+	return nil
+}
+
+func (b *Bzulip) JoinChannel(channel config.ChannelInfo) error {
+	return nil
+}
+
+func (b *Bzulip) Send(msg config.Message) (string, error) {
+	b.Log.Debugf("=> Receiving %#v", msg)
+
+	// Delete message
+	if msg.Event == config.EVENT_MSG_DELETE {
+		if msg.ID == "" {
+			return "", nil
+		}
+		_, err := b.bot.UpdateMessage(msg.ID, "")
+		return "", err
+	}
+
+	// Upload a file if it exists
+	if msg.Extra != nil {
+		for _, rmsg := range helper.HandleExtra(&msg, b.General) {
+			b.sendMessage(rmsg)
+		}
+		if len(msg.Extra["file"]) > 0 {
+			return b.handleUploadFile(&msg)
+		}
+	}
+
+	// edit the message if we have a msg ID
+	if msg.ID != "" {
+		_, err := b.bot.UpdateMessage(msg.ID, msg.Username+msg.Text)
+		return "", err
+	}
+
+	// Post normal message
+	return b.sendMessage(msg)
+}
+
+func (b *Bzulip) getChannel(id int) string {
+	if name, ok := b.streams[id]; ok {
+		return name
+	}
+	streams, err := b.bot.GetRawStreams()
+	if err != nil {
+		b.Log.Errorf("getChannel: %#v", err)
+		return ""
+	}
+	for _, stream := range streams.Streams {
+		b.streams[stream.StreamID] = stream.Name
+	}
+	if name, ok := b.streams[id]; ok {
+		return name
+	}
+	return ""
+}
+
+func (b *Bzulip) handleQueue() error {
+	for {
+		messages, _ := b.q.GetEvents()
+		for _, m := range messages {
+			b.Log.Debugf("== Receiving %#v", m)
+			// ignore our own messages
+			if m.SenderEmail == b.GetString("login") {
+				continue
+			}
+			rmsg := config.Message{Username: m.SenderFullName, Text: m.Content, Channel: b.getChannel(m.StreamID), Account: b.Account, UserID: strconv.Itoa(m.SenderID), Avatar: m.AvatarURL}
+			b.Log.Debugf("<= Sending message from %s on %s to gateway", rmsg.Username, b.Account)
+			b.Log.Debugf("<= Message is %#v", rmsg)
+			b.Remote <- rmsg
+			b.q.LastEventID = m.ID
+		}
+		time.Sleep(time.Second * 3)
+	}
+}
+
+func (b *Bzulip) sendMessage(msg config.Message) (string, error) {
+	topic := "matterbridge"
+	if b.GetString("topic") != "" {
+		topic = b.GetString("topic")
+	}
+	m := gzb.Message{
+		Stream:  msg.Channel,
+		Topic:   topic,
+		Content: msg.Username + msg.Text,
+	}
+	resp, err := b.bot.Message(m)
+	if err != nil {
+		return "", err
+	}
+	if resp != nil {
+		defer resp.Body.Close()
+		res, err := ioutil.ReadAll(resp.Body)
+		if err != nil {
+			return "", err
+		}
+		var jr struct {
+			ID int `json:"id"`
+		}
+		err = json.Unmarshal(res, &jr)
+		if err != nil {
+			return "", err
+		}
+		return strconv.Itoa(jr.ID), nil
+	}
+	return "", nil
+}
+
+func (b *Bzulip) handleUploadFile(msg *config.Message) (string, error) {
+	for _, f := range msg.Extra["file"] {
+		fi := f.(config.FileInfo)
+		if fi.Comment != "" {
+			msg.Text += fi.Comment + ": "
+		}
+		if fi.URL != "" {
+			msg.Text = fi.URL
+			if fi.Comment != "" {
+				msg.Text = fi.Comment + ": " + fi.URL
+			}
+		}
+		_, err := b.sendMessage(*msg)
+		if err != nil {
+			return "", err
+		}
+	}
+	return "", nil
+}

changelog.md

@@ -1,10 +1,282 @@
+# v1.11.1
+
+## New features
+* slack: Add support for slack channels by ID. Closes #436
+* discord: Clip too long messages sent to discord (discord). Closes #440
+
+## Bugfix
+* general: fix possible panic on downloads that are too big #448
+* general: Fix avatar uploads to work with MediaDownloadPath. Closes #454
+* discord: allow receiving of topic changes/channel leave/joins from other bridges through the webhook
+* discord: Add a space before url in file uploads (discord). Closes #461
+* discord:  Skip empty messages being sent with the webhook (discord). #469
+* mattermost: Use nickname instead of username if defined (mattermost). Closes #452
+* irc: Stop numbers being stripped after non-color control codes (irc) (#465)
+* slack: Use UserID to look for avatar instead of username (slack). Closes #472
+
+# v1.11.0
+
+## New features
+* general: Add config option MediaDownloadPath (#443). See `MediaDownloadPath` in matterbridge.toml.sample
+* general: Add MediaDownloadBlacklist option. Closes #442. See `MediaDownloadBlacklist` in matterbridge.toml.sample
+* xmpp: Add channel password support for XMPP (#451)
+* xmpp: Add message correction support for XMPP (#437)
+* telegram: Add support for MessageFormat=htmlnick (telegram). #444
+* mattermost: Add support for mattermost 5.x
+
+## Enhancements
+* slack: Add Title from attachment slack message (#446)
+* irc: Prevent white or black color codes (irc) (#434)
+
+## Bugfix
+* slack: Fix regexp in replaceMention (slack). (#435)
+* irc: Reconnect on quit. (irc) See #431 (#445)
+* sshchat: Ignore messages from ourself. (sshchat) Closes #439
+
+# v1.10.1
+## New features
+* irc: Colorize username sent to IRC using its crc32 IEEE checksum (#423). See `ColorNicks` in matterbridge.toml.sample
+* irc: Add support for CJK to/from utf-8 (irc). #400
+* telegram: Add QuoteFormat option (telegram). Closes #413. See `QuoteFormat` in matterbridge.toml.sample
+* xmpp: Send attached files to XMPP in different message with OOB data and without body (#421)
+
+## Bugfix
+* general: updated irc/xmpp/telegram libraries
+* mattermost/slack/rocketchat: Fix iconurl regression. Closes #430
+* mattermost/slack: Use uuid instead of userid. Fixes #429
+* slack: Avatar spoofing from Slack to Discord with uppercase in nick doesn't work (#433)
+* irc: Fix format string bug (irc) (#428)
+
+# v1.10.0
+## New features
+* general: Add support for reloading all settings automatically after changing config except connection and gateway configuration. Closes #373
+* zulip: New protocol support added (https://zulipchat.com)
+
+## Enhancements
+* general: Handle file comment better
+* steam: Handle file uploads to mediaserver (steam)
+* slack: Properly set Slack user who initiated slash command (#394)
+
+## Bugfix
+* general: Use only alphanumeric for file uploads to mediaserver. Closes #416
+* general: Fix crash on invalid filenames
+* general: Fix regression in ReplaceMessages and ReplaceNicks. Closes #407
+* telegram: Fix possible nil when using channels (telegram). #410
+* telegram: Fix panic (telegram). Closes #410
+* telegram: Handle channel posts correctly
+* mattermost: Update GetFileLinks to API_V4
+
+# v1.9.1
+## New features
+* telegram: Add QuoteDisable option (telegram). Closes #399. See QuoteDisable in matterbridge.toml.sample
+## Enhancements
+* discord: Send mediaserver link to Discord in Webhook mode (discord) (#405)
+* mattermost: Print list of valid team names when team not found (#390)
+* slack: Strip markdown URLs with blank text (slack) (#392)
+## Bugfix
+* slack/mattermost: Make our callbackid more unique. Fixes issue with running multiple matterbridge on the same channel (slack,mattermost)
+* telegram: fix newlines in multiline messages #399
+* telegram: Revert #378
+
+# v1.9.0 (the refactor release)
+## New features
+* general: better debug messages
+* general: better support for environment variables override
+* general: Ability to disable sending join/leave messages to other gateways. #382
+* slack: Allow Slack @usergroups to be parsed as human-friendly names #379
+* slack: Provide better context for shared posts from Slack<=>Slack enhancement #369
+* telegram: Convert nicks automatically into HTML when MessageFormat is set to HTML #378
+* irc: Add DebugLevel option 
+
+## Bugfix
+* slack: Ignore restricted_action on channel join (slack). Closes #387
+* slack: Add slack attachment support to matterhook
+* slack: Update userlist on join (slack). Closes #372
+
+# v1.8.0
+## New features
+* general: Send chat notification if media is too big to be re-uploaded to MediaServer. See #359
+* general: Download (and upload) avatar images from mattermost and telegram when mediaserver is configured. Closes #362
+* general: Add label support in RemoteNickFormat
+* general: Prettier info/debug log output
+* mattermost: Download files and reupload to supported bridges (mattermost). Closes #357
+* slack: Add ShowTopicChange option. Allow/disable topic change messages (currently only from slack). Closes #353
+* slack: Add support for file comments (slack). Closes #346
+* telegram: Add comment to file upload from telegram. Show comments on all bridges. Closes #358
+* telegram: Add markdown support (telegram). #355
+* api: Give api access to whole config.Message (and events). Closes #374
+
+## Bugfix
+* discord: Check for a valid WebhookURL (discord). Closes #367
+* discord: Fix role mention replace issues
+* irc: Truncate messages sent to IRC based on byte count (#368)
+* mattermost: Add file download urls also to mattermost webhooks #356
+* telegram: Fix panic on nil messages (telegram). Closes #366
+* telegram: Fix the UseInsecureURL text (telegram). Closes #184
+
+# v1.7.1
+## Bugfix
+* telegram: Enable Long Polling for Telegram. Reduces bandwidth consumption. (#350)
+
+# v1.7.0
+## New features
+* matrix: Add support for deleting messages from/to matrix (matrix). Closes #320
+* xmpp: Ignore <subject> messages (xmpp). #272
+* irc: Add twitch support (irc) to README / wiki
+
+## Bugfix
+* general: Change RemoteNickFormat replacement order. Closes #336
+* general: Make edits/delete work for bridges that gets reused. Closes #342
+* general: Lowercase irc channels in config. Closes #348
+* matrix: Fix possible panics (matrix). Closes #333
+* matrix: Add an extension to images without one (matrix). #331
+* api: Obey the Gateway value from the json (api). Closes #344
+* xmpp: Print only debug messages when specified (xmpp). Closes #345
+* xmpp: Allow xmpp to receive the extra messages (file uploads) when text is empty. #295
+
+# v1.6.3
+## Bugfix
+* slack: Fix connection issues
+* slack: Add more debug messages
+* irc: Convert received IRC channel names to lowercase. Fixes #329 (#330)
+
+# v1.6.2
+## Bugfix
+* mattermost: Crashes while connecting to Mattermost (regression). Closes #327
+
+# v1.6.1
+## Bugfix
+* general: Display of nicks not longer working (regression). Closes #323
+
+# v1.6.0
+## New features
+* sshchat: New protocol support added (https://github.com/shazow/ssh-chat)
+* general: Allow specifying maximum download size of media using MediaDownloadSize (slack,telegram,matrix)
+* api: Add (simple, one listener) long-polling support (api). Closes #307
+* telegram: Add support for forwarded messages. Closes #313
+* telegram: Add support for Audio/Voice files (telegram). Closes #314
+* irc: Add RejoinDelay option. Delay to rejoin after channel kick (irc). Closes #322
+
+## Bugfix
+* telegram: Also use HTML in edited messages (telegram). Closes #315
+* matrix: Fix panic (matrix). Closes #316
+
+# v1.5.1
+
+## Bugfix
+* irc: Fix irc ACTION regression (irc). Closes #306
+* irc: Split on UTF-8 for MessageSplit (irc). Closes #308
+
+# v1.5.0
+## New features
+* general: remote mediaserver support. See MediaServerDownload and MediaServerUpload in matterbridge.toml.sample
+  more information on https://github.com/42wim/matterbridge/wiki/Mediaserver-setup-%5Badvanced%5D
+* general: Add support for ReplaceNicks using regexp to replace nicks. Closes #269 (see matterbridge.toml.sample)
+* general: Add support for ReplaceMessages using regexp to replace messages. #269 (see matterbridge.toml.sample)
+* irc: Add MessageSplit option to split messages on MessageLength (irc). Closes #281
+* matrix: Add support for uploading images/video (matrix). Closes #302
+* matrix: Add support for uploaded images/video (matrix) 
+
+## Bugfix
+* telegram: Add webp extension to stickers if necessary (telegram)
+* mattermost: Break when re-login fails (mattermost)
+
+# v1.4.1
+## Bugfix
+* telegram: fix issue with uploading for images/documents/stickers
+* slack: remove double messages sent to other bridges when uploading files
+* irc: Fix strict user handling of girc (irc). Closes #298 
+
+# v1.4.0
+## Breaking changes
+* general: `[general]` settings don't override the specific bridge settings
+
+## New features
+* irc: Replace sorcix/irc and go-ircevent with girc, this should be give better reconnects
+* steam: Add support for bridging to individual steam chats. (steam) (#294)
+* telegram: Download files from telegram and reupload to supported bridges (telegram). #278
+* slack: Add support to upload files to slack, from bridges with private urls like slack/mattermost/telegram. (slack)
+* discord: Add support to upload files to discord, from bridges with private urls like slack/mattermost/telegram. (discord)
+* general: Add systemd service file (#291)
+* general: Add support for DEBUG=1 envvar to enable debug. Closes #283
+* general: Add StripNick option, only allow alphanumerical nicks. Closes #285
+
+## Bugfix
+* gitter: Use room.URI instead of room.Name. (gitter) (#293)
+* slack: Allow slack messages with variables (eg. @here) to be formatted correctly. (slack) (#288)
+* slack: Resolve slack channel to human-readable name. (slack) (#282)
+* slack: Use DisplayName instead of deprecated username (slack). Closes #276
+* slack: Allowed Slack bridge to extract simpler link format. (#287)
+* irc: Strip irc colors correct, strip also ctrl chars (irc)
+
+# v1.3.1
+## New features
+* Support mattermost 4.3.0 and every other 4.x as api4 should be stable (mattermost)
+## Bugfix
+* Use bot username if specified (slack). Closes #273
+
+# v1.3.0
+## New features
+* Relay slack_attachments from mattermost to slack (slack). Closes #260
+* Add support for quoting previous message when replying (telegram). #237
+* Add support for Quakenet auth (irc). Closes #263
+* Download files (max size 1MB) from slack and reupload to mattermost (slack/mattermost). Closes #255
+
+## Enhancements
+* Backoff for 60 seconds when reconnecting too fast (irc) #267
+* Use override username if specified (mattermost). #260
+
+## Bugfix
+* Try to not forward slack unfurls. Closes #266
+
+# v1.2.0
+## Breaking changes
+* If you're running a discord bridge, update to this release before 16 october otherwise
+it will stop working. (see https://discordapp.com/developers/docs/reference)
+
+## New features
+* general: Add delete support. (actually delete the messages on bridges that support it)
+    (mattermost,discord,gitter,slack,telegram)
+
+## Bugfix
+* Do not break messages on newline (slack). Closes #258 
+* Update telegram library
+* Update discord library (supports v6 API now). Old API is deprecated on 16 October
+
+# v1.1.2
+## New features
+* general: also build darwin binaries
+* mattermost: add support for mattermost 4.2.x
+
+## Bugfix 
+* mattermost: Send images when text is empty regression. (mattermost). Closes #254
+* slack: also send the first messsage after connect. #252
+
+# v1.1.1
+## Bugfix
+* mattermost: fix public links
+
+# v1.1.0
+## New features
+* general: Add better editing support. (actually edit the messages on bridges that support it)
+	(mattermost,discord,gitter,slack,telegram)
+* mattermost: use API v4 (removes support for mattermost < 3.8)
+* mattermost: add support for personal access tokens (since mattermost 4.1)
+	Use ```Token="yourtoken"``` in mattermost config
+	See https://docs.mattermost.com/developer/personal-access-tokens.html for more info
+* matrix: Relay notices (matrix). Closes #243
+* irc: Add a charset option. Closes #247
+
+## Bugfix
+* slack: Handle leave/join events (slack). Closes #246
+* slack: Replace mentions from other bridges. (slack). Closes #233
+* gitter: remove ZWSP after messages
+
 # v1.0.1
 ## New features
 * mattermost: add support for mattermost 4.1.x
 * discord: allow a webhookURL per channel #239
 
-## Bugfix
-
 # v1.0.0
 ## New features
 * general: Add action support for slack,mattermost,irc,gitter,matrix,xmpp,discord. #199

ci/bintray.sh

@@ -1,10 +1,11 @@
 #!/bin/bash
-go version |grep go1.8 || exit
+go version |grep go1.10 || exit
 VERSION=$(git describe --tags)
 mkdir ci/binaries
-GOOS=windows GOARCH=amd64 go build -ldflags "-s -w -X main.githash=$(git log --pretty=format:'%h' -n 1)" -o ci/binaries/matterbridge-$VERSION-win64.exe
-GOOS=linux GOARCH=amd64 go build -ldflags "-s -w -X main.githash=$(git log --pretty=format:'%h' -n 1)" -o ci/binaries/matterbridge-$VERSION-linux64
+GOOS=windows GOARCH=amd64 go build -ldflags "-s -w -X main.githash=$(git log --pretty=format:'%h' -n 1)" -o ci/binaries/matterbridge-$VERSION-windows-amd64.exe
+GOOS=linux GOARCH=amd64 go build -ldflags "-s -w -X main.githash=$(git log --pretty=format:'%h' -n 1)" -o ci/binaries/matterbridge-$VERSION-linux-amd64
 GOOS=linux GOARCH=arm go build -ldflags "-s -w -X main.githash=$(git log --pretty=format:'%h' -n 1)" -o ci/binaries/matterbridge-$VERSION-linux-arm
+GOOS=darwin GOARCH=amd64 go build -ldflags "-s -w -X main.githash=$(git log --pretty=format:'%h' -n 1)" -o ci/binaries/matterbridge-$VERSION-darwin-amd64
 cd ci
 cat > deploy.json <<EOF
 {

contrib/matterbridge.service

@@ -0,0 +1,11 @@
+[Unit]
+Description=matterbridge
+After=network.target
+
+[Service]
+ExecStart=/usr/bin/matterbridge -conf /etc/matterbridge/bridge.toml
+User=matterbridge
+Group=matterbridge
+
+[Install]
+WantedBy=multi-user.target

docker/arm/Dockerfile

@@ -0,0 +1,11 @@
+FROM cmosh/alpine-arm:edge
+ENTRYPOINT ["/bin/matterbridge"]
+
+COPY . /go/src/github.com/42wim/matterbridge
+RUN apk update && apk add go git gcc musl-dev ca-certificates \
+        && cd /go/src/github.com/42wim/matterbridge \
+        && export GOPATH=/go \
+        && go get \
+        && go build -x -ldflags "-X main.githash=$(git log --pretty=format:'%h' -n 1)" -o /bin/matterbridge \
+        && rm -rf /go \
+        && apk del --purge git go gcc musl-dev

gateway/gateway.go

@@ -1,15 +1,37 @@
 package gateway
 
 import (
+	"bytes"
 	"fmt"
+	"io/ioutil"
+	"net/http"
+	"os"
+
 	"github.com/42wim/matterbridge/bridge"
+	"github.com/42wim/matterbridge/bridge/api"
 	"github.com/42wim/matterbridge/bridge/config"
-	log "github.com/Sirupsen/logrus"
+	bdiscord "github.com/42wim/matterbridge/bridge/discord"
+	bgitter "github.com/42wim/matterbridge/bridge/gitter"
+	birc "github.com/42wim/matterbridge/bridge/irc"
+	bmatrix "github.com/42wim/matterbridge/bridge/matrix"
+	bmattermost "github.com/42wim/matterbridge/bridge/mattermost"
+	brocketchat "github.com/42wim/matterbridge/bridge/rocketchat"
+	bslack "github.com/42wim/matterbridge/bridge/slack"
+	bsshchat "github.com/42wim/matterbridge/bridge/sshchat"
+	bsteam "github.com/42wim/matterbridge/bridge/steam"
+	btelegram "github.com/42wim/matterbridge/bridge/telegram"
+	bxmpp "github.com/42wim/matterbridge/bridge/xmpp"
+	bzulip "github.com/42wim/matterbridge/bridge/zulip"
+	"github.com/hashicorp/golang-lru"
+	log "github.com/sirupsen/logrus"
 	//	"github.com/davecgh/go-spew/spew"
-	"github.com/peterhellberg/emojilib"
+	"crypto/sha1"
+	"path/filepath"
 	"regexp"
 	"strings"
 	"time"
+
+	"github.com/peterhellberg/emojilib"
 )
 
 type Gateway struct {
@@ -21,11 +43,42 @@ type Gateway struct {
 	ChannelOptions map[string]config.ChannelOptions
 	Message        chan config.Message
 	Name           string
+	Messages       *lru.Cache
+}
+
+type BrMsgID struct {
+	br        *bridge.Bridge
+	ID        string
+	ChannelID string
+}
+
+var flog *log.Entry
+
+var bridgeMap = map[string]bridge.Factory{
+	"api":        api.New,
+	"discord":    bdiscord.New,
+	"gitter":     bgitter.New,
+	"irc":        birc.New,
+	"mattermost": bmattermost.New,
+	"matrix":     bmatrix.New,
+	"rocketchat": brocketchat.New,
+	"slack":      bslack.New,
+	"sshchat":    bsshchat.New,
+	"steam":      bsteam.New,
+	"telegram":   btelegram.New,
+	"xmpp":       bxmpp.New,
+	"zulip":      bzulip.New,
+}
+
+func init() {
+	flog = log.WithFields(log.Fields{"prefix": "gateway"})
 }
 
 func New(cfg config.Gateway, r *Router) *Gateway {
 	gw := &Gateway{Channels: make(map[string]*config.ChannelInfo), Message: r.Message,
 		Router: r, Bridges: make(map[string]*bridge.Bridge), Config: r.Config}
+	cache, _ := lru.New(5000)
+	gw.Messages = cache
 	gw.AddConfig(&cfg)
 	return gw
 }
@@ -33,7 +86,14 @@ func New(cfg config.Gateway, r *Router) *Gateway {
 func (gw *Gateway) AddBridge(cfg *config.Bridge) error {
 	br := gw.Router.getBridge(cfg.Account)
 	if br == nil {
-		br = bridge.New(gw.Config, cfg, gw.Message)
+		br = bridge.New(cfg)
+		br.Config = gw.Router.Config
+		br.General = &gw.General
+		// set logging
+		br.Log = log.WithFields(log.Fields{"prefix": "bridge"})
+		brconfig := &bridge.Config{Remote: gw.Message, Log: log.WithFields(log.Fields{"prefix": br.Protocol}), Bridge: br}
+		// add the actual bridger for this protocol to this bridge using the bridgeMap
+		br.Bridger = bridgeMap[br.Protocol](brconfig)
 	}
 	gw.mapChannelsToBridge(br)
 	gw.Bridges[cfg.Account] = br
@@ -65,10 +125,10 @@ func (gw *Gateway) reconnectBridge(br *bridge.Bridge) {
 	br.Disconnect()
 	time.Sleep(time.Second * 5)
 RECONNECT:
-	log.Infof("Reconnecting %s", br.Account)
+	flog.Infof("Reconnecting %s", br.Account)
 	err := br.Connect()
 	if err != nil {
-		log.Errorf("Reconnection failed: %s. Trying again in 60 seconds", err)
+		flog.Errorf("Reconnection failed: %s. Trying again in 60 seconds", err)
 		time.Sleep(time.Second * 60)
 		goto RECONNECT
 	}
@@ -81,6 +141,10 @@ func (gw *Gateway) mapChannelConfig(cfg []config.Bridge, direction string) {
 		if isApi(br.Account) {
 			br.Channel = "api"
 		}
+		// make sure to lowercase irc channels in config #348
+		if strings.HasPrefix(br.Account, "irc.") {
+			br.Channel = strings.ToLower(br.Channel)
+		}
 		ID := br.Channel + br.Account
 		if _, ok := gw.Channels[ID]; !ok {
 			channel := &config.ChannelInfo{Name: br.Channel, Direction: direction, ID: ID, Options: br.Options, Account: br.Account,
@@ -106,6 +170,12 @@ func (gw *Gateway) mapChannels() error {
 
 func (gw *Gateway) getDestChannel(msg *config.Message, dest bridge.Bridge) []config.ChannelInfo {
 	var channels []config.ChannelInfo
+
+	// for messages received from the api check that the gateway is the specified one
+	if msg.Protocol == "api" && gw.Name != msg.Gateway {
+		return channels
+	}
+
 	// if source channel is in only, do nothing
 	for _, channel := range gw.Channels {
 		// lookup the channel from the message
@@ -122,7 +192,7 @@ func (gw *Gateway) getDestChannel(msg *config.Message, dest bridge.Bridge) []con
 			continue
 		}
 
-		// do samechannelgateway logic
+		// do samechannelgateway flogic
 		if channel.SameChannel[msg.Gateway] {
 			if msg.Channel == channel.Name && msg.Account != dest.Account {
 				channels = append(channels, *channel)
@@ -136,37 +206,87 @@ func (gw *Gateway) getDestChannel(msg *config.Message, dest bridge.Bridge) []con
 	return channels
 }
 
-func (gw *Gateway) handleMessage(msg config.Message, dest *bridge.Bridge) {
-	// only relay join/part when configged
-	if msg.Event == config.EVENT_JOIN_LEAVE && !gw.Bridges[dest.Account].Config.ShowJoinPart {
-		return
+func (gw *Gateway) handleMessage(msg config.Message, dest *bridge.Bridge) []*BrMsgID {
+	var brMsgIDs []*BrMsgID
+
+	// if we have an attached file, or other info
+	if msg.Extra != nil {
+		if len(msg.Extra[config.EVENT_FILE_FAILURE_SIZE]) != 0 {
+			if msg.Text == "" {
+				return brMsgIDs
+			}
+		}
 	}
+
+	// Avatar downloads are only relevant for telegram and mattermost for now
+	if msg.Event == config.EVENT_AVATAR_DOWNLOAD {
+		if dest.Protocol != "mattermost" &&
+			dest.Protocol != "telegram" {
+			return brMsgIDs
+		}
+	}
+
+	// only relay join/part when configured
+	if msg.Event == config.EVENT_JOIN_LEAVE && !gw.Bridges[dest.Account].GetBool("ShowJoinPart") {
+		return brMsgIDs
+	}
+
+	// only relay topic change when configured
+	if msg.Event == config.EVENT_TOPIC_CHANGE && !gw.Bridges[dest.Account].GetBool("ShowTopicChange") {
+		return brMsgIDs
+	}
+
 	// broadcast to every out channel (irc QUIT)
 	if msg.Channel == "" && msg.Event != config.EVENT_JOIN_LEAVE {
-		log.Debug("empty channel")
-		return
+		flog.Debug("empty channel")
+		return brMsgIDs
 	}
+
 	originchannel := msg.Channel
 	origmsg := msg
 	channels := gw.getDestChannel(&msg, *dest)
 	for _, channel := range channels {
-		// do not send to ourself
-		if channel.ID == getChannelID(origmsg) {
-			continue
+		// Only send the avatar download event to ourselves.
+		if msg.Event == config.EVENT_AVATAR_DOWNLOAD {
+			if channel.ID != getChannelID(origmsg) {
+				continue
+			}
+		} else {
+			// do not send to ourself for any other event
+			if channel.ID == getChannelID(origmsg) {
+				continue
+			}
 		}
-		log.Debugf("Sending %#v from %s (%s) to %s (%s)", msg, msg.Account, originchannel, dest.Account, channel.Name)
+		flog.Debugf("=> Sending %#v from %s (%s) to %s (%s)", msg, msg.Account, originchannel, dest.Account, channel.Name)
 		msg.Channel = channel.Name
 		msg.Avatar = gw.modifyAvatar(origmsg, dest)
 		msg.Username = gw.modifyUsername(origmsg, dest)
+		msg.ID = ""
+		if res, ok := gw.Messages.Get(origmsg.ID); ok {
+			IDs := res.([]*BrMsgID)
+			for _, id := range IDs {
+				// check protocol, bridge name and channelname
+				// for people that reuse the same bridge multiple times. see #342
+				if dest.Protocol == id.br.Protocol && dest.Name == id.br.Name && channel.ID == id.ChannelID {
+					msg.ID = id.ID
+				}
+			}
+		}
 		// for api we need originchannel as channel
 		if dest.Protocol == "api" {
 			msg.Channel = originchannel
 		}
-		err := dest.Send(msg)
+		mID, err := dest.Send(msg)
 		if err != nil {
-			fmt.Println(err)
+			flog.Error(err)
+		}
+		// append the message ID (mID) from this bridge (dest) to our brMsgIDs slice
+		if mID != "" {
+			flog.Debugf("mID %s: %s", dest.Account, mID)
+			brMsgIDs = append(brMsgIDs, &BrMsgID{dest, mID, channel.ID})
 		}
 	}
+	return brMsgIDs
 }
 
 func (gw *Gateway) ignoreMessage(msg *config.Message) bool {
@@ -174,26 +294,39 @@ func (gw *Gateway) ignoreMessage(msg *config.Message) bool {
 	if _, ok := gw.Bridges[msg.Account]; !ok {
 		return true
 	}
+
+	// check if we need to ignore a empty message
 	if msg.Text == "" {
-		log.Debugf("ignoring empty message %#v from %s", msg, msg.Account)
+		// we have an attachment or actual bytes, do not ignore
+		if msg.Extra != nil &&
+			(msg.Extra["attachments"] != nil ||
+				len(msg.Extra["file"]) > 0 ||
+				len(msg.Extra[config.EVENT_FILE_FAILURE_SIZE]) > 0) {
+			return false
+		}
+		flog.Debugf("ignoring empty message %#v from %s", msg, msg.Account)
 		return true
 	}
-	for _, entry := range strings.Fields(gw.Bridges[msg.Account].Config.IgnoreNicks) {
+
+	// is the username in IgnoreNicks field
+	for _, entry := range strings.Fields(gw.Bridges[msg.Account].GetString("IgnoreNicks")) {
 		if msg.Username == entry {
-			log.Debugf("ignoring %s from %s", msg.Username, msg.Account)
+			flog.Debugf("ignoring %s from %s", msg.Username, msg.Account)
 			return true
 		}
 	}
+
+	// does the message match regex in IgnoreMessages field
 	// TODO do not compile regexps everytime
-	for _, entry := range strings.Fields(gw.Bridges[msg.Account].Config.IgnoreMessages) {
+	for _, entry := range strings.Fields(gw.Bridges[msg.Account].GetString("IgnoreMessages")) {
 		if entry != "" {
 			re, err := regexp.Compile(entry)
 			if err != nil {
-				log.Errorf("incorrect regexp %s for %s", entry, msg.Account)
+				flog.Errorf("incorrect regexp %s for %s", entry, msg.Account)
 				continue
 			}
 			if re.MatchString(msg.Text) {
-				log.Debugf("matching %s. ignoring %s from %s", entry, msg.Text, msg.Account)
+				flog.Debugf("matching %s. ignoring %s from %s", entry, msg.Text, msg.Account)
 				return true
 			}
 		}
@@ -204,10 +337,28 @@ func (gw *Gateway) ignoreMessage(msg *config.Message) bool {
 func (gw *Gateway) modifyUsername(msg config.Message, dest *bridge.Bridge) string {
 	br := gw.Bridges[msg.Account]
 	msg.Protocol = br.Protocol
-	nick := gw.Config.General.RemoteNickFormat
-	if nick == "" {
-		nick = dest.Config.RemoteNickFormat
+	if gw.Config.General.StripNick || dest.GetBool("StripNick") {
+		re := regexp.MustCompile("[^a-zA-Z0-9]+")
+		msg.Username = re.ReplaceAllString(msg.Username, "")
 	}
+	nick := dest.GetString("RemoteNickFormat")
+	if nick == "" {
+		nick = gw.Config.General.RemoteNickFormat
+	}
+
+	// loop to replace nicks
+	for _, outer := range br.GetStringSlice2D("ReplaceNicks") {
+		search := outer[0]
+		replace := outer[1]
+		// TODO move compile to bridge init somewhere
+		re, err := regexp.Compile(search)
+		if err != nil {
+			flog.Errorf("regexp in %s failed: %s", msg.Account, err)
+			break
+		}
+		msg.Username = re.ReplaceAllString(msg.Username, replace)
+	}
+
 	if len(msg.Username) > 0 {
 		// fix utf-8 issue #193
 		i := 0
@@ -220,16 +371,18 @@ func (gw *Gateway) modifyUsername(msg config.Message, dest *bridge.Bridge) strin
 		}
 		nick = strings.Replace(nick, "{NOPINGNICK}", msg.Username[:i]+"​"+msg.Username[i:], -1)
 	}
-	nick = strings.Replace(nick, "{NICK}", msg.Username, -1)
+
 	nick = strings.Replace(nick, "{BRIDGE}", br.Name, -1)
 	nick = strings.Replace(nick, "{PROTOCOL}", br.Protocol, -1)
+	nick = strings.Replace(nick, "{LABEL}", br.GetString("Label"), -1)
+	nick = strings.Replace(nick, "{NICK}", msg.Username, -1)
 	return nick
 }
 
 func (gw *Gateway) modifyAvatar(msg config.Message, dest *bridge.Bridge) string {
 	iconurl := gw.Config.General.IconURL
 	if iconurl == "" {
-		iconurl = dest.Config.IconURL
+		iconurl = dest.GetString("IconURL")
 	}
 	iconurl = strings.Replace(iconurl, "{NICK}", msg.Username, -1)
 	if msg.Avatar == "" {
@@ -241,17 +394,115 @@ func (gw *Gateway) modifyAvatar(msg config.Message, dest *bridge.Bridge) string
 func (gw *Gateway) modifyMessage(msg *config.Message) {
 	// replace :emoji: to unicode
 	msg.Text = emojilib.Replace(msg.Text)
-	msg.Gateway = gw.Name
+
+	br := gw.Bridges[msg.Account]
+	// loop to replace messages
+	for _, outer := range br.GetStringSlice2D("ReplaceMessages") {
+		search := outer[0]
+		replace := outer[1]
+		// TODO move compile to bridge init somewhere
+		re, err := regexp.Compile(search)
+		if err != nil {
+			flog.Errorf("regexp in %s failed: %s", msg.Account, err)
+			break
+		}
+		msg.Text = re.ReplaceAllString(msg.Text, replace)
+	}
+
+	// messages from api have Gateway specified, don't overwrite
+	if msg.Protocol != "api" {
+		msg.Gateway = gw.Name
+	}
 }
 
-func getChannelID(msg config.Message) string {
-	return msg.Channel + msg.Account
+// handleFiles uploads or places all files on the given msg to the MediaServer and
+// adds the new URL of the file on the MediaServer onto the given msg.
+func (gw *Gateway) handleFiles(msg *config.Message) {
+	reg := regexp.MustCompile("[^a-zA-Z0-9]+")
+
+	// If we don't have a attachfield or we don't have a mediaserver configured return
+	if msg.Extra == nil || (gw.Config.General.MediaServerUpload == "" && gw.Config.General.MediaDownloadPath == "") {
+		return
+	}
+
+	// If we don't have files, nothing to upload.
+	if len(msg.Extra["file"]) == 0 {
+		return
+	}
+
+	client := &http.Client{
+		Timeout: time.Second * 5,
+	}
+
+	for i, f := range msg.Extra["file"] {
+		fi := f.(config.FileInfo)
+		ext := filepath.Ext(fi.Name)
+		fi.Name = fi.Name[0 : len(fi.Name)-len(ext)]
+		fi.Name = reg.ReplaceAllString(fi.Name, "_")
+		fi.Name = fi.Name + ext
+
+		sha1sum := fmt.Sprintf("%x", sha1.Sum(*fi.Data))[:8]
+
+		if gw.Config.General.MediaServerUpload != "" {
+			// Use MediaServerUpload. Upload using a PUT HTTP request and basicauth.
+
+			url := gw.Config.General.MediaServerUpload + "/" + sha1sum + "/" + fi.Name
+
+			req, err := http.NewRequest("PUT", url, bytes.NewReader(*fi.Data))
+			if err != nil {
+				flog.Errorf("mediaserver upload failed, could not create request: %#v", err)
+				continue
+			}
+
+			flog.Debugf("mediaserver upload url: %s", url)
+
+			req.Header.Set("Content-Type", "binary/octet-stream")
+			_, err = client.Do(req)
+			if err != nil {
+				flog.Errorf("mediaserver upload failed, could not Do request: %#v", err)
+				continue
+			}
+		} else {
+			// Use MediaServerPath. Place the file on the current filesystem.
+
+			dir := gw.Config.General.MediaDownloadPath + "/" + sha1sum
+			err := os.Mkdir(dir, os.ModePerm)
+			if err != nil && !os.IsExist(err) {
+				flog.Errorf("mediaserver path failed, could not mkdir: %s %#v", err, err)
+				continue
+			}
+
+			path := dir + "/" + fi.Name
+			flog.Debugf("mediaserver path placing file: %s", path)
+
+			err = ioutil.WriteFile(path, *fi.Data, os.ModePerm)
+			if err != nil {
+				flog.Errorf("mediaserver path failed, could not writefile: %s %#v", err, err)
+				continue
+			}
+		}
+
+		// Download URL.
+		durl := gw.Config.General.MediaServerDownload + "/" + sha1sum + "/" + fi.Name
+
+		flog.Debugf("mediaserver download URL = %s", durl)
+
+		// We uploaded/placed the file successfully. Add the SHA and URL.
+		extra := msg.Extra["file"][i].(config.FileInfo)
+		extra.URL = durl
+		extra.SHA = sha1sum
+		msg.Extra["file"][i] = extra
+	}
 }
 
 func (gw *Gateway) validGatewayDest(msg *config.Message, channel *config.ChannelInfo) bool {
 	return msg.Gateway == gw.Name
 }
 
+func getChannelID(msg config.Message) string {
+	return msg.Channel + msg.Account
+}
+
 func isApi(account string) bool {
 	return strings.HasPrefix(account, "api.")
 }

gateway/gateway_test.go

@@ -2,15 +2,15 @@ package gateway
 
 import (
 	"fmt"
-	"github.com/42wim/matterbridge/bridge/config"
-	"github.com/BurntSushi/toml"
-	"github.com/stretchr/testify/assert"
 	"strconv"
 
+	"github.com/42wim/matterbridge/bridge/config"
+	"github.com/stretchr/testify/assert"
+
 	"testing"
 )
 
-var testconfig = `
+var testconfig = []byte(`
 [irc.freenode]
 [mattermost.test]
 [gitter.42wim]
@@ -37,9 +37,9 @@ var testconfig = `
     [[gateway.inout]]
     account="slack.test"
     channel="testing"
-	`
+	`)
 
-var testconfig2 = `
+var testconfig2 = []byte(`
 [irc.freenode]
 [mattermost.test]
 [gitter.42wim]
@@ -80,8 +80,9 @@ var testconfig2 = `
     [[gateway.out]]
     account = "discord.test"
     channel = "general2"
-	`
-var testconfig3 = `
+	`)
+
+var testconfig3 = []byte(`
 [irc.zzz]
 [telegram.zzz]
 [slack.zzz]
@@ -149,13 +150,10 @@ enable=true
     [[gateway.inout]]
     account="telegram.zzz"
     channel="--333333333333"
-`
+`)
 
-func maketestRouter(input string) *Router {
-	var cfg *config.Config
-	if _, err := toml.Decode(input, &cfg); err != nil {
-		fmt.Println(err)
-	}
+func maketestRouter(input []byte) *Router {
+	cfg := config.NewConfigFromString(input)
 	r, err := NewRouter(cfg)
 	if err != nil {
 		fmt.Println(err)
@@ -163,14 +161,7 @@ func maketestRouter(input string) *Router {
 	return r
 }
 func TestNewRouter(t *testing.T) {
-	var cfg *config.Config
-	if _, err := toml.Decode(testconfig, &cfg); err != nil {
-		fmt.Println(err)
-	}
-	r, err := NewRouter(cfg)
-	if err != nil {
-		fmt.Println(err)
-	}
+	r := maketestRouter(testconfig)
 	assert.Equal(t, 1, len(r.Gateways))
 	assert.Equal(t, 4, len(r.Gateways["bridge1"].Bridges))
 	assert.Equal(t, 4, len(r.Gateways["bridge1"].Channels))

gateway/router.go

@@ -2,10 +2,10 @@ package gateway
 
 import (
 	"fmt"
+
 	"github.com/42wim/matterbridge/bridge"
 	"github.com/42wim/matterbridge/bridge/config"
-	"github.com/42wim/matterbridge/gateway/samechannel"
-	log "github.com/Sirupsen/logrus"
+	samechannelgateway "github.com/42wim/matterbridge/gateway/samechannel"
 	//	"github.com/davecgh/go-spew/spew"
 	"time"
 )
@@ -17,10 +17,7 @@ type Router struct {
 }
 
 func NewRouter(cfg *config.Config) (*Router, error) {
-	r := &Router{}
-	r.Config = cfg
-	r.Message = make(chan config.Message)
-	r.Gateways = make(map[string]*Gateway)
+	r := &Router{Message: make(chan config.Message), Gateways: make(map[string]*Gateway), Config: cfg}
 	sgw := samechannelgateway.New(cfg)
 	gwconfigs := sgw.GetConfig()
 
@@ -42,12 +39,13 @@ func NewRouter(cfg *config.Config) (*Router, error) {
 func (r *Router) Start() error {
 	m := make(map[string]*bridge.Bridge)
 	for _, gw := range r.Gateways {
+		flog.Infof("Parsing gateway %s", gw.Name)
 		for _, br := range gw.Bridges {
 			m[br.Account] = br
 		}
 	}
 	for _, br := range m {
-		log.Infof("Starting bridge: %s ", br.Account)
+		flog.Infof("Starting bridge: %s ", br.Account)
 		err := br.Connect()
 		if err != nil {
 			return fmt.Errorf("Bridge %s failed to start: %v", br.Account, err)
@@ -94,11 +92,18 @@ func (r *Router) handleReceive() {
 			}
 		}
 		for _, gw := range r.Gateways {
+			// record all the message ID's of the different bridges
+			var msgIDs []*BrMsgID
 			if !gw.ignoreMessage(&msg) {
 				msg.Timestamp = time.Now()
 				gw.modifyMessage(&msg)
+				gw.handleFiles(&msg)
 				for _, br := range gw.Bridges {
-					gw.handleMessage(msg, br)
+					msgIDs = append(msgIDs, gw.handleMessage(msg, br)...)
+				}
+				// only add the message ID if it doesn't already exists
+				if _, ok := gw.Messages.Get(msg.ID); !ok && msg.ID != "" {
+					gw.Messages.Add(msg.ID, msgIDs)
 				}
 			}
 		}

gateway/samechannel/samechannel_test.go

@@ -2,6 +2,7 @@ package samechannelgateway
 
 import (
 	"fmt"
+
 	"github.com/42wim/matterbridge/bridge/config"
 	"github.com/BurntSushi/toml"
 	"github.com/stretchr/testify/assert"

matterbridge.go

@@ -3,23 +3,24 @@ package main
 import (
 	"flag"
 	"fmt"
+	"os"
+	"strings"
+
 	"github.com/42wim/matterbridge/bridge/config"
 	"github.com/42wim/matterbridge/gateway"
-	log "github.com/Sirupsen/logrus"
 	"github.com/google/gops/agent"
-	"strings"
+	log "github.com/sirupsen/logrus"
+	prefixed "github.com/x-cray/logrus-prefixed-formatter"
 )
 
 var (
-	version = "1.0.1"
+	version = "1.11.1"
 	githash string
 )
 
-func init() {
-	log.SetFormatter(&log.TextFormatter{FullTimestamp: true})
-}
-
 func main() {
+	log.SetFormatter(&prefixed.TextFormatter{PrefixPadding: 13, DisableColors: true, FullTimestamp: true})
+	flog := log.WithFields(log.Fields{"prefix": "main"})
 	flagConfig := flag.String("conf", "matterbridge.toml", "config file")
 	flagDebug := flag.Bool("debug", false, "enable debug")
 	flagVersion := flag.Bool("version", false, "show version")
@@ -33,23 +34,25 @@ func main() {
 		fmt.Printf("version: %s %s\n", version, githash)
 		return
 	}
-	if *flagDebug {
-		log.Info("Enabling debug")
+	if *flagDebug || os.Getenv("DEBUG") == "1" {
+		log.SetFormatter(&prefixed.TextFormatter{PrefixPadding: 13, DisableColors: true, FullTimestamp: false, ForceFormatting: true})
+		flog.Info("Enabling debug")
 		log.SetLevel(log.DebugLevel)
 	}
-	log.Printf("Running version %s %s", version, githash)
+	flog.Printf("Running version %s %s", version, githash)
 	if strings.Contains(version, "-dev") {
-		log.Println("WARNING: THIS IS A DEVELOPMENT VERSION. Things may break.")
+		flog.Println("WARNING: THIS IS A DEVELOPMENT VERSION. Things may break.")
 	}
 	cfg := config.NewConfig(*flagConfig)
+	cfg.General.Debug = *flagDebug
 	r, err := gateway.NewRouter(cfg)
 	if err != nil {
-		log.Fatalf("Starting gateway failed: %s", err)
+		flog.Fatalf("Starting gateway failed: %s", err)
 	}
 	err = r.Start()
 	if err != nil {
-		log.Fatalf("Starting gateway failed: %s", err)
+		flog.Fatalf("Starting gateway failed: %s", err)
 	}
-	log.Printf("Gateway(s) started succesfully. Now relaying messages")
+	flog.Printf("Gateway(s) started succesfully. Now relaying messages")
 	select {}
 }

matterbridge.toml.sample

@@ -32,16 +32,41 @@ UseSASL=false
 #OPTIONAL (default false)
 SkipTLSVerify=true
 
+#If you know your charset, you can specify it manually. 
+#Otherwise it tries to detect this automatically. Select one below
+# "iso-8859-2:1987", "iso-8859-9:1989", "866", "latin9", "iso-8859-10:1992", "iso-ir-109", "hebrew", 
+# "cp932", "iso-8859-15", "cp437", "utf-16be", "iso-8859-3:1988", "windows-1251", "utf16", "latin6", 
+# "latin3", "iso-8859-1:1987", "iso-8859-9", "utf-16le", "big5", "cp819", "asmo-708", "utf-8", 
+# "ibm437", "iso-ir-157", "iso-ir-144", "latin4", "850", "iso-8859-5", "iso-8859-5:1988", "l3", 
+# "windows-31j", "utf8", "iso-8859-3", "437", "greek", "iso-8859-8", "l6", "l9-iso-8859-15", 
+# "iso-8859-2", "latin2", "iso-ir-100", "iso-8859-6", "arabic", "iso-ir-148", "us-ascii", "x-sjis", 
+# "utf16be", "iso-8859-8:1988", "utf16le", "l4", "utf-16", "iso-ir-138", "iso-8859-7", "iso-8859-7:1987", 
+# "windows-1252", "l2", "koi8-r", "iso8859-1", "latin1", "ecma-114", "iso-ir-110", "elot-928", 
+# "iso-ir-126", "iso-8859-1", "iso-ir-127", "cp850", "cyrillic", "greek8", "windows-1250", "iso-latin-1", 
+# "l5", "ibm866", "cp866", "ms-kanji", "ibm850", "ecma-118", "iso-ir-101", "ibm819", "l1", "iso-8859-6:1987", 
+# "latin5", "ascii", "sjis", "iso-8859-10", "iso-8859-4", "iso-8859-4:1988", "shift-jis
+# The select charset will be converted to utf-8 when sent to other bridges.
+#OPTIONAL (default "")
+Charset=""
+
 #Your nick on irc. 
 #REQUIRED
 Nick="matterbot"
 
 #If you registered your bot with a service like Nickserv on freenode. 
 #Also being used when UseSASL=true
+#
+#Note: if you want do to quakenet auth, set NickServNick="Q@CServe.quakenet.org"
 #OPTIONAL
 NickServNick="nickserv"
 NickServPassword="secret"
 
+#OPTIONAL only used for quakenet auth
+NickServUsername="username"
+
+## RELOADABLE SETTINGS
+## Settings below can be reloaded by editing the file
+
 #Flood control
 #Delay in milliseconds between each message send to the IRC server
 #OPTIONAL (default 1300)
@@ -58,6 +83,19 @@ MessageQueue=30
 #OPTIONAL (default 400)
 MessageLength=400
 
+#Split messages on MessageLength instead of showing the <message clipped>
+#WARNING: this could lead to flooding
+#OPTIONAL (default false)
+MessageSplit=false
+
+#Delay in seconds to rejoin a channel when kicked
+#OPTIONAL (default 0)
+RejoinDelay=0
+
+#ColorNicks will show each nickname in a different color.
+#Only works in IRC right now.
+ColorNicks=false
+
 #Nicks you want to ignore. 
 #Messages from those users will not be sent to other bridges.
 #OPTIONAL
@@ -69,19 +107,56 @@ IgnoreNicks="ircspammer1 ircspammer2"
 #OPTIONAL (example below ignores messages starting with ~~ or messages containing badword
 IgnoreMessages="^~~ badword"
 
+#messages you want to replace.
+#it replaces outgoing messages from the bridge.
+#so you need to place it by the sending bridge definition.
+#regular expressions supported
+#some examples:
+#this replaces cat => dog and sleep => awake
+#replacemessages=[ ["cat","dog"], ["sleep","awake"] ]
+#this replaces every number with number.  123 => numbernumbernumber
+#replacemessages=[ ["[0-9]","number"] ]
+#optional (default empty)
+ReplaceMessages=[ ["cat","dog"] ]
+
+#nicks you want to replace.
+#see replacemessages for syntaxa
+#optional (default empty)
+ReplaceNicks=[ ["user--","user"] ]
+
+#extra label that can be used in the RemoteNickFormat
+#optional (default empty)
+Label=""
+
 #RemoteNickFormat defines how remote users appear on this bridge 
 #The string "{NICK}" (case sensitive) will be replaced by the actual nick / username.
 #The string "{BRIDGE}" (case sensitive) will be replaced by the sending bridge
+#The string "{LABEL}" (case sensitive) will be replaced by label= field of the sending bridge
 #The string "{PROTOCOL}" (case sensitive) will be replaced by the protocol used by the bridge
 #The string "{NOPINGNICK}" (case sensitive) will be replaced by the actual nick / username, but with a ZWSP inside the nick, so the irc user with the same nick won't get pinged. See https://github.com/42wim/matterbridge/issues/175 for more information
 #OPTIONAL (default empty)
 RemoteNickFormat="[{PROTOCOL}] <{NICK}> "
 
 #Enable to show users joins/parts from other bridges 
-#Only works hiding/show messages from irc and mattermost bridge for now
+#Currently works for messages from the following bridges: irc, mattermost, slack
 #OPTIONAL (default false)
 ShowJoinPart=false
 
+#Do not send joins/parts to other bridges
+#Currently works for messages from the following bridges: irc, mattermost, slack
+#OPTIONAL (default false)
+NoSendJoinPart=false
+
+#StripNick only allows alphanumerical nicks. See https://github.com/42wim/matterbridge/issues/285
+#It will strip other characters from the nick
+#OPTIONAL (default false)
+StripNick=false
+
+#Enable to show topic changes from other bridges 
+#Only works hiding/show topic changes from slack bridge for now
+#OPTIONAL (default false)
+ShowTopicChange=false
+
 ###################################################################
 #XMPP section
 ###################################################################
@@ -116,6 +191,9 @@ Nick="xmppbot"
 #OPTIONAL (default false)
 SkipTLSVerify=true
 
+## RELOADABLE SETTINGS
+## Settings below can be reloaded by editing the file
+
 #Nicks you want to ignore. 
 #Messages from those users will not be sent to other bridges.
 #OPTIONAL
@@ -127,18 +205,49 @@ IgnoreNicks="ircspammer1 ircspammer2"
 #OPTIONAL (example below ignores messages starting with ~~ or messages containing badword
 IgnoreMessages="^~~ badword"
 
+#Messages you want to replace.
+#It replaces outgoing messages from the bridge.
+#So you need to place it by the sending bridge definition.
+#Regular expressions supported
+#Some examples:
+#This replaces cat => dog and sleep => awake
+#ReplaceMessages=[ ["cat","dog"], ["sleep","awake"] ]
+#This Replaces every number with number.  123 => numbernumbernumber
+#ReplaceMessages=[ ["[0-9]","number"] ]
+#OPTIONAL (default empty)
+ReplaceMessages=[ ["cat","dog"] ]
+
+#Nicks you want to replace.
+#See ReplaceMessages for syntaxA
+#OPTIONAL (default empty)
+ReplaceNicks=[ ["user--","user"] ]
+
+#extra label that can be used in the RemoteNickFormat
+#optional (default empty)
+Label=""
+
 #RemoteNickFormat defines how remote users appear on this bridge 
 #The string "{NICK}" (case sensitive) will be replaced by the actual nick / username.
 #The string "{BRIDGE}" (case sensitive) will be replaced by the sending bridge
+#The string "{LABEL}" (case sensitive) will be replaced by label= field of the sending bridge
 #The string "{PROTOCOL}" (case sensitive) will be replaced by the protocol used by the bridge
 #OPTIONAL (default empty)
 RemoteNickFormat="[{PROTOCOL}] <{NICK}> "
 
 #Enable to show users joins/parts from other bridges 
-#Only works hiding/show messages from irc and mattermost bridge for now
+#Currently works for messages from the following bridges: irc, mattermost, slack
 #OPTIONAL (default false)
 ShowJoinPart=false
 
+#StripNick only allows alphanumerical nicks. See https://github.com/42wim/matterbridge/issues/285
+#It will strip other characters from the nick
+#OPTIONAL (default false)
+StripNick=false
+
+#Enable to show topic changes from other bridges 
+#Only works hiding/show topic changes from slack bridge for now
+#OPTIONAL (default false)
+ShowTopicChange=false
 
 ###################################################################
 #hipchat section
@@ -166,6 +275,9 @@ Muc="conf.hipchat.com"
 #REQUIRED
 Nick="yourlogin"
 
+## RELOADABLE SETTINGS
+## Settings below can be reloaded by editing the file
+
 #Nicks you want to ignore. 
 #Messages from those users will not be sent to other bridges.
 #OPTIONAL
@@ -177,18 +289,49 @@ IgnoreNicks="spammer1 spammer2"
 #OPTIONAL (example below ignores messages starting with ~~ or messages containing badword
 IgnoreMessages="^~~ badword"
 
+#messages you want to replace.
+#it replaces outgoing messages from the bridge.
+#so you need to place it by the sending bridge definition.
+#regular expressions supported
+#some examples:
+#this replaces cat => dog and sleep => awake
+#replacemessages=[ ["cat","dog"], ["sleep","awake"] ]
+#this replaces every number with number.  123 => numbernumbernumber
+#replacemessages=[ ["[0-9]","number"] ]
+#optional (default empty)
+ReplaceMessages=[ ["cat","dog"] ]
+
+#nicks you want to replace.
+#see replacemessages for syntaxa
+#optional (default empty)
+ReplaceNicks=[ ["user--","user"] ]
+
+#extra label that can be used in the RemoteNickFormat
+#optional (default empty)
+Label=""
+
 #RemoteNickFormat defines how remote users appear on this bridge 
 #The string "{NICK}" (case sensitive) will be replaced by the actual nick / username.
 #The string "{BRIDGE}" (case sensitive) will be replaced by the sending bridge
+#The string "{LABEL}" (case sensitive) will be replaced by label= field of the sending bridge
 #The string "{PROTOCOL}" (case sensitive) will be replaced by the protocol used by the bridge
 #OPTIONAL (default empty)
 RemoteNickFormat="[{PROTOCOL}/{BRIDGE}] <{NICK}> "
 
 #Enable to show users joins/parts from other bridges 
-#Only works hiding/show messages from irc and mattermost bridge for now
+#Currently works for messages from the following bridges: irc, mattermost, slack
 #OPTIONAL (default false)
 ShowJoinPart=false
 
+#StripNick only allows alphanumerical nicks. See https://github.com/42wim/matterbridge/issues/285
+#It will strip other characters from the nick
+#OPTIONAL (default false)
+StripNick=false
+
+#Enable to show topic changes from other bridges 
+#Only works hiding/show topic changes from slack bridge for now
+#OPTIONAL (default false)
+ShowTopicChange=false
 
 ###################################################################
 #mattermost section
@@ -213,6 +356,11 @@ Team="yourteam"
 Login="yourlogin"
 Password="yourpass"
 
+#personal access token of the bot.
+#new feature since mattermost 4.1. See https://docs.mattermost.com/developer/personal-access-tokens.html
+#OPTIONAL (you can use token instead of login/password)
+#Token="abcdefghijklm"
+
 #Enable this to make a http connection (instead of https) to your mattermost. 
 #OPTIONAL (default false)
 NoTLS=false
@@ -247,6 +395,9 @@ IconURL="http://youricon.png"
 #OPTIONAL (default false)
 SkipTLSVerify=true
 
+## RELOADABLE SETTINGS
+## Settings below can be reloaded by editing the file
+
 #how to format the list of IRC nicks when displayed in mattermost. 
 #Possible options are "table" and "plain"
 #OPTIONAL (default plain)
@@ -282,18 +433,55 @@ IgnoreNicks="ircspammer1 ircspammer2"
 #OPTIONAL (example below ignores messages starting with ~~ or messages containing badword
 IgnoreMessages="^~~ badword"
 
+#messages you want to replace.
+#it replaces outgoing messages from the bridge.
+#so you need to place it by the sending bridge definition.
+#regular expressions supported
+#some examples:
+#this replaces cat => dog and sleep => awake
+#replacemessages=[ ["cat","dog"], ["sleep","awake"] ]
+#this replaces every number with number.  123 => numbernumbernumber
+#replacemessages=[ ["[0-9]","number"] ]
+#optional (default empty)
+ReplaceMessages=[ ["cat","dog"] ]
+
+#nicks you want to replace.
+#see replacemessages for syntaxa
+#optional (default empty)
+ReplaceNicks=[ ["user--","user"] ]
+
+#extra label that can be used in the RemoteNickFormat
+#optional (default empty)
+Label=""
+
 #RemoteNickFormat defines how remote users appear on this bridge 
 #The string "{NICK}" (case sensitive) will be replaced by the actual nick / username.
 #The string "{BRIDGE}" (case sensitive) will be replaced by the sending bridge
+#The string "{LABEL}" (case sensitive) will be replaced by label= field of the sending bridge
 #The string "{PROTOCOL}" (case sensitive) will be replaced by the protocol used by the bridge
 #OPTIONAL (default empty)
 RemoteNickFormat="[{PROTOCOL}] <{NICK}> "
 
 #Enable to show users joins/parts from other bridges 
-#Only works hiding/show messages from irc and mattermost bridge for now
+#Currently works for messages from the following bridges: irc, mattermost, slack
 #OPTIONAL (default false)
 ShowJoinPart=false
 
+#Do not send joins/parts to other bridges
+#Currently works for messages from the following bridges: irc, mattermost, slack
+#OPTIONAL (default false)
+NoSendJoinPart=false
+
+#StripNick only allows alphanumerical nicks. See https://github.com/42wim/matterbridge/issues/285
+#It will strip other characters from the nick
+#OPTIONAL (default false)
+StripNick=false
+
+#Enable to show topic changes from other bridges 
+#Only works hiding/show topic changes from slack bridge for now
+#OPTIONAL (default false)
+ShowTopicChange=false
+
 ###################################################################
 #Gitter section
 #Best to make a dedicated gitter account for the bot.
@@ -310,6 +498,9 @@ ShowJoinPart=false
 #REQUIRED
 Token="Yourtokenhere"
 
+## RELOADABLE SETTINGS
+## Settings below can be reloaded by editing the file
+
 #Nicks you want to ignore. 
 #Messages from those users will not be sent to other bridges.
 #OPTIONAL
@@ -321,18 +512,50 @@ IgnoreNicks="ircspammer1 ircspammer2"
 #OPTIONAL (example below ignores messages starting with ~~ or messages containing badword
 IgnoreMessages="^~~ badword"
 
+#messages you want to replace.
+#it replaces outgoing messages from the bridge.
+#so you need to place it by the sending bridge definition.
+#regular expressions supported
+#some examples:
+#this replaces cat => dog and sleep => awake
+#replacemessages=[ ["cat","dog"], ["sleep","awake"] ]
+#this replaces every number with number.  123 => numbernumbernumber
+#replacemessages=[ ["[0-9]","number"] ]
+#optional (default empty)
+ReplaceMessages=[ ["cat","dog"] ]
+
+#nicks you want to replace.
+#see replacemessages for syntaxa
+#optional (default empty)
+ReplaceNicks=[ ["user--","user"] ]
+
+#extra label that can be used in the RemoteNickFormat
+#optional (default empty)
+Label=""
+
 #RemoteNickFormat defines how remote users appear on this bridge 
 #The string "{NICK}" (case sensitive) will be replaced by the actual nick / username.
 #The string "{BRIDGE}" (case sensitive) will be replaced by the sending bridge
+#The string "{LABEL}" (case sensitive) will be replaced by label= field of the sending bridge
 #The string "{PROTOCOL}" (case sensitive) will be replaced by the protocol used by the bridge
 #OPTIONAL (default empty)
 RemoteNickFormat="[{PROTOCOL}] <{NICK}> "
 
 #Enable to show users joins/parts from other bridges 
-#Only works hiding/show messages from irc and mattermost bridge for now
+#Currently works for messages from the following bridges: irc, mattermost, slack
 #OPTIONAL (default false)
 ShowJoinPart=false
 
+#StripNick only allows alphanumerical nicks. See https://github.com/42wim/matterbridge/issues/285
+#It will strip other characters from the nick
+#OPTIONAL (default false)
+StripNick=false
+
+#Enable to show topic changes from other bridges 
+#Only works hiding/show topic changes from slack bridge for now
+#OPTIONAL (default false)
+ShowTopicChange=false
+
 ###################################################################
 #slack section
 ###################################################################
@@ -361,7 +584,6 @@ WebhookURL="https://hooks.slack.com/services/yourhook"
 #AND DEDICATED BOT USER WHEN POSSIBLE!
 #Address to listen on for outgoing webhook requests from slack
 #See account settings - integrations - outgoing webhooks on slack
-#This setting will not be used when useAPI is eanbled
 #webhooks
 #OPTIONAL
 WebhookBindAddress="0.0.0.0:9999"
@@ -369,10 +591,14 @@ WebhookBindAddress="0.0.0.0:9999"
 #Icon that will be showed in slack
 #The string "{NICK}" (case sensitive) will be replaced by the actual nick / username.
 #The string "{BRIDGE}" (case sensitive) will be replaced by the sending bridge
+#The string "{LABEL}" (case sensitive) will be replaced by label= field of the sending bridge
 #The string "{PROTOCOL}" (case sensitive) will be replaced by the protocol used by the bridge
 #OPTIONAL
 IconURL="https://robohash.org/{NICK}.png?size=48x48"
 
+## RELOADABLE SETTINGS
+## Settings below can be reloaded by editing the file
+
 #how to format the list of IRC nicks when displayed in slack
 #Possible options are "table" and "plain"
 #OPTIONAL (default plain)
@@ -408,18 +634,55 @@ IgnoreNicks="ircspammer1 ircspammer2"
 #OPTIONAL (example below ignores messages starting with ~~ or messages containing badword
 IgnoreMessages="^~~ badword"
 
+#messages you want to replace.
+#it replaces outgoing messages from the bridge.
+#so you need to place it by the sending bridge definition.
+#regular expressions supported
+#some examples:
+#this replaces cat => dog and sleep => awake
+#replacemessages=[ ["cat","dog"], ["sleep","awake"] ]
+#this replaces every number with number.  123 => numbernumbernumber
+#replacemessages=[ ["[0-9]","number"] ]
+#optional (default empty)
+ReplaceMessages=[ ["cat","dog"] ]
+
+#nicks you want to replace.
+#see replacemessages for syntaxa
+#optional (default empty)
+ReplaceNicks=[ ["user--","user"] ]
+
+#extra label that can be used in the RemoteNickFormat
+#optional (default empty)
+Label=""
+
 #RemoteNickFormat defines how remote users appear on this bridge 
 #The string "{NICK}" (case sensitive) will be replaced by the actual nick / username.
 #The string "{BRIDGE}" (case sensitive) will be replaced by the sending bridge
+#The string "{LABEL}" (case sensitive) will be replaced by label= field of the sending bridge
 #The string "{PROTOCOL}" (case sensitive) will be replaced by the protocol used by the bridge
 #OPTIONAL (default empty)
 RemoteNickFormat="[{PROTOCOL}] <{NICK}> "
 
 #Enable to show users joins/parts from other bridges 
-#Only works hiding/show messages from irc and mattermost bridge for now
+#Currently works for messages from the following bridges: irc, mattermost, slack
 #OPTIONAL (default false)
 ShowJoinPart=false
 
+#Do not send joins/parts to other bridges
+#Currently works for messages from the following bridges: irc, mattermost, slack
+#OPTIONAL (default false)
+NoSendJoinPart=false
+
+#StripNick only allows alphanumerical nicks. See https://github.com/42wim/matterbridge/issues/285
+#It will strip other characters from the nick
+#OPTIONAL (default false)
+StripNick=false
+
+#Enable to show topic changes from other bridges 
+#Only works hiding/show topic changes from slack bridge for now
+#OPTIONAL (default false)
+ShowTopicChange=false
+
 ###################################################################
 #discord section
 ###################################################################
@@ -439,6 +702,9 @@ Token="Yourtokenhere"
 #REQUIRED
 Server="yourservername"
 
+## RELOADABLE SETTINGS
+## Settings below can be reloaded by editing the file
+
 #Shows title, description and URL of embedded messages (sent by other bots)
 #OPTIONAL (default false)
 ShowEmbeds=false
@@ -471,18 +737,50 @@ IgnoreNicks="ircspammer1 ircspammer2"
 #OPTIONAL (example below ignores messages starting with ~~ or messages containing badword
 IgnoreMessages="^~~ badword"
 
+#messages you want to replace.
+#it replaces outgoing messages from the bridge.
+#so you need to place it by the sending bridge definition.
+#regular expressions supported
+#some examples:
+#this replaces cat => dog and sleep => awake
+#replacemessages=[ ["cat","dog"], ["sleep","awake"] ]
+#this replaces every number with number.  123 => numbernumbernumber
+#replacemessages=[ ["[0-9]","number"] ]
+#optional (default empty)
+ReplaceMessages=[ ["cat","dog"] ]
+
+#nicks you want to replace.
+#see replacemessages for syntaxa
+#optional (default empty)
+ReplaceNicks=[ ["user--","user"] ]
+
+#extra label that can be used in the RemoteNickFormat
+#optional (default empty)
+Label=""
+
 #RemoteNickFormat defines how remote users appear on this bridge 
 #The string "{NICK}" (case sensitive) will be replaced by the actual nick / username.
 #The string "{BRIDGE}" (case sensitive) will be replaced by the sending bridge
+#The string "{LABEL}" (case sensitive) will be replaced by label= field of the sending bridge
 #The string "{PROTOCOL}" (case sensitive) will be replaced by the protocol used by the bridge
 #OPTIONAL (default empty)
 RemoteNickFormat="[{PROTOCOL}] <{NICK}> "
 
 #Enable to show users joins/parts from other bridges 
-#Only works hiding/show messages from irc and mattermost bridge for now
+#Currently works for messages from the following bridges: irc, mattermost, slack
 #OPTIONAL (default false)
 ShowJoinPart=false
 
+#StripNick only allows alphanumerical nicks. See https://github.com/42wim/matterbridge/issues/285
+#It will strip other characters from the nick
+#OPTIONAL (default false)
+StripNick=false
+
+#Enable to show topic changes from other bridges 
+#Only works hiding/show topic changes from slack bridge for now
+#OPTIONAL (default false)
+ShowTopicChange=false
+
 ###################################################################
 #telegram section
 ###################################################################
@@ -497,9 +795,14 @@ ShowJoinPart=false
 #REQUIRED
 Token="Yourtokenhere"
 
+## RELOADABLE SETTINGS
+## Settings below can be reloaded by editing the file
+
 #OPTIONAL (default empty)
-#Only supported format is "HTML", messages will be sent in html parsemode.
+#Supported formats are "HTML", "Markdown" and "HTMLNick"
 #See https://core.telegram.org/bots/api#html-style
+#See https://core.telegram.org/bots/api#markdown-style
+#HTMLNick only allows HTML for the nick, the message itself will be html-escaped
 MessageFormat=""
 
 #If enabled use the "First Name" as username. If this is empty use the Username
@@ -514,6 +817,14 @@ UseFirstName=false
 #OPTIONAL (default false)
 UseInsecureURL=false
 
+#Disable quoted/reply messages
+#OPTIONAL (default false)
+QuoteDisable=false
+
+#Format quoted/reply messages
+#OPTIONAL (default "{MESSAGE} (re @{QUOTENICK}: {QUOTEMESSAGE})")
+QuoteFormat="{MESSAGE} (re @{QUOTENICK}: {QUOTEMESSAGE})"
+
 #Disable sending of edits to other bridges
 #OPTIONAL (default false)
 EditDisable=false
@@ -533,18 +844,54 @@ IgnoreNicks="spammer1 spammer2"
 #OPTIONAL (example below ignores messages starting with ~~ or messages containing badword
 IgnoreMessages="^~~ badword"
 
+#messages you want to replace.
+#it replaces outgoing messages from the bridge.
+#so you need to place it by the sending bridge definition.
+#regular expressions supported
+#some examples:
+#this replaces cat => dog and sleep => awake
+#replacemessages=[ ["cat","dog"], ["sleep","awake"] ]
+#this replaces every number with number.  123 => numbernumbernumber
+#replacemessages=[ ["[0-9]","number"] ]
+#optional (default empty)
+ReplaceMessages=[ ["cat","dog"] ]
+
+#nicks you want to replace.
+#see replacemessages for syntaxa
+#optional (default empty)
+ReplaceNicks=[ ["user--","user"] ]
+
+#extra label that can be used in the RemoteNickFormat
+#optional (default empty)
+Label=""
+
 #RemoteNickFormat defines how remote users appear on this bridge 
 #The string "{NICK}" (case sensitive) will be replaced by the actual nick / username.
 #The string "{BRIDGE}" (case sensitive) will be replaced by the sending bridge
+#The string "{LABEL}" (case sensitive) will be replaced by label= field of the sending bridge
 #The string "{PROTOCOL}" (case sensitive) will be replaced by the protocol used by the bridge
+#
+#WARNING: if you have set MessageFormat="HTML" be sure that this format matches the guidelines
+#on https://core.telegram.org/bots/api#html-style otherwise the message will not go through to
+#telegram! eg <{NICK}> should be &lt;{NICK}&gt;
+#
 #OPTIONAL (default empty)
 RemoteNickFormat="[{PROTOCOL}] <{NICK}> "
 
 #Enable to show users joins/parts from other bridges 
-#Only works hiding/show messages from irc and mattermost bridge for now
+#Currently works for messages from the following bridges: irc, mattermost, slack
 #OPTIONAL (default false)
 ShowJoinPart=false
 
+#StripNick only allows alphanumerical nicks. See https://github.com/42wim/matterbridge/issues/285
+#It will strip other characters from the nick
+#OPTIONAL (default false)
+StripNick=false
+
+#Enable to show topic changes from other bridges 
+#Only works hiding/show topic changes from slack bridge for now
+#OPTIONAL (default false)
+ShowTopicChange=false
 
 ###################################################################
 #rocketchat section
@@ -579,6 +926,9 @@ NoTLS=false
 #OPTIONAL (default false)
 SkipTLSVerify=true
 
+## RELOADABLE SETTINGS
+## Settings below can be reloaded by editing the file
+
 #Whether to prefix messages from other bridges to rocketchat with the sender's nick. 
 #Useful if username overrides for incoming webhooks isn't enabled on the 
 #rocketchat server. If you set PrefixMessagesWithNick to true, each message 
@@ -597,18 +947,50 @@ IgnoreNicks="ircspammer1 ircspammer2"
 #OPTIONAL (example below ignores messages starting with ~~ or messages containing badword
 IgnoreMessages="^~~ badword"
 
+#messages you want to replace.
+#it replaces outgoing messages from the bridge.
+#so you need to place it by the sending bridge definition.
+#regular expressions supported
+#some examples:
+#this replaces cat => dog and sleep => awake
+#replacemessages=[ ["cat","dog"], ["sleep","awake"] ]
+#this replaces every number with number.  123 => numbernumbernumber
+#replacemessages=[ ["[0-9]","number"] ]
+#optional (default empty)
+ReplaceMessages=[ ["cat","dog"] ]
+
+#nicks you want to replace.
+#see replacemessages for syntaxa
+#optional (default empty)
+ReplaceNicks=[ ["user--","user"] ]
+
+#extra label that can be used in the RemoteNickFormat
+#optional (default empty)
+Label=""
+
 #RemoteNickFormat defines how remote users appear on this bridge 
 #The string "{NICK}" (case sensitive) will be replaced by the actual nick / username.
 #The string "{BRIDGE}" (case sensitive) will be replaced by the sending bridge
+#The string "{LABEL}" (case sensitive) will be replaced by label= field of the sending bridge
 #The string "{PROTOCOL}" (case sensitive) will be replaced by the protocol used by the bridge
 #OPTIONAL (default empty)
 RemoteNickFormat="[{PROTOCOL}] <{NICK}> "
 
 #Enable to show users joins/parts from other bridges 
-#Only works hiding/show messages from irc and mattermost bridge for now
+#Currently works for messages from the following bridges: irc, mattermost, slack
 #OPTIONAL (default false)
 ShowJoinPart=false
 
+#StripNick only allows alphanumerical nicks. See https://github.com/42wim/matterbridge/issues/285
+#It will strip other characters from the nick
+#OPTIONAL (default false)
+StripNick=false
+
+#Enable to show topic changes from other bridges 
+#Only works hiding/show topic changes from slack bridge for now
+#OPTIONAL (default false)
+ShowTopicChange=false
+
 ###################################################################
 #matrix section
 ###################################################################
@@ -634,6 +1016,9 @@ Password="yourpass"
 #OPTIONAL (default false)
 NoHomeServerSuffix=false
 
+## RELOADABLE SETTINGS
+## Settings below can be reloaded by editing the file
+
 #Whether to prefix messages from other bridges to matrix with the sender's nick. 
 #Useful if username overrides for incoming webhooks isn't enabled on the 
 #matrix server. If you set PrefixMessagesWithNick to true, each message 
@@ -652,18 +1037,50 @@ IgnoreNicks="spammer1 spammer2"
 #OPTIONAL (example below ignores messages starting with ~~ or messages containing badword
 IgnoreMessages="^~~ badword"
 
+#messages you want to replace.
+#it replaces outgoing messages from the bridge.
+#so you need to place it by the sending bridge definition.
+#regular expressions supported
+#some examples:
+#this replaces cat => dog and sleep => awake
+#replacemessages=[ ["cat","dog"], ["sleep","awake"] ]
+#this replaces every number with number.  123 => numbernumbernumber
+#replacemessages=[ ["[0-9]","number"] ]
+#optional (default empty)
+ReplaceMessages=[ ["cat","dog"] ]
+
+#nicks you want to replace.
+#see replacemessages for syntaxa
+#optional (default empty)
+ReplaceNicks=[ ["user--","user"] ]
+
+#extra label that can be used in the RemoteNickFormat
+#optional (default empty)
+Label=""
+
 #RemoteNickFormat defines how remote users appear on this bridge 
 #The string "{NICK}" (case sensitive) will be replaced by the actual nick / username.
 #The string "{BRIDGE}" (case sensitive) will be replaced by the sending bridge
+#The string "{LABEL}" (case sensitive) will be replaced by label= field of the sending bridge
 #The string "{PROTOCOL}" (case sensitive) will be replaced by the protocol used by the bridge
 #OPTIONAL (default empty)
 RemoteNickFormat="[{PROTOCOL}] <{NICK}> "
 
 #Enable to show users joins/parts from other bridges 
-#Only works hiding/show messages from irc and mattermost bridge for now
+#Currently works for messages from the following bridges: irc, mattermost, slack
 #OPTIONAL (default false)
 ShowJoinPart=false
 
+#StripNick only allows alphanumerical nicks. See https://github.com/42wim/matterbridge/issues/285
+#It will strip other characters from the nick
+#OPTIONAL (default false)
+StripNick=false
+
+#Enable to show topic changes from other bridges 
+#Only works hiding/show topic changes from slack bridge for now
+#OPTIONAL (default false)
+ShowTopicChange=false
+
 ###################################################################
 #steam section
 ###################################################################
@@ -683,6 +1100,9 @@ Password="yourpass"
 #OPTIONAL 
 Authcode="ABCE12"
 
+## RELOADABLE SETTINGS
+## Settings below can be reloaded by editing the file
+
 #Whether to prefix messages from other bridges to matrix with the sender's nick. 
 #Useful if username overrides for incoming webhooks isn't enabled on the 
 #matrix server. If you set PrefixMessagesWithNick to true, each message 
@@ -701,18 +1121,133 @@ IgnoreNicks="spammer1 spammer2"
 #OPTIONAL (example below ignores messages starting with ~~ or messages containing badword
 IgnoreMessages="^~~ badword"
 
+#messages you want to replace.
+#it replaces outgoing messages from the bridge.
+#so you need to place it by the sending bridge definition.
+#regular expressions supported
+#some examples:
+#this replaces cat => dog and sleep => awake
+#replacemessages=[ ["cat","dog"], ["sleep","awake"] ]
+#this replaces every number with number.  123 => numbernumbernumber
+#replacemessages=[ ["[0-9]","number"] ]
+#optional (default empty)
+ReplaceMessages=[ ["cat","dog"] ]
+
+#nicks you want to replace.
+#see replacemessages for syntaxa
+#optional (default empty)
+ReplaceNicks=[ ["user--","user"] ]
+
+#extra label that can be used in the RemoteNickFormat
+#optional (default empty)
+Label=""
+
 #RemoteNickFormat defines how remote users appear on this bridge 
 #The string "{NICK}" (case sensitive) will be replaced by the actual nick / username.
 #The string "{BRIDGE}" (case sensitive) will be replaced by the sending bridge
+#The string "{LABEL}" (case sensitive) will be replaced by label= field of the sending bridge
 #The string "{PROTOCOL}" (case sensitive) will be replaced by the protocol used by the bridge
 #OPTIONAL (default empty)
 RemoteNickFormat="[{PROTOCOL}] <{NICK}> "
 
 #Enable to show users joins/parts from other bridges 
-#Only works hiding/show messages from irc and mattermost bridge for now
+#Currently works for messages from the following bridges: irc, mattermost, slack
 #OPTIONAL (default false)
 ShowJoinPart=false
 
+#StripNick only allows alphanumerical nicks. See https://github.com/42wim/matterbridge/issues/285
+#It will strip other characters from the nick
+#OPTIONAL (default false)
+StripNick=false
+
+#Enable to show topic changes from other bridges 
+#Only works hiding/show topic changes from slack bridge for now
+#OPTIONAL (default false)
+ShowTopicChange=false
+
+###################################################################
+#zulip section
+###################################################################
+[zulip]
+#You can configure multiple servers "[zulip.name]" or "[zulip.name2]"
+#In this example we use [zulip.streamchat]
+#REQUIRED
+
+[zulip.streamchat]
+#Token to connect with zulip API (called bot API key in Settings - Your bots)
+#REQUIRED
+Token="Yourtokenhere"
+
+#Username of the bot, normally called yourbot-bot@yourserver.zulipchat.com 
+#See username in Settings - Your bots 
+#REQUIRED
+Login="yourbot-bot@yourserver.zulipchat.com"
+
+#Servername of your zulip instance
+#REQUIRED 
+Server="https://yourserver.zulipchat.com"
+
+#Topic of the messages matterbridge will use
+#OPTIONAL (default "matterbridge")
+Topic="matterbridge"
+
+## RELOADABLE SETTINGS
+## Settings below can be reloaded by editing the file
+
+#Nicks you want to ignore. 
+#Messages from those users will not be sent to other bridges.
+#OPTIONAL
+IgnoreNicks="spammer1 spammer2"
+
+#Messages you want to ignore. 
+#Messages matching these regexp will be ignored and not sent to other bridges
+#See https://regex-golang.appspot.com/assets/html/index.html for more regex info
+#OPTIONAL (example below ignores messages starting with ~~ or messages containing badword
+IgnoreMessages="^~~ badword"
+
+#messages you want to replace.
+#it replaces outgoing messages from the bridge.
+#so you need to place it by the sending bridge definition.
+#regular expressions supported
+#some examples:
+#this replaces cat => dog and sleep => awake
+#replacemessages=[ ["cat","dog"], ["sleep","awake"] ]
+#this replaces every number with number.  123 => numbernumbernumber
+#replacemessages=[ ["[0-9]","number"] ]
+#optional (default empty)
+ReplaceMessages=[ ["cat","dog"] ]
+
+#nicks you want to replace.
+#see replacemessages for syntaxa
+#optional (default empty)
+ReplaceNicks=[ ["user--","user"] ]
+
+#extra label that can be used in the RemoteNickFormat
+#optional (default empty)
+Label=""
+
+#RemoteNickFormat defines how remote users appear on this bridge 
+#The string "{NICK}" (case sensitive) will be replaced by the actual nick / username.
+#The string "{BRIDGE}" (case sensitive) will be replaced by the sending bridge
+#The string "{LABEL}" (case sensitive) will be replaced by label= field of the sending bridge
+#The string "{PROTOCOL}" (case sensitive) will be replaced by the protocol used by the bridge
+#OPTIONAL (default empty)
+RemoteNickFormat="[{PROTOCOL}] <{NICK}> "
+
+#Enable to show users joins/parts from other bridges 
+#Currently works for messages from the following bridges: irc, mattermost, slack
+#OPTIONAL (default false)
+ShowJoinPart=false
+
+#StripNick only allows alphanumerical nicks. See https://github.com/42wim/matterbridge/issues/285
+#It will strip other characters from the nick
+#OPTIONAL (default false)
+StripNick=false
+
+#Enable to show topic changes from other bridges 
+#Only works hiding/show topic changes from slack bridge for now
+#OPTIONAL (default false)
+ShowTopicChange=false
 
 ###################################################################
 #API
@@ -735,9 +1270,14 @@ Buffer=1000
 #OPTIONAL (no authorization if token is empty)
 Token="mytoken"
 
+#extra label that can be used in the RemoteNickFormat
+#optional (default empty)
+Label=""
+
 #RemoteNickFormat defines how remote users appear on this bridge 
 #The string "{NICK}" (case sensitive) will be replaced by the actual nick / username.
 #The string "{BRIDGE}" (case sensitive) will be replaced by the sending bridge
+#The string "{LABEL}" (case sensitive) will be replaced by label= field of the sending bridge
 #The string "{PROTOCOL}" (case sensitive) will be replaced by the protocol used by the bridge
 #OPTIONAL (default empty)
 RemoteNickFormat="{NICK}"
@@ -747,15 +1287,60 @@ RemoteNickFormat="{NICK}"
 ###################################################################
 #General configuration
 ###################################################################
-#Settings here override specific settings for each protocol
+# Settings here are defaults that each protocol can override
 [general]
+
+## RELOADABLE SETTINGS
+## Settings below can be reloaded by editing the file
+
 #RemoteNickFormat defines how remote users appear on this bridge 
 #The string "{NICK}" (case sensitive) will be replaced by the actual nick / username.
 #The string "{BRIDGE}" (case sensitive) will be replaced by the sending bridge
+#The string "{LABEL}" (case sensitive) will be replaced by label= field of the sending bridge
 #The string "{PROTOCOL}" (case sensitive) will be replaced by the protocol used by the bridge
 #OPTIONAL (default empty)
 RemoteNickFormat="[{PROTOCOL}] <{NICK}> "
 
+#StripNick only allows alphanumerical nicks. See https://github.com/42wim/matterbridge/issues/285
+#It will strip other characters from the nick
+#OPTIONAL (default false)
+StripNick=false
+
+
+#MediaServerUpload (or MediaDownloadPath) and MediaServerDownload are used for uploading
+#images/files/video to a remote "mediaserver" (a webserver like caddy for example).
+#When configured images/files uploaded on bridges like mattermost, slack, telegram will be
+#downloaded and uploaded again to MediaServerUpload URL
+#MediaDownloadPath is the filesystem path where the media file will be placed, instead of uploaded,
+#for if Matterbridge has write access to the directory your webserver is serving.
+#It is an alternative to MediaServerUpload.
+#The MediaServerDownload will be used so that bridges without native uploading support:
+#gitter, irc and xmpp will be shown links to the files on MediaServerDownload
+#
+#More information https://github.com/42wim/matterbridge/wiki/Mediaserver-setup-%5Badvanced%5D
+#OPTIONAL (default empty)
+MediaServerUpload="https://user:pass@yourserver.com/upload"
+#OPTIONAL (default empty)
+MediaDownloadPath="/srv/http/yourserver.com/public/download"
+#OPTIONAL (default empty)
+MediaServerDownload="https://youserver.com/download"
+
+#MediaDownloadSize is the maximum size of attachments, videos, images
+#matterbridge will download and upload this file to bridges that also support uploading files.
+#eg downloading from slack to upload it to mattermost
+#
+#It will only download from bridges that don't have public links available, which are for the moment
+#slack, telegram, matrix and mattermost
+#
+#OPTIONAL (default 1000000 (1 megabyte))
+MediaDownloadSize=1000000
+
+#MediaDownloadBlacklist allows you to blacklist specific files from being downloaded.
+#Filenames matching these regexp will not be download/uploaded to the mediaserver
+#You can use regex for this, see https://regex-golang.appspot.com/assets/html/index.html for more regex info
+#OPTIONAL (default empty)
+MediaDownloadBlacklist=[".html$",".htm$"]
+
 ###################################################################
 #Gateway configuration
 ###################################################################
@@ -791,7 +1376,8 @@ enable=true
     #mattermost - channel (the channel name as seen in the URL, not the displayname)
     #gitter     - username/room 
     #xmpp       - channel
-    #slack      - channel (the channel name as seen in the URL, not the displayname)
+    #slack      - channel (without the #)
+    #           - ID:C123456 (where C123456 is the channel ID) does not work with webhook
     #discord    - channel (without the #)
     #           - ID:123456789 (where 123456789 is the channel ID) 
     #               (https://github.com/42wim/matterbridge/issues/57)
@@ -803,13 +1389,14 @@ enable=true
     #           - encrypted rooms are not supported in matrix
     #steam      - chatid (a large number). 
     #             The number in the URL when you click "enter chat room" in the browser
+    #zulip      - stream (without the #)
     #                  
     #REQUIRED
     channel="#testing"
 
-        #OPTIONAL - only used for IRC protocol at the moment
+        #OPTIONAL - only used for IRC and XMPP protocols at the moment
         [gateway.in.options]
-        #OPTIONAL - your irc channel key
+        #OPTIONAL - your irc / xmpp channel key
         key="yourkey"
 
 
@@ -818,9 +1405,9 @@ enable=true
     account="irc.freenode"
     channel="#testing"
 
-        #OPTIONAL - only used for IRC protocol at the moment
+        #OPTIONAL - only used for IRC and XMPP protocols at the moment
         [gateway.out.options]
-        #OPTIONAL - your irc channel key
+        #OPTIONAL - your irc / xmpp channel key
         key="yourkey"
 
     #[[gateway.inout]] can be used when then channel will be used to receive from 
@@ -829,9 +1416,9 @@ enable=true
     account="mattermost.work"
     channel="off-topic"
 
-        #OPTIONAL - only used for IRC protocol at the moment
+        #OPTIONAL - only used for IRC and XMPP protocols at the moment
         [gateway.inout.options]
-        #OPTIONAL - your irc channel key
+        #OPTIONAL - your irc / xmpp channel key
         key="yourkey"
 
     [[gateway.inout]]

matterbridge.toml.simple

@@ -6,7 +6,6 @@
 
 [mattermost]
     [mattermost.work]
-    useAPI=true
     #do not prefix it wit http:// or https://
     Server="yourmattermostserver.domain" 
     Team="yourteam"

matterclient/matterclient.go

@@ -9,12 +9,12 @@ import (
 	"net/http"
 	"net/http/cookiejar"
 	"net/url"
-	"strconv"
 	"strings"
 	"sync"
 	"time"
 
-	log "github.com/Sirupsen/logrus"
+	log "github.com/sirupsen/logrus"
+	prefixed "github.com/x-cray/logrus-prefixed-formatter"
 
 	"github.com/gorilla/websocket"
 	"github.com/hashicorp/golang-lru"
@@ -45,8 +45,8 @@ type Message struct {
 type Team struct {
 	Team         *model.Team
 	Id           string
-	Channels     *model.ChannelList
-	MoreChannels *model.ChannelList
+	Channels     []*model.Channel
+	MoreChannels []*model.Channel
 	Users        map[string]*model.User
 }
 
@@ -55,7 +55,7 @@ type MMClient struct {
 	*Credentials
 	Team          *Team
 	OtherTeams    []*Team
-	Client        *model.Client
+	Client        *model.Client4
 	User          *model.User
 	Users         map[string]*model.User
 	MessageChan   chan *Message
@@ -74,12 +74,16 @@ type MMClient struct {
 func New(login, pass, team, server string) *MMClient {
 	cred := &Credentials{Login: login, Pass: pass, Team: team, Server: server}
 	mmclient := &MMClient{Credentials: cred, MessageChan: make(chan *Message, 100), Users: make(map[string]*model.User)}
-	mmclient.log = log.WithFields(log.Fields{"module": "matterclient"})
-	log.SetFormatter(&log.TextFormatter{FullTimestamp: true})
+	log.SetFormatter(&prefixed.TextFormatter{PrefixPadding: 13, DisableColors: true})
+	mmclient.log = log.WithFields(log.Fields{"prefix": "matterclient"})
 	mmclient.lruCache, _ = lru.New(500)
 	return mmclient
 }
 
+func (m *MMClient) SetDebugLog() {
+	log.SetFormatter(&prefixed.TextFormatter{PrefixPadding: 13, DisableColors: true, FullTimestamp: false, ForceFormatting: true})
+}
+
 func (m *MMClient) SetLogLevel(level string) {
 	l, err := log.ParseLevel(level)
 	if err != nil {
@@ -109,21 +113,21 @@ func (m *MMClient) Login() error {
 		uriScheme = "http://"
 	}
 	// login to mattermost
-	m.Client = model.NewClient(uriScheme + m.Credentials.Server)
+	m.Client = model.NewAPIv4Client(uriScheme + m.Credentials.Server)
 	m.Client.HttpClient.Transport = &http.Transport{TLSClientConfig: &tls.Config{InsecureSkipVerify: m.SkipTLSVerify}, Proxy: http.ProxyFromEnvironment}
 	m.Client.HttpClient.Timeout = time.Second * 10
 
 	for {
 		d := b.Duration()
 		// bogus call to get the serverversion
-		_, err := m.Client.GetClientProperties()
-		if err != nil {
-			return fmt.Errorf("%#v", err.Error())
+		_, resp := m.Client.Logout()
+		if resp.Error != nil {
+			return fmt.Errorf("%#v", resp.Error.Error())
 		}
-		if firstConnection && !supportedVersion(m.Client.ServerVersion) {
-			return fmt.Errorf("unsupported mattermost version: %s", m.Client.ServerVersion)
+		if firstConnection && !supportedVersion(resp.ServerVersion) {
+			return fmt.Errorf("unsupported mattermost version: %s", resp.ServerVersion)
 		}
-		m.ServerVersion = m.Client.ServerVersion
+		m.ServerVersion = resp.ServerVersion
 		if m.ServerVersion == "" {
 			m.log.Debugf("Server not up yet, reconnecting in %s", d)
 			time.Sleep(d)
@@ -134,30 +138,33 @@ func (m *MMClient) Login() error {
 	}
 	b.Reset()
 
-	var myinfo *model.Result
+	var resp *model.Response
+	//var myinfo *model.Result
 	var appErr *model.AppError
 	var logmsg = "trying login"
 	for {
 		m.log.Debugf("%s %s %s %s", logmsg, m.Credentials.Team, m.Credentials.Login, m.Credentials.Server)
 		if strings.Contains(m.Credentials.Pass, model.SESSION_COOKIE_TOKEN) {
-			m.log.Debugf(logmsg+" with %s", model.SESSION_COOKIE_TOKEN)
+			m.log.Debugf(logmsg + " with token")
 			token := strings.Split(m.Credentials.Pass, model.SESSION_COOKIE_TOKEN+"=")
 			if len(token) != 2 {
 				return errors.New("incorrect MMAUTHTOKEN. valid input is MMAUTHTOKEN=yourtoken")
 			}
 			m.Client.HttpClient.Jar = m.createCookieJar(token[1])
-			m.Client.MockSession(token[1])
-			myinfo, appErr = m.Client.GetMe("")
-			if appErr != nil {
-				return errors.New(appErr.DetailedError)
+			m.Client.AuthToken = token[1]
+			m.Client.AuthType = model.HEADER_BEARER
+			m.User, resp = m.Client.GetMe("")
+			if resp.Error != nil {
+				return resp.Error
 			}
-			if myinfo.Data.(*model.User) == nil {
+			if m.User == nil {
 				m.log.Errorf("LOGIN TOKEN: %s is invalid", m.Credentials.Pass)
 				return errors.New("invalid " + model.SESSION_COOKIE_TOKEN)
 			}
 		} else {
-			_, appErr = m.Client.Login(m.Credentials.Login, m.Credentials.Pass)
+			m.User, resp = m.Client.Login(m.Credentials.Login, m.Credentials.Pass)
 		}
+		appErr = resp.Error
 		if appErr != nil {
 			d := b.Duration()
 			m.log.Debug(appErr.DetailedError)
@@ -183,10 +190,12 @@ func (m *MMClient) Login() error {
 	}
 
 	if m.Team == nil {
-		return errors.New("team not found")
+		validTeamNames := make([]string, len(m.OtherTeams))
+		for i, t := range m.OtherTeams {
+			validTeamNames[i] = t.Team.Name
+		}
+		return fmt.Errorf("Team '%s' not found in %v", m.Credentials.Team, validTeamNames)
 	}
-	// set our team id as default route
-	m.Client.SetTeamId(m.Team.Id)
 
 	m.wsConnect()
 
@@ -207,7 +216,7 @@ func (m *MMClient) wsConnect() {
 	}
 
 	// setup websocket connection
-	wsurl := wsScheme + m.Credentials.Server + model.API_URL_SUFFIX_V3 + "/users/websocket"
+	wsurl := wsScheme + m.Credentials.Server + model.API_URL_SUFFIX_V4 + "/websocket"
 	header := http.Header{}
 	header.Set(model.HEADER_AUTH, "BEARER "+m.Client.AuthToken)
 
@@ -241,9 +250,9 @@ func (m *MMClient) Logout() error {
 		m.log.Debug("Not invalidating session in logout, credential is a token")
 		return nil
 	}
-	_, err := m.Client.Logout()
-	if err != nil {
-		return err
+	_, resp := m.Client.Logout()
+	if resp.Error != nil {
+		return resp.Error
 	}
 	return nil
 }
@@ -277,6 +286,13 @@ func (m *MMClient) WsReceiver() {
 			// check if we didn't empty the message
 			if msg.Text != "" {
 				m.MessageChan <- msg
+				continue
+			}
+			// if we have file attached but the message is empty, also send it
+			if msg.Post != nil {
+				if msg.Text != "" || len(msg.Post.FileIds) > 0 || msg.Post.Type == "slack_attachment" {
+					m.MessageChan <- msg
+				}
 			}
 			continue
 		}
@@ -292,8 +308,13 @@ func (m *MMClient) WsReceiver() {
 
 func (m *MMClient) parseMessage(rmsg *Message) {
 	switch rmsg.Raw.Event {
-	case model.WEBSOCKET_EVENT_POSTED, model.WEBSOCKET_EVENT_POST_EDITED:
+	case model.WEBSOCKET_EVENT_POSTED, model.WEBSOCKET_EVENT_POST_EDITED, model.WEBSOCKET_EVENT_POST_DELETED:
 		m.parseActionPost(rmsg)
+	case "user_updated":
+		user := rmsg.Raw.Data["user"].(map[string]interface{})
+		if _, ok := user["id"].(string); ok {
+			m.UpdateUser(user["id"].(string))
+		}
 		/*
 			case model.ACTION_USER_REMOVED:
 				m.handleWsActionUserRemoved(&rmsg)
@@ -349,33 +370,34 @@ func (m *MMClient) parseActionPost(rmsg *Message) {
 }
 
 func (m *MMClient) UpdateUsers() error {
-	mmusers, err := m.Client.GetProfiles(0, 50000, "")
-	if err != nil {
-		return errors.New(err.DetailedError)
+	mmusers, resp := m.Client.GetUsers(0, 50000, "")
+	if resp.Error != nil {
+		return errors.New(resp.Error.DetailedError)
 	}
 	m.Lock()
-	m.Users = mmusers.Data.(map[string]*model.User)
+	for _, user := range mmusers {
+		m.Users[user.Id] = user
+	}
 	m.Unlock()
 	return nil
 }
 
 func (m *MMClient) UpdateChannels() error {
-	mmchannels, err := m.Client.GetChannels("")
-	if err != nil {
-		return errors.New(err.DetailedError)
-	}
-	var mmchannels2 *model.Result
-	if m.mmVersion() >= 3.08 {
-		mmchannels2, err = m.Client.GetMoreChannelsPage(0, 5000)
-	} else {
-		mmchannels2, err = m.Client.GetMoreChannels("")
-	}
-	if err != nil {
-		return errors.New(err.DetailedError)
+	mmchannels, resp := m.Client.GetChannelsForTeamForUser(m.Team.Id, m.User.Id, "")
+	if resp.Error != nil {
+		return errors.New(resp.Error.DetailedError)
 	}
 	m.Lock()
-	m.Team.Channels = mmchannels.Data.(*model.ChannelList)
-	m.Team.MoreChannels = mmchannels2.Data.(*model.ChannelList)
+	m.Team.Channels = mmchannels
+	m.Unlock()
+
+	mmchannels, resp = m.Client.GetPublicChannelsForTeam(m.Team.Id, 0, 5000, "")
+	if resp.Error != nil {
+		return errors.New(resp.Error.DetailedError)
+	}
+
+	m.Lock()
+	m.Team.MoreChannels = mmchannels
 	m.Unlock()
 	return nil
 }
@@ -388,14 +410,14 @@ func (m *MMClient) GetChannelName(channelId string) string {
 			continue
 		}
 		if t.Channels != nil {
-			for _, channel := range *t.Channels {
+			for _, channel := range t.Channels {
 				if channel.Id == channelId {
 					return channel.Name
 				}
 			}
 		}
 		if t.MoreChannels != nil {
-			for _, channel := range *t.MoreChannels {
+			for _, channel := range t.MoreChannels {
 				if channel.Id == channelId {
 					return channel.Name
 				}
@@ -413,7 +435,7 @@ func (m *MMClient) GetChannelId(name string, teamId string) string {
 	}
 	for _, t := range m.OtherTeams {
 		if t.Id == teamId {
-			for _, channel := range append(*t.Channels, *t.MoreChannels...) {
+			for _, channel := range append(t.Channels, t.MoreChannels...) {
 				if channel.Name == name {
 					return channel.Id
 				}
@@ -427,7 +449,7 @@ func (m *MMClient) GetChannelTeamId(id string) string {
 	m.RLock()
 	defer m.RUnlock()
 	for _, t := range append(m.OtherTeams, m.Team) {
-		for _, channel := range append(*t.Channels, *t.MoreChannels...) {
+		for _, channel := range append(t.Channels, t.MoreChannels...) {
 			if channel.Id == id {
 				return channel.TeamId
 			}
@@ -440,7 +462,7 @@ func (m *MMClient) GetChannelHeader(channelId string) string {
 	m.RLock()
 	defer m.RUnlock()
 	for _, t := range m.OtherTeams {
-		for _, channel := range append(*t.Channels, *t.MoreChannels...) {
+		for _, channel := range append(t.Channels, t.MoreChannels...) {
 			if channel.Id == channelId {
 				return channel.Header
 			}
@@ -450,55 +472,85 @@ func (m *MMClient) GetChannelHeader(channelId string) string {
 	return ""
 }
 
-func (m *MMClient) PostMessage(channelId string, text string) {
+func (m *MMClient) PostMessage(channelId string, text string) (string, error) {
 	post := &model.Post{ChannelId: channelId, Message: text}
-	m.Client.CreatePost(post)
+	res, resp := m.Client.CreatePost(post)
+	if resp.Error != nil {
+		return "", resp.Error
+	}
+	return res.Id, nil
+}
+
+func (m *MMClient) PostMessageWithFiles(channelId string, text string, fileIds []string) (string, error) {
+	post := &model.Post{ChannelId: channelId, Message: text, FileIds: fileIds}
+	res, resp := m.Client.CreatePost(post)
+	if resp.Error != nil {
+		return "", resp.Error
+	}
+	return res.Id, nil
+}
+
+func (m *MMClient) EditMessage(postId string, text string) (string, error) {
+	post := &model.Post{Message: text}
+	res, resp := m.Client.UpdatePost(postId, post)
+	if resp.Error != nil {
+		return "", resp.Error
+	}
+	return res.Id, nil
+}
+
+func (m *MMClient) DeleteMessage(postId string) error {
+	_, resp := m.Client.DeletePost(postId)
+	if resp.Error != nil {
+		return resp.Error
+	}
+	return nil
 }
 
 func (m *MMClient) JoinChannel(channelId string) error {
 	m.RLock()
 	defer m.RUnlock()
-	for _, c := range *m.Team.Channels {
+	for _, c := range m.Team.Channels {
 		if c.Id == channelId {
 			m.log.Debug("Not joining ", channelId, " already joined.")
 			return nil
 		}
 	}
 	m.log.Debug("Joining ", channelId)
-	_, err := m.Client.JoinChannel(channelId)
-	if err != nil {
-		return errors.New("failed to join")
+	_, resp := m.Client.AddChannelMember(channelId, m.User.Id)
+	if resp.Error != nil {
+		return resp.Error
 	}
 	return nil
 }
 
 func (m *MMClient) GetPostsSince(channelId string, time int64) *model.PostList {
-	res, err := m.Client.GetPostsSince(channelId, time)
-	if err != nil {
+	res, resp := m.Client.GetPostsSince(channelId, time)
+	if resp.Error != nil {
 		return nil
 	}
-	return res.Data.(*model.PostList)
+	return res
 }
 
 func (m *MMClient) SearchPosts(query string) *model.PostList {
-	res, err := m.Client.SearchPosts(query, false)
-	if err != nil {
+	res, resp := m.Client.SearchPosts(m.Team.Id, query, false)
+	if resp.Error != nil {
 		return nil
 	}
-	return res.Data.(*model.PostList)
+	return res
 }
 
 func (m *MMClient) GetPosts(channelId string, limit int) *model.PostList {
-	res, err := m.Client.GetPosts(channelId, 0, limit, "")
-	if err != nil {
+	res, resp := m.Client.GetPostsForChannel(channelId, 0, limit, "")
+	if resp.Error != nil {
 		return nil
 	}
-	return res.Data.(*model.PostList)
+	return res
 }
 
 func (m *MMClient) GetPublicLink(filename string) string {
-	res, err := m.Client.GetPublicLink(filename)
-	if err != nil {
+	res, resp := m.Client.GetFileLink(filename)
+	if resp.Error != nil {
 		return ""
 	}
 	return res
@@ -507,8 +559,8 @@ func (m *MMClient) GetPublicLink(filename string) string {
 func (m *MMClient) GetPublicLinks(filenames []string) []string {
 	var output []string
 	for _, f := range filenames {
-		res, err := m.Client.GetPublicLink(f)
-		if err != nil {
+		res, resp := m.Client.GetFileLink(f)
+		if resp.Error != nil {
 			continue
 		}
 		output = append(output, res)
@@ -524,10 +576,10 @@ func (m *MMClient) GetFileLinks(filenames []string) []string {
 
 	var output []string
 	for _, f := range filenames {
-		res, err := m.Client.GetPublicLink(f)
-		if err != nil {
+		res, resp := m.Client.GetFileLink(f)
+		if resp.Error != nil {
 			// public links is probably disabled, create the link ourselves
-			output = append(output, uriScheme+m.Credentials.Server+model.API_URL_SUFFIX_V3+"/files/"+f+"/get")
+			output = append(output, uriScheme+m.Credentials.Server+model.API_URL_SUFFIX_V4+"/files/"+f)
 			continue
 		}
 		output = append(output, res)
@@ -536,42 +588,43 @@ func (m *MMClient) GetFileLinks(filenames []string) []string {
 }
 
 func (m *MMClient) UpdateChannelHeader(channelId string, header string) {
-	data := make(map[string]string)
-	data["channel_id"] = channelId
-	data["channel_header"] = header
+	channel := &model.Channel{Id: channelId, Header: header}
 	m.log.Debugf("updating channelheader %#v, %#v", channelId, header)
-	_, err := m.Client.UpdateChannelHeader(data)
-	if err != nil {
-		log.Error(err)
+	_, resp := m.Client.UpdateChannel(channel)
+	if resp.Error != nil {
+		log.Error(resp.Error)
 	}
 }
 
 func (m *MMClient) UpdateLastViewed(channelId string) {
 	m.log.Debugf("posting lastview %#v", channelId)
-	if m.mmVersion() >= 3.08 {
-		view := model.ChannelView{ChannelId: channelId}
-		res, _ := m.Client.ViewChannel(view)
-		if !res {
-			m.log.Errorf("ChannelView update for %s failed", channelId)
-		}
-		return
-	}
-	_, err := m.Client.UpdateLastViewedAt(channelId, true)
-	if err != nil {
-		m.log.Error(err)
+	view := &model.ChannelView{ChannelId: channelId}
+	_, resp := m.Client.ViewChannel(m.User.Id, view)
+	if resp.Error != nil {
+		m.log.Errorf("ChannelView update for %s failed: %s", channelId, resp.Error)
 	}
 }
 
+func (m *MMClient) UpdateUserNick(nick string) error {
+	user := m.User
+	user.Nickname = nick
+	_, resp := m.Client.UpdateUser(user)
+	if resp.Error != nil {
+		return resp.Error
+	}
+	return nil
+}
+
 func (m *MMClient) UsernamesInChannel(channelId string) []string {
-	res, err := m.Client.GetProfilesInChannel(channelId, 0, 50000, "")
-	if err != nil {
-		m.log.Errorf("UsernamesInChannel(%s) failed: %s", channelId, err)
+	res, resp := m.Client.GetChannelMembers(channelId, 0, 50000, "")
+	if resp.Error != nil {
+		m.log.Errorf("UsernamesInChannel(%s) failed: %s", channelId, resp.Error)
 		return []string{}
 	}
-	members := res.Data.(map[string]*model.User)
+	allusers := m.GetUsers()
 	result := []string{}
-	for _, member := range members {
-		result = append(result, member.Nickname)
+	for _, member := range *res {
+		result = append(result, allusers[member.UserId].Nickname)
 	}
 	return result
 }
@@ -595,22 +648,15 @@ func (m *MMClient) createCookieJar(token string) *cookiejar.Jar {
 func (m *MMClient) SendDirectMessage(toUserId string, msg string) {
 	m.log.Debugf("SendDirectMessage to %s, msg %s", toUserId, msg)
 	// create DM channel (only happens on first message)
-	_, err := m.Client.CreateDirectChannel(toUserId)
-	if err != nil {
-		m.log.Debugf("SendDirectMessage to %#v failed: %s", toUserId, err)
+	_, resp := m.Client.CreateDirectChannel(m.User.Id, toUserId)
+	if resp.Error != nil {
+		m.log.Debugf("SendDirectMessage to %#v failed: %s", toUserId, resp.Error)
 		return
 	}
 	channelName := model.GetDMNameFromIds(toUserId, m.User.Id)
 
 	// update our channels
-	mmchannels, err := m.Client.GetChannels("")
-	if err != nil {
-		m.log.Debug("SendDirectMessage: Couldn't update channels")
-		return
-	}
-	m.Lock()
-	m.Team.Channels = mmchannels.Data.(*model.ChannelList)
-	m.Unlock()
+	m.UpdateChannels()
 
 	// build & send the message
 	msg = strings.Replace(msg, "\r", "", -1)
@@ -636,10 +682,10 @@ func (m *MMClient) GetChannels() []*model.Channel {
 	defer m.RUnlock()
 	var channels []*model.Channel
 	// our primary team channels first
-	channels = append(channels, *m.Team.Channels...)
+	channels = append(channels, m.Team.Channels...)
 	for _, t := range m.OtherTeams {
 		if t.Id != m.Team.Id {
-			channels = append(channels, *t.Channels...)
+			channels = append(channels, t.Channels...)
 		}
 	}
 	return channels
@@ -651,7 +697,7 @@ func (m *MMClient) GetMoreChannels() []*model.Channel {
 	defer m.RUnlock()
 	var channels []*model.Channel
 	for _, t := range m.OtherTeams {
-		channels = append(channels, *t.MoreChannels...)
+		channels = append(channels, t.MoreChannels...)
 	}
 	return channels
 }
@@ -662,9 +708,9 @@ func (m *MMClient) GetTeamFromChannel(channelId string) string {
 	defer m.RUnlock()
 	var channels []*model.Channel
 	for _, t := range m.OtherTeams {
-		channels = append(channels, *t.Channels...)
+		channels = append(channels, t.Channels...)
 		if t.MoreChannels != nil {
-			channels = append(channels, *t.MoreChannels...)
+			channels = append(channels, t.MoreChannels...)
 		}
 		for _, c := range channels {
 			if c.Id == channelId {
@@ -678,12 +724,11 @@ func (m *MMClient) GetTeamFromChannel(channelId string) string {
 func (m *MMClient) GetLastViewedAt(channelId string) int64 {
 	m.RLock()
 	defer m.RUnlock()
-	res, err := m.Client.GetChannel(channelId, "")
-	if err != nil {
+	res, resp := m.Client.GetChannelMember(channelId, m.User.Id, "")
+	if resp.Error != nil {
 		return model.GetMillis()
 	}
-	data := res.Data.(*model.ChannelData)
-	return data.Member.LastViewedAt
+	return res.LastViewedAt
 }
 
 func (m *MMClient) GetUsers() map[string]*model.User {
@@ -701,16 +746,25 @@ func (m *MMClient) GetUser(userId string) *model.User {
 	defer m.Unlock()
 	_, ok := m.Users[userId]
 	if !ok {
-		res, err := m.Client.GetProfilesByIds([]string{userId})
-		if err != nil {
+		res, resp := m.Client.GetUser(userId, "")
+		if resp.Error != nil {
 			return nil
 		}
-		u := res.Data.(map[string]*model.User)[userId]
-		m.Users[userId] = u
+		m.Users[userId] = res
 	}
 	return m.Users[userId]
 }
 
+func (m *MMClient) UpdateUser(userId string) {
+	m.Lock()
+	defer m.Unlock()
+	res, resp := m.Client.GetUser(userId, "")
+	if resp.Error != nil {
+		return
+	}
+	m.Users[userId] = res
+}
+
 func (m *MMClient) GetUserName(userId string) string {
 	user := m.GetUser(userId)
 	if user != nil {
@@ -719,37 +773,53 @@ func (m *MMClient) GetUserName(userId string) string {
 	return ""
 }
 
+func (m *MMClient) GetNickName(userId string) string {
+	user := m.GetUser(userId)
+	if user != nil {
+		return user.Nickname
+	}
+	return ""
+}
+
 func (m *MMClient) GetStatus(userId string) string {
-	res, err := m.Client.GetStatusesByIds([]string{userId})
-	if err != nil {
+	res, resp := m.Client.GetUserStatus(userId, "")
+	if resp.Error != nil {
 		return ""
 	}
-	status := res.Data.(map[string]string)
-	if status[userId] == model.STATUS_AWAY {
+	if res.Status == model.STATUS_AWAY {
 		return "away"
 	}
-	if status[userId] == model.STATUS_ONLINE {
+	if res.Status == model.STATUS_ONLINE {
 		return "online"
 	}
 	return "offline"
 }
 
+func (m *MMClient) UpdateStatus(userId string, status string) error {
+	_, resp := m.Client.UpdateUserStatus(userId, &model.Status{Status: status})
+	if resp.Error != nil {
+		return resp.Error
+	}
+	return nil
+}
+
 func (m *MMClient) GetStatuses() map[string]string {
-	var ok bool
+	var ids []string
 	statuses := make(map[string]string)
-	res, err := m.Client.GetStatuses()
-	if err != nil {
+	for id := range m.Users {
+		ids = append(ids, id)
+	}
+	res, resp := m.Client.GetUsersStatusesByIds(ids)
+	if resp.Error != nil {
 		return statuses
 	}
-	if statuses, ok = res.Data.(map[string]string); ok {
-		for userId, status := range statuses {
-			statuses[userId] = "offline"
-			if status == model.STATUS_AWAY {
-				statuses[userId] = "away"
-			}
-			if status == model.STATUS_ONLINE {
-				statuses[userId] = "online"
-			}
+	for _, status := range res {
+		statuses[status.UserId] = "offline"
+		if status.Status == model.STATUS_AWAY {
+			statuses[status.UserId] = "away"
+		}
+		if status.Status == model.STATUS_ONLINE {
+			statuses[status.UserId] = "online"
 		}
 	}
 	return statuses
@@ -759,6 +829,14 @@ func (m *MMClient) GetTeamId() string {
 	return m.Team.Id
 }
 
+func (m *MMClient) UploadFile(data []byte, channelId string, filename string) (string, error) {
+	f, resp := m.Client.UploadFile(data, channelId, filename)
+	if resp.Error != nil {
+		return "", resp.Error
+	}
+	return f.FileInfos[0].Id, nil
+}
+
 func (m *MMClient) StatusLoop() {
 	retries := 0
 	backoff := time.Second * 60
@@ -779,9 +857,14 @@ func (m *MMClient) StatusLoop() {
 				backoff = time.Second * 60
 			case <-time.After(time.Second * 5):
 				if retries > 3 {
+					m.log.Debug("StatusLoop() timeout")
 					m.Logout()
 					m.WsQuit = false
-					m.Login()
+					err := m.Login()
+					if err != nil {
+						log.Errorf("Login failed: %#v", err)
+						break
+					}
 					if m.OnWsConnect != nil {
 						m.OnWsConnect()
 					}
@@ -800,40 +883,39 @@ func (m *MMClient) StatusLoop() {
 func (m *MMClient) initUser() error {
 	m.Lock()
 	defer m.Unlock()
-	initLoad, err := m.Client.GetInitialLoad()
-	if err != nil {
-		return err
-	}
-	initData := initLoad.Data.(*model.InitialLoad)
-	m.User = initData.User
 	// we only load all team data on initial login.
 	// all other updates are for channels from our (primary) team only.
 	//m.log.Debug("initUser(): loading all team data")
-	for _, v := range initData.Teams {
-		m.Client.SetTeamId(v.Id)
-		mmusers, err := m.Client.GetProfiles(0, 50000, "")
-		if err != nil {
-			return errors.New(err.DetailedError)
+	teams, resp := m.Client.GetTeamsForUser(m.User.Id, "")
+	if resp.Error != nil {
+		return resp.Error
+	}
+	for _, team := range teams {
+		mmusers, resp := m.Client.GetUsersInTeam(team.Id, 0, 50000, "")
+		if resp.Error != nil {
+			return errors.New(resp.Error.DetailedError)
 		}
-		t := &Team{Team: v, Users: mmusers.Data.(map[string]*model.User), Id: v.Id}
-		mmchannels, err := m.Client.GetChannels("")
-		if err != nil {
-			return errors.New(err.DetailedError)
+		usermap := make(map[string]*model.User)
+		for _, user := range mmusers {
+			usermap[user.Id] = user
 		}
-		t.Channels = mmchannels.Data.(*model.ChannelList)
-		if m.mmVersion() >= 3.08 {
-			mmchannels, err = m.Client.GetMoreChannelsPage(0, 5000)
-		} else {
-			mmchannels, err = m.Client.GetMoreChannels("")
+
+		t := &Team{Team: team, Users: usermap, Id: team.Id}
+
+		mmchannels, resp := m.Client.GetChannelsForTeamForUser(team.Id, m.User.Id, "")
+		if resp.Error != nil {
+			return resp.Error
 		}
-		if err != nil {
-			return errors.New(err.DetailedError)
+		t.Channels = mmchannels
+		mmchannels, resp = m.Client.GetPublicChannelsForTeam(team.Id, 0, 5000, "")
+		if resp.Error != nil {
+			return resp.Error
 		}
-		t.MoreChannels = mmchannels.Data.(*model.ChannelList)
+		t.MoreChannels = mmchannels
 		m.OtherTeams = append(m.OtherTeams, t)
-		if v.Name == m.Credentials.Team {
+		if team.Name == m.Credentials.Team {
 			m.Team = t
-			m.log.Debugf("initUser(): found our team %s (id: %s)", v.Name, v.Id)
+			m.log.Debugf("initUser(): found our team %s (id: %s)", team.Name, team.Id)
 		}
 		// add all users
 		for k, v := range t.Users {
@@ -854,23 +936,12 @@ func (m *MMClient) sendWSRequest(action string, data map[string]interface{}) err
 	return nil
 }
 
-func (m *MMClient) mmVersion() float64 {
-	v, _ := strconv.ParseFloat(string(m.ServerVersion[0:2])+"0"+string(m.ServerVersion[2]), 64)
-	if string(m.ServerVersion[4]) == "." {
-		v, _ = strconv.ParseFloat(m.ServerVersion[0:4], 64)
-	}
-	return v
-}
-
 func supportedVersion(version string) bool {
-	if strings.HasPrefix(version, "3.5.0") ||
-		strings.HasPrefix(version, "3.6.0") ||
-		strings.HasPrefix(version, "3.7.0") ||
-		strings.HasPrefix(version, "3.8.0") ||
+	if strings.HasPrefix(version, "3.8.0") ||
 		strings.HasPrefix(version, "3.9.0") ||
 		strings.HasPrefix(version, "3.10.0") ||
-		strings.HasPrefix(version, "4.0") ||
-		strings.HasPrefix(version, "4.1") {
+		strings.HasPrefix(version, "4.") ||
+		strings.HasPrefix(version, "5.") {
 		return true
 	}
 	return false

matterhook/matterhook.go

@@ -6,24 +6,27 @@ import (
 	"crypto/tls"
 	"encoding/json"
 	"fmt"
-	"github.com/gorilla/schema"
 	"io"
 	"io/ioutil"
 	"log"
 	"net"
 	"net/http"
 	"time"
+
+	"github.com/gorilla/schema"
+	"github.com/nlopes/slack"
 )
 
 // OMessage for mattermost incoming webhook. (send to mattermost)
 type OMessage struct {
-	Channel     string      `json:"channel,omitempty"`
-	IconURL     string      `json:"icon_url,omitempty"`
-	IconEmoji   string      `json:"icon_emoji,omitempty"`
-	UserName    string      `json:"username,omitempty"`
-	Text        string      `json:"text"`
-	Attachments interface{} `json:"attachments,omitempty"`
-	Type        string      `json:"type,omitempty"`
+	Channel     string                 `json:"channel,omitempty"`
+	IconURL     string                 `json:"icon_url,omitempty"`
+	IconEmoji   string                 `json:"icon_emoji,omitempty"`
+	UserName    string                 `json:"username,omitempty"`
+	Text        string                 `json:"text"`
+	Attachments []slack.Attachment     `json:"attachments,omitempty"`
+	Type        string                 `json:"type,omitempty"`
+	Props       map[string]interface{} `json:"props"`
 }
 
 // IMessage for mattermost outgoing webhook. (received from mattermost)

vendor/github.com/42wim/go-gitter/gitter.go

@@ -205,17 +205,43 @@ func (gitter *Gitter) GetMessage(roomID, messageID string) (*Message, error) {
 }
 
 // SendMessage sends a message to a room
-func (gitter *Gitter) SendMessage(roomID, text string) error {
+func (gitter *Gitter) SendMessage(roomID, text string) (*Message, error) {
 
 	message := Message{Text: text}
 	body, _ := json.Marshal(message)
-	_, err := gitter.post(gitter.config.apiBaseURL+"rooms/"+roomID+"/chatMessages", body)
+	response, err := gitter.post(gitter.config.apiBaseURL+"rooms/"+roomID+"/chatMessages", body)
 	if err != nil {
 		gitter.log(err)
-		return err
+		return nil, err
 	}
 
-	return nil
+	err = json.Unmarshal(response, &message)
+	if err != nil {
+		gitter.log(err)
+		return nil, err
+	}
+
+	return &message, nil
+}
+
+// UpdateMessage updates a message in a room
+func (gitter *Gitter) UpdateMessage(roomID, msgID, text string) (*Message, error) {
+
+	message := Message{Text: text}
+	body, _ := json.Marshal(message)
+	response, err := gitter.put(gitter.config.apiBaseURL+"rooms/"+roomID+"/chatMessages/"+msgID, body)
+	if err != nil {
+		gitter.log(err)
+		return nil, err
+	}
+
+	err = json.Unmarshal(response, &message)
+	if err != nil {
+		gitter.log(err)
+		return nil, err
+	}
+
+	return &message, nil
 }
 
 // JoinRoom joins a room
@@ -265,7 +291,7 @@ func (gitter *Gitter) SearchRooms(room string) ([]Room, error) {
 		Results []Room `json:"results"`
 	}
 
-	response, err := gitter.get(gitter.config.apiBaseURL + "rooms?q=" + room )
+	response, err := gitter.get(gitter.config.apiBaseURL + "rooms?q=" + room)
 
 	if err != nil {
 		gitter.log(err)
@@ -414,6 +440,39 @@ func (gitter *Gitter) post(url string, body []byte) ([]byte, error) {
 	return result, nil
 }
 
+func (gitter *Gitter) put(url string, body []byte) ([]byte, error) {
+	r, err := http.NewRequest("PUT", url, bytes.NewBuffer(body))
+	if err != nil {
+		gitter.log(err)
+		return nil, err
+	}
+
+	r.Header.Set("Content-Type", "application/json")
+	r.Header.Set("Accept", "application/json")
+	r.Header.Set("Authorization", "Bearer "+gitter.config.token)
+
+	resp, err := gitter.config.client.Do(r)
+	if err != nil {
+		gitter.log(err)
+		return nil, err
+	}
+	defer resp.Body.Close()
+
+	if resp.StatusCode != http.StatusOK {
+		err = APIError{What: fmt.Sprintf("Status code: %v", resp.StatusCode)}
+		gitter.log(err)
+		return nil, err
+	}
+
+	result, err := ioutil.ReadAll(resp.Body)
+	if err != nil {
+		gitter.log(err)
+		return nil, err
+	}
+
+	return result, nil
+}
+
 func (gitter *Gitter) delete(url string) ([]byte, error) {
 	r, err := http.NewRequest("delete", url, nil)
 	if err != nil {

vendor/github.com/42wim/go-ircevent/irc.go

@@ -227,12 +227,17 @@ func (irc *Connection) isQuitting() bool {
 // Main loop to control the connection.
 func (irc *Connection) Loop() {
 	errChan := irc.ErrorChan()
+	connTime := time.Now()
 	for !irc.isQuitting() {
 		err := <-errChan
 		close(irc.end)
 		irc.Wait()
 		for !irc.isQuitting() {
 			irc.Log.Printf("Error, disconnected: %s\n", err)
+			if time.Now().Sub(connTime) < time.Second*5 {
+				irc.Log.Println("Rreconnecting too fast, sleeping 60 seconds")
+				time.Sleep(60 * time.Second)
+			}
 			if err = irc.Reconnect(); err != nil {
 				irc.Log.Printf("Error while reconnecting: %s\n", err)
 				time.Sleep(60 * time.Second)
@@ -241,6 +246,7 @@ func (irc *Connection) Loop() {
 				break
 			}
 		}
+		connTime = time.Now()
 	}
 }
 

vendor/github.com/Sirupsen/logrus/examples/hook/hook.go

@@ -1,30 +0,0 @@
-package main
-
-import (
-	"github.com/Sirupsen/logrus"
-	"gopkg.in/gemnasium/logrus-airbrake-hook.v2"
-)
-
-var log = logrus.New()
-
-func init() {
-	log.Formatter = new(logrus.TextFormatter) // default
-	log.Hooks.Add(airbrake.NewHook(123, "xyz", "development"))
-}
-
-func main() {
-	log.WithFields(logrus.Fields{
-		"animal": "walrus",
-		"size":   10,
-	}).Info("A group of walrus emerges from the ocean")
-
-	log.WithFields(logrus.Fields{
-		"omg":    true,
-		"number": 122,
-	}).Warn("The group's number increased tremendously!")
-
-	log.WithFields(logrus.Fields{
-		"omg":    true,
-		"number": 100,
-	}).Fatal("The ice breaks!")
-}

vendor/github.com/Sirupsen/logrus/hooks/test/test.go

@@ -1,67 +0,0 @@
-package test
-
-import (
-	"io/ioutil"
-
-	"github.com/Sirupsen/logrus"
-)
-
-// test.Hook is a hook designed for dealing with logs in test scenarios.
-type Hook struct {
-	Entries []*logrus.Entry
-}
-
-// Installs a test hook for the global logger.
-func NewGlobal() *Hook {
-
-	hook := new(Hook)
-	logrus.AddHook(hook)
-
-	return hook
-
-}
-
-// Installs a test hook for a given local logger.
-func NewLocal(logger *logrus.Logger) *Hook {
-
-	hook := new(Hook)
-	logger.Hooks.Add(hook)
-
-	return hook
-
-}
-
-// Creates a discarding logger and installs the test hook.
-func NewNullLogger() (*logrus.Logger, *Hook) {
-
-	logger := logrus.New()
-	logger.Out = ioutil.Discard
-
-	return logger, NewLocal(logger)
-
-}
-
-func (t *Hook) Fire(e *logrus.Entry) error {
-	t.Entries = append(t.Entries, e)
-	return nil
-}
-
-func (t *Hook) Levels() []logrus.Level {
-	return logrus.AllLevels
-}
-
-// LastEntry returns the last entry that was logged or nil.
-func (t *Hook) LastEntry() (l *logrus.Entry) {
-
-	if i := len(t.Entries) - 1; i < 0 {
-		return nil
-	} else {
-		return t.Entries[i]
-	}
-
-}
-
-// Reset removes all Entries from this test hook.
-func (t *Hook) Reset() {
-	t.Entries = make([]*logrus.Entry, 0)
-}

vendor/github.com/Sirupsen/logrus/terminal_appengine.go

@@ -1,10 +0,0 @@
-// +build appengine
-
-package logrus
-
-import "io"
-
-// IsTerminal returns true if stderr's file descriptor is a terminal.
-func IsTerminal(f io.Writer) bool {
-	return true
-}

vendor/github.com/Sirupsen/logrus/terminal_bsd.go

@@ -1,10 +0,0 @@
-// +build darwin freebsd openbsd netbsd dragonfly
-// +build !appengine
-
-package logrus
-
-import "syscall"
-
-const ioctlReadTermios = syscall.TIOCGETA
-
-type Termios syscall.Termios

vendor/github.com/Sirupsen/logrus/terminal_notwindows.go

@@ -1,28 +0,0 @@
-// Based on ssh/terminal:
-// Copyright 2011 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-// +build linux darwin freebsd openbsd netbsd dragonfly
-// +build !appengine
-
-package logrus
-
-import (
-	"io"
-	"os"
-	"syscall"
-	"unsafe"
-)
-
-// IsTerminal returns true if stderr's file descriptor is a terminal.
-func IsTerminal(f io.Writer) bool {
-	var termios Termios
-	switch v := f.(type) {
-	case *os.File:
-		_, _, err := syscall.Syscall6(syscall.SYS_IOCTL, uintptr(v.Fd()), ioctlReadTermios, uintptr(unsafe.Pointer(&termios)), 0, 0, 0)
-		return err == 0
-	default:
-		return false
-	}
-}

vendor/github.com/Sirupsen/logrus/terminal_solaris.go

@@ -1,21 +0,0 @@
-// +build solaris,!appengine
-
-package logrus
-
-import (
-	"io"
-	"os"
-
-	"golang.org/x/sys/unix"
-)
-
-// IsTerminal returns true if the given file descriptor is a terminal.
-func IsTerminal(f io.Writer) bool {
-	switch v := f.(type) {
-	case *os.File:
-		_, err := unix.IoctlGetTermios(int(v.Fd()), unix.TCGETA)
-		return err == nil
-	default:
-		return false
-	}
-}

vendor/github.com/Sirupsen/logrus/terminal_windows.go

@@ -1,33 +0,0 @@
-// Based on ssh/terminal:
-// Copyright 2011 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-// +build windows,!appengine
-
-package logrus
-
-import (
-	"io"
-	"os"
-	"syscall"
-	"unsafe"
-)
-
-var kernel32 = syscall.NewLazyDLL("kernel32.dll")
-
-var (
-	procGetConsoleMode = kernel32.NewProc("GetConsoleMode")
-)
-
-// IsTerminal returns true if stderr's file descriptor is a terminal.
-func IsTerminal(f io.Writer) bool {
-	switch v := f.(type) {
-	case *os.File:
-		var st uint32
-		r, _, e := syscall.Syscall(procGetConsoleMode.Addr(), 2, uintptr(v.Fd()), uintptr(unsafe.Pointer(&st)), 0)
-		return r != 0 && e == 0
-	default:
-		return false
-	}
-}

vendor/github.com/armon/consul-api/LICENSE

@@ -0,0 +1,362 @@
+Mozilla Public License, version 2.0
+
+1. Definitions
+
+1.1. "Contributor"
+
+     means each individual or legal entity that creates, contributes to the
+     creation of, or owns Covered Software.
+
+1.2. "Contributor Version"
+
+     means the combination of the Contributions of others (if any) used by a
+     Contributor and that particular Contributor's Contribution.
+
+1.3. "Contribution"
+
+     means Covered Software of a particular Contributor.
+
+1.4. "Covered Software"
+
+     means Source Code Form to which the initial Contributor has attached the
+     notice in Exhibit A, the Executable Form of such Source Code Form, and
+     Modifications of such Source Code Form, in each case including portions
+     thereof.
+
+1.5. "Incompatible With Secondary Licenses"
+     means
+
+     a. that the initial Contributor has attached the notice described in
+        Exhibit B to the Covered Software; or
+
+     b. that the Covered Software was made available under the terms of
+        version 1.1 or earlier of the License, but not also under the terms of
+        a Secondary License.
+
+1.6. "Executable Form"
+
+     means any form of the work other than Source Code Form.
+
+1.7. "Larger Work"
+
+     means a work that combines Covered Software with other material, in a
+     separate file or files, that is not Covered Software.
+
+1.8. "License"
+
+     means this document.
+
+1.9. "Licensable"
+
+     means having the right to grant, to the maximum extent possible, whether
+     at the time of the initial grant or subsequently, any and all of the
+     rights conveyed by this License.
+
+1.10. "Modifications"
+
+     means any of the following:
+
+     a. any file in Source Code Form that results from an addition to,
+        deletion from, or modification of the contents of Covered Software; or
+
+     b. any new file in Source Code Form that contains any Covered Software.
+
+1.11. "Patent Claims" of a Contributor
+
+      means any patent claim(s), including without limitation, method,
+      process, and apparatus claims, in any patent Licensable by such
+      Contributor that would be infringed, but for the grant of the License,
+      by the making, using, selling, offering for sale, having made, import,
+      or transfer of either its Contributions or its Contributor Version.
+
+1.12. "Secondary License"
+
+      means either the GNU General Public License, Version 2.0, the GNU Lesser
+      General Public License, Version 2.1, the GNU Affero General Public
+      License, Version 3.0, or any later versions of those licenses.
+
+1.13. "Source Code Form"
+
+      means the form of the work preferred for making modifications.
+
+1.14. "You" (or "Your")
+
+      means an individual or a legal entity exercising rights under this
+      License. For legal entities, "You" includes any entity that controls, is
+      controlled by, or is under common control with You. For purposes of this
+      definition, "control" means (a) the power, direct or indirect, to cause
+      the direction or management of such entity, whether by contract or
+      otherwise, or (b) ownership of more than fifty percent (50%) of the
+      outstanding shares or beneficial ownership of such entity.
+
+
+2. License Grants and Conditions
+
+2.1. Grants
+
+     Each Contributor hereby grants You a world-wide, royalty-free,
+     non-exclusive license:
+
+     a. under intellectual property rights (other than patent or trademark)
+        Licensable by such Contributor to use, reproduce, make available,
+        modify, display, perform, distribute, and otherwise exploit its
+        Contributions, either on an unmodified basis, with Modifications, or
+        as part of a Larger Work; and
+
+     b. under Patent Claims of such Contributor to make, use, sell, offer for
+        sale, have made, import, and otherwise transfer either its
+        Contributions or its Contributor Version.
+
+2.2. Effective Date
+
+     The licenses granted in Section 2.1 with respect to any Contribution
+     become effective for each Contribution on the date the Contributor first
+     distributes such Contribution.
+
+2.3. Limitations on Grant Scope
+
+     The licenses granted in this Section 2 are the only rights granted under
+     this License. No additional rights or licenses will be implied from the
+     distribution or licensing of Covered Software under this License.
+     Notwithstanding Section 2.1(b) above, no patent license is granted by a
+     Contributor:
+
+     a. for any code that a Contributor has removed from Covered Software; or
+
+     b. for infringements caused by: (i) Your and any other third party's
+        modifications of Covered Software, or (ii) the combination of its
+        Contributions with other software (except as part of its Contributor
+        Version); or
+
+     c. under Patent Claims infringed by Covered Software in the absence of
+        its Contributions.
+
+     This License does not grant any rights in the trademarks, service marks,
+     or logos of any Contributor (except as may be necessary to comply with
+     the notice requirements in Section 3.4).
+
+2.4. Subsequent Licenses
+
+     No Contributor makes additional grants as a result of Your choice to
+     distribute the Covered Software under a subsequent version of this
+     License (see Section 10.2) or under the terms of a Secondary License (if
+     permitted under the terms of Section 3.3).
+
+2.5. Representation
+
+     Each Contributor represents that the Contributor believes its
+     Contributions are its original creation(s) or it has sufficient rights to
+     grant the rights to its Contributions conveyed by this License.
+
+2.6. Fair Use
+
+     This License is not intended to limit any rights You have under
+     applicable copyright doctrines of fair use, fair dealing, or other
+     equivalents.
+
+2.7. Conditions
+
+     Sections 3.1, 3.2, 3.3, and 3.4 are conditions of the licenses granted in
+     Section 2.1.
+
+
+3. Responsibilities
+
+3.1. Distribution of Source Form
+
+     All distribution of Covered Software in Source Code Form, including any
+     Modifications that You create or to which You contribute, must be under
+     the terms of this License. You must inform recipients that the Source
+     Code Form of the Covered Software is governed by the terms of this
+     License, and how they can obtain a copy of this License. You may not
+     attempt to alter or restrict the recipients' rights in the Source Code
+     Form.
+
+3.2. Distribution of Executable Form
+
+     If You distribute Covered Software in Executable Form then:
+
+     a. such Covered Software must also be made available in Source Code Form,
+        as described in Section 3.1, and You must inform recipients of the
+        Executable Form how they can obtain a copy of such Source Code Form by
+        reasonable means in a timely manner, at a charge no more than the cost
+        of distribution to the recipient; and
+
+     b. You may distribute such Executable Form under the terms of this
+        License, or sublicense it under different terms, provided that the
+        license for the Executable Form does not attempt to limit or alter the
+        recipients' rights in the Source Code Form under this License.
+
+3.3. Distribution of a Larger Work
+
+     You may create and distribute a Larger Work under terms of Your choice,
+     provided that You also comply with the requirements of this License for
+     the Covered Software. If the Larger Work is a combination of Covered
+     Software with a work governed by one or more Secondary Licenses, and the
+     Covered Software is not Incompatible With Secondary Licenses, this
+     License permits You to additionally distribute such Covered Software
+     under the terms of such Secondary License(s), so that the recipient of
+     the Larger Work may, at their option, further distribute the Covered
+     Software under the terms of either this License or such Secondary
+     License(s).
+
+3.4. Notices
+
+     You may not remove or alter the substance of any license notices
+     (including copyright notices, patent notices, disclaimers of warranty, or
+     limitations of liability) contained within the Source Code Form of the
+     Covered Software, except that You may alter any license notices to the
+     extent required to remedy known factual inaccuracies.
+
+3.5. Application of Additional Terms
+
+     You may choose to offer, and to charge a fee for, warranty, support,
+     indemnity or liability obligations to one or more recipients of Covered
+     Software. However, You may do so only on Your own behalf, and not on
+     behalf of any Contributor. You must make it absolutely clear that any
+     such warranty, support, indemnity, or liability obligation is offered by
+     You alone, and You hereby agree to indemnify every Contributor for any
+     liability incurred by such Contributor as a result of warranty, support,
+     indemnity or liability terms You offer. You may include additional
+     disclaimers of warranty and limitations of liability specific to any
+     jurisdiction.
+
+4. Inability to Comply Due to Statute or Regulation
+
+   If it is impossible for You to comply with any of the terms of this License
+   with respect to some or all of the Covered Software due to statute,
+   judicial order, or regulation then You must: (a) comply with the terms of
+   this License to the maximum extent possible; and (b) describe the
+   limitations and the code they affect. Such description must be placed in a
+   text file included with all distributions of the Covered Software under
+   this License. Except to the extent prohibited by statute or regulation,
+   such description must be sufficiently detailed for a recipient of ordinary
+   skill to be able to understand it.
+
+5. Termination
+
+5.1. The rights granted under this License will terminate automatically if You
+     fail to comply with any of its terms. However, if You become compliant,
+     then the rights granted under this License from a particular Contributor
+     are reinstated (a) provisionally, unless and until such Contributor
+     explicitly and finally terminates Your grants, and (b) on an ongoing
+     basis, if such Contributor fails to notify You of the non-compliance by
+     some reasonable means prior to 60 days after You have come back into
+     compliance. Moreover, Your grants from a particular Contributor are
+     reinstated on an ongoing basis if such Contributor notifies You of the
+     non-compliance by some reasonable means, this is the first time You have
+     received notice of non-compliance with this License from such
+     Contributor, and You become compliant prior to 30 days after Your receipt
+     of the notice.
+
+5.2. If You initiate litigation against any entity by asserting a patent
+     infringement claim (excluding declaratory judgment actions,
+     counter-claims, and cross-claims) alleging that a Contributor Version
+     directly or indirectly infringes any patent, then the rights granted to
+     You by any and all Contributors for the Covered Software under Section
+     2.1 of this License shall terminate.
+
+5.3. In the event of termination under Sections 5.1 or 5.2 above, all end user
+     license agreements (excluding distributors and resellers) which have been
+     validly granted by You or Your distributors under this License prior to
+     termination shall survive termination.
+
+6. Disclaimer of Warranty
+
+   Covered Software is provided under this License on an "as is" basis,
+   without warranty of any kind, either expressed, implied, or statutory,
+   including, without limitation, warranties that the Covered Software is free
+   of defects, merchantable, fit for a particular purpose or non-infringing.
+   The entire risk as to the quality and performance of the Covered Software
+   is with You. Should any Covered Software prove defective in any respect,
+   You (not any Contributor) assume the cost of any necessary servicing,
+   repair, or correction. This disclaimer of warranty constitutes an essential
+   part of this License. No use of  any Covered Software is authorized under
+   this License except under this disclaimer.
+
+7. Limitation of Liability
+
+   Under no circumstances and under no legal theory, whether tort (including
+   negligence), contract, or otherwise, shall any Contributor, or anyone who
+   distributes Covered Software as permitted above, be liable to You for any
+   direct, indirect, special, incidental, or consequential damages of any
+   character including, without limitation, damages for lost profits, loss of
+   goodwill, work stoppage, computer failure or malfunction, or any and all
+   other commercial damages or losses, even if such party shall have been
+   informed of the possibility of such damages. This limitation of liability
+   shall not apply to liability for death or personal injury resulting from
+   such party's negligence to the extent applicable law prohibits such
+   limitation. Some jurisdictions do not allow the exclusion or limitation of
+   incidental or consequential damages, so this exclusion and limitation may
+   not apply to You.
+
+8. Litigation
+
+   Any litigation relating to this License may be brought only in the courts
+   of a jurisdiction where the defendant maintains its principal place of
+   business and such litigation shall be governed by laws of that
+   jurisdiction, without reference to its conflict-of-law provisions. Nothing
+   in this Section shall prevent a party's ability to bring cross-claims or
+   counter-claims.
+
+9. Miscellaneous
+
+   This License represents the complete agreement concerning the subject
+   matter hereof. If any provision of this License is held to be
+   unenforceable, such provision shall be reformed only to the extent
+   necessary to make it enforceable. Any law or regulation which provides that
+   the language of a contract shall be construed against the drafter shall not
+   be used to construe this License against a Contributor.
+
+
+10. Versions of the License
+
+10.1. New Versions
+
+      Mozilla Foundation is the license steward. Except as provided in Section
+      10.3, no one other than the license steward has the right to modify or
+      publish new versions of this License. Each version will be given a
+      distinguishing version number.
+
+10.2. Effect of New Versions
+
+      You may distribute the Covered Software under the terms of the version
+      of the License under which You originally received the Covered Software,
+      or under the terms of any subsequent version published by the license
+      steward.
+
+10.3. Modified Versions
+
+      If you create software not governed by this License, and you want to
+      create a new license for such software, you may create and use a
+      modified version of this License if you rename the license and remove
+      any references to the name of the license steward (except to note that
+      such modified license differs from this License).
+
+10.4. Distributing Source Code Form that is Incompatible With Secondary
+      Licenses If You choose to distribute Source Code Form that is
+      Incompatible With Secondary Licenses under the terms of this version of
+      the License, the notice described in Exhibit B of this License must be
+      attached.
+
+Exhibit A - Source Code Form License Notice
+
+      This Source Code Form is subject to the
+      terms of the Mozilla Public License, v.
+      2.0. If a copy of the MPL was not
+      distributed with this file, You can
+      obtain one at
+      http://mozilla.org/MPL/2.0/.
+
+If it is not possible or desirable to put the notice in a particular file,
+then You may include the notice in a location (such as a LICENSE file in a
+relevant directory) where a recipient would be likely to look for such a
+notice.
+
+You may add additional accurate notices of copyright ownership.
+
+Exhibit B - "Incompatible With Secondary Licenses" Notice
+
+      This Source Code Form is "Incompatible
+      With Secondary Licenses", as defined by
+      the Mozilla Public License, v. 2.0.

vendor/github.com/armon/consul-api/acl.go

@@ -0,0 +1,140 @@
+package consulapi
+
+const (
+	// ACLCLientType is the client type token
+	ACLClientType = "client"
+
+	// ACLManagementType is the management type token
+	ACLManagementType = "management"
+)
+
+// ACLEntry is used to represent an ACL entry
+type ACLEntry struct {
+	CreateIndex uint64
+	ModifyIndex uint64
+	ID          string
+	Name        string
+	Type        string
+	Rules       string
+}
+
+// ACL can be used to query the ACL endpoints
+type ACL struct {
+	c *Client
+}
+
+// ACL returns a handle to the ACL endpoints
+func (c *Client) ACL() *ACL {
+	return &ACL{c}
+}
+
+// Create is used to generate a new token with the given parameters
+func (a *ACL) Create(acl *ACLEntry, q *WriteOptions) (string, *WriteMeta, error) {
+	r := a.c.newRequest("PUT", "/v1/acl/create")
+	r.setWriteOptions(q)
+	r.obj = acl
+	rtt, resp, err := requireOK(a.c.doRequest(r))
+	if err != nil {
+		return "", nil, err
+	}
+	defer resp.Body.Close()
+
+	wm := &WriteMeta{RequestTime: rtt}
+	var out struct{ ID string }
+	if err := decodeBody(resp, &out); err != nil {
+		return "", nil, err
+	}
+	return out.ID, wm, nil
+}
+
+// Update is used to update the rules of an existing token
+func (a *ACL) Update(acl *ACLEntry, q *WriteOptions) (*WriteMeta, error) {
+	r := a.c.newRequest("PUT", "/v1/acl/update")
+	r.setWriteOptions(q)
+	r.obj = acl
+	rtt, resp, err := requireOK(a.c.doRequest(r))
+	if err != nil {
+		return nil, err
+	}
+	defer resp.Body.Close()
+
+	wm := &WriteMeta{RequestTime: rtt}
+	return wm, nil
+}
+
+// Destroy is used to destroy a given ACL token ID
+func (a *ACL) Destroy(id string, q *WriteOptions) (*WriteMeta, error) {
+	r := a.c.newRequest("PUT", "/v1/acl/destroy/"+id)
+	r.setWriteOptions(q)
+	rtt, resp, err := requireOK(a.c.doRequest(r))
+	if err != nil {
+		return nil, err
+	}
+	resp.Body.Close()
+
+	wm := &WriteMeta{RequestTime: rtt}
+	return wm, nil
+}
+
+// Clone is used to return a new token cloned from an existing one
+func (a *ACL) Clone(id string, q *WriteOptions) (string, *WriteMeta, error) {
+	r := a.c.newRequest("PUT", "/v1/acl/clone/"+id)
+	r.setWriteOptions(q)
+	rtt, resp, err := requireOK(a.c.doRequest(r))
+	if err != nil {
+		return "", nil, err
+	}
+	defer resp.Body.Close()
+
+	wm := &WriteMeta{RequestTime: rtt}
+	var out struct{ ID string }
+	if err := decodeBody(resp, &out); err != nil {
+		return "", nil, err
+	}
+	return out.ID, wm, nil
+}
+
+// Info is used to query for information about an ACL token
+func (a *ACL) Info(id string, q *QueryOptions) (*ACLEntry, *QueryMeta, error) {
+	r := a.c.newRequest("GET", "/v1/acl/info/"+id)
+	r.setQueryOptions(q)
+	rtt, resp, err := requireOK(a.c.doRequest(r))
+	if err != nil {
+		return nil, nil, err
+	}
+	defer resp.Body.Close()
+
+	qm := &QueryMeta{}
+	parseQueryMeta(resp, qm)
+	qm.RequestTime = rtt
+
+	var entries []*ACLEntry
+	if err := decodeBody(resp, &entries); err != nil {
+		return nil, nil, err
+	}
+	if len(entries) > 0 {
+		return entries[0], qm, nil
+	}
+	return nil, qm, nil
+}
+
+// List is used to get all the ACL tokens
+func (a *ACL) List(q *QueryOptions) ([]*ACLEntry, *QueryMeta, error) {
+	r := a.c.newRequest("GET", "/v1/acl/list")
+	r.setQueryOptions(q)
+	rtt, resp, err := requireOK(a.c.doRequest(r))
+	if err != nil {
+		return nil, nil, err
+	}
+	defer resp.Body.Close()
+
+	qm := &QueryMeta{}
+	parseQueryMeta(resp, qm)
+	qm.RequestTime = rtt
+
+	var entries []*ACLEntry
+	if err := decodeBody(resp, &entries); err != nil {
+		return nil, nil, err
+	}
+	return entries, qm, nil
+}

vendor/github.com/armon/consul-api/agent.go

@@ -0,0 +1,272 @@
+package consulapi
+
+import (
+	"fmt"
+)
+
+// AgentCheck represents a check known to the agent
+type AgentCheck struct {
+	Node        string
+	CheckID     string
+	Name        string
+	Status      string
+	Notes       string
+	Output      string
+	ServiceID   string
+	ServiceName string
+}
+
+// AgentService represents a service known to the agent
+type AgentService struct {
+	ID      string
+	Service string
+	Tags    []string
+	Port    int
+}
+
+// AgentMember represents a cluster member known to the agent
+type AgentMember struct {
+	Name        string
+	Addr        string
+	Port        uint16
+	Tags        map[string]string
+	Status      int
+	ProtocolMin uint8
+	ProtocolMax uint8
+	ProtocolCur uint8
+	DelegateMin uint8
+	DelegateMax uint8
+	DelegateCur uint8
+}
+
+// AgentServiceRegistration is used to register a new service
+type AgentServiceRegistration struct {
+	ID    string   `json:",omitempty"`
+	Name  string   `json:",omitempty"`
+	Tags  []string `json:",omitempty"`
+	Port  int      `json:",omitempty"`
+	Check *AgentServiceCheck
+}
+
+// AgentCheckRegistration is used to register a new check
+type AgentCheckRegistration struct {
+	ID    string `json:",omitempty"`
+	Name  string `json:",omitempty"`
+	Notes string `json:",omitempty"`
+	AgentServiceCheck
+}
+
+// AgentServiceCheck is used to create an associated
+// check for a service
+type AgentServiceCheck struct {
+	Script   string `json:",omitempty"`
+	Interval string `json:",omitempty"`
+	TTL      string `json:",omitempty"`
+}
+
+// Agent can be used to query the Agent endpoints
+type Agent struct {
+	c *Client
+
+	// cache the node name
+	nodeName string
+}
+
+// Agent returns a handle to the agent endpoints
+func (c *Client) Agent() *Agent {
+	return &Agent{c: c}
+}
+
+// Self is used to query the agent we are speaking to for
+// information about itself
+func (a *Agent) Self() (map[string]map[string]interface{}, error) {
+	r := a.c.newRequest("GET", "/v1/agent/self")
+	_, resp, err := requireOK(a.c.doRequest(r))
+	if err != nil {
+		return nil, err
+	}
+	defer resp.Body.Close()
+
+	var out map[string]map[string]interface{}
+	if err := decodeBody(resp, &out); err != nil {
+		return nil, err
+	}
+	return out, nil
+}
+
+// NodeName is used to get the node name of the agent
+func (a *Agent) NodeName() (string, error) {
+	if a.nodeName != "" {
+		return a.nodeName, nil
+	}
+	info, err := a.Self()
+	if err != nil {
+		return "", err
+	}
+	name := info["Config"]["NodeName"].(string)
+	a.nodeName = name
+	return name, nil
+}
+
+// Checks returns the locally registered checks
+func (a *Agent) Checks() (map[string]*AgentCheck, error) {
+	r := a.c.newRequest("GET", "/v1/agent/checks")
+	_, resp, err := requireOK(a.c.doRequest(r))
+	if err != nil {
+		return nil, err
+	}
+	defer resp.Body.Close()
+
+	var out map[string]*AgentCheck
+	if err := decodeBody(resp, &out); err != nil {
+		return nil, err
+	}
+	return out, nil
+}
+
+// Services returns the locally registered services
+func (a *Agent) Services() (map[string]*AgentService, error) {
+	r := a.c.newRequest("GET", "/v1/agent/services")
+	_, resp, err := requireOK(a.c.doRequest(r))
+	if err != nil {
+		return nil, err
+	}
+	defer resp.Body.Close()
+
+	var out map[string]*AgentService
+	if err := decodeBody(resp, &out); err != nil {
+		return nil, err
+	}
+	return out, nil
+}
+
+// Members returns the known gossip members. The WAN
+// flag can be used to query a server for WAN members.
+func (a *Agent) Members(wan bool) ([]*AgentMember, error) {
+	r := a.c.newRequest("GET", "/v1/agent/members")
+	if wan {
+		r.params.Set("wan", "1")
+	}
+	_, resp, err := requireOK(a.c.doRequest(r))
+	if err != nil {
+		return nil, err
+	}
+	defer resp.Body.Close()
+
+	var out []*AgentMember
+	if err := decodeBody(resp, &out); err != nil {
+		return nil, err
+	}
+	return out, nil
+}
+
+// ServiceRegister is used to register a new service with
+// the local agent
+func (a *Agent) ServiceRegister(service *AgentServiceRegistration) error {
+	r := a.c.newRequest("PUT", "/v1/agent/service/register")
+	r.obj = service
+	_, resp, err := requireOK(a.c.doRequest(r))
+	if err != nil {
+		return err
+	}
+	resp.Body.Close()
+	return nil
+}
+
+// ServiceDeregister is used to deregister a service with
+// the local agent
+func (a *Agent) ServiceDeregister(serviceID string) error {
+	r := a.c.newRequest("PUT", "/v1/agent/service/deregister/"+serviceID)
+	_, resp, err := requireOK(a.c.doRequest(r))
+	if err != nil {
+		return err
+	}
+	resp.Body.Close()
+	return nil
+}
+
+// PassTTL is used to set a TTL check to the passing state
+func (a *Agent) PassTTL(checkID, note string) error {
+	return a.UpdateTTL(checkID, note, "pass")
+}
+
+// WarnTTL is used to set a TTL check to the warning state
+func (a *Agent) WarnTTL(checkID, note string) error {
+	return a.UpdateTTL(checkID, note, "warn")
+}
+
+// FailTTL is used to set a TTL check to the failing state
+func (a *Agent) FailTTL(checkID, note string) error {
+	return a.UpdateTTL(checkID, note, "fail")
+}
+
+// UpdateTTL is used to update the TTL of a check
+func (a *Agent) UpdateTTL(checkID, note, status string) error {
+	switch status {
+	case "pass":
+	case "warn":
+	case "fail":
+	default:
+		return fmt.Errorf("Invalid status: %s", status)
+	}
+	endpoint := fmt.Sprintf("/v1/agent/check/%s/%s", status, checkID)
+	r := a.c.newRequest("PUT", endpoint)
+	r.params.Set("note", note)
+	_, resp, err := requireOK(a.c.doRequest(r))
+	if err != nil {
+		return err
+	}
+	resp.Body.Close()
+	return nil
+}
+
+// CheckRegister is used to register a new check with
+// the local agent
+func (a *Agent) CheckRegister(check *AgentCheckRegistration) error {
+	r := a.c.newRequest("PUT", "/v1/agent/check/register")
+	r.obj = check
+	_, resp, err := requireOK(a.c.doRequest(r))
+	if err != nil {
+		return err
+	}
+	resp.Body.Close()
+	return nil
+}
+
+// CheckDeregister is used to deregister a check with
+// the local agent
+func (a *Agent) CheckDeregister(checkID string) error {
+	r := a.c.newRequest("PUT", "/v1/agent/check/deregister/"+checkID)
+	_, resp, err := requireOK(a.c.doRequest(r))
+	if err != nil {
+		return err
+	}
+	resp.Body.Close()
+	return nil
+}
+
+// Join is used to instruct the agent to attempt a join to
+// another cluster member
+func (a *Agent) Join(addr string, wan bool) error {
+	r := a.c.newRequest("PUT", "/v1/agent/join/"+addr)
+	if wan {
+		r.params.Set("wan", "1")
+	}
+	_, resp, err := requireOK(a.c.doRequest(r))
+	if err != nil {
+		return err
+	}
+	resp.Body.Close()
+	return nil
+}
+
+// ForceLeave is used to have the agent eject a failed node
+func (a *Agent) ForceLeave(node string) error {
+	r := a.c.newRequest("PUT", "/v1/agent/force-leave/"+node)
+	_, resp, err := requireOK(a.c.doRequest(r))
+	if err != nil {
+		return err
+	}
+	resp.Body.Close()
+	return nil
+}

vendor/github.com/armon/consul-api/api.go

@@ -0,0 +1,323 @@
+package consulapi
+
+import (
+	"bytes"
+	"encoding/json"
+	"fmt"
+	"io"
+	"net/http"
+	"net/url"
+	"strconv"
+	"time"
+)
+
+// QueryOptions are used to parameterize a query
+type QueryOptions struct {
+	// Providing a datacenter overwrites the DC provided
+	// by the Config
+	Datacenter string
+
+	// AllowStale allows any Consul server (non-leader) to service
+	// a read. This allows for lower latency and higher throughput
+	AllowStale bool
+
+	// RequireConsistent forces the read to be fully consistent.
+	// This is more expensive but prevents ever performing a stale
+	// read.
+	RequireConsistent bool
+
+	// WaitIndex is used to enable a blocking query. Waits
+	// until the timeout or the next index is reached
+	WaitIndex uint64
+
+	// WaitTime is used to bound the duration of a wait.
+	// Defaults to that of the Config, but can be overriden.
+	WaitTime time.Duration
+
+	// Token is used to provide a per-request ACL token
+	// which overrides the agent's default token.
+	Token string
+}
+
+// WriteOptions are used to parameterize a write
+type WriteOptions struct {
+	// Providing a datacenter overwrites the DC provided
+	// by the Config
+	Datacenter string
+
+	// Token is used to provide a per-request ACL token
+	// which overrides the agent's default token.
+	Token string
+}
+
+// QueryMeta is used to return meta data about a query
+type QueryMeta struct {
+	// LastIndex. This can be used as a WaitIndex to perform
+	// a blocking query
+	LastIndex uint64
+
+	// Time of last contact from the leader for the
+	// server servicing the request
+	LastContact time.Duration
+
+	// Is there a known leader
+	KnownLeader bool
+
+	// How long did the request take
+	RequestTime time.Duration
+}
+
+// WriteMeta is used to return meta data about a write
+type WriteMeta struct {
+	// How long did the request take
+	RequestTime time.Duration
+}
+
+// HttpBasicAuth is used to authenticate http client with HTTP Basic Authentication
+type HttpBasicAuth struct {
+	// Username to use for HTTP Basic Authentication
+	Username string
+
+	// Password to use for HTTP Basic Authentication
+	Password string
+}
+
+// Config is used to configure the creation of a client
+type Config struct {
+	// Address is the address of the Consul server
+	Address string
+
+	// Scheme is the URI scheme for the Consul server
+	Scheme string
+
+	// Datacenter to use. If not provided, the default agent datacenter is used.
+	Datacenter string
+
+	// HttpClient is the client to use. Default will be
+	// used if not provided.
+	HttpClient *http.Client
+
+	// HttpAuth is the auth info to use for http access.
+	HttpAuth *HttpBasicAuth
+
+	// WaitTime limits how long a Watch will block. If not provided,
+	// the agent default values will be used.
+	WaitTime time.Duration
+
+	// Token is used to provide a per-request ACL token
+	// which overrides the agent's default token.
+	Token string
+}
+
+// DefaultConfig returns a default configuration for the client
+func DefaultConfig() *Config {
+	return &Config{
+		Address:    "127.0.0.1:8500",
+		Scheme:     "http",
+		HttpClient: http.DefaultClient,
+	}
+}
+
+// Client provides a client to the Consul API
+type Client struct {
+	config Config
+}
+
+// NewClient returns a new client
+func NewClient(config *Config) (*Client, error) {
+	// bootstrap the config
+	defConfig := DefaultConfig()
+
+	if len(config.Address) == 0 {
+		config.Address = defConfig.Address
+	}
+
+	if len(config.Scheme) == 0 {
+		config.Scheme = defConfig.Scheme
+	}
+
+	if config.HttpClient == nil {
+		config.HttpClient = defConfig.HttpClient
+	}
+
+	client := &Client{
+		config: *config,
+	}
+	return client, nil
+}
+
+// request is used to help build up a request
+type request struct {
+	config *Config
+	method string
+	url    *url.URL
+	params url.Values
+	body   io.Reader
+	obj    interface{}
+}
+
+// setQueryOptions is used to annotate the request with
+// additional query options
+func (r *request) setQueryOptions(q *QueryOptions) {
+	if q == nil {
+		return
+	}
+	if q.Datacenter != "" {
+		r.params.Set("dc", q.Datacenter)
+	}
+	if q.AllowStale {
+		r.params.Set("stale", "")
+	}
+	if q.RequireConsistent {
+		r.params.Set("consistent", "")
+	}
+	if q.WaitIndex != 0 {
+		r.params.Set("index", strconv.FormatUint(q.WaitIndex, 10))
+	}
+	if q.WaitTime != 0 {
+		r.params.Set("wait", durToMsec(q.WaitTime))
+	}
+	if q.Token != "" {
+		r.params.Set("token", q.Token)
+	}
+}
+
+// durToMsec converts a duration to a millisecond specified string
+func durToMsec(dur time.Duration) string {
+	return fmt.Sprintf("%dms", dur/time.Millisecond)
+}
+
+// setWriteOptions is used to annotate the request with
+// additional write options
+func (r *request) setWriteOptions(q *WriteOptions) {
+	if q == nil {
+		return
+	}
+	if q.Datacenter != "" {
+		r.params.Set("dc", q.Datacenter)
+	}
+	if q.Token != "" {
+		r.params.Set("token", q.Token)
+	}
+}
+
+// toHTTP converts the request to an HTTP request
+func (r *request) toHTTP() (*http.Request, error) {
+	// Encode the query parameters
+	r.url.RawQuery = r.params.Encode()
+
+	// Get the url sring
+	urlRaw := r.url.String()
+
+	// Check if we should encode the body
+	if r.body == nil && r.obj != nil {
+		if b, err := encodeBody(r.obj); err != nil {
+			return nil, err
+		} else {
+			r.body = b
+		}
+	}
+
+	// Create the HTTP request
+	req, err := http.NewRequest(r.method, urlRaw, r.body)
+
+	// Setup auth
+	if err == nil && r.config.HttpAuth != nil {
+		req.SetBasicAuth(r.config.HttpAuth.Username, r.config.HttpAuth.Password)
+	}
+
+	return req, err
+}
+
+// newRequest is used to create a new request
+func (c *Client) newRequest(method, path string) *request {
+	r := &request{
+		config: &c.config,
+		method: method,
+		url: &url.URL{
+			Scheme: c.config.Scheme,
+			Host:   c.config.Address,
+			Path:   path,
+		},
+		params: make(map[string][]string),
+	}
+	if c.config.Datacenter != "" {
+		r.params.Set("dc", c.config.Datacenter)
+	}
+	if c.config.WaitTime != 0 {
+		r.params.Set("wait", durToMsec(r.config.WaitTime))
+	}
+	if c.config.Token != "" {
+		r.params.Set("token", r.config.Token)
+	}
+	return r
+}
+
+// doRequest runs a request with our client
+func (c *Client) doRequest(r *request) (time.Duration, *http.Response, error) {
+	req, err := r.toHTTP()
+	if err != nil {
+		return 0, nil, err
+	}
+	start := time.Now()
+	resp, err := c.config.HttpClient.Do(req)
+	diff := time.Now().Sub(start)
+	return diff, resp, err
+}
+
+// parseQueryMeta is used to help parse query meta-data
+func parseQueryMeta(resp *http.Response, q *QueryMeta) error {
+	header := resp.Header
+
+	// Parse the X-Consul-Index
+	index, err := strconv.ParseUint(header.Get("X-Consul-Index"), 10, 64)
+	if err != nil {
+		return fmt.Errorf("Failed to parse X-Consul-Index: %v", err)
+	}
+	q.LastIndex = index
+
+	// Parse the X-Consul-LastContact
+	last, err := strconv.ParseUint(header.Get("X-Consul-LastContact"), 10, 64)
+	if err != nil {
+		return fmt.Errorf("Failed to parse X-Consul-LastContact: %v", err)
+	}
+	q.LastContact = time.Duration(last) * time.Millisecond
+
+	// Parse the X-Consul-KnownLeader
+	switch header.Get("X-Consul-KnownLeader") {
+	case "true":
+		q.KnownLeader = true
+	default:
+		q.KnownLeader = false
+	}
+	return nil
+}
+
+// decodeBody is used to JSON decode a body
+func decodeBody(resp *http.Response, out interface{}) error {
+	dec := json.NewDecoder(resp.Body)
+	return dec.Decode(out)
+}
+
+// encodeBody is used to encode a request body
+func encodeBody(obj interface{}) (io.Reader, error) {
+	buf := bytes.NewBuffer(nil)
+	enc := json.NewEncoder(buf)
+	if err := enc.Encode(obj); err != nil {
+		return nil, err
+	}
+	return buf, nil
+}
+
+// requireOK is used to wrap doRequest and check for a 200
+func requireOK(d time.Duration, resp *http.Response, e error) (time.Duration, *http.Response, error) {
+	if e != nil {
+		return d, resp, e
+	}
+	if resp.StatusCode != 200 {
+		var buf bytes.Buffer
+		io.Copy(&buf, resp.Body)
+		return d, resp, fmt.Errorf("Unexpected response code: %d (%s)", resp.StatusCode, buf.Bytes())
+	}
+	return d, resp, e
+}

vendor/github.com/armon/consul-api/catalog.go

@@ -0,0 +1,181 @@
+package consulapi
+
+type Node struct {
+	Node    string
+	Address string
+}
+
+type CatalogService struct {
+	Node        string
+	Address     string
+	ServiceID   string
+	ServiceName string
+	ServiceTags []string
+	ServicePort int
+}
+
+type CatalogNode struct {
+	Node     *Node
+	Services map[string]*AgentService
+}
+
+type CatalogRegistration struct {
+	Node       string
+	Address    string
+	Datacenter string
+	Service    *AgentService
+	Check      *AgentCheck
+}
+
+type CatalogDeregistration struct {
+	Node       string
+	Address    string
+	Datacenter string
+	ServiceID  string
+	CheckID    string
+}
+
+// Catalog can be used to query the Catalog endpoints
+type Catalog struct {
+	c *Client
+}
+
+// Catalog returns a handle to the catalog endpoints
+func (c *Client) Catalog() *Catalog {
+	return &Catalog{c}
+}
+
+func (c *Catalog) Register(reg *CatalogRegistration, q *WriteOptions) (*WriteMeta, error) {
+	r := c.c.newRequest("PUT", "/v1/catalog/register")
+	r.setWriteOptions(q)
+	r.obj = reg
+	rtt, resp, err := requireOK(c.c.doRequest(r))
+	if err != nil {
+		return nil, err
+	}
+	resp.Body.Close()
+
+	wm := &WriteMeta{}
+	wm.RequestTime = rtt
+
+	return wm, nil
+}
+
+func (c *Catalog) Deregister(dereg *CatalogDeregistration, q *WriteOptions) (*WriteMeta, error) {
+	r := c.c.newRequest("PUT", "/v1/catalog/deregister")
+	r.setWriteOptions(q)
+	r.obj = dereg
+	rtt, resp, err := requireOK(c.c.doRequest(r))
+	if err != nil {
+		return nil, err
+	}
+	resp.Body.Close()
+
+	wm := &WriteMeta{}
+	wm.RequestTime = rtt
+
+	return wm, nil
+}
+
+// Datacenters is used to query for all the known datacenters
+func (c *Catalog) Datacenters() ([]string, error) {
+	r := c.c.newRequest("GET", "/v1/catalog/datacenters")
+	_, resp, err := requireOK(c.c.doRequest(r))
+	if err != nil {
+		return nil, err
+	}
+	defer resp.Body.Close()
+
+	var out []string
+	if err := decodeBody(resp, &out); err != nil {
+		return nil, err
+	}
+	return out, nil
+}
+
+// Nodes is used to query all the known nodes
+func (c *Catalog) Nodes(q *QueryOptions) ([]*Node, *QueryMeta, error) {
+	r := c.c.newRequest("GET", "/v1/catalog/nodes")
+	r.setQueryOptions(q)
+	rtt, resp, err := requireOK(c.c.doRequest(r))
+	if err != nil {
+		return nil, nil, err
+	}
+	defer resp.Body.Close()
+
+	qm := &QueryMeta{}
+	parseQueryMeta(resp, qm)
+	qm.RequestTime = rtt
+
+	var out []*Node
+	if err := decodeBody(resp, &out); err != nil {
+		return nil, nil, err
+	}
+	return out, qm, nil
+}
+
+// Services is used to query for all known services
+func (c *Catalog) Services(q *QueryOptions) (map[string][]string, *QueryMeta, error) {
+	r := c.c.newRequest("GET", "/v1/catalog/services")
+	r.setQueryOptions(q)
+	rtt, resp, err := requireOK(c.c.doRequest(r))
+	if err != nil {
+		return nil, nil, err
+	}
+	defer resp.Body.Close()
+
+	qm := &QueryMeta{}
+	parseQueryMeta(resp, qm)
+	qm.RequestTime = rtt
+
+	var out map[string][]string
+	if err := decodeBody(resp, &out); err != nil {
+		return nil, nil, err
+	}
+	return out, qm, nil
+}
+
+// Service is used to query catalog entries for a given service
+func (c *Catalog) Service(service, tag string, q *QueryOptions) ([]*CatalogService, *QueryMeta, error) {
+	r := c.c.newRequest("GET", "/v1/catalog/service/"+service)
+	r.setQueryOptions(q)
+	if tag != "" {
+		r.params.Set("tag", tag)
+	}
+	rtt, resp, err := requireOK(c.c.doRequest(r))
+	if err != nil {
+		return nil, nil, err
+	}
+	defer resp.Body.Close()
+
+	qm := &QueryMeta{}
+	parseQueryMeta(resp, qm)
+	qm.RequestTime = rtt
+
+	var out []*CatalogService
+	if err := decodeBody(resp, &out); err != nil {
+		return nil, nil, err
+	}
+	return out, qm, nil
+}
+
+// Node is used to query for service information about a single node
+func (c *Catalog) Node(node string, q *QueryOptions) (*CatalogNode, *QueryMeta, error) {
+	r := c.c.newRequest("GET", "/v1/catalog/node/"+node)
+	r.setQueryOptions(q)
+	rtt, resp, err := requireOK(c.c.doRequest(r))
+	if err != nil {
+		return nil, nil, err
+	}
+	defer resp.Body.Close()
+
+	qm := &QueryMeta{}
+	parseQueryMeta(resp, qm)
+	qm.RequestTime = rtt
+
+	var out *CatalogNode
+	if err := decodeBody(resp, &out); err != nil {
+		return nil, nil, err
+	}
+	return out, qm, nil
+}

vendor/github.com/armon/consul-api/event.go

@@ -0,0 +1,104 @@
+package consulapi
+
+import (
+	"bytes"
+	"strconv"
+)
+
+// Event can be used to query the Event endpoints
+type Event struct {
+	c *Client
+}
+
+// UserEvent represents an event that was fired by the user
+type UserEvent struct {
+	ID            string
+	Name          string
+	Payload       []byte
+	NodeFilter    string
+	ServiceFilter string
+	TagFilter     string
+	Version       int
+	LTime         uint64
+}
+
+// Event returns a handle to the event endpoints
+func (c *Client) Event() *Event {
+	return &Event{c}
+}
+
+// Fire is used to fire a new user event. Only the Name, Payload and Filters
+// are respected. This returns the ID or an associated error. Cross DC requests
+// are supported.
+func (e *Event) Fire(params *UserEvent, q *WriteOptions) (string, *WriteMeta, error) {
+	r := e.c.newRequest("PUT", "/v1/event/fire/"+params.Name)
+	r.setWriteOptions(q)
+	if params.NodeFilter != "" {
+		r.params.Set("node", params.NodeFilter)
+	}
+	if params.ServiceFilter != "" {
+		r.params.Set("service", params.ServiceFilter)
+	}
+	if params.TagFilter != "" {
+		r.params.Set("tag", params.TagFilter)
+	}
+	if params.Payload != nil {
+		r.body = bytes.NewReader(params.Payload)
+	}
+
+	rtt, resp, err := requireOK(e.c.doRequest(r))
+	if err != nil {
+		return "", nil, err
+	}
+	defer resp.Body.Close()
+
+	wm := &WriteMeta{RequestTime: rtt}
+	var out UserEvent
+	if err := decodeBody(resp, &out); err != nil {
+		return "", nil, err
+	}
+	return out.ID, wm, nil
+}
+
+// List is used to get the most recent events an agent has received.
+// This list can be optionally filtered by the name. This endpoint supports
+// quasi-blocking queries. The index is not monotonic, nor does it provide provide
+// LastContact or KnownLeader.
+func (e *Event) List(name string, q *QueryOptions) ([]*UserEvent, *QueryMeta, error) {
+	r := e.c.newRequest("GET", "/v1/event/list")
+	r.setQueryOptions(q)
+	if name != "" {
+		r.params.Set("name", name)
+	}
+	rtt, resp, err := requireOK(e.c.doRequest(r))
+	if err != nil {
+		return nil, nil, err
+	}
+	defer resp.Body.Close()
+
+	qm := &QueryMeta{}
+	parseQueryMeta(resp, qm)
+	qm.RequestTime = rtt
+
+	var entries []*UserEvent
+	if err := decodeBody(resp, &entries); err != nil {
+		return nil, nil, err
+	}
+	return entries, qm, nil
+}
+
+// IDToIndex is a bit of a hack. This simulates the index generation to
+// convert an event ID into a WaitIndex.
+func (e *Event) IDToIndex(uuid string) uint64 {
+	lower := uuid[0:8] + uuid[9:13] + uuid[14:18]
+	upper := uuid[19:23] + uuid[24:36]
+	lowVal, err := strconv.ParseUint(lower, 16, 64)
+	if err != nil {
+		panic("Failed to convert " + lower)
+	}
+	highVal, err := strconv.ParseUint(upper, 16, 64)
+	if err != nil {
+		panic("Failed to convert " + upper)
+	}
+	return lowVal ^ highVal
+}

vendor/github.com/armon/consul-api/health.go

@@ -0,0 +1,136 @@
+package consulapi
+
+import (
+	"fmt"
+)
+
+// HealthCheck is used to represent a single check
+type HealthCheck struct {
+	Node        string
+	CheckID     string
+	Name        string
+	Status      string
+	Notes       string
+	Output      string
+	ServiceID   string
+	ServiceName string
+}
+
+// ServiceEntry is used for the health service endpoint
+type ServiceEntry struct {
+	Node    *Node
+	Service *AgentService
+	Checks  []*HealthCheck
+}
+
+// Health can be used to query the Health endpoints
+type Health struct {
+	c *Client
+}
+
+// Health returns a handle to the health endpoints
+func (c *Client) Health() *Health {
+	return &Health{c}
+}
+
+// Node is used to query for checks belonging to a given node
+func (h *Health) Node(node string, q *QueryOptions) ([]*HealthCheck, *QueryMeta, error) {
+	r := h.c.newRequest("GET", "/v1/health/node/"+node)
+	r.setQueryOptions(q)
+	rtt, resp, err := requireOK(h.c.doRequest(r))
+	if err != nil {
+		return nil, nil, err
+	}
+	defer resp.Body.Close()
+
+	qm := &QueryMeta{}
+	parseQueryMeta(resp, qm)
+	qm.RequestTime = rtt
+
+	var out []*HealthCheck
+	if err := decodeBody(resp, &out); err != nil {
+		return nil, nil, err
+	}
+	return out, qm, nil
+}
+
+// Checks is used to return the checks associated with a service
+func (h *Health) Checks(service string, q *QueryOptions) ([]*HealthCheck, *QueryMeta, error) {
+	r := h.c.newRequest("GET", "/v1/health/checks/"+service)
+	r.setQueryOptions(q)
+	rtt, resp, err := requireOK(h.c.doRequest(r))
+	if err != nil {
+		return nil, nil, err
+	}
+	defer resp.Body.Close()
+
+	qm := &QueryMeta{}
+	parseQueryMeta(resp, qm)
+	qm.RequestTime = rtt
+
+	var out []*HealthCheck
+	if err := decodeBody(resp, &out); err != nil {
+		return nil, nil, err
+	}
+	return out, qm, nil
+}
+
+// Service is used to query health information along with service info
+// for a given service. It can optionally do server-side filtering on a tag
+// or nodes with passing health checks only.
+func (h *Health) Service(service, tag string, passingOnly bool, q *QueryOptions) ([]*ServiceEntry, *QueryMeta, error) {
+	r := h.c.newRequest("GET", "/v1/health/service/"+service)
+	r.setQueryOptions(q)
+	if tag != "" {
+		r.params.Set("tag", tag)
+	}
+	if passingOnly {
+		r.params.Set("passing", "1")
+	}
+	rtt, resp, err := requireOK(h.c.doRequest(r))
+	if err != nil {
+		return nil, nil, err
+	}
+	defer resp.Body.Close()
+
+	qm := &QueryMeta{}
+	parseQueryMeta(resp, qm)
+	qm.RequestTime = rtt
+
+	var out []*ServiceEntry
+	if err := decodeBody(resp, &out); err != nil {
+		return nil, nil, err
+	}
+	return out, qm, nil
+}
+
+// State is used to retrieve all the checks in a given state.
+// The wildcard "any" state can also be used for all checks.
+func (h *Health) State(state string, q *QueryOptions) ([]*HealthCheck, *QueryMeta, error) {
+	switch state {
+	case "any":
+	case "warning":
+	case "critical":
+	case "passing":
+	case "unknown":
+	default:
+		return nil, nil, fmt.Errorf("Unsupported state: %v", state)
+	}
+	r := h.c.newRequest("GET", "/v1/health/state/"+state)
+	r.setQueryOptions(q)
+	rtt, resp, err := requireOK(h.c.doRequest(r))
+	if err != nil {
+		return nil, nil, err
+	}
+	defer resp.Body.Close()
+
+	qm := &QueryMeta{}
+	parseQueryMeta(resp, qm)
+	qm.RequestTime = rtt
+
+	var out []*HealthCheck
+	if err := decodeBody(resp, &out); err != nil {
+		return nil, nil, err
+	}
+	return out, qm, nil
+}

vendor/github.com/armon/consul-api/kv.go

@@ -0,0 +1,219 @@
+package consulapi
+
+import (
+	"bytes"
+	"fmt"
+	"io"
+	"net/http"
+	"strconv"
+	"strings"
+)
+
+// KVPair is used to represent a single K/V entry
+type KVPair struct {
+	Key         string
+	CreateIndex uint64
+	ModifyIndex uint64
+	LockIndex   uint64
+	Flags       uint64
+	Value       []byte
+	Session     string
+}
+
+// KVPairs is a list of KVPair objects
+type KVPairs []*KVPair
+
+// KV is used to manipulate the K/V API
+type KV struct {
+	c *Client
+}
+
+// KV is used to return a handle to the K/V apis
+func (c *Client) KV() *KV {
+	return &KV{c}
+}
+
+// Get is used to lookup a single key
+func (k *KV) Get(key string, q *QueryOptions) (*KVPair, *QueryMeta, error) {
+	resp, qm, err := k.getInternal(key, nil, q)
+	if err != nil {
+		return nil, nil, err
+	}
+	if resp == nil {
+		return nil, qm, nil
+	}
+	defer resp.Body.Close()
+
+	var entries []*KVPair
+	if err := decodeBody(resp, &entries); err != nil {
+		return nil, nil, err
+	}
+	if len(entries) > 0 {
+		return entries[0], qm, nil
+	}
+	return nil, qm, nil
+}
+
+// List is used to lookup all keys under a prefix
+func (k *KV) List(prefix string, q *QueryOptions) (KVPairs, *QueryMeta, error) {
+	resp, qm, err := k.getInternal(prefix, map[string]string{"recurse": ""}, q)
+	if err != nil {
+		return nil, nil, err
+	}
+	if resp == nil {
+		return nil, qm, nil
+	}
+	defer resp.Body.Close()
+
+	var entries []*KVPair
+	if err := decodeBody(resp, &entries); err != nil {
+		return nil, nil, err
+	}
+	return entries, qm, nil
+}
+
+// Keys is used to list all the keys under a prefix. Optionally,
+// a separator can be used to limit the responses.
+func (k *KV) Keys(prefix, separator string, q *QueryOptions) ([]string, *QueryMeta, error) {
+	params := map[string]string{"keys": ""}
+	if separator != "" {
+		params["separator"] = separator
+	}
+	resp, qm, err := k.getInternal(prefix, params, q)
+	if err != nil {
+		return nil, nil, err
+	}
+	if resp == nil {
+		return nil, qm, nil
+	}
+	defer resp.Body.Close()
+
+	var entries []string
+	if err := decodeBody(resp, &entries); err != nil {
+		return nil, nil, err
+	}
+	return entries, qm, nil
+}
+
+func (k *KV) getInternal(key string, params map[string]string, q *QueryOptions) (*http.Response, *QueryMeta, error) {
+	r := k.c.newRequest("GET", "/v1/kv/"+key)
+	r.setQueryOptions(q)
+	for param, val := range params {
+		r.params.Set(param, val)
+	}
+	rtt, resp, err := k.c.doRequest(r)
+	if err != nil {
+		return nil, nil, err
+	}
+
+	qm := &QueryMeta{}
+	parseQueryMeta(resp, qm)
+	qm.RequestTime = rtt
+
+	if resp.StatusCode == 404 {
+		resp.Body.Close()
+		return nil, qm, nil
+	} else if resp.StatusCode != 200 {
+		resp.Body.Close()
+		return nil, nil, fmt.Errorf("Unexpected response code: %d", resp.StatusCode)
+	}
+	return resp, qm, nil
+}
+
+// Put is used to write a new value. Only the
+// Key, Flags and Value is respected.
+func (k *KV) Put(p *KVPair, q *WriteOptions) (*WriteMeta, error) {
+	params := make(map[string]string, 1)
+	if p.Flags != 0 {
+		params["flags"] = strconv.FormatUint(p.Flags, 10)
+	}
+	_, wm, err := k.put(p.Key, params, p.Value, q)
+	return wm, err
+}
+
+// CAS is used for a Check-And-Set operation. The Key,
+// ModifyIndex, Flags and Value are respected. Returns true
+// on success or false on failures.
+func (k *KV) CAS(p *KVPair, q *WriteOptions) (bool, *WriteMeta, error) {
+	params := make(map[string]string, 2)
+	if p.Flags != 0 {
+		params["flags"] = strconv.FormatUint(p.Flags, 10)
+	}
+	params["cas"] = strconv.FormatUint(p.ModifyIndex, 10)
+	return k.put(p.Key, params, p.Value, q)
+}
+
+// Acquire is used for a lock acquisiiton operation. The Key,
+// Flags, Value and Session are respected. Returns true
+// on success or false on failures.
+func (k *KV) Acquire(p *KVPair, q *WriteOptions) (bool, *WriteMeta, error) {
+	params := make(map[string]string, 2)
+	if p.Flags != 0 {
+		params["flags"] = strconv.FormatUint(p.Flags, 10)
+	}
+	params["acquire"] = p.Session
+	return k.put(p.Key, params, p.Value, q)
+}
+
+// Release is used for a lock release operation. The Key,
+// Flags, Value and Session are respected. Returns true
+// on success or false on failures.
+func (k *KV) Release(p *KVPair, q *WriteOptions) (bool, *WriteMeta, error) {
+	params := make(map[string]string, 2)
+	if p.Flags != 0 {
+		params["flags"] = strconv.FormatUint(p.Flags, 10)
+	}
+	params["release"] = p.Session
+	return k.put(p.Key, params, p.Value, q)
+}
+
+func (k *KV) put(key string, params map[string]string, body []byte, q *WriteOptions) (bool, *WriteMeta, error) {
+	r := k.c.newRequest("PUT", "/v1/kv/"+key)
+	r.setWriteOptions(q)
+	for param, val := range params {
+		r.params.Set(param, val)
+	}
+	r.body = bytes.NewReader(body)
+	rtt, resp, err := requireOK(k.c.doRequest(r))
+	if err != nil {
+		return false, nil, err
+	}
+	defer resp.Body.Close()
+
+	qm := &WriteMeta{}
+	qm.RequestTime = rtt
+
+	var buf bytes.Buffer
+	if _, err := io.Copy(&buf, resp.Body); err != nil {
+		return false, nil, fmt.Errorf("Failed to read response: %v", err)
+	}
+	res := strings.Contains(string(buf.Bytes()), "true")
+	return res, qm, nil
+}
+
+// Delete is used to delete a single key
+func (k *KV) Delete(key string, w *WriteOptions) (*WriteMeta, error) {
+	return k.deleteInternal(key, nil, w)
+}
+
+// DeleteTree is used to delete all keys under a prefix
+func (k *KV) DeleteTree(prefix string, w *WriteOptions) (*WriteMeta, error) {
+	return k.deleteInternal(prefix, []string{"recurse"}, w)
+}
+
+func (k *KV) deleteInternal(key string, params []string, q *WriteOptions) (*WriteMeta, error) {
+	r := k.c.newRequest("DELETE", "/v1/kv/"+key)
+	r.setWriteOptions(q)
+	for _, param := range params {
+		r.params.Set(param, "")
+	}
+	rtt, resp, err := requireOK(k.c.doRequest(r))
+	if err != nil {
+		return nil, err
+	}
+	resp.Body.Close()
+
+	qm := &WriteMeta{}
+	qm.RequestTime = rtt
+	return qm, nil
+}

vendor/github.com/armon/consul-api/session.go

@@ -0,0 +1,204 @@
+package consulapi
+
+import (
+	"time"
+)
+
+// SessionEntry represents a session in consul
+type SessionEntry struct {
+	CreateIndex uint64
+	ID          string
+	Name        string
+	Node        string
+	Checks      []string
+	LockDelay   time.Duration
+	Behavior    string
+	TTL         string
+}
+
+// Session can be used to query the Session endpoints
+type Session struct {
+	c *Client
+}
+
+// Session returns a handle to the session endpoints
+func (c *Client) Session() *Session {
+	return &Session{c}
+}
+
+// CreateNoChecks is like Create but is used specifically to create
+// a session with no associated health checks.
+func (s *Session) CreateNoChecks(se *SessionEntry, q *WriteOptions) (string, *WriteMeta, error) {
+	body := make(map[string]interface{})
+	body["Checks"] = []string{}
+	if se != nil {
+		if se.Name != "" {
+			body["Name"] = se.Name
+		}
+		if se.Node != "" {
+			body["Node"] = se.Node
+		}
+		if se.LockDelay != 0 {
+			body["LockDelay"] = durToMsec(se.LockDelay)
+		}
+		if se.Behavior != "" {
+			body["Behavior"] = se.Behavior
+		}
+		if se.TTL != "" {
+			body["TTL"] = se.TTL
+		}
+	}
+	return s.create(body, q)
+
+}
+
+// Create makes a new session. Providing a session entry can
+// customize the session. It can also be nil to use defaults.
+func (s *Session) Create(se *SessionEntry, q *WriteOptions) (string, *WriteMeta, error) {
+	var obj interface{}
+	if se != nil {
+		body := make(map[string]interface{})
+		obj = body
+		if se.Name != "" {
+			body["Name"] = se.Name
+		}
+		if se.Node != "" {
+			body["Node"] = se.Node
+		}
+		if se.LockDelay != 0 {
+			body["LockDelay"] = durToMsec(se.LockDelay)
+		}
+		if len(se.Checks) > 0 {
+			body["Checks"] = se.Checks
+		}
+		if se.Behavior != "" {
+			body["Behavior"] = se.Behavior
+		}
+		if se.TTL != "" {
+			body["TTL"] = se.TTL
+		}
+	}
+	return s.create(obj, q)
+}
+
+func (s *Session) create(obj interface{}, q *WriteOptions) (string, *WriteMeta, error) {
+	r := s.c.newRequest("PUT", "/v1/session/create")
+	r.setWriteOptions(q)
+	r.obj = obj
+	rtt, resp, err := requireOK(s.c.doRequest(r))
+	if err != nil {
+		return "", nil, err
+	}
+	defer resp.Body.Close()
+
+	wm := &WriteMeta{RequestTime: rtt}
+	var out struct{ ID string }
+	if err := decodeBody(resp, &out); err != nil {
+		return "", nil, err
+	}
+	return out.ID, wm, nil
+}
+
+// Destroy invalides a given session
+func (s *Session) Destroy(id string, q *WriteOptions) (*WriteMeta, error) {
+	r := s.c.newRequest("PUT", "/v1/session/destroy/"+id)
+	r.setWriteOptions(q)
+	rtt, resp, err := requireOK(s.c.doRequest(r))
+	if err != nil {
+		return nil, err
+	}
+	resp.Body.Close()
+
+	wm := &WriteMeta{RequestTime: rtt}
+	return wm, nil
+}
+
+// Renew renews the TTL on a given session
+func (s *Session) Renew(id string, q *WriteOptions) (*SessionEntry, *WriteMeta, error) {
+	r := s.c.newRequest("PUT", "/v1/session/renew/"+id)
+	r.setWriteOptions(q)
+	rtt, resp, err := requireOK(s.c.doRequest(r))
+	if err != nil {
+		return nil, nil, err
+	}
+	defer resp.Body.Close()
+
+	wm := &WriteMeta{RequestTime: rtt}
+
+	var entries []*SessionEntry
+	if err := decodeBody(resp, &entries); err != nil {
+		return nil, wm, err
+	}
+
+	if len(entries) > 0 {
+		return entries[0], wm, nil
+	}
+	return nil, wm, nil
+}
+
+// Info looks up a single session
+func (s *Session) Info(id string, q *QueryOptions) (*SessionEntry, *QueryMeta, error) {
+	r := s.c.newRequest("GET", "/v1/session/info/"+id)
+	r.setQueryOptions(q)
+	rtt, resp, err := requireOK(s.c.doRequest(r))
+	if err != nil {
+		return nil, nil, err
+	}
+	defer resp.Body.Close()
+
+	qm := &QueryMeta{}
+	parseQueryMeta(resp, qm)
+	qm.RequestTime = rtt
+
+	var entries []*SessionEntry
+	if err := decodeBody(resp, &entries); err != nil {
+		return nil, nil, err
+	}
+
+	if len(entries) > 0 {
+		return entries[0], qm, nil
+	}
+	return nil, qm, nil
+}
+
+// List gets sessions for a node
+func (s *Session) Node(node string, q *QueryOptions) ([]*SessionEntry, *QueryMeta, error) {
+	r := s.c.newRequest("GET", "/v1/session/node/"+node)
+	r.setQueryOptions(q)
+	rtt, resp, err := requireOK(s.c.doRequest(r))
+	if err != nil {
+		return nil, nil, err
+	}
+	defer resp.Body.Close()
+
+	qm := &QueryMeta{}
+	parseQueryMeta(resp, qm)
+	qm.RequestTime = rtt
+
+	var entries []*SessionEntry
+	if err := decodeBody(resp, &entries); err != nil {
+		return nil, nil, err
+	}
+	return entries, qm, nil
+}
+
+// List gets all active sessions
+func (s *Session) List(q *QueryOptions) ([]*SessionEntry, *QueryMeta, error) {
+	r := s.c.newRequest("GET", "/v1/session/list")
+	r.setQueryOptions(q)
+	rtt, resp, err := requireOK(s.c.doRequest(r))
+	if err != nil {
+		return nil, nil, err
+	}
+	defer resp.Body.Close()
+
+	qm := &QueryMeta{}
+	parseQueryMeta(resp, qm)
+	qm.RequestTime = rtt
+
+	var entries []*SessionEntry
+	if err := decodeBody(resp, &entries); err != nil {
+		return nil, nil, err
+	}
+	return entries, qm, nil
+}

vendor/github.com/armon/consul-api/status.go

@@ -0,0 +1,43 @@
+package consulapi
+
+// Status can be used to query the Status endpoints
+type Status struct {
+	c *Client
+}
+
+// Status returns a handle to the status endpoints
+func (c *Client) Status() *Status {
+	return &Status{c}
+}
+
+// Leader is used to query for a known leader
+func (s *Status) Leader() (string, error) {
+	r := s.c.newRequest("GET", "/v1/status/leader")
+	_, resp, err := requireOK(s.c.doRequest(r))
+	if err != nil {
+		return "", err
+	}
+	defer resp.Body.Close()
+
+	var leader string
+	if err := decodeBody(resp, &leader); err != nil {
+		return "", err
+	}
+	return leader, nil
+}
+
+// Peers is used to query for a known raft peers
+func (s *Status) Peers() ([]string, error) {
+	r := s.c.newRequest("GET", "/v1/status/peers")
+	_, resp, err := requireOK(s.c.doRequest(r))
+	if err != nil {
+		return nil, err
+	}
+	defer resp.Body.Close()
+
+	var peers []string
+	if err := decodeBody(resp, &peers); err != nil {
+		return nil, err
+	}
+	return peers, nil
+}

vendor/github.com/bwmarrin/discordgo/discord.go

@@ -21,7 +21,7 @@ import (
 )
 
 // VERSION of DiscordGo, follows Semantic Versioning. (http://semver.org/)
-const VERSION = "0.16.0"
+const VERSION = "0.18.0"
 
 // ErrMFA will be risen by New when the user has 2FA.
 var ErrMFA = errors.New("account has 2FA enabled")
@@ -50,7 +50,7 @@ func New(args ...interface{}) (s *Session, err error) {
 	// Create an empty Session interface.
 	s = &Session{
 		State:                  NewState(),
-		ratelimiter:            NewRatelimiter(),
+		Ratelimiter:            NewRatelimiter(),
 		StateEnabled:           true,
 		Compress:               true,
 		ShouldReconnectOnError: true,
@@ -59,6 +59,7 @@ func New(args ...interface{}) (s *Session, err error) {
 		MaxRestRetries:         3,
 		Client:                 &http.Client{Timeout: (20 * time.Second)},
 		sequence:               new(int64),
+		LastHeartbeatAck:       time.Now().UTC(),
 	}
 
 	// If no arguments are passed return the empty Session interface.

vendor/github.com/bwmarrin/discordgo/endpoints.go

@@ -11,6 +11,9 @@
 
 package discordgo
 
+// APIVersion is the Discord API version used for the REST and Websocket API.
+var APIVersion = "6"
+
 // Known Discord API Endpoints.
 var (
 	EndpointStatus     = "https://status.discordapp.com/api/v2/"
@@ -18,13 +21,14 @@ var (
 	EndpointSmActive   = EndpointSm + "active.json"
 	EndpointSmUpcoming = EndpointSm + "upcoming.json"
 
-	EndpointDiscord  = "https://discordapp.com/"
-	EndpointAPI      = EndpointDiscord + "api/"
-	EndpointGuilds   = EndpointAPI + "guilds/"
-	EndpointChannels = EndpointAPI + "channels/"
-	EndpointUsers    = EndpointAPI + "users/"
-	EndpointGateway  = EndpointAPI + "gateway"
-	EndpointWebhooks = EndpointAPI + "webhooks/"
+	EndpointDiscord    = "https://discordapp.com/"
+	EndpointAPI        = EndpointDiscord + "api/v" + APIVersion + "/"
+	EndpointGuilds     = EndpointAPI + "guilds/"
+	EndpointChannels   = EndpointAPI + "channels/"
+	EndpointUsers      = EndpointAPI + "users/"
+	EndpointGateway    = EndpointAPI + "gateway"
+	EndpointGatewayBot = EndpointGateway + "/bot"
+	EndpointWebhooks   = EndpointAPI + "webhooks/"
 
 	EndpointCDN             = "https://cdn.discordapp.com/"
 	EndpointCDNAttachments  = EndpointCDN + "attachments/"
@@ -54,19 +58,19 @@ var (
 	EndpointReport       = EndpointAPI + "report"
 	EndpointIntegrations = EndpointAPI + "integrations"
 
-	EndpointUser              = func(uID string) string { return EndpointUsers + uID }
-	EndpointUserAvatar        = func(uID, aID string) string { return EndpointCDNAvatars + uID + "/" + aID + ".png" }
-	EndpointUserSettings      = func(uID string) string { return EndpointUsers + uID + "/settings" }
-	EndpointUserGuilds        = func(uID string) string { return EndpointUsers + uID + "/guilds" }
-	EndpointUserGuild         = func(uID, gID string) string { return EndpointUsers + uID + "/guilds/" + gID }
-	EndpointUserGuildSettings = func(uID, gID string) string { return EndpointUsers + uID + "/guilds/" + gID + "/settings" }
-	EndpointUserChannels      = func(uID string) string { return EndpointUsers + uID + "/channels" }
-	EndpointUserDevices       = func(uID string) string { return EndpointUsers + uID + "/devices" }
-	EndpointUserConnections   = func(uID string) string { return EndpointUsers + uID + "/connections" }
-	EndpointUserNotes         = func(uID string) string { return EndpointUsers + "@me/notes/" + uID }
+	EndpointUser               = func(uID string) string { return EndpointUsers + uID }
+	EndpointUserAvatar         = func(uID, aID string) string { return EndpointCDNAvatars + uID + "/" + aID + ".png" }
+	EndpointUserAvatarAnimated = func(uID, aID string) string { return EndpointCDNAvatars + uID + "/" + aID + ".gif" }
+	EndpointUserSettings       = func(uID string) string { return EndpointUsers + uID + "/settings" }
+	EndpointUserGuilds         = func(uID string) string { return EndpointUsers + uID + "/guilds" }
+	EndpointUserGuild          = func(uID, gID string) string { return EndpointUsers + uID + "/guilds/" + gID }
+	EndpointUserGuildSettings  = func(uID, gID string) string { return EndpointUsers + uID + "/guilds/" + gID + "/settings" }
+	EndpointUserChannels       = func(uID string) string { return EndpointUsers + uID + "/channels" }
+	EndpointUserDevices        = func(uID string) string { return EndpointUsers + uID + "/devices" }
+	EndpointUserConnections    = func(uID string) string { return EndpointUsers + uID + "/connections" }
+	EndpointUserNotes          = func(uID string) string { return EndpointUsers + "@me/notes/" + uID }
 
 	EndpointGuild                = func(gID string) string { return EndpointGuilds + gID }
-	EndpointGuildInivtes         = func(gID string) string { return EndpointGuilds + gID + "/invites" }
 	EndpointGuildChannels        = func(gID string) string { return EndpointGuilds + gID + "/channels" }
 	EndpointGuildMembers         = func(gID string) string { return EndpointGuilds + gID + "/members" }
 	EndpointGuildMember          = func(gID, uID string) string { return EndpointGuilds + gID + "/members/" + uID }
@@ -93,7 +97,7 @@ var (
 	EndpointChannelMessages           = func(cID string) string { return EndpointChannels + cID + "/messages" }
 	EndpointChannelMessage            = func(cID, mID string) string { return EndpointChannels + cID + "/messages/" + mID }
 	EndpointChannelMessageAck         = func(cID, mID string) string { return EndpointChannels + cID + "/messages/" + mID + "/ack" }
-	EndpointChannelMessagesBulkDelete = func(cID string) string { return EndpointChannel(cID) + "/messages/bulk_delete" }
+	EndpointChannelMessagesBulkDelete = func(cID string) string { return EndpointChannel(cID) + "/messages/bulk-delete" }
 	EndpointChannelMessagesPins       = func(cID string) string { return EndpointChannel(cID) + "/pins" }
 	EndpointChannelMessagePin         = func(cID, mID string) string { return EndpointChannel(cID) + "/pins/" + mID }
 
@@ -103,6 +107,9 @@ var (
 	EndpointWebhook         = func(wID string) string { return EndpointWebhooks + wID }
 	EndpointWebhookToken    = func(wID, token string) string { return EndpointWebhooks + wID + "/" + token }
 
+	EndpointMessageReactionsAll = func(cID, mID string) string {
+		return EndpointChannelMessage(cID, mID) + "/reactions"
+	}
 	EndpointMessageReactions = func(cID, mID, eID string) string {
 		return EndpointChannelMessage(cID, mID) + "/reactions/" + eID
 	}
@@ -114,6 +121,8 @@ var (
 	EndpointRelationship        = func(uID string) string { return EndpointRelationships() + "/" + uID }
 	EndpointRelationshipsMutual = func(uID string) string { return EndpointUsers + uID + "/relationships" }
 
+	EndpointGuildCreate = EndpointAPI + "guilds"
+
 	EndpointInvite = func(iID string) string { return EndpointAPI + "invite/" + iID }
 
 	EndpointIntegrationsJoin = func(iID string) string { return EndpointAPI + "integrations/" + iID + "/join" }

vendor/github.com/bwmarrin/discordgo/event.go

@@ -6,7 +6,7 @@ type EventHandler interface {
 	Type() string
 
 	// Handle is called whenever an event of Type() happens.
-	// It is the recievers responsibility to type assert that the interface
+	// It is the receivers responsibility to type assert that the interface
 	// is the expected struct.
 	Handle(*Session, interface{})
 }
@@ -156,12 +156,20 @@ func (s *Session) removeEventHandlerInstance(t string, ehi *eventHandlerInstance
 // Handles calling permanent and once handlers for an event type.
 func (s *Session) handle(t string, i interface{}) {
 	for _, eh := range s.handlers[t] {
-		go eh.eventHandler.Handle(s, i)
+		if s.SyncEvents {
+			eh.eventHandler.Handle(s, i)
+		} else {
+			go eh.eventHandler.Handle(s, i)
+		}
 	}
 
 	if len(s.onceHandlers[t]) > 0 {
 		for _, eh := range s.onceHandlers[t] {
-			go eh.eventHandler.Handle(s, i)
+			if s.SyncEvents {
+				eh.eventHandler.Handle(s, i)
+			} else {
+				go eh.eventHandler.Handle(s, i)
+			}
 		}
 		s.onceHandlers[t] = nil
 	}
@@ -216,7 +224,7 @@ func (s *Session) onInterface(i interface{}) {
 	case *VoiceStateUpdate:
 		go s.onVoiceStateUpdate(t)
 	}
-	err := s.State.onInterface(s, i)
+	err := s.State.OnInterface(s, i)
 	if err != nil {
 		s.log(LogDebug, "error dispatching internal event, %s", err)
 	}

vendor/github.com/bwmarrin/discordgo/examples/appmaker/main.go

@@ -79,7 +79,7 @@ func main() {
 	ap.Name = Name
 	ap, err = dg.ApplicationCreate(ap)
 	if err != nil {
-		fmt.Println("error creating new applicaiton,", err)
+		fmt.Println("error creating new application,", err)
 		return
 	}
 

vendor/github.com/bwmarrin/discordgo/logging.go

@@ -23,7 +23,7 @@ const (
 	LogError int = iota
 
 	// LogWarning level is used for very abnormal events and errors that are
-	// also returend to a calling function.
+	// also returned to a calling function.
 	LogWarning
 
 	// LogInformational level is used for normal non-error activity
@@ -34,26 +34,34 @@ const (
 	LogDebug
 )
 
+// Logger can be used to replace the standard logging for discordgo
+var Logger func(msgL, caller int, format string, a ...interface{})
+
 // msglog provides package wide logging consistancy for discordgo
 // the format, a...  portion this command follows that of fmt.Printf
 //   msgL   : LogLevel of the message
 //   caller : 1 + the number of callers away from the message source
 //   format : Printf style message format
-//   a ...  : comma seperated list of values to pass
+//   a ...  : comma separated list of values to pass
 func msglog(msgL, caller int, format string, a ...interface{}) {
 
-	pc, file, line, _ := runtime.Caller(caller)
+	if Logger != nil {
+		Logger(msgL, caller, format, a...)
+	} else {
 
-	files := strings.Split(file, "/")
-	file = files[len(files)-1]
+		pc, file, line, _ := runtime.Caller(caller)
 
-	name := runtime.FuncForPC(pc).Name()
-	fns := strings.Split(name, ".")
-	name = fns[len(fns)-1]
+		files := strings.Split(file, "/")
+		file = files[len(files)-1]
 
-	msg := fmt.Sprintf(format, a...)
+		name := runtime.FuncForPC(pc).Name()
+		fns := strings.Split(name, ".")
+		name = fns[len(fns)-1]
 
-	log.Printf("[DG%d] %s:%d:%s() %s\n", msgL, file, line, name, msg)
+		msg := fmt.Sprintf(format, a...)
+
+		log.Printf("[DG%d] %s:%d:%s() %s\n", msgL, file, line, name, msg)
+	}
 }
 
 // helper function that wraps msglog for the Session struct

vendor/github.com/bwmarrin/discordgo/message.go

@@ -10,9 +10,24 @@
 package discordgo
 
 import (
-	"fmt"
 	"io"
 	"regexp"
+	"strings"
+)
+
+// MessageType is the type of Message
+type MessageType int
+
+// Block contains the valid known MessageType values
+const (
+	MessageTypeDefault MessageType = iota
+	MessageTypeRecipientAdd
+	MessageTypeRecipientRemove
+	MessageTypeCall
+	MessageTypeChannelNameChange
+	MessageTypeChannelIconChange
+	MessageTypeChannelPinnedMessage
+	MessageTypeGuildMemberJoin
 )
 
 // A Message stores all data related to a specific Discord message.
@@ -30,12 +45,14 @@ type Message struct {
 	Embeds          []*MessageEmbed      `json:"embeds"`
 	Mentions        []*User              `json:"mentions"`
 	Reactions       []*MessageReactions  `json:"reactions"`
+	Type            MessageType          `json:"type"`
 }
 
 // File stores info about files you e.g. send in messages.
 type File struct {
-	Name   string
-	Reader io.Reader
+	Name        string
+	ContentType string
+	Reader      io.Reader
 }
 
 // MessageSend stores all parameters you can send with ChannelMessageSendComplex.
@@ -43,7 +60,10 @@ type MessageSend struct {
 	Content string        `json:"content,omitempty"`
 	Embed   *MessageEmbed `json:"embed,omitempty"`
 	Tts     bool          `json:"tts"`
-	File    *File         `json:"file"`
+	Files   []*File       `json:"-"`
+
+	// TODO: Remove this when compatibility is not required.
+	File *File `json:"-"`
 }
 
 // MessageEdit is used to chain parameters via ChannelMessageEditComplex, which
@@ -168,13 +188,65 @@ type MessageReactions struct {
 
 // ContentWithMentionsReplaced will replace all @<id> mentions with the
 // username of the mention.
-func (m *Message) ContentWithMentionsReplaced() string {
-	if m.Mentions == nil {
-		return m.Content
-	}
-	content := m.Content
+func (m *Message) ContentWithMentionsReplaced() (content string) {
+	content = m.Content
+
 	for _, user := range m.Mentions {
-		content = regexp.MustCompile(fmt.Sprintf("<@!?(%s)>", user.ID)).ReplaceAllString(content, "@"+user.Username)
+		content = strings.NewReplacer(
+			"<@"+user.ID+">", "@"+user.Username,
+			"<@!"+user.ID+">", "@"+user.Username,
+		).Replace(content)
 	}
-	return content
+	return
+}
+
+var patternChannels = regexp.MustCompile("<#[^>]*>")
+
+// ContentWithMoreMentionsReplaced will replace all @<id> mentions with the
+// username of the mention, but also role IDs and more.
+func (m *Message) ContentWithMoreMentionsReplaced(s *Session) (content string, err error) {
+	content = m.Content
+
+	if !s.StateEnabled {
+		content = m.ContentWithMentionsReplaced()
+		return
+	}
+
+	channel, err := s.State.Channel(m.ChannelID)
+	if err != nil {
+		content = m.ContentWithMentionsReplaced()
+		return
+	}
+
+	for _, user := range m.Mentions {
+		nick := user.Username
+
+		member, err := s.State.Member(channel.GuildID, user.ID)
+		if err == nil && member.Nick != "" {
+			nick = member.Nick
+		}
+
+		content = strings.NewReplacer(
+			"<@"+user.ID+">", "@"+user.Username,
+			"<@!"+user.ID+">", "@"+nick,
+		).Replace(content)
+	}
+	for _, roleID := range m.MentionRoles {
+		role, err := s.State.Role(channel.GuildID, roleID)
+		if err != nil || !role.Mentionable {
+			continue
+		}
+
+		content = strings.Replace(content, "<@&"+role.ID+">", "@"+role.Name, -1)
+	}
+
+	content = patternChannels.ReplaceAllStringFunc(content, func(mention string) string {
+		channel, err := s.State.Channel(mention[2 : len(mention)-1])
+		if err != nil || channel.Type == ChannelTypeGuildVoice {
+			return mention
+		}
+
+		return "#" + channel.Name
+	})
+	return
 }

vendor/github.com/bwmarrin/discordgo/ratelimit.go

@@ -3,17 +3,26 @@ package discordgo
 import (
 	"net/http"
 	"strconv"
+	"strings"
 	"sync"
 	"sync/atomic"
 	"time"
 )
 
+// customRateLimit holds information for defining a custom rate limit
+type customRateLimit struct {
+	suffix   string
+	requests int
+	reset    time.Duration
+}
+
 // RateLimiter holds all ratelimit buckets
 type RateLimiter struct {
 	sync.Mutex
-	global          *int64
-	buckets         map[string]*Bucket
-	globalRateLimit time.Duration
+	global           *int64
+	buckets          map[string]*Bucket
+	globalRateLimit  time.Duration
+	customRateLimits []*customRateLimit
 }
 
 // NewRatelimiter returns a new RateLimiter
@@ -22,11 +31,18 @@ func NewRatelimiter() *RateLimiter {
 	return &RateLimiter{
 		buckets: make(map[string]*Bucket),
 		global:  new(int64),
+		customRateLimits: []*customRateLimit{
+			&customRateLimit{
+				suffix:   "//reactions//",
+				requests: 1,
+				reset:    200 * time.Millisecond,
+			},
+		},
 	}
 }
 
-// getBucket retrieves or creates a bucket
-func (r *RateLimiter) getBucket(key string) *Bucket {
+// GetBucket retrieves or creates a bucket
+func (r *RateLimiter) GetBucket(key string) *Bucket {
 	r.Lock()
 	defer r.Unlock()
 
@@ -35,36 +51,54 @@ func (r *RateLimiter) getBucket(key string) *Bucket {
 	}
 
 	b := &Bucket{
-		remaining: 1,
+		Remaining: 1,
 		Key:       key,
 		global:    r.global,
 	}
 
+	// Check if there is a custom ratelimit set for this bucket ID.
+	for _, rl := range r.customRateLimits {
+		if strings.HasSuffix(b.Key, rl.suffix) {
+			b.customRateLimit = rl
+			break
+		}
+	}
+
 	r.buckets[key] = b
 	return b
 }
 
-// LockBucket Locks until a request can be made
-func (r *RateLimiter) LockBucket(bucketID string) *Bucket {
-
-	b := r.getBucket(bucketID)
-
-	b.Lock()
-
+// GetWaitTime returns the duration you should wait for a Bucket
+func (r *RateLimiter) GetWaitTime(b *Bucket, minRemaining int) time.Duration {
 	// If we ran out of calls and the reset time is still ahead of us
 	// then we need to take it easy and relax a little
-	if b.remaining < 1 && b.reset.After(time.Now()) {
-		time.Sleep(b.reset.Sub(time.Now()))
-
+	if b.Remaining < minRemaining && b.reset.After(time.Now()) {
+		return b.reset.Sub(time.Now())
 	}
 
 	// Check for global ratelimits
 	sleepTo := time.Unix(0, atomic.LoadInt64(r.global))
 	if now := time.Now(); now.Before(sleepTo) {
-		time.Sleep(sleepTo.Sub(now))
+		return sleepTo.Sub(now)
 	}
 
-	b.remaining--
+	return 0
+}
+
+// LockBucket Locks until a request can be made
+func (r *RateLimiter) LockBucket(bucketID string) *Bucket {
+	return r.LockBucketObject(r.GetBucket(bucketID))
+}
+
+// LockBucketObject Locks an already resolved bucket until a request can be made
+func (r *RateLimiter) LockBucketObject(b *Bucket) *Bucket {
+	b.Lock()
+
+	if wait := r.GetWaitTime(b, 1); wait > 0 {
+		time.Sleep(wait)
+	}
+
+	b.Remaining--
 	return b
 }
 
@@ -72,17 +106,33 @@ func (r *RateLimiter) LockBucket(bucketID string) *Bucket {
 type Bucket struct {
 	sync.Mutex
 	Key       string
-	remaining int
+	Remaining int
 	limit     int
 	reset     time.Time
 	global    *int64
+
+	lastReset       time.Time
+	customRateLimit *customRateLimit
+	Userdata        interface{}
 }
 
 // Release unlocks the bucket and reads the headers to update the buckets ratelimit info
 // and locks up the whole thing in case if there's a global ratelimit.
 func (b *Bucket) Release(headers http.Header) error {
-
 	defer b.Unlock()
+
+	// Check if the bucket uses a custom ratelimiter
+	if rl := b.customRateLimit; rl != nil {
+		if time.Now().Sub(b.lastReset) >= rl.reset {
+			b.Remaining = rl.requests - 1
+			b.lastReset = time.Now()
+		}
+		if b.Remaining < 1 {
+			b.reset = time.Now().Add(rl.reset)
+		}
+		return nil
+	}
+
 	if headers == nil {
 		return nil
 	}
@@ -137,7 +187,7 @@ func (b *Bucket) Release(headers http.Header) error {
 		if err != nil {
 			return err
 		}
-		b.remaining = int(parsedRemaining)
+		b.Remaining = int(parsedRemaining)
 	}
 
 	return nil

vendor/github.com/bwmarrin/discordgo/restapi.go

@@ -23,6 +23,7 @@ import (
 	"log"
 	"mime/multipart"
 	"net/http"
+	"net/textproto"
 	"net/url"
 	"strconv"
 	"strings"
@@ -64,9 +65,11 @@ func (s *Session) request(method, urlStr, contentType string, b []byte, bucketID
 	if bucketID == "" {
 		bucketID = strings.SplitN(urlStr, "?", 2)[0]
 	}
+	return s.RequestWithLockedBucket(method, urlStr, contentType, b, s.Ratelimiter.LockBucket(bucketID), sequence)
+}
 
-	bucket := s.ratelimiter.LockBucket(bucketID)
-
+// RequestWithLockedBucket makes a request using a bucket that's already been locked
+func (s *Session) RequestWithLockedBucket(method, urlStr, contentType string, b []byte, bucket *Bucket, sequence int) (response []byte, err error) {
 	if s.Debug {
 		log.Printf("API REQUEST %8s :: %s\n", method, urlStr)
 		log.Printf("API REQUEST  PAYLOAD :: [%s]\n", string(b))
@@ -138,7 +141,7 @@ func (s *Session) request(method, urlStr, contentType string, b []byte, bucketID
 		if sequence < s.MaxRestRetries {
 
 			s.log(LogInformational, "%s Failed (%s), Retrying...", urlStr, resp.Status)
-			response, err = s.request(method, urlStr, contentType, b, bucketID, sequence+1)
+			response, err = s.RequestWithLockedBucket(method, urlStr, contentType, b, s.Ratelimiter.LockBucketObject(bucket), sequence+1)
 		} else {
 			err = fmt.Errorf("Exceeded Max retries HTTP %s, %s", resp.Status, response)
 		}
@@ -157,7 +160,7 @@ func (s *Session) request(method, urlStr, contentType string, b []byte, bucketID
 		// we can make the above smarter
 		// this method can cause longer delays than required
 
-		response, err = s.request(method, urlStr, contentType, b, bucketID, sequence)
+		response, err = s.RequestWithLockedBucket(method, urlStr, contentType, b, s.Ratelimiter.LockBucketObject(bucket), sequence)
 
 	default: // Error condition
 		err = newRestError(req, resp, response)
@@ -309,8 +312,8 @@ func (s *Session) UserUpdate(email, password, username, avatar, newPassword stri
 	// If left blank, avatar will be set to null/blank
 
 	data := struct {
-		Email       string `json:"email"`
-		Password    string `json:"password"`
+		Email       string `json:"email,omitempty"`
+		Password    string `json:"password,omitempty"`
 		Username    string `json:"username,omitempty"`
 		Avatar      string `json:"avatar,omitempty"`
 		NewPassword string `json:"new_password,omitempty"`
@@ -584,7 +587,7 @@ func (s *Session) GuildCreate(name string) (st *Guild, err error) {
 		Name string `json:"name"`
 	}{name}
 
-	body, err := s.RequestWithBucketID("POST", EndpointGuilds, data, EndpointGuilds)
+	body, err := s.RequestWithBucketID("POST", EndpointGuildCreate, data, EndpointGuildCreate)
 	if err != nil {
 		return
 	}
@@ -763,7 +766,21 @@ func (s *Session) GuildMember(guildID, userID string) (st *Member, err error) {
 // userID    : The ID of a User
 func (s *Session) GuildMemberDelete(guildID, userID string) (err error) {
 
-	_, err = s.RequestWithBucketID("DELETE", EndpointGuildMember(guildID, userID), nil, EndpointGuildMember(guildID, ""))
+	return s.GuildMemberDeleteWithReason(guildID, userID, "")
+}
+
+// GuildMemberDeleteWithReason removes the given user from the given guild.
+// guildID   : The ID of a Guild.
+// userID    : The ID of a User
+// reason    : The reason for the kick
+func (s *Session) GuildMemberDeleteWithReason(guildID, userID, reason string) (err error) {
+
+	uri := EndpointGuildMember(guildID, userID)
+	if reason != "" {
+		uri += "?reason=" + url.QueryEscape(reason)
+	}
+
+	_, err = s.RequestWithBucketID("DELETE", uri, nil, EndpointGuildMember(guildID, ""))
 	return
 }
 
@@ -892,7 +909,7 @@ func (s *Session) GuildChannelsReorder(guildID string, channels []*Channel) (err
 // GuildInvites returns an array of Invite structures for the given guild
 // guildID   : The ID of a Guild.
 func (s *Session) GuildInvites(guildID string) (st []*Invite, err error) {
-	body, err := s.RequestWithBucketID("GET", EndpointGuildInvites(guildID), nil, EndpointGuildInivtes(guildID))
+	body, err := s.RequestWithBucketID("GET", EndpointGuildInvites(guildID), nil, EndpointGuildInvites(guildID))
 	if err != nil {
 		return
 	}
@@ -942,6 +959,7 @@ func (s *Session) GuildRoleEdit(guildID, roleID, name string, color int, hoist b
 	// Prevent sending a color int that is too big.
 	if color > 0xFFFFFF {
 		err = fmt.Errorf("color value cannot be larger than 0xFFFFFF")
+		return nil, err
 	}
 
 	data := struct {
@@ -1005,6 +1023,9 @@ func (s *Session) GuildPruneCount(guildID string, days uint32) (count uint32, er
 
 	uri := EndpointGuildPrune(guildID) + fmt.Sprintf("?days=%d", days)
 	body, err := s.RequestWithBucketID("GET", uri, nil, EndpointGuildPrune(guildID))
+	if err != nil {
+		return
+	}
 
 	err = unmarshal(body, &p)
 	if err != nil {
@@ -1189,7 +1210,7 @@ func (s *Session) GuildEmbedEdit(guildID string, enabled bool, channelID string)
 // Functions specific to Discord Channels
 // ------------------------------------------------------------------------------------------------
 
-// Channel returns a Channel strucutre of a specific Channel.
+// Channel returns a Channel structure of a specific Channel.
 // channelID  : The ID of the Channel you want returned.
 func (s *Session) Channel(channelID string) (st *Channel, err error) {
 	body, err := s.RequestWithBucketID("GET", EndpointChannel(channelID), nil, EndpointChannel(channelID))
@@ -1204,12 +1225,16 @@ func (s *Session) Channel(channelID string) (st *Channel, err error) {
 // ChannelEdit edits the given channel
 // channelID  : The ID of a Channel
 // name       : The new name to assign the channel.
-func (s *Session) ChannelEdit(channelID, name string) (st *Channel, err error) {
-
-	data := struct {
-		Name string `json:"name"`
-	}{name}
+func (s *Session) ChannelEdit(channelID, name string) (*Channel, error) {
+	return s.ChannelEditComplex(channelID, &ChannelEdit{
+		Name: name,
+	})
+}
 
+// ChannelEditComplex edits an existing channel, replacing the parameters entirely with ChannelEdit struct
+// channelID  : The ID of a Channel
+// data          : The channel struct to send
+func (s *Session) ChannelEditComplex(channelID string, data *ChannelEdit) (st *Channel, err error) {
 	body, err := s.RequestWithBucketID("PATCH", EndpointChannel(channelID), data, EndpointChannel(channelID))
 	if err != nil {
 		return
@@ -1316,6 +1341,8 @@ func (s *Session) ChannelMessageSend(channelID string, content string) (*Message
 	})
 }
 
+var quoteEscaper = strings.NewReplacer("\\", "\\\\", `"`, "\\\"")
+
 // ChannelMessageSendComplex sends a message to the given channel.
 // channelID : The ID of a Channel.
 // data      : The message struct to send.
@@ -1326,48 +1353,62 @@ func (s *Session) ChannelMessageSendComplex(channelID string, data *MessageSend)
 
 	endpoint := EndpointChannelMessages(channelID)
 
-	var response []byte
+	// TODO: Remove this when compatibility is not required.
+	files := data.Files
 	if data.File != nil {
+		if files == nil {
+			files = []*File{data.File}
+		} else {
+			err = fmt.Errorf("cannot specify both File and Files")
+			return
+		}
+	}
+
+	var response []byte
+	if len(files) > 0 {
 		body := &bytes.Buffer{}
 		bodywriter := multipart.NewWriter(body)
 
-		// What's a better way of doing this? Reflect? Generator? I'm open to suggestions
-
-		if data.Content != "" {
-			if err = bodywriter.WriteField("content", data.Content); err != nil {
-				return
-			}
-		}
-
-		if data.Embed != nil {
-			var embed []byte
-			embed, err = json.Marshal(data.Embed)
-			if err != nil {
-				return
-			}
-			err = bodywriter.WriteField("embed", string(embed))
-			if err != nil {
-				return
-			}
-		}
-
-		if data.Tts {
-			if err = bodywriter.WriteField("tts", "true"); err != nil {
-				return
-			}
-		}
-
-		var writer io.Writer
-		writer, err = bodywriter.CreateFormFile("file", data.File.Name)
+		var payload []byte
+		payload, err = json.Marshal(data)
 		if err != nil {
 			return
 		}
 
-		_, err = io.Copy(writer, data.File.Reader)
+		var p io.Writer
+
+		h := make(textproto.MIMEHeader)
+		h.Set("Content-Disposition", `form-data; name="payload_json"`)
+		h.Set("Content-Type", "application/json")
+
+		p, err = bodywriter.CreatePart(h)
 		if err != nil {
 			return
 		}
 
+		if _, err = p.Write(payload); err != nil {
+			return
+		}
+
+		for i, file := range files {
+			h := make(textproto.MIMEHeader)
+			h.Set("Content-Disposition", fmt.Sprintf(`form-data; name="file%d"; filename="%s"`, i, quoteEscaper.Replace(file.Name)))
+			contentType := file.ContentType
+			if contentType == "" {
+				contentType = "application/octet-stream"
+			}
+			h.Set("Content-Type", contentType)
+
+			p, err = bodywriter.CreatePart(h)
+			if err != nil {
+				return
+			}
+
+			if _, err = io.Copy(p, file.Reader); err != nil {
+				return
+			}
+		}
+
 		err = bodywriter.Close()
 		if err != nil {
 			return
@@ -1445,7 +1486,7 @@ func (s *Session) ChannelMessageDelete(channelID, messageID string) (err error)
 }
 
 // ChannelMessagesBulkDelete bulk deletes the messages from the channel for the provided messageIDs.
-// If only one messageID is in the slice call channelMessageDelete funciton.
+// If only one messageID is in the slice call channelMessageDelete function.
 // If the slice is empty do nothing.
 // channelID : The ID of the channel for the messages to delete.
 // messages  : The IDs of the messages to be deleted. A slice of string IDs. A maximum of 100 messages.
@@ -1538,16 +1579,14 @@ func (s *Session) ChannelInvites(channelID string) (st []*Invite, err error) {
 
 // ChannelInviteCreate creates a new invite for the given channel.
 // channelID   : The ID of a Channel
-// i           : An Invite struct with the values MaxAge, MaxUses, Temporary,
-//               and XkcdPass defined.
+// i           : An Invite struct with the values MaxAge, MaxUses and Temporary defined.
 func (s *Session) ChannelInviteCreate(channelID string, i Invite) (st *Invite, err error) {
 
 	data := struct {
-		MaxAge    int    `json:"max_age"`
-		MaxUses   int    `json:"max_uses"`
-		Temporary bool   `json:"temporary"`
-		XKCDPass  string `json:"xkcdpass"`
-	}{i.MaxAge, i.MaxUses, i.Temporary, i.XkcdPass}
+		MaxAge    int  `json:"max_age"`
+		MaxUses   int  `json:"max_uses"`
+		Temporary bool `json:"temporary"`
+	}{i.MaxAge, i.MaxUses, i.Temporary}
 
 	body, err := s.RequestWithBucketID("POST", EndpointChannelInvites(channelID), data, EndpointChannelInvites(channelID))
 	if err != nil {
@@ -1587,7 +1626,7 @@ func (s *Session) ChannelPermissionDelete(channelID, targetID string) (err error
 // ------------------------------------------------------------------------------------------------
 
 // Invite returns an Invite structure of the given invite
-// inviteID : The invite code (or maybe xkcdpass?)
+// inviteID : The invite code
 func (s *Session) Invite(inviteID string) (st *Invite, err error) {
 
 	body, err := s.RequestWithBucketID("GET", EndpointInvite(inviteID), nil, EndpointInvite(""))
@@ -1600,7 +1639,7 @@ func (s *Session) Invite(inviteID string) (st *Invite, err error) {
 }
 
 // InviteDelete deletes an existing invite
-// inviteID   : the code (or maybe xkcdpass?) of an invite
+// inviteID   : the code of an invite
 func (s *Session) InviteDelete(inviteID string) (st *Invite, err error) {
 
 	body, err := s.RequestWithBucketID("DELETE", EndpointInvite(inviteID), nil, EndpointInvite(""))
@@ -1613,7 +1652,7 @@ func (s *Session) InviteDelete(inviteID string) (st *Invite, err error) {
 }
 
 // InviteAccept accepts an Invite to a Guild or Channel
-// inviteID : The invite code (or maybe xkcdpass?)
+// inviteID : The invite code
 func (s *Session) InviteAccept(inviteID string) (st *Invite, err error) {
 
 	body, err := s.RequestWithBucketID("POST", EndpointInvite(inviteID), nil, EndpointInvite(""))
@@ -1685,6 +1724,28 @@ func (s *Session) Gateway() (gateway string, err error) {
 	return
 }
 
+// GatewayBot returns the websocket Gateway address and the recommended number of shards
+func (s *Session) GatewayBot() (st *GatewayBotResponse, err error) {
+
+	response, err := s.RequestWithBucketID("GET", EndpointGatewayBot, nil, EndpointGatewayBot)
+	if err != nil {
+		return
+	}
+
+	err = unmarshal(response, &st)
+	if err != nil {
+		return
+	}
+
+	// Ensure the gateway always has a trailing slash.
+	// MacOS will fail to connect if we add query params without a trailing slash on the base domain.
+	if !strings.HasSuffix(st.URL, "/") {
+		st.URL += "/"
+	}
+
+	return
+}
+
 // Functions specific to Webhooks
 
 // WebhookCreate returns a new Webhook.
@@ -1810,14 +1871,9 @@ func (s *Session) WebhookEditWithToken(webhookID, token, name, avatar string) (s
 
 // WebhookDelete deletes a webhook for a given ID
 // webhookID: The ID of a webhook.
-func (s *Session) WebhookDelete(webhookID string) (st *Webhook, err error) {
+func (s *Session) WebhookDelete(webhookID string) (err error) {
 
-	body, err := s.RequestWithBucketID("DELETE", EndpointWebhook(webhookID), nil, EndpointWebhooks)
-	if err != nil {
-		return
-	}
-
-	err = unmarshal(body, &st)
+	_, err = s.RequestWithBucketID("DELETE", EndpointWebhook(webhookID), nil, EndpointWebhooks)
 
 	return
 }
@@ -1875,6 +1931,16 @@ func (s *Session) MessageReactionRemove(channelID, messageID, emojiID, userID st
 	return err
 }
 
+// MessageReactionsRemoveAll deletes all reactions from a message
+// channelID : The channel ID
+// messageID : The message ID.
+func (s *Session) MessageReactionsRemoveAll(channelID, messageID string) error {
+
+	_, err := s.RequestWithBucketID("DELETE", EndpointMessageReactionsAll(channelID, messageID), nil, EndpointMessageReactionsAll(channelID, messageID))
+
+	return err
+}
+
 // MessageReactions gets all the users reactions for a specific emoji.
 // channelID : The channel ID.
 // messageID : The message ID.

vendor/github.com/bwmarrin/discordgo/state.go

@@ -42,6 +42,7 @@ type State struct {
 
 	guildMap   map[string]*Guild
 	channelMap map[string]*Channel
+	memberMap  map[string]map[string]*Member
 }
 
 // NewState creates an empty state.
@@ -59,9 +60,18 @@ func NewState() *State {
 		TrackPresences: true,
 		guildMap:       make(map[string]*Guild),
 		channelMap:     make(map[string]*Channel),
+		memberMap:      make(map[string]map[string]*Member),
 	}
 }
 
+func (s *State) createMemberMap(guild *Guild) {
+	members := make(map[string]*Member)
+	for _, m := range guild.Members {
+		members[m.User.ID] = m
+	}
+	s.memberMap[guild.ID] = members
+}
+
 // GuildAdd adds a guild to the current world state, or
 // updates it if it already exists.
 func (s *State) GuildAdd(guild *Guild) error {
@@ -77,6 +87,14 @@ func (s *State) GuildAdd(guild *Guild) error {
 		s.channelMap[c.ID] = c
 	}
 
+	// If this guild contains a new member slice, we must regenerate the member map so the pointers stay valid
+	if guild.Members != nil {
+		s.createMemberMap(guild)
+	} else if _, ok := s.memberMap[guild.ID]; !ok {
+		// Even if we have no new member slice, we still initialize the member map for this guild if it doesn't exist
+		s.memberMap[guild.ID] = make(map[string]*Member)
+	}
+
 	if g, ok := s.guildMap[guild.ID]; ok {
 		// We are about to replace `g` in the state with `guild`, but first we need to
 		// make sure we preserve any fields that the `guild` doesn't contain from `g`.
@@ -271,14 +289,19 @@ func (s *State) MemberAdd(member *Member) error {
 	s.Lock()
 	defer s.Unlock()
 
-	for i, m := range guild.Members {
-		if m.User.ID == member.User.ID {
-			guild.Members[i] = member
-			return nil
-		}
+	members, ok := s.memberMap[member.GuildID]
+	if !ok {
+		return ErrStateNotFound
+	}
+
+	m, ok := members[member.User.ID]
+	if !ok {
+		members[member.User.ID] = member
+		guild.Members = append(guild.Members, member)
+	} else {
+		*m = *member // Update the actual data, which will also update the member pointer in the slice
 	}
 
-	guild.Members = append(guild.Members, member)
 	return nil
 }
 
@@ -296,6 +319,17 @@ func (s *State) MemberRemove(member *Member) error {
 	s.Lock()
 	defer s.Unlock()
 
+	members, ok := s.memberMap[member.GuildID]
+	if !ok {
+		return ErrStateNotFound
+	}
+
+	_, ok = members[member.User.ID]
+	if !ok {
+		return ErrStateNotFound
+	}
+	delete(members, member.User.ID)
+
 	for i, m := range guild.Members {
 		if m.User.ID == member.User.ID {
 			guild.Members = append(guild.Members[:i], guild.Members[i+1:]...)
@@ -312,18 +346,17 @@ func (s *State) Member(guildID, userID string) (*Member, error) {
 		return nil, ErrNilState
 	}
 
-	guild, err := s.Guild(guildID)
-	if err != nil {
-		return nil, err
-	}
-
 	s.RLock()
 	defer s.RUnlock()
 
-	for _, m := range guild.Members {
-		if m.User.ID == userID {
-			return m, nil
-		}
+	members, ok := s.memberMap[guildID]
+	if !ok {
+		return nil, ErrStateNotFound
+	}
+
+	m, ok := members[userID]
+	if ok {
+		return m, nil
 	}
 
 	return nil, ErrStateNotFound
@@ -427,7 +460,7 @@ func (s *State) ChannelAdd(channel *Channel) error {
 		return nil
 	}
 
-	if channel.IsPrivate {
+	if channel.Type == ChannelTypeDM || channel.Type == ChannelTypeGroupDM {
 		s.PrivateChannels = append(s.PrivateChannels, channel)
 	} else {
 		guild, ok := s.guildMap[channel.GuildID]
@@ -454,7 +487,7 @@ func (s *State) ChannelRemove(channel *Channel) error {
 		return err
 	}
 
-	if channel.IsPrivate {
+	if channel.Type == ChannelTypeDM || channel.Type == ChannelTypeGroupDM {
 		s.Lock()
 		defer s.Unlock()
 
@@ -498,7 +531,7 @@ func (s *State) PrivateChannel(channelID string) (*Channel, error) {
 	return s.Channel(channelID)
 }
 
-// Channel gets a channel by ID, it will look in all guilds an private channels.
+// Channel gets a channel by ID, it will look in all guilds and private channels.
 func (s *State) Channel(channelID string) (*Channel, error) {
 	if s == nil {
 		return nil, ErrNilState
@@ -735,6 +768,7 @@ func (s *State) onReady(se *Session, r *Ready) (err error) {
 
 	for _, g := range s.Guilds {
 		s.guildMap[g.ID] = g
+		s.createMemberMap(g)
 
 		for _, c := range g.Channels {
 			s.channelMap[c.ID] = c
@@ -748,8 +782,8 @@ func (s *State) onReady(se *Session, r *Ready) (err error) {
 	return nil
 }
 
-// onInterface handles all events related to states.
-func (s *State) onInterface(se *Session, i interface{}) (err error) {
+// OnInterface handles all events related to states.
+func (s *State) OnInterface(se *Session, i interface{}) (err error) {
 	if s == nil {
 		return ErrNilState
 	}
@@ -782,6 +816,13 @@ func (s *State) onInterface(se *Session, i interface{}) (err error) {
 		if s.TrackMembers {
 			err = s.MemberRemove(t.Member)
 		}
+	case *GuildMembersChunk:
+		if s.TrackMembers {
+			for i := range t.Members {
+				t.Members[i].GuildID = t.GuildID
+				err = s.MemberAdd(t.Members[i])
+			}
+		}
 	case *GuildRoleCreate:
 		if s.TrackRoles {
 			err = s.RoleAdd(t.GuildID, t.Role)

vendor/github.com/bwmarrin/discordgo/structs.go

@@ -14,7 +14,6 @@ package discordgo
 import (
 	"encoding/json"
 	"net/http"
-	"strconv"
 	"sync"
 	"time"
 
@@ -50,6 +49,10 @@ type Session struct {
 	// active guilds and the members of the guilds.
 	StateEnabled bool
 
+	// Whether or not to call event handlers synchronously.
+	// e.g false = launch event handlers in their own goroutines.
+	SyncEvents bool
+
 	// Exposed but should not be modified by User.
 
 	// Whether the Data Websocket is ready
@@ -78,6 +81,12 @@ type Session struct {
 	// The http client used for REST requests
 	Client *http.Client
 
+	// Stores the last HeartbeatAck that was recieved (in UTC)
+	LastHeartbeatAck time.Time
+
+	// used to deal with rate limits
+	Ratelimiter *RateLimiter
+
 	// Event handlers
 	handlersMu   sync.RWMutex
 	handlers     map[string][]*eventHandlerInstance
@@ -89,9 +98,6 @@ type Session struct {
 	// When nil, the session is not listening.
 	listening chan interface{}
 
-	// used to deal with rate limits
-	ratelimiter *RateLimiter
-
 	// sequence tracks the current gateway api websocket sequence number
 	sequence *int64
 
@@ -136,25 +142,50 @@ type Invite struct {
 	MaxAge    int       `json:"max_age"`
 	Uses      int       `json:"uses"`
 	MaxUses   int       `json:"max_uses"`
-	XkcdPass  string    `json:"xkcdpass"`
 	Revoked   bool      `json:"revoked"`
 	Temporary bool      `json:"temporary"`
+	Unique    bool      `json:"unique"`
 }
 
+// ChannelType is the type of a Channel
+type ChannelType int
+
+// Block contains known ChannelType values
+const (
+	ChannelTypeGuildText ChannelType = iota
+	ChannelTypeDM
+	ChannelTypeGuildVoice
+	ChannelTypeGroupDM
+	ChannelTypeGuildCategory
+)
+
 // A Channel holds all data related to an individual Discord channel.
 type Channel struct {
 	ID                   string                 `json:"id"`
 	GuildID              string                 `json:"guild_id"`
 	Name                 string                 `json:"name"`
 	Topic                string                 `json:"topic"`
-	Type                 string                 `json:"type"`
+	Type                 ChannelType            `json:"type"`
 	LastMessageID        string                 `json:"last_message_id"`
+	NSFW                 bool                   `json:"nsfw"`
 	Position             int                    `json:"position"`
 	Bitrate              int                    `json:"bitrate"`
-	IsPrivate            bool                   `json:"is_private"`
-	Recipient            *User                  `json:"recipient"`
+	Recipients           []*User                `json:"recipients"`
 	Messages             []*Message             `json:"-"`
 	PermissionOverwrites []*PermissionOverwrite `json:"permission_overwrites"`
+	ParentID             string                 `json:"parent_id"`
+}
+
+// A ChannelEdit holds Channel Feild data for a channel edit.
+type ChannelEdit struct {
+	Name                 string                 `json:"name,omitempty"`
+	Topic                string                 `json:"topic,omitempty"`
+	NSFW                 bool                   `json:"nsfw,omitempty"`
+	Position             int                    `json:"position"`
+	Bitrate              int                    `json:"bitrate,omitempty"`
+	UserLimit            int                    `json:"user_limit,omitempty"`
+	PermissionOverwrites []*PermissionOverwrite `json:"permission_overwrites,omitempty"`
+	ParentID             string                 `json:"parent_id,omitempty"`
 }
 
 // A PermissionOverwrite holds permission overwrite data for a Channel
@@ -172,6 +203,7 @@ type Emoji struct {
 	Roles         []string `json:"roles"`
 	Managed       bool     `json:"managed"`
 	RequireColons bool     `json:"require_colons"`
+	Animated      bool     `json:"animated"`
 }
 
 // APIName returns an correctly formatted API name for use in the MessageReactions endpoints.
@@ -185,7 +217,7 @@ func (e *Emoji) APIName() string {
 	return e.ID
 }
 
-// VerificationLevel type defination
+// VerificationLevel type definition
 type VerificationLevel int
 
 // Constants for VerificationLevel levels from 0 to 3 inclusive
@@ -292,47 +324,61 @@ type Presence struct {
 	Game   *Game    `json:"game"`
 	Nick   string   `json:"nick"`
 	Roles  []string `json:"roles"`
+	Since  *int     `json:"since"`
 }
 
+// GameType is the type of "game" (see GameType* consts) in the Game struct
+type GameType int
+
+// Valid GameType values
+const (
+	GameTypeGame GameType = iota
+	GameTypeStreaming
+)
+
 // A Game struct holds the name of the "playing .." game for a user
 type Game struct {
-	Name string `json:"name"`
-	Type int    `json:"type"`
-	URL  string `json:"url"`
+	Name          string     `json:"name"`
+	Type          GameType   `json:"type"`
+	URL           string     `json:"url,omitempty"`
+	Details       string     `json:"details,omitempty"`
+	State         string     `json:"state,omitempty"`
+	TimeStamps    TimeStamps `json:"timestamps,omitempty"`
+	Assets        Assets     `json:"assets,omitempty"`
+	ApplicationID string     `json:"application_id,omitempty"`
+	Instance      int8       `json:"instance,omitempty"`
+	// TODO: Party and Secrets (unknown structure)
 }
 
-// UnmarshalJSON unmarshals json to Game struct
-func (g *Game) UnmarshalJSON(bytes []byte) error {
-	temp := &struct {
-		Name json.Number     `json:"name"`
-		Type json.RawMessage `json:"type"`
-		URL  string          `json:"url"`
+// A TimeStamps struct contains start and end times used in the rich presence "playing .." Game
+type TimeStamps struct {
+	EndTimestamp   int64 `json:"end,omitempty"`
+	StartTimestamp int64 `json:"start,omitempty"`
+}
+
+// UnmarshalJSON unmarshals JSON into TimeStamps struct
+func (t *TimeStamps) UnmarshalJSON(b []byte) error {
+	temp := struct {
+		End   float64 `json:"end,omitempty"`
+		Start float64 `json:"start,omitempty"`
 	}{}
-	err := json.Unmarshal(bytes, temp)
+	err := json.Unmarshal(b, &temp)
 	if err != nil {
 		return err
 	}
-	g.URL = temp.URL
-	g.Name = temp.Name.String()
-
-	if temp.Type != nil {
-		err = json.Unmarshal(temp.Type, &g.Type)
-		if err == nil {
-			return nil
-		}
-
-		s := ""
-		err = json.Unmarshal(temp.Type, &s)
-		if err == nil {
-			g.Type, err = strconv.Atoi(s)
-		}
-
-		return err
-	}
-
+	t.EndTimestamp = int64(temp.End)
+	t.StartTimestamp = int64(temp.Start)
 	return nil
 }
 
+// An Assets struct contains assets and labels used in the rich presence "playing .." Game
+type Assets struct {
+	LargeImageID string `json:"large_image,omitempty"`
+	SmallImageID string `json:"small_image,omitempty"`
+	LargeText    string `json:"large_text,omitempty"`
+	SmallText    string `json:"small_text,omitempty"`
+}
+
 // A Member stores user information for Guild members.
 type Member struct {
 	GuildID  string   `json:"guild_id"`
@@ -363,7 +409,7 @@ type Settings struct {
 	DeveloperMode          bool               `json:"developer_mode"`
 }
 
-// Status type defination
+// Status type definition
 type Status string
 
 // Constants for Status with the different current available status
@@ -509,6 +555,12 @@ type MessageReaction struct {
 	ChannelID string `json:"channel_id"`
 }
 
+// GatewayBotResponse stores the data for the gateway/bot response
+type GatewayBotResponse struct {
+	URL    string `json:"url"`
+	Shards int    `json:"shards"`
+}
+
 // Constants for the different bit offsets of text channel permissions
 const (
 	PermissionReadMessages = 1 << (iota + 10)
@@ -579,3 +631,56 @@ const (
 		PermissionManageServer |
 		PermissionAdministrator
 )
+
+// Block contains Discord JSON Error Response codes
+const (
+	ErrCodeUnknownAccount     = 10001
+	ErrCodeUnknownApplication = 10002
+	ErrCodeUnknownChannel     = 10003
+	ErrCodeUnknownGuild       = 10004
+	ErrCodeUnknownIntegration = 10005
+	ErrCodeUnknownInvite      = 10006
+	ErrCodeUnknownMember      = 10007
+	ErrCodeUnknownMessage     = 10008
+	ErrCodeUnknownOverwrite   = 10009
+	ErrCodeUnknownProvider    = 10010
+	ErrCodeUnknownRole        = 10011
+	ErrCodeUnknownToken       = 10012
+	ErrCodeUnknownUser        = 10013
+	ErrCodeUnknownEmoji       = 10014
+
+	ErrCodeBotsCannotUseEndpoint  = 20001
+	ErrCodeOnlyBotsCanUseEndpoint = 20002
+
+	ErrCodeMaximumGuildsReached     = 30001
+	ErrCodeMaximumFriendsReached    = 30002
+	ErrCodeMaximumPinsReached       = 30003
+	ErrCodeMaximumGuildRolesReached = 30005
+	ErrCodeTooManyReactions         = 30010
+
+	ErrCodeUnauthorized = 40001
+
+	ErrCodeMissingAccess                             = 50001
+	ErrCodeInvalidAccountType                        = 50002
+	ErrCodeCannotExecuteActionOnDMChannel            = 50003
+	ErrCodeEmbedCisabled                             = 50004
+	ErrCodeCannotEditFromAnotherUser                 = 50005
+	ErrCodeCannotSendEmptyMessage                    = 50006
+	ErrCodeCannotSendMessagesToThisUser              = 50007
+	ErrCodeCannotSendMessagesInVoiceChannel          = 50008
+	ErrCodeChannelVerificationLevelTooHigh           = 50009
+	ErrCodeOAuth2ApplicationDoesNotHaveBot           = 50010
+	ErrCodeOAuth2ApplicationLimitReached             = 50011
+	ErrCodeInvalidOAuthState                         = 50012
+	ErrCodeMissingPermissions                        = 50013
+	ErrCodeInvalidAuthenticationToken                = 50014
+	ErrCodeNoteTooLong                               = 50015
+	ErrCodeTooFewOrTooManyMessagesToDelete           = 50016
+	ErrCodeCanOnlyPinMessageToOriginatingChannel     = 50019
+	ErrCodeCannotExecuteActionOnSystemMessage        = 50021
+	ErrCodeMessageProvidedTooOldForBulkDelete        = 50034
+	ErrCodeInvalidFormBody                           = 50035
+	ErrCodeInviteAcceptedToGuildApplicationsBotNotIn = 50036
+
+	ErrCodeReactionBlocked = 90001
+)

vendor/github.com/bwmarrin/discordgo/user.go

@@ -1,6 +1,9 @@
 package discordgo
 
-import "fmt"
+import (
+	"fmt"
+	"strings"
+)
 
 // A User stores all data for an individual Discord user.
 type User struct {
@@ -24,3 +27,21 @@ func (u *User) String() string {
 func (u *User) Mention() string {
 	return fmt.Sprintf("<@%s>", u.ID)
 }
+
+// AvatarURL returns a URL to the user's avatar.
+//    size:    The size of the user's avatar as a power of two
+//             if size is an empty string, no size parameter will
+//             be added to the URL.
+func (u *User) AvatarURL(size string) string {
+	var URL string
+	if strings.HasPrefix(u.Avatar, "a_") {
+		URL = EndpointUserAvatarAnimated(u.ID, u.Avatar)
+	} else {
+		URL = EndpointUserAvatar(u.ID, u.Avatar)
+	}
+
+	if size != "" {
+		return URL + "?size=" + size
+	}
+	return URL
+}

vendor/github.com/bwmarrin/discordgo/voice.go

@@ -13,7 +13,6 @@ import (
 	"encoding/binary"
 	"encoding/json"
 	"fmt"
-	"log"
 	"net"
 	"strings"
 	"sync"
@@ -69,7 +68,7 @@ type VoiceConnection struct {
 	voiceSpeakingUpdateHandlers []VoiceSpeakingUpdateHandler
 }
 
-// VoiceSpeakingUpdateHandler type provides a function defination for the
+// VoiceSpeakingUpdateHandler type provides a function definition for the
 // VoiceSpeakingUpdate event
 type VoiceSpeakingUpdateHandler func(vc *VoiceConnection, vs *VoiceSpeakingUpdate)
 
@@ -104,7 +103,7 @@ func (v *VoiceConnection) Speaking(b bool) (err error) {
 	defer v.Unlock()
 	if err != nil {
 		v.speaking = false
-		log.Println("Speaking() write json error:", err)
+		v.log(LogError, "Speaking() write json error:", err)
 		return
 	}
 
@@ -181,7 +180,7 @@ func (v *VoiceConnection) Close() {
 		v.log(LogInformational, "closing udp")
 		err := v.udpConn.Close()
 		if err != nil {
-			log.Println("error closing udp connection: ", err)
+			v.log(LogError, "error closing udp connection: ", err)
 		}
 		v.udpConn = nil
 	}
@@ -247,7 +246,7 @@ type voiceOP2 struct {
 }
 
 // WaitUntilConnected waits for the Voice Connection to
-// become ready, if it does not become ready it retuns an err
+// become ready, if it does not become ready it returns an err
 func (v *VoiceConnection) waitUntilConnected() error {
 
 	v.log(LogInformational, "called")
@@ -796,7 +795,7 @@ func (v *VoiceConnection) opusReceiver(udpConn *net.UDPConn, close <-chan struct
 		}
 
 		// For now, skip anything except audio.
-		if rlen < 12 || recvbuf[0] != 0x80 {
+		if rlen < 12 || (recvbuf[0] != 0x80 && recvbuf[0] != 0x90) {
 			continue
 		}
 
@@ -810,8 +809,17 @@ func (v *VoiceConnection) opusReceiver(udpConn *net.UDPConn, close <-chan struct
 		copy(nonce[:], recvbuf[0:12])
 		p.Opus, _ = secretbox.Open(nil, recvbuf[12:rlen], &nonce, &v.op4.SecretKey)
 
+		if len(p.Opus) > 8 && recvbuf[0] == 0x90 {
+			// Extension bit is set, first 8 bytes is the extended header
+			p.Opus = p.Opus[8:]
+		}
+
 		if c != nil {
-			c <- &p
+			select {
+			case c <- &p:
+			case <-close:
+				return
+			}
 		}
 	}
 }
@@ -849,7 +857,7 @@ func (v *VoiceConnection) reconnect() {
 		}
 
 		if v.session.DataReady == false || v.session.wsConn == nil {
-			v.log(LogInformational, "cannot reconenct to channel %s with unready session", v.ChannelID)
+			v.log(LogInformational, "cannot reconnect to channel %s with unready session", v.ChannelID)
 			continue
 		}
 

vendor/github.com/bwmarrin/discordgo/wsapi.go

@@ -46,19 +46,114 @@ type resumePacket struct {
 	} `json:"d"`
 }
 
-// Open opens a websocket connection to Discord.
-func (s *Session) Open() (err error) {
-
+// Open creates a websocket connection to Discord.
+// See: https://discordapp.com/developers/docs/topics/gateway#connecting
+func (s *Session) Open() error {
 	s.log(LogInformational, "called")
 
+	var err error
+
+	// Prevent Open or other major Session functions from
+	// being called while Open is still running.
 	s.Lock()
-	defer func() {
+	defer s.Unlock()
+
+	// If the websock is already open, bail out here.
+	if s.wsConn != nil {
+		return ErrWSAlreadyOpen
+	}
+
+	// Get the gateway to use for the Websocket connection
+	if s.gateway == "" {
+		s.gateway, err = s.Gateway()
 		if err != nil {
-			s.Unlock()
+			return err
+		}
+
+		// Add the version and encoding to the URL
+		s.gateway = s.gateway + "?v=" + APIVersion + "&encoding=json"
+	}
+
+	// Connect to the Gateway
+	s.log(LogInformational, "connecting to gateway %s", s.gateway)
+	header := http.Header{}
+	header.Add("accept-encoding", "zlib")
+	s.wsConn, _, err = websocket.DefaultDialer.Dial(s.gateway, header)
+	if err != nil {
+		s.log(LogWarning, "error connecting to gateway %s, %s", s.gateway, err)
+		s.gateway = "" // clear cached gateway
+		s.wsConn = nil // Just to be safe.
+		return err
+	}
+
+	defer func() {
+		// because of this, all code below must set err to the error
+		// when exiting with an error :)  Maybe someone has a better
+		// way :)
+		if err != nil {
+			s.wsConn.Close()
+			s.wsConn = nil
 		}
 	}()
 
+	// The first response from Discord should be an Op 10 (Hello) Packet.
+	// When processed by onEvent the heartbeat goroutine will be started.
+	mt, m, err := s.wsConn.ReadMessage()
+	if err != nil {
+		return err
+	}
+	e, err := s.onEvent(mt, m)
+	if err != nil {
+		return err
+	}
+	if e.Operation != 10 {
+		err = fmt.Errorf("expecting Op 10, got Op %d instead", e.Operation)
+		return err
+	}
+	s.log(LogInformational, "Op 10 Hello Packet received from Discord")
+	s.LastHeartbeatAck = time.Now().UTC()
+	var h helloOp
+	if err = json.Unmarshal(e.RawData, &h); err != nil {
+		err = fmt.Errorf("error unmarshalling helloOp, %s", err)
+		return err
+	}
+
+	// Now we send either an Op 2 Identity if this is a brand new
+	// connection or Op 6 Resume if we are resuming an existing connection.
+	sequence := atomic.LoadInt64(s.sequence)
+	if s.sessionID == "" && sequence == 0 {
+
+		// Send Op 2 Identity Packet
+		err = s.identify()
+		if err != nil {
+			err = fmt.Errorf("error sending identify packet to gateway, %s, %s", s.gateway, err)
+			return err
+		}
+
+	} else {
+
+		// Send Op 6 Resume Packet
+		p := resumePacket{}
+		p.Op = 6
+		p.Data.Token = s.Token
+		p.Data.SessionID = s.sessionID
+		p.Data.Sequence = sequence
+
+		s.log(LogInformational, "sending resume packet to gateway")
+		s.wsMutex.Lock()
+		err = s.wsConn.WriteJSON(p)
+		s.wsMutex.Unlock()
+		if err != nil {
+			err = fmt.Errorf("error sending gateway resume packet, %s, %s", s.gateway, err)
+			return err
+		}
+
+	}
+
 	// A basic state is a hard requirement for Voice.
+	// We create it here so the below READY/RESUMED packet can populate
+	// the state :)
+	// XXX: Move to New() func?
 	if s.State == nil {
 		state := NewState()
 		state.TrackChannels = false
@@ -69,76 +164,42 @@ func (s *Session) Open() (err error) {
 		s.State = state
 	}
 
-	if s.wsConn != nil {
-		err = ErrWSAlreadyOpen
-		return
+	// Now Discord should send us a READY or RESUMED packet.
+	mt, m, err = s.wsConn.ReadMessage()
+	if err != nil {
+		return err
 	}
+	e, err = s.onEvent(mt, m)
+	if err != nil {
+		return err
+	}
+	if e.Type != `READY` && e.Type != `RESUMED` {
+		// This is not fatal, but it does not follow their API documentation.
+		s.log(LogWarning, "Expected READY/RESUMED, instead got:\n%#v\n", e)
+	}
+	s.log(LogInformational, "First Packet:\n%#v\n", e)
 
+	s.log(LogInformational, "We are now connected to Discord, emitting connect event")
+	s.handleEvent(connectEventType, &Connect{})
+
+	// A VoiceConnections map is a hard requirement for Voice.
+	// XXX: can this be moved to when opening a voice connection?
 	if s.VoiceConnections == nil {
 		s.log(LogInformational, "creating new VoiceConnections map")
 		s.VoiceConnections = make(map[string]*VoiceConnection)
 	}
 
-	// Get the gateway to use for the Websocket connection
-	if s.gateway == "" {
-		s.gateway, err = s.Gateway()
-		if err != nil {
-			return
-		}
-
-		// Add the version and encoding to the URL
-		s.gateway = fmt.Sprintf("%s?v=5&encoding=json", s.gateway)
-	}
-
-	header := http.Header{}
-	header.Add("accept-encoding", "zlib")
-
-	s.log(LogInformational, "connecting to gateway %s", s.gateway)
-	s.wsConn, _, err = websocket.DefaultDialer.Dial(s.gateway, header)
-	if err != nil {
-		s.log(LogWarning, "error connecting to gateway %s, %s", s.gateway, err)
-		s.gateway = "" // clear cached gateway
-		// TODO: should we add a retry block here?
-		return
-	}
-
-	sequence := atomic.LoadInt64(s.sequence)
-	if s.sessionID != "" && sequence > 0 {
-
-		p := resumePacket{}
-		p.Op = 6
-		p.Data.Token = s.Token
-		p.Data.SessionID = s.sessionID
-		p.Data.Sequence = sequence
-
-		s.log(LogInformational, "sending resume packet to gateway")
-		err = s.wsConn.WriteJSON(p)
-		if err != nil {
-			s.log(LogWarning, "error sending gateway resume packet, %s, %s", s.gateway, err)
-			return
-		}
-
-	} else {
-
-		err = s.identify()
-		if err != nil {
-			s.log(LogWarning, "error sending gateway identify packet, %s, %s", s.gateway, err)
-			return
-		}
-	}
-
-	// Create listening outside of listen, as it needs to happen inside the mutex
-	// lock.
+	// Create listening chan outside of listen, as it needs to happen inside the
+	// mutex lock and needs to exist before calling heartbeat and listen
+	// go rountines.
 	s.listening = make(chan interface{})
+
+	// Start sending heartbeats and reading messages from Discord.
+	go s.heartbeat(s.wsConn, s.listening, h.HeartbeatInterval)
 	go s.listen(s.wsConn, s.listening)
 
-	s.Unlock()
-
-	s.log(LogInformational, "emit connect event")
-	s.handleEvent(connectEventType, &Connect{})
-
 	s.log(LogInformational, "exiting")
-	return
+	return nil
 }
 
 // listen polls the websocket connection for events, it will stop when the
@@ -199,10 +260,13 @@ type helloOp struct {
 	Trace             []string      `json:"_trace"`
 }
 
+// FailedHeartbeatAcks is the Number of heartbeat intervals to wait until forcing a connection restart.
+const FailedHeartbeatAcks time.Duration = 5 * time.Millisecond
+
 // heartbeat sends regular heartbeats to Discord so it knows the client
 // is still connected.  If you do not send these heartbeats Discord will
 // disconnect the websocket connection after a few seconds.
-func (s *Session) heartbeat(wsConn *websocket.Conn, listening <-chan interface{}, i time.Duration) {
+func (s *Session) heartbeat(wsConn *websocket.Conn, listening <-chan interface{}, heartbeatIntervalMsec time.Duration) {
 
 	s.log(LogInformational, "called")
 
@@ -211,20 +275,26 @@ func (s *Session) heartbeat(wsConn *websocket.Conn, listening <-chan interface{}
 	}
 
 	var err error
-	ticker := time.NewTicker(i * time.Millisecond)
+	ticker := time.NewTicker(heartbeatIntervalMsec * time.Millisecond)
 	defer ticker.Stop()
 
 	for {
+		s.RLock()
+		last := s.LastHeartbeatAck
+		s.RUnlock()
 		sequence := atomic.LoadInt64(s.sequence)
 		s.log(LogInformational, "sending gateway websocket heartbeat seq %d", sequence)
 		s.wsMutex.Lock()
 		err = wsConn.WriteJSON(heartbeatOp{1, sequence})
 		s.wsMutex.Unlock()
-		if err != nil {
-			s.log(LogError, "error sending heartbeat to gateway %s, %s", s.gateway, err)
-			s.Lock()
-			s.DataReady = false
-			s.Unlock()
+		if err != nil || time.Now().UTC().Sub(last) > (heartbeatIntervalMsec*FailedHeartbeatAcks) {
+			if err != nil {
+				s.log(LogError, "error sending heartbeat to gateway %s, %s", s.gateway, err)
+			} else {
+				s.log(LogError, "haven't gotten a heartbeat ACK in %v, triggering a reconnection", time.Now().UTC().Sub(last))
+			}
+			s.Close()
+			s.reconnect()
 			return
 		}
 		s.Lock()
@@ -240,14 +310,17 @@ func (s *Session) heartbeat(wsConn *websocket.Conn, listening <-chan interface{}
 	}
 }
 
-type updateStatusData struct {
-	IdleSince *int  `json:"idle_since"`
-	Game      *Game `json:"game"`
+// UpdateStatusData ia provided to UpdateStatusComplex()
+type UpdateStatusData struct {
+	IdleSince *int   `json:"since"`
+	Game      *Game  `json:"game"`
+	AFK       bool   `json:"afk"`
+	Status    string `json:"status"`
 }
 
 type updateStatusOp struct {
 	Op   int              `json:"op"`
-	Data updateStatusData `json:"d"`
+	Data UpdateStatusData `json:"d"`
 }
 
 // UpdateStreamingStatus is used to update the user's streaming status.
@@ -259,21 +332,18 @@ func (s *Session) UpdateStreamingStatus(idle int, game string, url string) (err
 
 	s.log(LogInformational, "called")
 
-	s.RLock()
-	defer s.RUnlock()
-	if s.wsConn == nil {
-		return ErrWSNotFound
+	usd := UpdateStatusData{
+		Status: "online",
 	}
 
-	var usd updateStatusData
 	if idle > 0 {
 		usd.IdleSince = &idle
 	}
 
 	if game != "" {
-		gameType := 0
+		gameType := GameTypeGame
 		if url != "" {
-			gameType = 1
+			gameType = GameTypeStreaming
 		}
 		usd.Game = &Game{
 			Name: game,
@@ -282,6 +352,18 @@ func (s *Session) UpdateStreamingStatus(idle int, game string, url string) (err
 		}
 	}
 
+	return s.UpdateStatusComplex(usd)
+}
+
+// UpdateStatusComplex allows for sending the raw status update data untouched by discordgo.
+func (s *Session) UpdateStatusComplex(usd UpdateStatusData) (err error) {
+
+	s.RLock()
+	defer s.RUnlock()
+	if s.wsConn == nil {
+		return ErrWSNotFound
+	}
+
 	s.wsMutex.Lock()
 	err = s.wsConn.WriteJSON(updateStatusOp{3, usd})
 	s.wsMutex.Unlock()
@@ -343,9 +425,7 @@ func (s *Session) RequestGuildMembers(guildID, query string, limit int) (err err
 //
 // If you use the AddHandler() function to register a handler for the
 // "OnEvent" event then all events will be passed to that handler.
-//
-// TODO: You may also register a custom event handler entirely using...
-func (s *Session) onEvent(messageType int, message []byte) {
+func (s *Session) onEvent(messageType int, message []byte) (*Event, error) {
 
 	var err error
 	var reader io.Reader
@@ -357,7 +437,7 @@ func (s *Session) onEvent(messageType int, message []byte) {
 		z, err2 := zlib.NewReader(reader)
 		if err2 != nil {
 			s.log(LogError, "error uncompressing websocket message, %s", err)
-			return
+			return nil, err2
 		}
 
 		defer func() {
@@ -375,7 +455,7 @@ func (s *Session) onEvent(messageType int, message []byte) {
 	decoder := json.NewDecoder(reader)
 	if err = decoder.Decode(&e); err != nil {
 		s.log(LogError, "error decoding websocket message, %s", err)
-		return
+		return e, err
 	}
 
 	s.log(LogDebug, "Op: %d, Seq: %d, Type: %s, Data: %s\n\n", e.Operation, e.Sequence, e.Type, string(e.RawData))
@@ -389,16 +469,19 @@ func (s *Session) onEvent(messageType int, message []byte) {
 		s.wsMutex.Unlock()
 		if err != nil {
 			s.log(LogError, "error sending heartbeat in response to Op1")
-			return
+			return e, err
 		}
 
-		return
+		return e, nil
 	}
 
 	// Reconnect
 	// Must immediately disconnect from gateway and reconnect to new gateway.
 	if e.Operation == 7 {
-		// TODO
+		s.log(LogInformational, "Closing and reconnecting in response to Op7")
+		s.Close()
+		s.reconnect()
+		return e, nil
 	}
 
 	// Invalid Session
@@ -410,20 +493,23 @@ func (s *Session) onEvent(messageType int, message []byte) {
 		err = s.identify()
 		if err != nil {
 			s.log(LogWarning, "error sending gateway identify packet, %s, %s", s.gateway, err)
-			return
+			return e, err
 		}
 
-		return
+		return e, nil
 	}
 
 	if e.Operation == 10 {
-		var h helloOp
-		if err = json.Unmarshal(e.RawData, &h); err != nil {
-			s.log(LogError, "error unmarshalling helloOp, %s", err)
-		} else {
-			go s.heartbeat(s.wsConn, s.listening, h.HeartbeatInterval)
-		}
-		return
+		// Op10 is handled by Open()
+		return e, nil
+	}
+
+	if e.Operation == 11 {
+		s.Lock()
+		s.LastHeartbeatAck = time.Now().UTC()
+		s.Unlock()
+		s.log(LogInformational, "got heartbeat ACK")
+		return e, nil
 	}
 
 	// Do not try to Dispatch a non-Dispatch Message
@@ -431,7 +517,7 @@ func (s *Session) onEvent(messageType int, message []byte) {
 		// But we probably should be doing something with them.
 		// TEMP
 		s.log(LogWarning, "unknown Op: %d, Seq: %d, Type: %s, Data: %s, message: %s", e.Operation, e.Sequence, e.Type, string(e.RawData), string(message))
-		return
+		return e, nil
 	}
 
 	// Store the message sequence
@@ -460,6 +546,8 @@ func (s *Session) onEvent(messageType int, message []byte) {
 
 	// For legacy reasons, we send the raw event also, this could be useful for handling unknown events.
 	s.handleEvent(eventEventType, e)
+
+	return e, nil
 }
 
 // ------------------------------------------------------------------------------------------------
@@ -585,7 +673,7 @@ func (s *Session) onVoiceServerUpdate(st *VoiceServerUpdate) {
 	voice.GuildID = st.GuildID
 	voice.Unlock()
 
-	// Open a conenction to the voice server
+	// Open a connection to the voice server
 	err := voice.open()
 	if err != nil {
 		s.log(LogError, "onVoiceServerUpdate voice.open, %s", err)
@@ -688,6 +776,13 @@ func (s *Session) reconnect() {
 				return
 			}
 
+			// Certain race conditions can call reconnect() twice. If this happens, we
+			// just break out of the reconnect loop
+			if err == ErrWSAlreadyOpen {
+				s.log(LogInformational, "Websocket already exists, no need to reconnect")
+				return
+			}
+
 			s.log(LogError, "error reconnecting to gateway, %s", err)
 
 			<-time.After(wait * time.Second)

vendor/github.com/coreos/etcd/client/LICENSE

@@ -0,0 +1,202 @@
+
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

vendor/github.com/coreos/etcd/client/auth_role.go

@@ -0,0 +1,236 @@
+// Copyright 2015 The etcd Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package client
+
+import (
+	"bytes"
+	"context"
+	"encoding/json"
+	"net/http"
+	"net/url"
+)
+
+type Role struct {
+	Role        string       `json:"role"`
+	Permissions Permissions  `json:"permissions"`
+	Grant       *Permissions `json:"grant,omitempty"`
+	Revoke      *Permissions `json:"revoke,omitempty"`
+}
+
+type Permissions struct {
+	KV rwPermission `json:"kv"`
+}
+
+type rwPermission struct {
+	Read  []string `json:"read"`
+	Write []string `json:"write"`
+}
+
+type PermissionType int
+
+const (
+	ReadPermission PermissionType = iota
+	WritePermission
+	ReadWritePermission
+)
+
+// NewAuthRoleAPI constructs a new AuthRoleAPI that uses HTTP to
+// interact with etcd's role creation and modification features.
+func NewAuthRoleAPI(c Client) AuthRoleAPI {
+	return &httpAuthRoleAPI{
+		client: c,
+	}
+}
+
+type AuthRoleAPI interface {
+	// AddRole adds a role.
+	AddRole(ctx context.Context, role string) error
+
+	// RemoveRole removes a role.
+	RemoveRole(ctx context.Context, role string) error
+
+	// GetRole retrieves role details.
+	GetRole(ctx context.Context, role string) (*Role, error)
+
+	// GrantRoleKV grants a role some permission prefixes for the KV store.
+	GrantRoleKV(ctx context.Context, role string, prefixes []string, permType PermissionType) (*Role, error)
+
+	// RevokeRoleKV revokes some permission prefixes for a role on the KV store.
+	RevokeRoleKV(ctx context.Context, role string, prefixes []string, permType PermissionType) (*Role, error)
+
+	// ListRoles lists roles.
+	ListRoles(ctx context.Context) ([]string, error)
+}
+
+type httpAuthRoleAPI struct {
+	client httpClient
+}
+
+type authRoleAPIAction struct {
+	verb string
+	name string
+	role *Role
+}
+
+type authRoleAPIList struct{}
+
+func (list *authRoleAPIList) HTTPRequest(ep url.URL) *http.Request {
+	u := v2AuthURL(ep, "roles", "")
+	req, _ := http.NewRequest("GET", u.String(), nil)
+	req.Header.Set("Content-Type", "application/json")
+	return req
+}
+
+func (l *authRoleAPIAction) HTTPRequest(ep url.URL) *http.Request {
+	u := v2AuthURL(ep, "roles", l.name)
+	if l.role == nil {
+		req, _ := http.NewRequest(l.verb, u.String(), nil)
+		return req
+	}
+	b, err := json.Marshal(l.role)
+	if err != nil {
+		panic(err)
+	}
+	body := bytes.NewReader(b)
+	req, _ := http.NewRequest(l.verb, u.String(), body)
+	req.Header.Set("Content-Type", "application/json")
+	return req
+}
+
+func (r *httpAuthRoleAPI) ListRoles(ctx context.Context) ([]string, error) {
+	resp, body, err := r.client.Do(ctx, &authRoleAPIList{})
+	if err != nil {
+		return nil, err
+	}
+	if err = assertStatusCode(resp.StatusCode, http.StatusOK); err != nil {
+		return nil, err
+	}
+	var roleList struct {
+		Roles []Role `json:"roles"`
+	}
+	if err = json.Unmarshal(body, &roleList); err != nil {
+		return nil, err
+	}
+	ret := make([]string, 0, len(roleList.Roles))
+	for _, r := range roleList.Roles {
+		ret = append(ret, r.Role)
+	}
+	return ret, nil
+}
+
+func (r *httpAuthRoleAPI) AddRole(ctx context.Context, rolename string) error {
+	role := &Role{
+		Role: rolename,
+	}
+	return r.addRemoveRole(ctx, &authRoleAPIAction{
+		verb: "PUT",
+		name: rolename,
+		role: role,
+	})
+}
+
+func (r *httpAuthRoleAPI) RemoveRole(ctx context.Context, rolename string) error {
+	return r.addRemoveRole(ctx, &authRoleAPIAction{
+		verb: "DELETE",
+		name: rolename,
+	})
+}
+
+func (r *httpAuthRoleAPI) addRemoveRole(ctx context.Context, req *authRoleAPIAction) error {
+	resp, body, err := r.client.Do(ctx, req)
+	if err != nil {
+		return err
+	}
+	if err := assertStatusCode(resp.StatusCode, http.StatusOK, http.StatusCreated); err != nil {
+		var sec authError
+		err := json.Unmarshal(body, &sec)
+		if err != nil {
+			return err
+		}
+		return sec
+	}
+	return nil
+}
+
+func (r *httpAuthRoleAPI) GetRole(ctx context.Context, rolename string) (*Role, error) {
+	return r.modRole(ctx, &authRoleAPIAction{
+		verb: "GET",
+		name: rolename,
+	})
+}
+
+func buildRWPermission(prefixes []string, permType PermissionType) rwPermission {
+	var out rwPermission
+	switch permType {
+	case ReadPermission:
+		out.Read = prefixes
+	case WritePermission:
+		out.Write = prefixes
+	case ReadWritePermission:
+		out.Read = prefixes
+		out.Write = prefixes
+	}
+	return out
+}
+
+func (r *httpAuthRoleAPI) GrantRoleKV(ctx context.Context, rolename string, prefixes []string, permType PermissionType) (*Role, error) {
+	rwp := buildRWPermission(prefixes, permType)
+	role := &Role{
+		Role: rolename,
+		Grant: &Permissions{
+			KV: rwp,
+		},
+	}
+	return r.modRole(ctx, &authRoleAPIAction{
+		verb: "PUT",
+		name: rolename,
+		role: role,
+	})
+}
+
+func (r *httpAuthRoleAPI) RevokeRoleKV(ctx context.Context, rolename string, prefixes []string, permType PermissionType) (*Role, error) {
+	rwp := buildRWPermission(prefixes, permType)
+	role := &Role{
+		Role: rolename,
+		Revoke: &Permissions{
+			KV: rwp,
+		},
+	}
+	return r.modRole(ctx, &authRoleAPIAction{
+		verb: "PUT",
+		name: rolename,
+		role: role,
+	})
+}
+
+func (r *httpAuthRoleAPI) modRole(ctx context.Context, req *authRoleAPIAction) (*Role, error) {
+	resp, body, err := r.client.Do(ctx, req)
+	if err != nil {
+		return nil, err
+	}
+	if err = assertStatusCode(resp.StatusCode, http.StatusOK); err != nil {
+		var sec authError
+		err = json.Unmarshal(body, &sec)
+		if err != nil {
+			return nil, err
+		}
+		return nil, sec
+	}
+	var role Role
+	if err = json.Unmarshal(body, &role); err != nil {
+		return nil, err
+	}
+	return &role, nil
+}

vendor/github.com/coreos/etcd/client/auth_user.go

@@ -0,0 +1,319 @@
+// Copyright 2015 The etcd Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package client
+
+import (
+	"bytes"
+	"context"
+	"encoding/json"
+	"net/http"
+	"net/url"
+	"path"
+)
+
+var (
+	defaultV2AuthPrefix = "/v2/auth"
+)
+
+type User struct {
+	User     string   `json:"user"`
+	Password string   `json:"password,omitempty"`
+	Roles    []string `json:"roles"`
+	Grant    []string `json:"grant,omitempty"`
+	Revoke   []string `json:"revoke,omitempty"`
+}
+
+// userListEntry is the user representation given by the server for ListUsers
+type userListEntry struct {
+	User  string `json:"user"`
+	Roles []Role `json:"roles"`
+}
+
+type UserRoles struct {
+	User  string `json:"user"`
+	Roles []Role `json:"roles"`
+}
+
+func v2AuthURL(ep url.URL, action string, name string) *url.URL {
+	if name != "" {
+		ep.Path = path.Join(ep.Path, defaultV2AuthPrefix, action, name)
+		return &ep
+	}
+	ep.Path = path.Join(ep.Path, defaultV2AuthPrefix, action)
+	return &ep
+}
+
+// NewAuthAPI constructs a new AuthAPI that uses HTTP to
+// interact with etcd's general auth features.
+func NewAuthAPI(c Client) AuthAPI {
+	return &httpAuthAPI{
+		client: c,
+	}
+}
+
+type AuthAPI interface {
+	// Enable auth.
+	Enable(ctx context.Context) error
+
+	// Disable auth.
+	Disable(ctx context.Context) error
+}
+
+type httpAuthAPI struct {
+	client httpClient
+}
+
+func (s *httpAuthAPI) Enable(ctx context.Context) error {
+	return s.enableDisable(ctx, &authAPIAction{"PUT"})
+}
+
+func (s *httpAuthAPI) Disable(ctx context.Context) error {
+	return s.enableDisable(ctx, &authAPIAction{"DELETE"})
+}
+
+func (s *httpAuthAPI) enableDisable(ctx context.Context, req httpAction) error {
+	resp, body, err := s.client.Do(ctx, req)
+	if err != nil {
+		return err
+	}
+	if err = assertStatusCode(resp.StatusCode, http.StatusOK, http.StatusCreated); err != nil {
+		var sec authError
+		err = json.Unmarshal(body, &sec)
+		if err != nil {
+			return err
+		}
+		return sec
+	}
+	return nil
+}
+
+type authAPIAction struct {
+	verb string
+}
+
+func (l *authAPIAction) HTTPRequest(ep url.URL) *http.Request {
+	u := v2AuthURL(ep, "enable", "")
+	req, _ := http.NewRequest(l.verb, u.String(), nil)
+	return req
+}
+
+type authError struct {
+	Message string `json:"message"`
+	Code    int    `json:"-"`
+}
+
+func (e authError) Error() string {
+	return e.Message
+}
+
+// NewAuthUserAPI constructs a new AuthUserAPI that uses HTTP to
+// interact with etcd's user creation and modification features.
+func NewAuthUserAPI(c Client) AuthUserAPI {
+	return &httpAuthUserAPI{
+		client: c,
+	}
+}
+
+type AuthUserAPI interface {
+	// AddUser adds a user.
+	AddUser(ctx context.Context, username string, password string) error
+
+	// RemoveUser removes a user.
+	RemoveUser(ctx context.Context, username string) error
+
+	// GetUser retrieves user details.
+	GetUser(ctx context.Context, username string) (*User, error)
+
+	// GrantUser grants a user some permission roles.
+	GrantUser(ctx context.Context, username string, roles []string) (*User, error)
+
+	// RevokeUser revokes some permission roles from a user.
+	RevokeUser(ctx context.Context, username string, roles []string) (*User, error)
+
+	// ChangePassword changes the user's password.
+	ChangePassword(ctx context.Context, username string, password string) (*User, error)
+
+	// ListUsers lists the users.
+	ListUsers(ctx context.Context) ([]string, error)
+}
+
+type httpAuthUserAPI struct {
+	client httpClient
+}
+
+type authUserAPIAction struct {
+	verb     string
+	username string
+	user     *User
+}
+
+type authUserAPIList struct{}
+
+func (list *authUserAPIList) HTTPRequest(ep url.URL) *http.Request {
+	u := v2AuthURL(ep, "users", "")
+	req, _ := http.NewRequest("GET", u.String(), nil)
+	req.Header.Set("Content-Type", "application/json")
+	return req
+}
+
+func (l *authUserAPIAction) HTTPRequest(ep url.URL) *http.Request {
+	u := v2AuthURL(ep, "users", l.username)
+	if l.user == nil {
+		req, _ := http.NewRequest(l.verb, u.String(), nil)
+		return req
+	}
+	b, err := json.Marshal(l.user)
+	if err != nil {
+		panic(err)
+	}
+	body := bytes.NewReader(b)
+	req, _ := http.NewRequest(l.verb, u.String(), body)
+	req.Header.Set("Content-Type", "application/json")
+	return req
+}
+
+func (u *httpAuthUserAPI) ListUsers(ctx context.Context) ([]string, error) {
+	resp, body, err := u.client.Do(ctx, &authUserAPIList{})
+	if err != nil {
+		return nil, err
+	}
+	if err = assertStatusCode(resp.StatusCode, http.StatusOK); err != nil {
+		var sec authError
+		err = json.Unmarshal(body, &sec)
+		if err != nil {
+			return nil, err
+		}
+		return nil, sec
+	}
+
+	var userList struct {
+		Users []userListEntry `json:"users"`
+	}
+
+	if err = json.Unmarshal(body, &userList); err != nil {
+		return nil, err
+	}
+
+	ret := make([]string, 0, len(userList.Users))
+	for _, u := range userList.Users {
+		ret = append(ret, u.User)
+	}
+	return ret, nil
+}
+
+func (u *httpAuthUserAPI) AddUser(ctx context.Context, username string, password string) error {
+	user := &User{
+		User:     username,
+		Password: password,
+	}
+	return u.addRemoveUser(ctx, &authUserAPIAction{
+		verb:     "PUT",
+		username: username,
+		user:     user,
+	})
+}
+
+func (u *httpAuthUserAPI) RemoveUser(ctx context.Context, username string) error {
+	return u.addRemoveUser(ctx, &authUserAPIAction{
+		verb:     "DELETE",
+		username: username,
+	})
+}
+
+func (u *httpAuthUserAPI) addRemoveUser(ctx context.Context, req *authUserAPIAction) error {
+	resp, body, err := u.client.Do(ctx, req)
+	if err != nil {
+		return err
+	}
+	if err = assertStatusCode(resp.StatusCode, http.StatusOK, http.StatusCreated); err != nil {
+		var sec authError
+		err = json.Unmarshal(body, &sec)
+		if err != nil {
+			return err
+		}
+		return sec
+	}
+	return nil
+}
+
+func (u *httpAuthUserAPI) GetUser(ctx context.Context, username string) (*User, error) {
+	return u.modUser(ctx, &authUserAPIAction{
+		verb:     "GET",
+		username: username,
+	})
+}
+
+func (u *httpAuthUserAPI) GrantUser(ctx context.Context, username string, roles []string) (*User, error) {
+	user := &User{
+		User:  username,
+		Grant: roles,
+	}
+	return u.modUser(ctx, &authUserAPIAction{
+		verb:     "PUT",
+		username: username,
+		user:     user,
+	})
+}
+
+func (u *httpAuthUserAPI) RevokeUser(ctx context.Context, username string, roles []string) (*User, error) {
+	user := &User{
+		User:   username,
+		Revoke: roles,
+	}
+	return u.modUser(ctx, &authUserAPIAction{
+		verb:     "PUT",
+		username: username,
+		user:     user,
+	})
+}
+
+func (u *httpAuthUserAPI) ChangePassword(ctx context.Context, username string, password string) (*User, error) {
+	user := &User{
+		User:     username,
+		Password: password,
+	}
+	return u.modUser(ctx, &authUserAPIAction{
+		verb:     "PUT",
+		username: username,
+		user:     user,
+	})
+}
+
+func (u *httpAuthUserAPI) modUser(ctx context.Context, req *authUserAPIAction) (*User, error) {
+	resp, body, err := u.client.Do(ctx, req)
+	if err != nil {
+		return nil, err
+	}
+	if err = assertStatusCode(resp.StatusCode, http.StatusOK); err != nil {
+		var sec authError
+		err = json.Unmarshal(body, &sec)
+		if err != nil {
+			return nil, err
+		}
+		return nil, sec
+	}
+	var user User
+	if err = json.Unmarshal(body, &user); err != nil {
+		var userR UserRoles
+		if urerr := json.Unmarshal(body, &userR); urerr != nil {
+			return nil, err
+		}
+		user.User = userR.User
+		for _, r := range userR.Roles {
+			user.Roles = append(user.Roles, r.Role)
+		}
+	}
+	return &user, nil
+}

vendor/github.com/coreos/etcd/client/cancelreq.go

@@ -0,0 +1,18 @@
+// Copyright 2015 The Go Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+
+// borrowed from golang/net/context/ctxhttp/cancelreq.go
+
+package client
+
+import "net/http"
+
+func requestCanceler(tr CancelableTransport, req *http.Request) func() {
+	ch := make(chan struct{})
+	req.Cancel = ch
+
+	return func() {
+		close(ch)
+	}
+}

vendor/github.com/coreos/etcd/client/client.go

@@ -0,0 +1,710 @@
+// Copyright 2015 The etcd Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package client
+
+import (
+	"context"
+	"encoding/json"
+	"errors"
+	"fmt"
+	"io/ioutil"
+	"math/rand"
+	"net"
+	"net/http"
+	"net/url"
+	"sort"
+	"strconv"
+	"sync"
+	"time"
+
+	"github.com/coreos/etcd/version"
+)
+
+var (
+	ErrNoEndpoints           = errors.New("client: no endpoints available")
+	ErrTooManyRedirects      = errors.New("client: too many redirects")
+	ErrClusterUnavailable    = errors.New("client: etcd cluster is unavailable or misconfigured")
+	ErrNoLeaderEndpoint      = errors.New("client: no leader endpoint available")
+	errTooManyRedirectChecks = errors.New("client: too many redirect checks")
+
+	// oneShotCtxValue is set on a context using WithValue(&oneShotValue) so
+	// that Do() will not retry a request
+	oneShotCtxValue interface{}
+)
+
+var DefaultRequestTimeout = 5 * time.Second
+
+var DefaultTransport CancelableTransport = &http.Transport{
+	Proxy: http.ProxyFromEnvironment,
+	Dial: (&net.Dialer{
+		Timeout:   30 * time.Second,
+		KeepAlive: 30 * time.Second,
+	}).Dial,
+	TLSHandshakeTimeout: 10 * time.Second,
+}
+
+type EndpointSelectionMode int
+
+const (
+	// EndpointSelectionRandom is the default value of the 'SelectionMode'.
+	// As the name implies, the client object will pick a node from the members
+	// of the cluster in a random fashion. If the cluster has three members, A, B,
+	// and C, the client picks any node from its three members as its request
+	// destination.
+	EndpointSelectionRandom EndpointSelectionMode = iota
+
+	// If 'SelectionMode' is set to 'EndpointSelectionPrioritizeLeader',
+	// requests are sent directly to the cluster leader. This reduces
+	// forwarding roundtrips compared to making requests to etcd followers
+	// who then forward them to the cluster leader. In the event of a leader
+	// failure, however, clients configured this way cannot prioritize among
+	// the remaining etcd followers. Therefore, when a client sets 'SelectionMode'
+	// to 'EndpointSelectionPrioritizeLeader', it must use 'client.AutoSync()' to
+	// maintain its knowledge of current cluster state.
+	//
+	// This mode should be used with Client.AutoSync().
+	EndpointSelectionPrioritizeLeader
+)
+
+type Config struct {
+	// Endpoints defines a set of URLs (schemes, hosts and ports only)
+	// that can be used to communicate with a logical etcd cluster. For
+	// example, a three-node cluster could be provided like so:
+	//
+	// 	Endpoints: []string{
+	//		"http://node1.example.com:2379",
+	//		"http://node2.example.com:2379",
+	//		"http://node3.example.com:2379",
+	//	}
+	//
+	// If multiple endpoints are provided, the Client will attempt to
+	// use them all in the event that one or more of them are unusable.
+	//
+	// If Client.Sync is ever called, the Client may cache an alternate
+	// set of endpoints to continue operation.
+	Endpoints []string
+
+	// Transport is used by the Client to drive HTTP requests. If not
+	// provided, DefaultTransport will be used.
+	Transport CancelableTransport
+
+	// CheckRedirect specifies the policy for handling HTTP redirects.
+	// If CheckRedirect is not nil, the Client calls it before
+	// following an HTTP redirect. The sole argument is the number of
+	// requests that have already been made. If CheckRedirect returns
+	// an error, Client.Do will not make any further requests and return
+	// the error back it to the caller.
+	//
+	// If CheckRedirect is nil, the Client uses its default policy,
+	// which is to stop after 10 consecutive requests.
+	CheckRedirect CheckRedirectFunc
+
+	// Username specifies the user credential to add as an authorization header
+	Username string
+
+	// Password is the password for the specified user to add as an authorization header
+	// to the request.
+	Password string
+
+	// HeaderTimeoutPerRequest specifies the time limit to wait for response
+	// header in a single request made by the Client. The timeout includes
+	// connection time, any redirects, and header wait time.
+	//
+	// For non-watch GET request, server returns the response body immediately.
+	// For PUT/POST/DELETE request, server will attempt to commit request
+	// before responding, which is expected to take `100ms + 2 * RTT`.
+	// For watch request, server returns the header immediately to notify Client
+	// watch start. But if server is behind some kind of proxy, the response
+	// header may be cached at proxy, and Client cannot rely on this behavior.
+	//
+	// Especially, wait request will ignore this timeout.
+	//
+	// One API call may send multiple requests to different etcd servers until it
+	// succeeds. Use context of the API to specify the overall timeout.
+	//
+	// A HeaderTimeoutPerRequest of zero means no timeout.
+	HeaderTimeoutPerRequest time.Duration
+
+	// SelectionMode is an EndpointSelectionMode enum that specifies the
+	// policy for choosing the etcd cluster node to which requests are sent.
+	SelectionMode EndpointSelectionMode
+}
+
+func (cfg *Config) transport() CancelableTransport {
+	if cfg.Transport == nil {
+		return DefaultTransport
+	}
+	return cfg.Transport
+}
+
+func (cfg *Config) checkRedirect() CheckRedirectFunc {
+	if cfg.CheckRedirect == nil {
+		return DefaultCheckRedirect
+	}
+	return cfg.CheckRedirect
+}
+
+// CancelableTransport mimics net/http.Transport, but requires that
+// the object also support request cancellation.
+type CancelableTransport interface {
+	http.RoundTripper
+	CancelRequest(req *http.Request)
+}
+
+type CheckRedirectFunc func(via int) error
+
+// DefaultCheckRedirect follows up to 10 redirects, but no more.
+var DefaultCheckRedirect CheckRedirectFunc = func(via int) error {
+	if via > 10 {
+		return ErrTooManyRedirects
+	}
+	return nil
+}
+
+type Client interface {
+	// Sync updates the internal cache of the etcd cluster's membership.
+	Sync(context.Context) error
+
+	// AutoSync periodically calls Sync() every given interval.
+	// The recommended sync interval is 10 seconds to 1 minute, which does
+	// not bring too much overhead to server and makes client catch up the
+	// cluster change in time.
+	//
+	// The example to use it:
+	//
+	//  for {
+	//      err := client.AutoSync(ctx, 10*time.Second)
+	//      if err == context.DeadlineExceeded || err == context.Canceled {
+	//          break
+	//      }
+	//      log.Print(err)
+	//  }
+	AutoSync(context.Context, time.Duration) error
+
+	// Endpoints returns a copy of the current set of API endpoints used
+	// by Client to resolve HTTP requests. If Sync has ever been called,
+	// this may differ from the initial Endpoints provided in the Config.
+	Endpoints() []string
+
+	// SetEndpoints sets the set of API endpoints used by Client to resolve
+	// HTTP requests. If the given endpoints are not valid, an error will be
+	// returned
+	SetEndpoints(eps []string) error
+
+	// GetVersion retrieves the current etcd server and cluster version
+	GetVersion(ctx context.Context) (*version.Versions, error)
+
+	httpClient
+}
+
+func New(cfg Config) (Client, error) {
+	c := &httpClusterClient{
+		clientFactory: newHTTPClientFactory(cfg.transport(), cfg.checkRedirect(), cfg.HeaderTimeoutPerRequest),
+		rand:          rand.New(rand.NewSource(int64(time.Now().Nanosecond()))),
+		selectionMode: cfg.SelectionMode,
+	}
+	if cfg.Username != "" {
+		c.credentials = &credentials{
+			username: cfg.Username,
+			password: cfg.Password,
+		}
+	}
+	if err := c.SetEndpoints(cfg.Endpoints); err != nil {
+		return nil, err
+	}
+	return c, nil
+}
+
+type httpClient interface {
+	Do(context.Context, httpAction) (*http.Response, []byte, error)
+}
+
+func newHTTPClientFactory(tr CancelableTransport, cr CheckRedirectFunc, headerTimeout time.Duration) httpClientFactory {
+	return func(ep url.URL) httpClient {
+		return &redirectFollowingHTTPClient{
+			checkRedirect: cr,
+			client: &simpleHTTPClient{
+				transport:     tr,
+				endpoint:      ep,
+				headerTimeout: headerTimeout,
+			},
+		}
+	}
+}
+
+type credentials struct {
+	username string
+	password string
+}
+
+type httpClientFactory func(url.URL) httpClient
+
+type httpAction interface {
+	HTTPRequest(url.URL) *http.Request
+}
+
+type httpClusterClient struct {
+	clientFactory httpClientFactory
+	endpoints     []url.URL
+	pinned        int
+	credentials   *credentials
+	sync.RWMutex
+	rand          *rand.Rand
+	selectionMode EndpointSelectionMode
+}
+
+func (c *httpClusterClient) getLeaderEndpoint(ctx context.Context, eps []url.URL) (string, error) {
+	ceps := make([]url.URL, len(eps))
+	copy(ceps, eps)
+
+	// To perform a lookup on the new endpoint list without using the current
+	// client, we'll copy it
+	clientCopy := &httpClusterClient{
+		clientFactory: c.clientFactory,
+		credentials:   c.credentials,
+		rand:          c.rand,
+
+		pinned:    0,
+		endpoints: ceps,
+	}
+
+	mAPI := NewMembersAPI(clientCopy)
+	leader, err := mAPI.Leader(ctx)
+	if err != nil {
+		return "", err
+	}
+	if len(leader.ClientURLs) == 0 {
+		return "", ErrNoLeaderEndpoint
+	}
+
+	return leader.ClientURLs[0], nil // TODO: how to handle multiple client URLs?
+}
+
+func (c *httpClusterClient) parseEndpoints(eps []string) ([]url.URL, error) {
+	if len(eps) == 0 {
+		return []url.URL{}, ErrNoEndpoints
+	}
+
+	neps := make([]url.URL, len(eps))
+	for i, ep := range eps {
+		u, err := url.Parse(ep)
+		if err != nil {
+			return []url.URL{}, err
+		}
+		neps[i] = *u
+	}
+	return neps, nil
+}
+
+func (c *httpClusterClient) SetEndpoints(eps []string) error {
+	neps, err := c.parseEndpoints(eps)
+	if err != nil {
+		return err
+	}
+
+	c.Lock()
+	defer c.Unlock()
+
+	c.endpoints = shuffleEndpoints(c.rand, neps)
+	// We're not doing anything for PrioritizeLeader here. This is
+	// due to not having a context meaning we can't call getLeaderEndpoint
+	// However, if you're using PrioritizeLeader, you've already been told
+	// to regularly call sync, where we do have a ctx, and can figure the
+	// leader. PrioritizeLeader is also quite a loose guarantee, so deal
+	// with it
+	c.pinned = 0
+
+	return nil
+}
+
+func (c *httpClusterClient) Do(ctx context.Context, act httpAction) (*http.Response, []byte, error) {
+	action := act
+	c.RLock()
+	leps := len(c.endpoints)
+	eps := make([]url.URL, leps)
+	n := copy(eps, c.endpoints)
+	pinned := c.pinned
+
+	if c.credentials != nil {
+		action = &authedAction{
+			act:         act,
+			credentials: *c.credentials,
+		}
+	}
+	c.RUnlock()
+
+	if leps == 0 {
+		return nil, nil, ErrNoEndpoints
+	}
+
+	if leps != n {
+		return nil, nil, errors.New("unable to pick endpoint: copy failed")
+	}
+
+	var resp *http.Response
+	var body []byte
+	var err error
+	cerr := &ClusterError{}
+	isOneShot := ctx.Value(&oneShotCtxValue) != nil
+
+	for i := pinned; i < leps+pinned; i++ {
+		k := i % leps
+		hc := c.clientFactory(eps[k])
+		resp, body, err = hc.Do(ctx, action)
+		if err != nil {
+			cerr.Errors = append(cerr.Errors, err)
+			if err == ctx.Err() {
+				return nil, nil, ctx.Err()
+			}
+			if err == context.Canceled || err == context.DeadlineExceeded {
+				return nil, nil, err
+			}
+		} else if resp.StatusCode/100 == 5 {
+			switch resp.StatusCode {
+			case http.StatusInternalServerError, http.StatusServiceUnavailable:
+				// TODO: make sure this is a no leader response
+				cerr.Errors = append(cerr.Errors, fmt.Errorf("client: etcd member %s has no leader", eps[k].String()))
+			default:
+				cerr.Errors = append(cerr.Errors, fmt.Errorf("client: etcd member %s returns server error [%s]", eps[k].String(), http.StatusText(resp.StatusCode)))
+			}
+			err = cerr.Errors[0]
+		}
+		if err != nil {
+			if !isOneShot {
+				continue
+			}
+			c.Lock()
+			c.pinned = (k + 1) % leps
+			c.Unlock()
+			return nil, nil, err
+		}
+		if k != pinned {
+			c.Lock()
+			c.pinned = k
+			c.Unlock()
+		}
+		return resp, body, nil
+	}
+
+	return nil, nil, cerr
+}
+
+func (c *httpClusterClient) Endpoints() []string {
+	c.RLock()
+	defer c.RUnlock()
+
+	eps := make([]string, len(c.endpoints))
+	for i, ep := range c.endpoints {
+		eps[i] = ep.String()
+	}
+
+	return eps
+}
+
+func (c *httpClusterClient) Sync(ctx context.Context) error {
+	mAPI := NewMembersAPI(c)
+	ms, err := mAPI.List(ctx)
+	if err != nil {
+		return err
+	}
+
+	var eps []string
+	for _, m := range ms {
+		eps = append(eps, m.ClientURLs...)
+	}
+
+	neps, err := c.parseEndpoints(eps)
+	if err != nil {
+		return err
+	}
+
+	npin := 0
+
+	switch c.selectionMode {
+	case EndpointSelectionRandom:
+		c.RLock()
+		eq := endpointsEqual(c.endpoints, neps)
+		c.RUnlock()
+
+		if eq {
+			return nil
+		}
+		// When items in the endpoint list changes, we choose a new pin
+		neps = shuffleEndpoints(c.rand, neps)
+	case EndpointSelectionPrioritizeLeader:
+		nle, err := c.getLeaderEndpoint(ctx, neps)
+		if err != nil {
+			return ErrNoLeaderEndpoint
+		}
+
+		for i, n := range neps {
+			if n.String() == nle {
+				npin = i
+				break
+			}
+		}
+	default:
+		return fmt.Errorf("invalid endpoint selection mode: %d", c.selectionMode)
+	}
+
+	c.Lock()
+	defer c.Unlock()
+	c.endpoints = neps
+	c.pinned = npin
+
+	return nil
+}
+
+func (c *httpClusterClient) AutoSync(ctx context.Context, interval time.Duration) error {
+	ticker := time.NewTicker(interval)
+	defer ticker.Stop()
+	for {
+		err := c.Sync(ctx)
+		if err != nil {
+			return err
+		}
+		select {
+		case <-ctx.Done():
+			return ctx.Err()
+		case <-ticker.C:
+		}
+	}
+}
+
+func (c *httpClusterClient) GetVersion(ctx context.Context) (*version.Versions, error) {
+	act := &getAction{Prefix: "/version"}
+
+	resp, body, err := c.Do(ctx, act)
+	if err != nil {
+		return nil, err
+	}
+
+	switch resp.StatusCode {
+	case http.StatusOK:
+		if len(body) == 0 {
+			return nil, ErrEmptyBody
+		}
+		var vresp version.Versions
+		if err := json.Unmarshal(body, &vresp); err != nil {
+			return nil, ErrInvalidJSON
+		}
+		return &vresp, nil
+	default:
+		var etcdErr Error
+		if err := json.Unmarshal(body, &etcdErr); err != nil {
+			return nil, ErrInvalidJSON
+		}
+		return nil, etcdErr
+	}
+}
+
+type roundTripResponse struct {
+	resp *http.Response
+	err  error
+}
+
+type simpleHTTPClient struct {
+	transport     CancelableTransport
+	endpoint      url.URL
+	headerTimeout time.Duration
+}
+
+func (c *simpleHTTPClient) Do(ctx context.Context, act httpAction) (*http.Response, []byte, error) {
+	req := act.HTTPRequest(c.endpoint)
+
+	if err := printcURL(req); err != nil {
+		return nil, nil, err
+	}
+
+	isWait := false
+	if req != nil && req.URL != nil {
+		ws := req.URL.Query().Get("wait")
+		if len(ws) != 0 {
+			var err error
+			isWait, err = strconv.ParseBool(ws)
+			if err != nil {
+				return nil, nil, fmt.Errorf("wrong wait value %s (%v for %+v)", ws, err, req)
+			}
+		}
+	}
+
+	var hctx context.Context
+	var hcancel context.CancelFunc
+	if !isWait && c.headerTimeout > 0 {
+		hctx, hcancel = context.WithTimeout(ctx, c.headerTimeout)
+	} else {
+		hctx, hcancel = context.WithCancel(ctx)
+	}
+	defer hcancel()
+
+	reqcancel := requestCanceler(c.transport, req)
+
+	rtchan := make(chan roundTripResponse, 1)
+	go func() {
+		resp, err := c.transport.RoundTrip(req)
+		rtchan <- roundTripResponse{resp: resp, err: err}
+		close(rtchan)
+	}()
+
+	var resp *http.Response
+	var err error
+
+	select {
+	case rtresp := <-rtchan:
+		resp, err = rtresp.resp, rtresp.err
+	case <-hctx.Done():
+		// cancel and wait for request to actually exit before continuing
+		reqcancel()
+		rtresp := <-rtchan
+		resp = rtresp.resp
+		switch {
+		case ctx.Err() != nil:
+			err = ctx.Err()
+		case hctx.Err() != nil:
+			err = fmt.Errorf("client: endpoint %s exceeded header timeout", c.endpoint.String())
+		default:
+			panic("failed to get error from context")
+		}
+	}
+
+	// always check for resp nil-ness to deal with possible
+	// race conditions between channels above
+	defer func() {
+		if resp != nil {
+			resp.Body.Close()
+		}
+	}()
+
+	if err != nil {
+		return nil, nil, err
+	}
+
+	var body []byte
+	done := make(chan struct{})
+	go func() {
+		body, err = ioutil.ReadAll(resp.Body)
+		done <- struct{}{}
+	}()
+
+	select {
+	case <-ctx.Done():
+		resp.Body.Close()
+		<-done
+		return nil, nil, ctx.Err()
+	case <-done:
+	}
+
+	return resp, body, err
+}
+
+type authedAction struct {
+	act         httpAction
+	credentials credentials
+}
+
+func (a *authedAction) HTTPRequest(url url.URL) *http.Request {
+	r := a.act.HTTPRequest(url)
+	r.SetBasicAuth(a.credentials.username, a.credentials.password)
+	return r
+}
+
+type redirectFollowingHTTPClient struct {
+	client        httpClient
+	checkRedirect CheckRedirectFunc
+}
+
+func (r *redirectFollowingHTTPClient) Do(ctx context.Context, act httpAction) (*http.Response, []byte, error) {
+	next := act
+	for i := 0; i < 100; i++ {
+		if i > 0 {
+			if err := r.checkRedirect(i); err != nil {
+				return nil, nil, err
+			}
+		}
+		resp, body, err := r.client.Do(ctx, next)
+		if err != nil {
+			return nil, nil, err
+		}
+		if resp.StatusCode/100 == 3 {
+			hdr := resp.Header.Get("Location")
+			if hdr == "" {
+				return nil, nil, fmt.Errorf("Location header not set")
+			}
+			loc, err := url.Parse(hdr)
+			if err != nil {
+				return nil, nil, fmt.Errorf("Location header not valid URL: %s", hdr)
+			}
+			next = &redirectedHTTPAction{
+				action:   act,
+				location: *loc,
+			}
+			continue
+		}
+		return resp, body, nil
+	}
+
+	return nil, nil, errTooManyRedirectChecks
+}
+
+type redirectedHTTPAction struct {
+	action   httpAction
+	location url.URL
+}
+
+func (r *redirectedHTTPAction) HTTPRequest(ep url.URL) *http.Request {
+	orig := r.action.HTTPRequest(ep)
+	orig.URL = &r.location
+	return orig
+}
+
+func shuffleEndpoints(r *rand.Rand, eps []url.URL) []url.URL {
+	// copied from Go 1.9<= rand.Rand.Perm
+	n := len(eps)
+	p := make([]int, n)
+	for i := 0; i < n; i++ {
+		j := r.Intn(i + 1)
+		p[i] = p[j]
+		p[j] = i
+	}
+	neps := make([]url.URL, n)
+	for i, k := range p {
+		neps[i] = eps[k]
+	}
+	return neps
+}
+
+func endpointsEqual(left, right []url.URL) bool {
+	if len(left) != len(right) {
+		return false
+	}
+
+	sLeft := make([]string, len(left))
+	sRight := make([]string, len(right))
+	for i, l := range left {
+		sLeft[i] = l.String()
+	}
+	for i, r := range right {
+		sRight[i] = r.String()
+	}
+
+	sort.Strings(sLeft)
+	sort.Strings(sRight)
+	for i := range sLeft {
+		if sLeft[i] != sRight[i] {
+			return false
+		}
+	}
+	return true
+}

vendor/github.com/coreos/etcd/client/cluster_error.go

@@ -0,0 +1,37 @@
+// Copyright 2015 The etcd Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package client
+
+import "fmt"
+
+type ClusterError struct {
+	Errors []error
+}
+
+func (ce *ClusterError) Error() string {
+	s := ErrClusterUnavailable.Error()
+	for i, e := range ce.Errors {
+		s += fmt.Sprintf("; error #%d: %s\n", i, e)
+	}
+	return s
+}
+
+func (ce *ClusterError) Detail() string {
+	s := ""
+	for i, e := range ce.Errors {
+		s += fmt.Sprintf("error #%d: %s\n", i, e)
+	}
+	return s
+}

vendor/github.com/coreos/etcd/client/curl.go

@@ -0,0 +1,70 @@
+// Copyright 2015 The etcd Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package client
+
+import (
+	"bytes"
+	"fmt"
+	"io/ioutil"
+	"net/http"
+	"os"
+)
+
+var (
+	cURLDebug = false
+)
+
+func EnablecURLDebug() {
+	cURLDebug = true
+}
+
+func DisablecURLDebug() {
+	cURLDebug = false
+}
+
+// printcURL prints the cURL equivalent request to stderr.
+// It returns an error if the body of the request cannot
+// be read.
+// The caller MUST cancel the request if there is an error.
+func printcURL(req *http.Request) error {
+	if !cURLDebug {
+		return nil
+	}
+	var (
+		command string
+		b       []byte
+		err     error
+	)
+
+	if req.URL != nil {
+		command = fmt.Sprintf("curl -X %s %s", req.Method, req.URL.String())
+	}
+
+	if req.Body != nil {
+		b, err = ioutil.ReadAll(req.Body)
+		if err != nil {
+			return err
+		}
+		command += fmt.Sprintf(" -d %q", string(b))
+	}
+
+	fmt.Fprintf(os.Stderr, "cURL Command: %s\n", command)
+
+	// reset body
+	body := bytes.NewBuffer(b)
+	req.Body = ioutil.NopCloser(body)
+
+	return nil
+}

vendor/github.com/coreos/etcd/client/discover.go

@@ -0,0 +1,40 @@
+// Copyright 2015 The etcd Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package client
+
+import (
+	"github.com/coreos/etcd/pkg/srv"
+)
+
+// Discoverer is an interface that wraps the Discover method.
+type Discoverer interface {
+	// Discover looks up the etcd servers for the domain.
+	Discover(domain string) ([]string, error)
+}
+
+type srvDiscover struct{}
+
+// NewSRVDiscover constructs a new Discoverer that uses the stdlib to lookup SRV records.
+func NewSRVDiscover() Discoverer {
+	return &srvDiscover{}
+}
+
+func (d *srvDiscover) Discover(domain string) ([]string, error) {
+	srvs, err := srv.GetClient("etcd-client", domain)
+	if err != nil {
+		return nil, err
+	}
+	return srvs.Endpoints, nil
+}

vendor/github.com/coreos/etcd/client/doc.go

@@ -0,0 +1,73 @@
+// Copyright 2015 The etcd Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+/*
+Package client provides bindings for the etcd APIs.
+
+Create a Config and exchange it for a Client:
+
+	import (
+		"net/http"
+		"context"
+
+		"github.com/coreos/etcd/client"
+	)
+
+	cfg := client.Config{
+		Endpoints: []string{"http://127.0.0.1:2379"},
+		Transport: DefaultTransport,
+	}
+
+	c, err := client.New(cfg)
+	if err != nil {
+		// handle error
+	}
+
+Clients are safe for concurrent use by multiple goroutines.
+
+Create a KeysAPI using the Client, then use it to interact with etcd:
+
+	kAPI := client.NewKeysAPI(c)
+
+	// create a new key /foo with the value "bar"
+	_, err = kAPI.Create(context.Background(), "/foo", "bar")
+	if err != nil {
+		// handle error
+	}
+
+	// delete the newly created key only if the value is still "bar"
+	_, err = kAPI.Delete(context.Background(), "/foo", &DeleteOptions{PrevValue: "bar"})
+	if err != nil {
+		// handle error
+	}
+
+Use a custom context to set timeouts on your operations:
+
+	import "time"
+
+	ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
+	defer cancel()
+
+	// set a new key, ignoring its previous state
+	_, err := kAPI.Set(ctx, "/ping", "pong", nil)
+	if err != nil {
+		if err == context.DeadlineExceeded {
+			// request took longer than 5s
+		} else {
+			// handle error
+		}
+	}
+
+*/
+package client

vendor/github.com/coreos/etcd/client/integration/doc.go

@@ -0,0 +1,17 @@
+// Copyright 2016 The etcd Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+// Package integration implements tests built upon embedded etcd, focusing on
+// the correctness of the etcd v2 client.
+package integration

vendor/github.com/coreos/etcd/client/keys.go

@@ -0,0 +1,681 @@
+// Copyright 2015 The etcd Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package client
+
+//go:generate codecgen -d 1819 -r "Node|Response|Nodes" -o keys.generated.go keys.go
+
+import (
+	"context"
+	"encoding/json"
+	"errors"
+	"fmt"
+	"net/http"
+	"net/url"
+	"strconv"
+	"strings"
+	"time"
+
+	"github.com/coreos/etcd/pkg/pathutil"
+	"github.com/ugorji/go/codec"
+)
+
+const (
+	ErrorCodeKeyNotFound  = 100
+	ErrorCodeTestFailed   = 101
+	ErrorCodeNotFile      = 102
+	ErrorCodeNotDir       = 104
+	ErrorCodeNodeExist    = 105
+	ErrorCodeRootROnly    = 107
+	ErrorCodeDirNotEmpty  = 108
+	ErrorCodeUnauthorized = 110
+
+	ErrorCodePrevValueRequired = 201
+	ErrorCodeTTLNaN            = 202
+	ErrorCodeIndexNaN          = 203
+	ErrorCodeInvalidField      = 209
+	ErrorCodeInvalidForm       = 210
+
+	ErrorCodeRaftInternal = 300
+	ErrorCodeLeaderElect  = 301
+
+	ErrorCodeWatcherCleared    = 400
+	ErrorCodeEventIndexCleared = 401
+)
+
+type Error struct {
+	Code    int    `json:"errorCode"`
+	Message string `json:"message"`
+	Cause   string `json:"cause"`
+	Index   uint64 `json:"index"`
+}
+
+func (e Error) Error() string {
+	return fmt.Sprintf("%v: %v (%v) [%v]", e.Code, e.Message, e.Cause, e.Index)
+}
+
+var (
+	ErrInvalidJSON = errors.New("client: response is invalid json. The endpoint is probably not valid etcd cluster endpoint.")
+	ErrEmptyBody   = errors.New("client: response body is empty")
+)
+
+// PrevExistType is used to define an existence condition when setting
+// or deleting Nodes.
+type PrevExistType string
+
+const (
+	PrevIgnore  = PrevExistType("")
+	PrevExist   = PrevExistType("true")
+	PrevNoExist = PrevExistType("false")
+)
+
+var (
+	defaultV2KeysPrefix = "/v2/keys"
+)
+
+// NewKeysAPI builds a KeysAPI that interacts with etcd's key-value
+// API over HTTP.
+func NewKeysAPI(c Client) KeysAPI {
+	return NewKeysAPIWithPrefix(c, defaultV2KeysPrefix)
+}
+
+// NewKeysAPIWithPrefix acts like NewKeysAPI, but allows the caller
+// to provide a custom base URL path. This should only be used in
+// very rare cases.
+func NewKeysAPIWithPrefix(c Client, p string) KeysAPI {
+	return &httpKeysAPI{
+		client: c,
+		prefix: p,
+	}
+}
+
+type KeysAPI interface {
+	// Get retrieves a set of Nodes from etcd
+	Get(ctx context.Context, key string, opts *GetOptions) (*Response, error)
+
+	// Set assigns a new value to a Node identified by a given key. The caller
+	// may define a set of conditions in the SetOptions. If SetOptions.Dir=true
+	// then value is ignored.
+	Set(ctx context.Context, key, value string, opts *SetOptions) (*Response, error)
+
+	// Delete removes a Node identified by the given key, optionally destroying
+	// all of its children as well. The caller may define a set of required
+	// conditions in an DeleteOptions object.
+	Delete(ctx context.Context, key string, opts *DeleteOptions) (*Response, error)
+
+	// Create is an alias for Set w/ PrevExist=false
+	Create(ctx context.Context, key, value string) (*Response, error)
+
+	// CreateInOrder is used to atomically create in-order keys within the given directory.
+	CreateInOrder(ctx context.Context, dir, value string, opts *CreateInOrderOptions) (*Response, error)
+
+	// Update is an alias for Set w/ PrevExist=true
+	Update(ctx context.Context, key, value string) (*Response, error)
+
+	// Watcher builds a new Watcher targeted at a specific Node identified
+	// by the given key. The Watcher may be configured at creation time
+	// through a WatcherOptions object. The returned Watcher is designed
+	// to emit events that happen to a Node, and optionally to its children.
+	Watcher(key string, opts *WatcherOptions) Watcher
+}
+
+type WatcherOptions struct {
+	// AfterIndex defines the index after-which the Watcher should
+	// start emitting events. For example, if a value of 5 is
+	// provided, the first event will have an index >= 6.
+	//
+	// Setting AfterIndex to 0 (default) means that the Watcher
+	// should start watching for events starting at the current
+	// index, whatever that may be.
+	AfterIndex uint64
+
+	// Recursive specifies whether or not the Watcher should emit
+	// events that occur in children of the given keyspace. If set
+	// to false (default), events will be limited to those that
+	// occur for the exact key.
+	Recursive bool
+}
+
+type CreateInOrderOptions struct {
+	// TTL defines a period of time after-which the Node should
+	// expire and no longer exist. Values <= 0 are ignored. Given
+	// that the zero-value is ignored, TTL cannot be used to set
+	// a TTL of 0.
+	TTL time.Duration
+}
+
+type SetOptions struct {
+	// PrevValue specifies what the current value of the Node must
+	// be in order for the Set operation to succeed.
+	//
+	// Leaving this field empty means that the caller wishes to
+	// ignore the current value of the Node. This cannot be used
+	// to compare the Node's current value to an empty string.
+	//
+	// PrevValue is ignored if Dir=true
+	PrevValue string
+
+	// PrevIndex indicates what the current ModifiedIndex of the
+	// Node must be in order for the Set operation to succeed.
+	//
+	// If PrevIndex is set to 0 (default), no comparison is made.
+	PrevIndex uint64
+
+	// PrevExist specifies whether the Node must currently exist
+	// (PrevExist) or not (PrevNoExist). If the caller does not
+	// care about existence, set PrevExist to PrevIgnore, or simply
+	// leave it unset.
+	PrevExist PrevExistType
+
+	// TTL defines a period of time after-which the Node should
+	// expire and no longer exist. Values <= 0 are ignored. Given
+	// that the zero-value is ignored, TTL cannot be used to set
+	// a TTL of 0.
+	TTL time.Duration
+
+	// Refresh set to true means a TTL value can be updated
+	// without firing a watch or changing the node value. A
+	// value must not be provided when refreshing a key.
+	Refresh bool
+
+	// Dir specifies whether or not this Node should be created as a directory.
+	Dir bool
+
+	// NoValueOnSuccess specifies whether the response contains the current value of the Node.
+	// If set, the response will only contain the current value when the request fails.
+	NoValueOnSuccess bool
+}
+
+type GetOptions struct {
+	// Recursive defines whether or not all children of the Node
+	// should be returned.
+	Recursive bool
+
+	// Sort instructs the server whether or not to sort the Nodes.
+	// If true, the Nodes are sorted alphabetically by key in
+	// ascending order (A to z). If false (default), the Nodes will
+	// not be sorted and the ordering used should not be considered
+	// predictable.
+	Sort bool
+
+	// Quorum specifies whether it gets the latest committed value that
+	// has been applied in quorum of members, which ensures external
+	// consistency (or linearizability).
+	Quorum bool
+}
+
+type DeleteOptions struct {
+	// PrevValue specifies what the current value of the Node must
+	// be in order for the Delete operation to succeed.
+	//
+	// Leaving this field empty means that the caller wishes to
+	// ignore the current value of the Node. This cannot be used
+	// to compare the Node's current value to an empty string.
+	PrevValue string
+
+	// PrevIndex indicates what the current ModifiedIndex of the
+	// Node must be in order for the Delete operation to succeed.
+	//
+	// If PrevIndex is set to 0 (default), no comparison is made.
+	PrevIndex uint64
+
+	// Recursive defines whether or not all children of the Node
+	// should be deleted. If set to true, all children of the Node
+	// identified by the given key will be deleted. If left unset
+	// or explicitly set to false, only a single Node will be
+	// deleted.
+	Recursive bool
+
+	// Dir specifies whether or not this Node should be removed as a directory.
+	Dir bool
+}
+
+type Watcher interface {
+	// Next blocks until an etcd event occurs, then returns a Response
+	// representing that event. The behavior of Next depends on the
+	// WatcherOptions used to construct the Watcher. Next is designed to
+	// be called repeatedly, each time blocking until a subsequent event
+	// is available.
+	//
+	// If the provided context is cancelled, Next will return a non-nil
+	// error. Any other failures encountered while waiting for the next
+	// event (connection issues, deserialization failures, etc) will
+	// also result in a non-nil error.
+	Next(context.Context) (*Response, error)
+}
+
+type Response struct {
+	// Action is the name of the operation that occurred. Possible values
+	// include get, set, delete, update, create, compareAndSwap,
+	// compareAndDelete and expire.
+	Action string `json:"action"`
+
+	// Node represents the state of the relevant etcd Node.
+	Node *Node `json:"node"`
+
+	// PrevNode represents the previous state of the Node. PrevNode is non-nil
+	// only if the Node existed before the action occurred and the action
+	// caused a change to the Node.
+	PrevNode *Node `json:"prevNode"`
+
+	// Index holds the cluster-level index at the time the Response was generated.
+	// This index is not tied to the Node(s) contained in this Response.
+	Index uint64 `json:"-"`
+
+	// ClusterID holds the cluster-level ID reported by the server.  This
+	// should be different for different etcd clusters.
+	ClusterID string `json:"-"`
+}
+
+type Node struct {
+	// Key represents the unique location of this Node (e.g. "/foo/bar").
+	Key string `json:"key"`
+
+	// Dir reports whether node describes a directory.
+	Dir bool `json:"dir,omitempty"`
+
+	// Value is the current data stored on this Node. If this Node
+	// is a directory, Value will be empty.
+	Value string `json:"value"`
+
+	// Nodes holds the children of this Node, only if this Node is a directory.
+	// This slice of will be arbitrarily deep (children, grandchildren, great-
+	// grandchildren, etc.) if a recursive Get or Watch request were made.
+	Nodes Nodes `json:"nodes"`
+
+	// CreatedIndex is the etcd index at-which this Node was created.
+	CreatedIndex uint64 `json:"createdIndex"`
+
+	// ModifiedIndex is the etcd index at-which this Node was last modified.
+	ModifiedIndex uint64 `json:"modifiedIndex"`
+
+	// Expiration is the server side expiration time of the key.
+	Expiration *time.Time `json:"expiration,omitempty"`
+
+	// TTL is the time to live of the key in second.
+	TTL int64 `json:"ttl,omitempty"`
+}
+
+func (n *Node) String() string {
+	return fmt.Sprintf("{Key: %s, CreatedIndex: %d, ModifiedIndex: %d, TTL: %d}", n.Key, n.CreatedIndex, n.ModifiedIndex, n.TTL)
+}
+
+// TTLDuration returns the Node's TTL as a time.Duration object
+func (n *Node) TTLDuration() time.Duration {
+	return time.Duration(n.TTL) * time.Second
+}
+
+type Nodes []*Node
+
+// interfaces for sorting
+
+func (ns Nodes) Len() int           { return len(ns) }
+func (ns Nodes) Less(i, j int) bool { return ns[i].Key < ns[j].Key }
+func (ns Nodes) Swap(i, j int)      { ns[i], ns[j] = ns[j], ns[i] }
+
+type httpKeysAPI struct {
+	client httpClient
+	prefix string
+}
+
+func (k *httpKeysAPI) Set(ctx context.Context, key, val string, opts *SetOptions) (*Response, error) {
+	act := &setAction{
+		Prefix: k.prefix,
+		Key:    key,
+		Value:  val,
+	}
+
+	if opts != nil {
+		act.PrevValue = opts.PrevValue
+		act.PrevIndex = opts.PrevIndex
+		act.PrevExist = opts.PrevExist
+		act.TTL = opts.TTL
+		act.Refresh = opts.Refresh
+		act.Dir = opts.Dir
+		act.NoValueOnSuccess = opts.NoValueOnSuccess
+	}
+
+	doCtx := ctx
+	if act.PrevExist == PrevNoExist {
+		doCtx = context.WithValue(doCtx, &oneShotCtxValue, &oneShotCtxValue)
+	}
+	resp, body, err := k.client.Do(doCtx, act)
+	if err != nil {
+		return nil, err
+	}
+
+	return unmarshalHTTPResponse(resp.StatusCode, resp.Header, body)
+}
+
+func (k *httpKeysAPI) Create(ctx context.Context, key, val string) (*Response, error) {
+	return k.Set(ctx, key, val, &SetOptions{PrevExist: PrevNoExist})
+}
+
+func (k *httpKeysAPI) CreateInOrder(ctx context.Context, dir, val string, opts *CreateInOrderOptions) (*Response, error) {
+	act := &createInOrderAction{
+		Prefix: k.prefix,
+		Dir:    dir,
+		Value:  val,
+	}
+
+	if opts != nil {
+		act.TTL = opts.TTL
+	}
+
+	resp, body, err := k.client.Do(ctx, act)
+	if err != nil {
+		return nil, err
+	}
+
+	return unmarshalHTTPResponse(resp.StatusCode, resp.Header, body)
+}
+
+func (k *httpKeysAPI) Update(ctx context.Context, key, val string) (*Response, error) {
+	return k.Set(ctx, key, val, &SetOptions{PrevExist: PrevExist})
+}
+
+func (k *httpKeysAPI) Delete(ctx context.Context, key string, opts *DeleteOptions) (*Response, error) {
+	act := &deleteAction{
+		Prefix: k.prefix,
+		Key:    key,
+	}
+
+	if opts != nil {
+		act.PrevValue = opts.PrevValue
+		act.PrevIndex = opts.PrevIndex
+		act.Dir = opts.Dir
+		act.Recursive = opts.Recursive
+	}
+
+	doCtx := context.WithValue(ctx, &oneShotCtxValue, &oneShotCtxValue)
+	resp, body, err := k.client.Do(doCtx, act)
+	if err != nil {
+		return nil, err
+	}
+
+	return unmarshalHTTPResponse(resp.StatusCode, resp.Header, body)
+}
+
+func (k *httpKeysAPI) Get(ctx context.Context, key string, opts *GetOptions) (*Response, error) {
+	act := &getAction{
+		Prefix: k.prefix,
+		Key:    key,
+	}
+
+	if opts != nil {
+		act.Recursive = opts.Recursive
+		act.Sorted = opts.Sort
+		act.Quorum = opts.Quorum
+	}
+
+	resp, body, err := k.client.Do(ctx, act)
+	if err != nil {
+		return nil, err
+	}
+
+	return unmarshalHTTPResponse(resp.StatusCode, resp.Header, body)
+}
+
+func (k *httpKeysAPI) Watcher(key string, opts *WatcherOptions) Watcher {
+	act := waitAction{
+		Prefix: k.prefix,
+		Key:    key,
+	}
+
+	if opts != nil {
+		act.Recursive = opts.Recursive
+		if opts.AfterIndex > 0 {
+			act.WaitIndex = opts.AfterIndex + 1
+		}
+	}
+
+	return &httpWatcher{
+		client:   k.client,
+		nextWait: act,
+	}
+}
+
+type httpWatcher struct {
+	client   httpClient
+	nextWait waitAction
+}
+
+func (hw *httpWatcher) Next(ctx context.Context) (*Response, error) {
+	for {
+		httpresp, body, err := hw.client.Do(ctx, &hw.nextWait)
+		if err != nil {
+			return nil, err
+		}
+
+		resp, err := unmarshalHTTPResponse(httpresp.StatusCode, httpresp.Header, body)
+		if err != nil {
+			if err == ErrEmptyBody {
+				continue
+			}
+			return nil, err
+		}
+
+		hw.nextWait.WaitIndex = resp.Node.ModifiedIndex + 1
+		return resp, nil
+	}
+}
+
+// v2KeysURL forms a URL representing the location of a key.
+// The endpoint argument represents the base URL of an etcd
+// server. The prefix is the path needed to route from the
+// provided endpoint's path to the root of the keys API
+// (typically "/v2/keys").
+func v2KeysURL(ep url.URL, prefix, key string) *url.URL {
+	// We concatenate all parts together manually. We cannot use
+	// path.Join because it does not reserve trailing slash.
+	// We call CanonicalURLPath to further cleanup the path.
+	if prefix != "" && prefix[0] != '/' {
+		prefix = "/" + prefix
+	}
+	if key != "" && key[0] != '/' {
+		key = "/" + key
+	}
+	ep.Path = pathutil.CanonicalURLPath(ep.Path + prefix + key)
+	return &ep
+}
+
+type getAction struct {
+	Prefix    string
+	Key       string
+	Recursive bool
+	Sorted    bool
+	Quorum    bool
+}
+
+func (g *getAction) HTTPRequest(ep url.URL) *http.Request {
+	u := v2KeysURL(ep, g.Prefix, g.Key)
+
+	params := u.Query()
+	params.Set("recursive", strconv.FormatBool(g.Recursive))
+	params.Set("sorted", strconv.FormatBool(g.Sorted))
+	params.Set("quorum", strconv.FormatBool(g.Quorum))
+	u.RawQuery = params.Encode()
+
+	req, _ := http.NewRequest("GET", u.String(), nil)
+	return req
+}
+
+type waitAction struct {
+	Prefix    string
+	Key       string
+	WaitIndex uint64
+	Recursive bool
+}
+
+func (w *waitAction) HTTPRequest(ep url.URL) *http.Request {
+	u := v2KeysURL(ep, w.Prefix, w.Key)
+
+	params := u.Query()
+	params.Set("wait", "true")
+	params.Set("waitIndex", strconv.FormatUint(w.WaitIndex, 10))
+	params.Set("recursive", strconv.FormatBool(w.Recursive))
+	u.RawQuery = params.Encode()
+
+	req, _ := http.NewRequest("GET", u.String(), nil)
+	return req
+}
+
+type setAction struct {
+	Prefix           string
+	Key              string
+	Value            string
+	PrevValue        string
+	PrevIndex        uint64
+	PrevExist        PrevExistType
+	TTL              time.Duration
+	Refresh          bool
+	Dir              bool
+	NoValueOnSuccess bool
+}
+
+func (a *setAction) HTTPRequest(ep url.URL) *http.Request {
+	u := v2KeysURL(ep, a.Prefix, a.Key)
+
+	params := u.Query()
+	form := url.Values{}
+
+	// we're either creating a directory or setting a key
+	if a.Dir {
+		params.Set("dir", strconv.FormatBool(a.Dir))
+	} else {
+		// These options are only valid for setting a key
+		if a.PrevValue != "" {
+			params.Set("prevValue", a.PrevValue)
+		}
+		form.Add("value", a.Value)
+	}
+
+	// Options which apply to both setting a key and creating a dir
+	if a.PrevIndex != 0 {
+		params.Set("prevIndex", strconv.FormatUint(a.PrevIndex, 10))
+	}
+	if a.PrevExist != PrevIgnore {
+		params.Set("prevExist", string(a.PrevExist))
+	}
+	if a.TTL > 0 {
+		form.Add("ttl", strconv.FormatUint(uint64(a.TTL.Seconds()), 10))
+	}
+
+	if a.Refresh {
+		form.Add("refresh", "true")
+	}
+	if a.NoValueOnSuccess {
+		params.Set("noValueOnSuccess", strconv.FormatBool(a.NoValueOnSuccess))
+	}
+
+	u.RawQuery = params.Encode()
+	body := strings.NewReader(form.Encode())
+
+	req, _ := http.NewRequest("PUT", u.String(), body)
+	req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
+
+	return req
+}
+
+type deleteAction struct {
+	Prefix    string
+	Key       string
+	PrevValue string
+	PrevIndex uint64
+	Dir       bool
+	Recursive bool
+}
+
+func (a *deleteAction) HTTPRequest(ep url.URL) *http.Request {
+	u := v2KeysURL(ep, a.Prefix, a.Key)
+
+	params := u.Query()
+	if a.PrevValue != "" {
+		params.Set("prevValue", a.PrevValue)
+	}
+	if a.PrevIndex != 0 {
+		params.Set("prevIndex", strconv.FormatUint(a.PrevIndex, 10))
+	}
+	if a.Dir {
+		params.Set("dir", "true")
+	}
+	if a.Recursive {
+		params.Set("recursive", "true")
+	}
+	u.RawQuery = params.Encode()
+
+	req, _ := http.NewRequest("DELETE", u.String(), nil)
+	req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
+
+	return req
+}
+
+type createInOrderAction struct {
+	Prefix string
+	Dir    string
+	Value  string
+	TTL    time.Duration
+}
+
+func (a *createInOrderAction) HTTPRequest(ep url.URL) *http.Request {
+	u := v2KeysURL(ep, a.Prefix, a.Dir)
+
+	form := url.Values{}
+	form.Add("value", a.Value)
+	if a.TTL > 0 {
+		form.Add("ttl", strconv.FormatUint(uint64(a.TTL.Seconds()), 10))
+	}
+	body := strings.NewReader(form.Encode())
+
+	req, _ := http.NewRequest("POST", u.String(), body)
+	req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
+	return req
+}
+
+func unmarshalHTTPResponse(code int, header http.Header, body []byte) (res *Response, err error) {
+	switch code {
+	case http.StatusOK, http.StatusCreated:
+		if len(body) == 0 {
+			return nil, ErrEmptyBody
+		}
+		res, err = unmarshalSuccessfulKeysResponse(header, body)
+	default:
+		err = unmarshalFailedKeysResponse(body)
+	}
+	return res, err
+}
+
+func unmarshalSuccessfulKeysResponse(header http.Header, body []byte) (*Response, error) {
+	var res Response
+	err := codec.NewDecoderBytes(body, new(codec.JsonHandle)).Decode(&res)
+	if err != nil {
+		return nil, ErrInvalidJSON
+	}
+	if header.Get("X-Etcd-Index") != "" {
+		res.Index, err = strconv.ParseUint(header.Get("X-Etcd-Index"), 10, 64)
+		if err != nil {
+			return nil, err
+		}
+	}
+	res.ClusterID = header.Get("X-Etcd-Cluster-ID")
+	return &res, nil
+}
+
+func unmarshalFailedKeysResponse(body []byte) error {
+	var etcdErr Error
+	if err := json.Unmarshal(body, &etcdErr); err != nil {
+		return ErrInvalidJSON
+	}
+	return etcdErr
+}

vendor/github.com/coreos/etcd/client/members.go

@@ -0,0 +1,303 @@
+// Copyright 2015 The etcd Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package client
+
+import (
+	"bytes"
+	"context"
+	"encoding/json"
+	"fmt"
+	"net/http"
+	"net/url"
+	"path"
+
+	"github.com/coreos/etcd/pkg/types"
+)
+
+var (
+	defaultV2MembersPrefix = "/v2/members"
+	defaultLeaderSuffix    = "/leader"
+)
+
+type Member struct {
+	// ID is the unique identifier of this Member.
+	ID string `json:"id"`
+
+	// Name is a human-readable, non-unique identifier of this Member.
+	Name string `json:"name"`
+
+	// PeerURLs represents the HTTP(S) endpoints this Member uses to
+	// participate in etcd's consensus protocol.
+	PeerURLs []string `json:"peerURLs"`
+
+	// ClientURLs represents the HTTP(S) endpoints on which this Member
+	// serves its client-facing APIs.
+	ClientURLs []string `json:"clientURLs"`
+}
+
+type memberCollection []Member
+
+func (c *memberCollection) UnmarshalJSON(data []byte) error {
+	d := struct {
+		Members []Member
+	}{}
+
+	if err := json.Unmarshal(data, &d); err != nil {
+		return err
+	}
+
+	if d.Members == nil {
+		*c = make([]Member, 0)
+		return nil
+	}
+
+	*c = d.Members
+	return nil
+}
+
+type memberCreateOrUpdateRequest struct {
+	PeerURLs types.URLs
+}
+
+func (m *memberCreateOrUpdateRequest) MarshalJSON() ([]byte, error) {
+	s := struct {
+		PeerURLs []string `json:"peerURLs"`
+	}{
+		PeerURLs: make([]string, len(m.PeerURLs)),
+	}
+
+	for i, u := range m.PeerURLs {
+		s.PeerURLs[i] = u.String()
+	}
+
+	return json.Marshal(&s)
+}
+
+// NewMembersAPI constructs a new MembersAPI that uses HTTP to
+// interact with etcd's membership API.
+func NewMembersAPI(c Client) MembersAPI {
+	return &httpMembersAPI{
+		client: c,
+	}
+}
+
+type MembersAPI interface {
+	// List enumerates the current cluster membership.
+	List(ctx context.Context) ([]Member, error)
+
+	// Add instructs etcd to accept a new Member into the cluster.
+	Add(ctx context.Context, peerURL string) (*Member, error)
+
+	// Remove demotes an existing Member out of the cluster.
+	Remove(ctx context.Context, mID string) error
+
+	// Update instructs etcd to update an existing Member in the cluster.
+	Update(ctx context.Context, mID string, peerURLs []string) error
+
+	// Leader gets current leader of the cluster
+	Leader(ctx context.Context) (*Member, error)
+}
+
+type httpMembersAPI struct {
+	client httpClient
+}
+
+func (m *httpMembersAPI) List(ctx context.Context) ([]Member, error) {
+	req := &membersAPIActionList{}
+	resp, body, err := m.client.Do(ctx, req)
+	if err != nil {
+		return nil, err
+	}
+
+	if err := assertStatusCode(resp.StatusCode, http.StatusOK); err != nil {
+		return nil, err
+	}
+
+	var mCollection memberCollection
+	if err := json.Unmarshal(body, &mCollection); err != nil {
+		return nil, err
+	}
+
+	return []Member(mCollection), nil
+}
+
+func (m *httpMembersAPI) Add(ctx context.Context, peerURL string) (*Member, error) {
+	urls, err := types.NewURLs([]string{peerURL})
+	if err != nil {
+		return nil, err
+	}
+
+	req := &membersAPIActionAdd{peerURLs: urls}
+	resp, body, err := m.client.Do(ctx, req)
+	if err != nil {
+		return nil, err
+	}
+
+	if err := assertStatusCode(resp.StatusCode, http.StatusCreated, http.StatusConflict); err != nil {
+		return nil, err
+	}
+
+	if resp.StatusCode != http.StatusCreated {
+		var merr membersError
+		if err := json.Unmarshal(body, &merr); err != nil {
+			return nil, err
+		}
+		return nil, merr
+	}
+
+	var memb Member
+	if err := json.Unmarshal(body, &memb); err != nil {
+		return nil, err
+	}
+
+	return &memb, nil
+}
+
+func (m *httpMembersAPI) Update(ctx context.Context, memberID string, peerURLs []string) error {
+	urls, err := types.NewURLs(peerURLs)
+	if err != nil {
+		return err
+	}
+
+	req := &membersAPIActionUpdate{peerURLs: urls, memberID: memberID}
+	resp, body, err := m.client.Do(ctx, req)
+	if err != nil {
+		return err
+	}
+
+	if err := assertStatusCode(resp.StatusCode, http.StatusNoContent, http.StatusNotFound, http.StatusConflict); err != nil {
+		return err
+	}
+
+	if resp.StatusCode != http.StatusNoContent {
+		var merr membersError
+		if err := json.Unmarshal(body, &merr); err != nil {
+			return err
+		}
+		return merr
+	}
+
+	return nil
+}
+
+func (m *httpMembersAPI) Remove(ctx context.Context, memberID string) error {
+	req := &membersAPIActionRemove{memberID: memberID}
+	resp, _, err := m.client.Do(ctx, req)
+	if err != nil {
+		return err
+	}
+
+	return assertStatusCode(resp.StatusCode, http.StatusNoContent, http.StatusGone)
+}
+
+func (m *httpMembersAPI) Leader(ctx context.Context) (*Member, error) {
+	req := &membersAPIActionLeader{}
+	resp, body, err := m.client.Do(ctx, req)
+	if err != nil {
+		return nil, err
+	}
+
+	if err := assertStatusCode(resp.StatusCode, http.StatusOK); err != nil {
+		return nil, err
+	}
+
+	var leader Member
+	if err := json.Unmarshal(body, &leader); err != nil {
+		return nil, err
+	}
+
+	return &leader, nil
+}
+
+type membersAPIActionList struct{}
+
+func (l *membersAPIActionList) HTTPRequest(ep url.URL) *http.Request {
+	u := v2MembersURL(ep)
+	req, _ := http.NewRequest("GET", u.String(), nil)
+	return req
+}
+
+type membersAPIActionRemove struct {
+	memberID string
+}
+
+func (d *membersAPIActionRemove) HTTPRequest(ep url.URL) *http.Request {
+	u := v2MembersURL(ep)
+	u.Path = path.Join(u.Path, d.memberID)
+	req, _ := http.NewRequest("DELETE", u.String(), nil)
+	return req
+}
+
+type membersAPIActionAdd struct {
+	peerURLs types.URLs
+}
+
+func (a *membersAPIActionAdd) HTTPRequest(ep url.URL) *http.Request {
+	u := v2MembersURL(ep)
+	m := memberCreateOrUpdateRequest{PeerURLs: a.peerURLs}
+	b, _ := json.Marshal(&m)
+	req, _ := http.NewRequest("POST", u.String(), bytes.NewReader(b))
+	req.Header.Set("Content-Type", "application/json")
+	return req
+}
+
+type membersAPIActionUpdate struct {
+	memberID string
+	peerURLs types.URLs
+}
+
+func (a *membersAPIActionUpdate) HTTPRequest(ep url.URL) *http.Request {
+	u := v2MembersURL(ep)
+	m := memberCreateOrUpdateRequest{PeerURLs: a.peerURLs}
+	u.Path = path.Join(u.Path, a.memberID)
+	b, _ := json.Marshal(&m)
+	req, _ := http.NewRequest("PUT", u.String(), bytes.NewReader(b))
+	req.Header.Set("Content-Type", "application/json")
+	return req
+}
+
+func assertStatusCode(got int, want ...int) (err error) {
+	for _, w := range want {
+		if w == got {
+			return nil
+		}
+	}
+	return fmt.Errorf("unexpected status code %d", got)
+}
+
+type membersAPIActionLeader struct{}
+
+func (l *membersAPIActionLeader) HTTPRequest(ep url.URL) *http.Request {
+	u := v2MembersURL(ep)
+	u.Path = path.Join(u.Path, defaultLeaderSuffix)
+	req, _ := http.NewRequest("GET", u.String(), nil)
+	return req
+}
+
+// v2MembersURL add the necessary path to the provided endpoint
+// to route requests to the default v2 members API.
+func v2MembersURL(ep url.URL) *url.URL {
+	ep.Path = path.Join(ep.Path, defaultV2MembersPrefix)
+	return &ep
+}
+
+type membersError struct {
+	Message string `json:"message"`
+	Code    int    `json:"-"`
+}
+
+func (e membersError) Error() string {
+	return e.Message
+}

vendor/github.com/coreos/etcd/client/util.go

@@ -0,0 +1,53 @@
+// Copyright 2016 The etcd Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package client
+
+import (
+	"regexp"
+)
+
+var (
+	roleNotFoundRegExp *regexp.Regexp
+	userNotFoundRegExp *regexp.Regexp
+)
+
+func init() {
+	roleNotFoundRegExp = regexp.MustCompile("auth: Role .* does not exist.")
+	userNotFoundRegExp = regexp.MustCompile("auth: User .* does not exist.")
+}
+
+// IsKeyNotFound returns true if the error code is ErrorCodeKeyNotFound.
+func IsKeyNotFound(err error) bool {
+	if cErr, ok := err.(Error); ok {
+		return cErr.Code == ErrorCodeKeyNotFound
+	}
+	return false
+}
+
+// IsRoleNotFound returns true if the error means role not found of v2 API.
+func IsRoleNotFound(err error) bool {
+	if ae, ok := err.(authError); ok {
+		return roleNotFoundRegExp.MatchString(ae.Message)
+	}
+	return false
+}
+
+// IsUserNotFound returns true if the error means user not found of v2 API.
+func IsUserNotFound(err error) bool {
+	if ae, ok := err.(authError); ok {
+		return userNotFoundRegExp.MatchString(ae.Message)
+	}
+	return false
+}

vendor/github.com/coreos/etcd/pkg/pathutil/LICENSE

@@ -0,0 +1,202 @@
+
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

vendor/github.com/coreos/etcd/pkg/pathutil/path.go

@@ -0,0 +1,31 @@
+// Copyright 2009 The Go Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+
+// Package pathutil implements utility functions for handling slash-separated
+// paths.
+package pathutil
+
+import "path"
+
+// CanonicalURLPath returns the canonical url path for p, which follows the rules:
+// 1. the path always starts with "/"
+// 2. replace multiple slashes with a single slash
+// 3. replace each '.' '..' path name element with equivalent one
+// 4. keep the trailing slash
+// The function is borrowed from stdlib http.cleanPath in server.go.
+func CanonicalURLPath(p string) string {
+	if p == "" {
+		return "/"
+	}
+	if p[0] != '/' {
+		p = "/" + p
+	}
+	np := path.Clean(p)
+	// path.Clean removes trailing slash except for root,
+	// put the trailing slash back if necessary.
+	if p[len(p)-1] == '/' && np != "/" {
+		np += "/"
+	}
+	return np
+}

vendor/github.com/coreos/etcd/pkg/srv/LICENSE

@@ -0,0 +1,202 @@
+
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

vendor/github.com/coreos/etcd/pkg/srv/srv.go

@@ -0,0 +1,130 @@
+// Copyright 2015 The etcd Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+// Package srv looks up DNS SRV records.
+package srv
+
+import (
+	"fmt"
+	"net"
+	"net/url"
+	"strings"
+
+	"github.com/coreos/etcd/pkg/types"
+)
+
+var (
+	// indirection for testing
+	lookupSRV      = net.LookupSRV // net.DefaultResolver.LookupSRV when ctxs don't conflict
+	resolveTCPAddr = net.ResolveTCPAddr
+)
+
+// GetCluster gets the cluster information via DNS discovery.
+// Also sees each entry as a separate instance.
+func GetCluster(serviceScheme, service, name, dns string, apurls types.URLs) ([]string, error) {
+	tempName := int(0)
+	tcp2ap := make(map[string]url.URL)
+
+	// First, resolve the apurls
+	for _, url := range apurls {
+		tcpAddr, err := resolveTCPAddr("tcp", url.Host)
+		if err != nil {
+			return nil, err
+		}
+		tcp2ap[tcpAddr.String()] = url
+	}
+
+	stringParts := []string{}
+	updateNodeMap := func(service, scheme string) error {
+		_, addrs, err := lookupSRV(service, "tcp", dns)
+		if err != nil {
+			return err
+		}
+		for _, srv := range addrs {
+			port := fmt.Sprintf("%d", srv.Port)
+			host := net.JoinHostPort(srv.Target, port)
+			tcpAddr, terr := resolveTCPAddr("tcp", host)
+			if terr != nil {
+				err = terr
+				continue
+			}
+			n := ""
+			url, ok := tcp2ap[tcpAddr.String()]
+			if ok {
+				n = name
+			}
+			if n == "" {
+				n = fmt.Sprintf("%d", tempName)
+				tempName++
+			}
+			// SRV records have a trailing dot but URL shouldn't.
+			shortHost := strings.TrimSuffix(srv.Target, ".")
+			urlHost := net.JoinHostPort(shortHost, port)
+			if ok && url.Scheme != scheme {
+				err = fmt.Errorf("bootstrap at %s from DNS for %s has scheme mismatch with expected peer %s", scheme+"://"+urlHost, service, url.String())
+			} else {
+				stringParts = append(stringParts, fmt.Sprintf("%s=%s://%s", n, scheme, urlHost))
+			}
+		}
+		if len(stringParts) == 0 {
+			return err
+		}
+		return nil
+	}
+
+	err := updateNodeMap(service, serviceScheme)
+	if err != nil {
+		return nil, fmt.Errorf("error querying DNS SRV records for _%s %s", service, err)
+	}
+	return stringParts, nil
+}
+
+type SRVClients struct {
+	Endpoints []string
+	SRVs      []*net.SRV
+}
+
+// GetClient looks up the client endpoints for a service and domain.
+func GetClient(service, domain string) (*SRVClients, error) {
+	var urls []*url.URL
+	var srvs []*net.SRV
+
+	updateURLs := func(service, scheme string) error {
+		_, addrs, err := lookupSRV(service, "tcp", domain)
+		if err != nil {
+			return err
+		}
+		for _, srv := range addrs {
+			urls = append(urls, &url.URL{
+				Scheme: scheme,
+				Host:   net.JoinHostPort(srv.Target, fmt.Sprintf("%d", srv.Port)),
+			})
+		}
+		srvs = append(srvs, addrs...)
+		return nil
+	}
+
+	errHTTPS := updateURLs(service+"-ssl", "https")
+	errHTTP := updateURLs(service, "http")
+
+	if errHTTPS != nil && errHTTP != nil {
+		return nil, fmt.Errorf("dns lookup errors: %s and %s", errHTTPS, errHTTP)
+	}
+
+	endpoints := make([]string, len(urls))
+	for i := range urls {
+		endpoints[i] = urls[i].String()
+	}
+	return &SRVClients{Endpoints: endpoints, SRVs: srvs}, nil
+}

vendor/github.com/coreos/etcd/pkg/types/LICENSE

@@ -0,0 +1,202 @@
+
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

vendor/github.com/coreos/etcd/pkg/types/doc.go

@@ -0,0 +1,17 @@
+// Copyright 2015 The etcd Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+// Package types declares various data types and implements type-checking
+// functions.
+package types

vendor/github.com/coreos/etcd/pkg/types/id.go

@@ -0,0 +1,41 @@
+// Copyright 2015 The etcd Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package types
+
+import (
+	"strconv"
+)
+
+// ID represents a generic identifier which is canonically
+// stored as a uint64 but is typically represented as a
+// base-16 string for input/output
+type ID uint64
+
+func (i ID) String() string {
+	return strconv.FormatUint(uint64(i), 16)
+}
+
+// IDFromString attempts to create an ID from a base-16 string.
+func IDFromString(s string) (ID, error) {
+	i, err := strconv.ParseUint(s, 16, 64)
+	return ID(i), err
+}
+
+// IDSlice implements the sort interface
+type IDSlice []ID
+
+func (p IDSlice) Len() int           { return len(p) }
+func (p IDSlice) Less(i, j int) bool { return uint64(p[i]) < uint64(p[j]) }
+func (p IDSlice) Swap(i, j int)      { p[i], p[j] = p[j], p[i] }

vendor/github.com/coreos/etcd/pkg/types/set.go

@@ -0,0 +1,178 @@
+// Copyright 2015 The etcd Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package types
+
+import (
+	"reflect"
+	"sort"
+	"sync"
+)
+
+type Set interface {
+	Add(string)
+	Remove(string)
+	Contains(string) bool
+	Equals(Set) bool
+	Length() int
+	Values() []string
+	Copy() Set
+	Sub(Set) Set
+}
+
+func NewUnsafeSet(values ...string) *unsafeSet {
+	set := &unsafeSet{make(map[string]struct{})}
+	for _, v := range values {
+		set.Add(v)
+	}
+	return set
+}
+
+func NewThreadsafeSet(values ...string) *tsafeSet {
+	us := NewUnsafeSet(values...)
+	return &tsafeSet{us, sync.RWMutex{}}
+}
+
+type unsafeSet struct {
+	d map[string]struct{}
+}
+
+// Add adds a new value to the set (no-op if the value is already present)
+func (us *unsafeSet) Add(value string) {
+	us.d[value] = struct{}{}
+}
+
+// Remove removes the given value from the set
+func (us *unsafeSet) Remove(value string) {
+	delete(us.d, value)
+}
+
+// Contains returns whether the set contains the given value
+func (us *unsafeSet) Contains(value string) (exists bool) {
+	_, exists = us.d[value]
+	return exists
+}
+
+// ContainsAll returns whether the set contains all given values
+func (us *unsafeSet) ContainsAll(values []string) bool {
+	for _, s := range values {
+		if !us.Contains(s) {
+			return false
+		}
+	}
+	return true
+}
+
+// Equals returns whether the contents of two sets are identical
+func (us *unsafeSet) Equals(other Set) bool {
+	v1 := sort.StringSlice(us.Values())
+	v2 := sort.StringSlice(other.Values())
+	v1.Sort()
+	v2.Sort()
+	return reflect.DeepEqual(v1, v2)
+}
+
+// Length returns the number of elements in the set
+func (us *unsafeSet) Length() int {
+	return len(us.d)
+}
+
+// Values returns the values of the Set in an unspecified order.
+func (us *unsafeSet) Values() (values []string) {
+	values = make([]string, 0)
+	for val := range us.d {
+		values = append(values, val)
+	}
+	return values
+}
+
+// Copy creates a new Set containing the values of the first
+func (us *unsafeSet) Copy() Set {
+	cp := NewUnsafeSet()
+	for val := range us.d {
+		cp.Add(val)
+	}
+
+	return cp
+}
+
+// Sub removes all elements in other from the set
+func (us *unsafeSet) Sub(other Set) Set {
+	oValues := other.Values()
+	result := us.Copy().(*unsafeSet)
+
+	for _, val := range oValues {
+		if _, ok := result.d[val]; !ok {
+			continue
+		}
+		delete(result.d, val)
+	}
+
+	return result
+}
+
+type tsafeSet struct {
+	us *unsafeSet
+	m  sync.RWMutex
+}
+
+func (ts *tsafeSet) Add(value string) {
+	ts.m.Lock()
+	defer ts.m.Unlock()
+	ts.us.Add(value)
+}
+
+func (ts *tsafeSet) Remove(value string) {
+	ts.m.Lock()
+	defer ts.m.Unlock()
+	ts.us.Remove(value)
+}
+
+func (ts *tsafeSet) Contains(value string) (exists bool) {
+	ts.m.RLock()
+	defer ts.m.RUnlock()
+	return ts.us.Contains(value)
+}
+
+func (ts *tsafeSet) Equals(other Set) bool {
+	ts.m.RLock()
+	defer ts.m.RUnlock()
+	return ts.us.Equals(other)
+}
+
+func (ts *tsafeSet) Length() int {
+	ts.m.RLock()
+	defer ts.m.RUnlock()
+	return ts.us.Length()
+}
+
+func (ts *tsafeSet) Values() (values []string) {
+	ts.m.RLock()
+	defer ts.m.RUnlock()
+	return ts.us.Values()
+}
+
+func (ts *tsafeSet) Copy() Set {
+	ts.m.RLock()
+	defer ts.m.RUnlock()
+	usResult := ts.us.Copy().(*unsafeSet)
+	return &tsafeSet{usResult, sync.RWMutex{}}
+}
+
+func (ts *tsafeSet) Sub(other Set) Set {
+	ts.m.RLock()
+	defer ts.m.RUnlock()
+	usResult := ts.us.Sub(other).(*unsafeSet)
+	return &tsafeSet{usResult, sync.RWMutex{}}
+}

vendor/github.com/coreos/etcd/pkg/types/slice.go

@@ -0,0 +1,22 @@
+// Copyright 2015 The etcd Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package types
+
+// Uint64Slice implements sort interface
+type Uint64Slice []uint64
+
+func (p Uint64Slice) Len() int           { return len(p) }
+func (p Uint64Slice) Less(i, j int) bool { return p[i] < p[j] }
+func (p Uint64Slice) Swap(i, j int)      { p[i], p[j] = p[j], p[i] }

vendor/github.com/coreos/etcd/pkg/types/urls.go

@@ -0,0 +1,82 @@
+// Copyright 2015 The etcd Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package types
+
+import (
+	"errors"
+	"fmt"
+	"net"
+	"net/url"
+	"sort"
+	"strings"
+)
+
+type URLs []url.URL
+
+func NewURLs(strs []string) (URLs, error) {
+	all := make([]url.URL, len(strs))
+	if len(all) == 0 {
+		return nil, errors.New("no valid URLs given")
+	}
+	for i, in := range strs {
+		in = strings.TrimSpace(in)
+		u, err := url.Parse(in)
+		if err != nil {
+			return nil, err
+		}
+		if u.Scheme != "http" && u.Scheme != "https" && u.Scheme != "unix" && u.Scheme != "unixs" {
+			return nil, fmt.Errorf("URL scheme must be http, https, unix, or unixs: %s", in)
+		}
+		if _, _, err := net.SplitHostPort(u.Host); err != nil {
+			return nil, fmt.Errorf(`URL address does not have the form "host:port": %s`, in)
+		}
+		if u.Path != "" {
+			return nil, fmt.Errorf("URL must not contain a path: %s", in)
+		}
+		all[i] = *u
+	}
+	us := URLs(all)
+	us.Sort()
+
+	return us, nil
+}
+
+func MustNewURLs(strs []string) URLs {
+	urls, err := NewURLs(strs)
+	if err != nil {
+		panic(err)
+	}
+	return urls
+}
+
+func (us URLs) String() string {
+	return strings.Join(us.StringSlice(), ",")
+}
+
+func (us *URLs) Sort() {
+	sort.Sort(us)
+}
+func (us URLs) Len() int           { return len(us) }
+func (us URLs) Less(i, j int) bool { return us[i].String() < us[j].String() }
+func (us URLs) Swap(i, j int)      { us[i], us[j] = us[j], us[i] }
+
+func (us URLs) StringSlice() []string {
+	out := make([]string, len(us))
+	for i := range us {
+		out[i] = us[i].String()
+	}
+
+	return out
+}

vendor/github.com/coreos/etcd/pkg/types/urlsmap.go

@@ -0,0 +1,107 @@
+// Copyright 2015 The etcd Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package types
+
+import (
+	"fmt"
+	"sort"
+	"strings"
+)
+
+// URLsMap is a map from a name to its URLs.
+type URLsMap map[string]URLs
+
+// NewURLsMap returns a URLsMap instantiated from the given string,
+// which consists of discovery-formatted names-to-URLs, like:
+// mach0=http://1.1.1.1:2380,mach0=http://2.2.2.2::2380,mach1=http://3.3.3.3:2380,mach2=http://4.4.4.4:2380
+func NewURLsMap(s string) (URLsMap, error) {
+	m := parse(s)
+
+	cl := URLsMap{}
+	for name, urls := range m {
+		us, err := NewURLs(urls)
+		if err != nil {
+			return nil, err
+		}
+		cl[name] = us
+	}
+	return cl, nil
+}
+
+// NewURLsMapFromStringMap takes a map of strings and returns a URLsMap. The
+// string values in the map can be multiple values separated by the sep string.
+func NewURLsMapFromStringMap(m map[string]string, sep string) (URLsMap, error) {
+	var err error
+	um := URLsMap{}
+	for k, v := range m {
+		um[k], err = NewURLs(strings.Split(v, sep))
+		if err != nil {
+			return nil, err
+		}
+	}
+	return um, nil
+}
+
+// String turns URLsMap into discovery-formatted name-to-URLs sorted by name.
+func (c URLsMap) String() string {
+	var pairs []string
+	for name, urls := range c {
+		for _, url := range urls {
+			pairs = append(pairs, fmt.Sprintf("%s=%s", name, url.String()))
+		}
+	}
+	sort.Strings(pairs)
+	return strings.Join(pairs, ",")
+}
+
+// URLs returns a list of all URLs.
+// The returned list is sorted in ascending lexicographical order.
+func (c URLsMap) URLs() []string {
+	var urls []string
+	for _, us := range c {
+		for _, u := range us {
+			urls = append(urls, u.String())
+		}
+	}
+	sort.Strings(urls)
+	return urls
+}
+
+// Len returns the size of URLsMap.
+func (c URLsMap) Len() int {
+	return len(c)
+}
+
+// parse parses the given string and returns a map listing the values specified for each key.
+func parse(s string) map[string][]string {
+	m := make(map[string][]string)
+	for s != "" {
+		key := s
+		if i := strings.IndexAny(key, ","); i >= 0 {
+			key, s = key[:i], key[i+1:]
+		} else {
+			s = ""
+		}
+		if key == "" {
+			continue
+		}
+		value := ""
+		if i := strings.Index(key, "="); i >= 0 {
+			key, value = key[:i], key[i+1:]
+		}
+		m[key] = append(m[key], value)
+	}
+	return m
+}

vendor/github.com/coreos/etcd/vendor/github.com/coreos/go-semver/semver/LICENSE

@@ -0,0 +1,202 @@
+
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

vendor/github.com/coreos/etcd/vendor/github.com/coreos/go-semver/semver/semver.go

@@ -0,0 +1,268 @@
+// Copyright 2013-2015 CoreOS, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+// Semantic Versions http://semver.org
+package semver
+
+import (
+	"bytes"
+	"errors"
+	"fmt"
+	"strconv"
+	"strings"
+)
+
+type Version struct {
+	Major      int64
+	Minor      int64
+	Patch      int64
+	PreRelease PreRelease
+	Metadata   string
+}
+
+type PreRelease string
+
+func splitOff(input *string, delim string) (val string) {
+	parts := strings.SplitN(*input, delim, 2)
+
+	if len(parts) == 2 {
+		*input = parts[0]
+		val = parts[1]
+	}
+
+	return val
+}
+
+func New(version string) *Version {
+	return Must(NewVersion(version))
+}
+
+func NewVersion(version string) (*Version, error) {
+	v := Version{}
+
+	if err := v.Set(version); err != nil {
+		return nil, err
+	}
+
+	return &v, nil
+}
+
+// Must is a helper for wrapping NewVersion and will panic if err is not nil.
+func Must(v *Version, err error) *Version {
+	if err != nil {
+		panic(err)
+	}
+	return v
+}
+
+// Set parses and updates v from the given version string. Implements flag.Value
+func (v *Version) Set(version string) error {
+	metadata := splitOff(&version, "+")
+	preRelease := PreRelease(splitOff(&version, "-"))
+	dotParts := strings.SplitN(version, ".", 3)
+
+	if len(dotParts) != 3 {
+		return fmt.Errorf("%s is not in dotted-tri format", version)
+	}
+
+	parsed := make([]int64, 3, 3)
+
+	for i, v := range dotParts[:3] {
+		val, err := strconv.ParseInt(v, 10, 64)
+		parsed[i] = val
+		if err != nil {
+			return err
+		}
+	}
+
+	v.Metadata = metadata
+	v.PreRelease = preRelease
+	v.Major = parsed[0]
+	v.Minor = parsed[1]
+	v.Patch = parsed[2]
+	return nil
+}
+
+func (v Version) String() string {
+	var buffer bytes.Buffer
+
+	fmt.Fprintf(&buffer, "%d.%d.%d", v.Major, v.Minor, v.Patch)
+
+	if v.PreRelease != "" {
+		fmt.Fprintf(&buffer, "-%s", v.PreRelease)
+	}
+
+	if v.Metadata != "" {
+		fmt.Fprintf(&buffer, "+%s", v.Metadata)
+	}
+
+	return buffer.String()
+}
+
+func (v *Version) UnmarshalYAML(unmarshal func(interface{}) error) error {
+	var data string
+	if err := unmarshal(&data); err != nil {
+		return err
+	}
+	return v.Set(data)
+}
+
+func (v Version) MarshalJSON() ([]byte, error) {
+	return []byte(`"` + v.String() + `"`), nil
+}
+
+func (v *Version) UnmarshalJSON(data []byte) error {
+	l := len(data)
+	if l == 0 || string(data) == `""` {
+		return nil
+	}
+	if l < 2 || data[0] != '"' || data[l-1] != '"' {
+		return errors.New("invalid semver string")
+	}
+	return v.Set(string(data[1 : l-1]))
+}
+
+// Compare tests if v is less than, equal to, or greater than versionB,
+// returning -1, 0, or +1 respectively.
+func (v Version) Compare(versionB Version) int {
+	if cmp := recursiveCompare(v.Slice(), versionB.Slice()); cmp != 0 {
+		return cmp
+	}
+	return preReleaseCompare(v, versionB)
+}
+
+// Equal tests if v is equal to versionB.
+func (v Version) Equal(versionB Version) bool {
+	return v.Compare(versionB) == 0
+}
+
+// LessThan tests if v is less than versionB.
+func (v Version) LessThan(versionB Version) bool {
+	return v.Compare(versionB) < 0
+}
+
+// Slice converts the comparable parts of the semver into a slice of integers.
+func (v Version) Slice() []int64 {
+	return []int64{v.Major, v.Minor, v.Patch}
+}
+
+func (p PreRelease) Slice() []string {
+	preRelease := string(p)
+	return strings.Split(preRelease, ".")
+}
+
+func preReleaseCompare(versionA Version, versionB Version) int {
+	a := versionA.PreRelease
+	b := versionB.PreRelease
+
+	/* Handle the case where if two versions are otherwise equal it is the
+	 * one without a PreRelease that is greater */
+	if len(a) == 0 && (len(b) > 0) {
+		return 1
+	} else if len(b) == 0 && (len(a) > 0) {
+		return -1
+	}
+
+	// If there is a prerelease, check and compare each part.
+	return recursivePreReleaseCompare(a.Slice(), b.Slice())
+}
+
+func recursiveCompare(versionA []int64, versionB []int64) int {
+	if len(versionA) == 0 {
+		return 0
+	}
+
+	a := versionA[0]
+	b := versionB[0]
+
+	if a > b {
+		return 1
+	} else if a < b {
+		return -1
+	}
+
+	return recursiveCompare(versionA[1:], versionB[1:])
+}
+
+func recursivePreReleaseCompare(versionA []string, versionB []string) int {
+	// A larger set of pre-release fields has a higher precedence than a smaller set,
+	// if all of the preceding identifiers are equal.
+	if len(versionA) == 0 {
+		if len(versionB) > 0 {
+			return -1
+		}
+		return 0
+	} else if len(versionB) == 0 {
+		// We're longer than versionB so return 1.
+		return 1
+	}
+
+	a := versionA[0]
+	b := versionB[0]
+
+	aInt := false
+	bInt := false
+
+	aI, err := strconv.Atoi(versionA[0])
+	if err == nil {
+		aInt = true
+	}
+
+	bI, err := strconv.Atoi(versionB[0])
+	if err == nil {
+		bInt = true
+	}
+
+	// Handle Integer Comparison
+	if aInt && bInt {
+		if aI > bI {
+			return 1
+		} else if aI < bI {
+			return -1
+		}
+	}
+
+	// Handle String Comparison
+	if a > b {
+		return 1
+	} else if a < b {
+		return -1
+	}
+
+	return recursivePreReleaseCompare(versionA[1:], versionB[1:])
+}
+
+// BumpMajor increments the Major field by 1 and resets all other fields to their default values
+func (v *Version) BumpMajor() {
+	v.Major += 1
+	v.Minor = 0
+	v.Patch = 0
+	v.PreRelease = PreRelease("")
+	v.Metadata = ""
+}
+
+// BumpMinor increments the Minor field by 1 and resets all other fields to their default values
+func (v *Version) BumpMinor() {
+	v.Minor += 1
+	v.Patch = 0
+	v.PreRelease = PreRelease("")
+	v.Metadata = ""
+}
+
+// BumpPatch increments the Patch field by 1 and resets all other fields to their default values
+func (v *Version) BumpPatch() {
+	v.Patch += 1
+	v.PreRelease = PreRelease("")
+	v.Metadata = ""
+}