Commit 51501f92 authored by David Lu's avatar David Lu Committed by enahum
Browse files

PLT-3753 Added Segment analytics (#3972)

parent 47d77d25
Mattermost Software Privacy Policy
For the purposes of this Mattermost Software Privacy Policy (the “Privacy Policy”) “you” means you as an individual user, or you as an entity using the software. “Mattermost” or “we” means Mattermost, Inc.
This privacy policy is designed to help you understand which information of yours may be collected by the Mattermost software and how that information is used by Mattermost.
By using the Mattermost software, you consent to the terms of this Privacy Policy.
What information do we collect?
As described below, the kinds of information that Mattermost collects and uses may depend on the configuration options selected by your administrator:
Security Fix Alert Feature:
Mattermost offers a Security Fix Alert feature that is designed to protect your system by attempting to alert you to relevant, high priority security fixes released for your system. You may opt out of receiving Security Alerts by switching off the feature in the [System Console or via the configuration file (see link)](https://docs.mattermost.com/administration/config-settings.html#enable-security-alerts). If you opt out of the Security Fix Alert feature, Mattermost will not collect information required for communication of relevant Security Fix Alerts for your installation and may be unable to contact you regarding such alerts.
When you deploy a Mattermost server with the Security Alert feature enabled, information about your installation that is needed to (i) determine whether your system is at risk of security issues from time-to-time, and (ii) permit notification to you of security fixes and updates to protect your system, will be collected over an encrypted channel and communicated to Mattermost. Such collected and communicated information includes configuration options, the version and type of software components deployed, such as the Mattermost server, database and operating system, activity indicators counting the number of active users and teams, the email addresses of system administrators, and a server identifier. If you are using an activated copy of Mattermost Enterprise Edition, Mattermost may also collect information about available enterprise features including commercial license key registration information.
Error and Diagnostics Reporting Feature:
Mattermost offers an Error and Diagnostics Reporting feature that, when enabled, permits the transmission of information to Mattermost about your system that is needed to diagnose errors and improve the functionality of Mattermost software for deployments with your usage pattern. This information will be collected over an encrypted channel, and include reports of critical errors, configuration options, the version and type of software components deployed, such as Mattermost server, database and operating system, anonymous usage statistics including changes to system preferences and creation of channels and posts. If you are using an activated copy of Mattermost Enterprise Edition, Mattermost may also collect information about usage of enterprise features as well as commercial license key registration information.
You may opt out of the Error and Diagnostics Reporting Feature by switching the feature off in the [System Console user interface or via the system configuration file (see link)]( https://docs.mattermost.com/administration/config-settings.html#enable-error-and-diagnostics-reporting). If you opt out of the Error and Diagnostics Reporting feature, Mattermost will not collect your information required for reporting of errors and diagnostics from your deployment and may be less able to improve system performance for deployments with your usage pattern in future.
Mattermost Hosted Push Notification Service:
If you choose not to compile your own Mattermost push notification service from the source code provided and instead to use the Hosted Push Notification Service (“HPNS”) provided by Mattermost, a privacy policy for HPNS is available here: https://about.mattermost.com/hpns-privacy/
What do we use your information for?
The information we collect from you may be used in the following ways:
To alert you of updates designed to protect your system from newly discovered potential security attacks.
To improve future versions of the Mattermost software for deployments with similar usage patterns as your installation — we continually strive to improve the performance, reliability and functionality of our product, and our efforts to do so are assisted the usage information and feedback we receive from you.
How do we protect your information?
Mattermost takes care to protect the information you provide as part of your use of the Mattermost software from misuse and unauthorized access or disclosure. These steps take into account the sensitivity of the information we collect, process and store, as well as the current state of technology. You may opt-out of the [Security Alerts (see link)](https://docs.mattermost.com/administration/config-settings.html#enable-security-alerts) and [Error and Diagnostics Reporting (see link)](https://docs.mattermost.com/administration/config-settings.html#enable-error-and-diagnostics) services any time from the System Console or via the configuration file, as described previously. Any information that we consider potentially sensitive is transmitted through encrypted channels and we follow generally accepted industry standards to protect the data collected by us, both during transmission and after we receive it.
Internally, we restrict access to all personally identifiable information we receive from You to our personnel that need access to the information in order to do their jobs and that are authorized to handle such information. Our staff is limited and all personnel are committed to adhere to our privacy and security policies. All personnel execute nondisclosure agreements, which provide explicit confidentiality protections. Any staff member who violates our privacy and/or security policies is subject to possible termination of contract and civil/criminal prosecution.
Do we disclose any information to outside parties?
We do not sell, trade, or otherwise transfer the information we collect from you to unaffiliated third parties. We do, however, share the information we collect from you with trusted third parties who assist us in operating our site, conducting our business, or servicing you, provided that those parties agree to keep your information confidential and secure. We also reserve the right to release your information when we believe that release is necessary to comply with the law, enforce our site policies, or protect our own or another’s intellectual property rights, property, or safety.
Changes to our privacy policy
If we decide to change this Privacy Policy, we will include those changes with updated versions of the software.
Last revised September 2016.
......@@ -61,7 +61,8 @@
"FileLevel": "INFO",
"FileFormat": "",
"FileLocation": "",
"EnableWebhookDebugging": true
"EnableWebhookDebugging": true,
"EnableDiagnostics": true
},
"PasswordSettings": {
"MinimumLength": 5,
......
hash: aa2fadc7f997a93e78d46d009d5695fb079697453d83e6a6dd7481d46ce73b7e
updated: 2016-05-12T19:14:13.836695608-04:00
hash: 5edc1c9990dbfc36e041dcdabeab2dd8fdd0c8b8b0540b54d515a0044f63952c
updated: 2016-09-06T15:41:59.168710473-04:00
imports:
- name: github.com/alecthomas/log4go
version: e5dc62318d9bd58682f1dceb53a4b24e8253682f
......@@ -42,6 +42,8 @@ imports:
version: 9c19ed558d5df4da88e2ade9c8940d742aef0e7e
- name: github.com/gorilla/websocket
version: 1f512fc3f05332ba7117626cdfb4e07474e58e60
- name: github.com/jehiah/go-strftime
version: 834e15c05a45371503440cc195bbd05c9a0968d9
- name: github.com/lib/pq
version: ee1442bda7bd1b6a84e913bdb421cb1874ec629d
subpackages:
......@@ -70,6 +72,10 @@ imports:
subpackages:
- exif
- tiff
- name: github.com/segmentio/analytics-go
version: bdb0aeca8a993b292b85c9ec17b5ce0ff81848c8
- name: github.com/segmentio/backo-go
version: 204274ad699c0983a70203a566887f17a717fef4
- name: github.com/vaughan0/go-ini
version: a98ad7ee00ec53921f08832bc06ecf7fd600e6a1
- name: golang.org/x/crypto
......@@ -77,6 +83,8 @@ imports:
subpackages:
- bcrypt
- blowfish
- name: github.com/xtgo/uuid
version: a0b114877d4caeffbd7f87e3757c17fce570fea7
- name: golang.org/x/image
version: f551d3a6b7fc11df315ad9e18b404280680f8bec
subpackages:
......@@ -99,4 +107,4 @@ imports:
- store
- name: gopkg.in/yaml.v2
version: a83829b6f1293c91addabc89d0571c246397bbf4
devImports: []
devImports: []
\ No newline at end of file
......@@ -39,3 +39,4 @@ import:
- package: gopkg.in/throttled/throttled.v1
subpackages:
- store
- package: github.com/segmentio/analytics-go
......@@ -87,6 +87,10 @@
"id": "api.admin.recycle_db_start.warn",
"translation": "Attempting to recycle the database connection"
},
{
"id": "utils.diagnostic.analytics_not_found.app_error",
"translation": "Analytics not initialized"
},
{
"id": "api.admin.remove_certificate.delete.app_error",
"translation": "An error occurred while deleting the certificate. Make sure the file config/{{.Filename}} exists."
......@@ -4219,6 +4223,10 @@
"id": "store.sql_session.analytics_session_count.app_error",
"translation": "We couldn't count the sessions"
},
{
"id": "store.sql_system.get_version.app_error",
"translation": "We couldn't get the database version"
},
{
"id": "store.sql_session.cleanup_expired_sessions.app_error",
"translation": "We encountered an error while deleting expired user sessions"
......
......@@ -279,6 +279,47 @@ func doSecurityAndDiagnostics() {
}
}
}
if *utils.Cfg.LogSettings.EnableDiagnostics {
utils.SendGeneralDiagnostics()
sendServerDiagnostics()
}
}
func sendServerDiagnostics() {
var userCount int64
var activeUserCount int64
var teamCount int64
if ucr := <-api.Srv.Store.User().GetTotalUsersCount(); ucr.Err == nil {
userCount = ucr.Data.(int64)
}
if ucr := <-api.Srv.Store.Status().GetTotalActiveUsersCount(); ucr.Err == nil {
activeUserCount = ucr.Data.(int64)
}
if tcr := <-api.Srv.Store.Team().AnalyticsTeamCount(); tcr.Err == nil {
teamCount = tcr.Data.(int64)
}
utils.SendDiagnostic(utils.TRACK_ACTIVITY, map[string]interface{}{
"users": userCount,
"active_users": activeUserCount,
"teams": teamCount,
})
edition := model.BuildEnterpriseReady
version := model.CurrentVersion
database := utils.Cfg.SqlSettings.DriverName
operatingSystem := runtime.GOOS
utils.SendDiagnostic(utils.TRACK_VERSION, map[string]interface{}{
"edition": edition,
"version": version,
"database": database,
"operating_system": operatingSystem,
})
}
func runSecurityAndDiagnosticsJob() {
......
......@@ -118,6 +118,7 @@ type LogSettings struct {
FileFormat string
FileLocation string
EnableWebhookDebugging bool
EnableDiagnostics *bool
}
type PasswordSettings struct {
......@@ -781,6 +782,11 @@ func (o *Config) SetDefaults() {
*o.LocalizationSettings.AvailableLocales = ""
}
if o.LogSettings.EnableDiagnostics == nil {
o.LogSettings.EnableDiagnostics = new(bool)
*o.LogSettings.EnableDiagnostics = true
}
if o.SamlSettings.Enable == nil {
o.SamlSettings.Enable = new(bool)
*o.SamlSettings.Enable = false
......
......@@ -46,6 +46,22 @@ type Features struct {
FutureFeatures *bool `json:"future_features"`
}
func (f *Features) ToMap() map[string]interface{} {
return map[string]interface{}{
"ldap": *f.LDAP,
"mfa": *f.MFA,
"google": *f.GoogleOAuth,
"office365": *f.Office365OAuth,
"compliance": *f.Compliance,
"cluster": *f.Cluster,
"custom_brand": *f.CustomBrand,
"mhpns": *f.MHPNS,
"saml": *f.SAML,
"password": *f.PasswordRequirements,
"future": *f.FutureFeatures,
}
}
func (f *Features) SetDefaults() {
if f.FutureFeatures == nil {
f.FutureFeatures = new(bool)
......
......@@ -4,8 +4,9 @@
package store
import (
"github.com/mattermost/platform/model"
"testing"
"github.com/mattermost/platform/model"
)
func TestSqlSystemStore(t *testing.T) {
......
......@@ -4,9 +4,10 @@
package store
import (
"time"
l4g "github.com/alecthomas/log4go"
"github.com/mattermost/platform/model"
"time"
)
type StoreResult struct {
......
......@@ -241,6 +241,7 @@ func getClientConfig(c *model.Config) map[string]string {
props["EnablePostIconOverride"] = strconv.FormatBool(c.ServiceSettings.EnablePostIconOverride)
props["EnableTesting"] = strconv.FormatBool(c.ServiceSettings.EnableTesting)
props["EnableDeveloper"] = strconv.FormatBool(*c.ServiceSettings.EnableDeveloper)
props["EnableDiagnostics"] = strconv.FormatBool(*c.LogSettings.EnableDiagnostics)
props["SendEmailNotifications"] = strconv.FormatBool(c.EmailSettings.SendEmailNotifications)
props["SendPushNotifications"] = strconv.FormatBool(*c.EmailSettings.SendPushNotifications)
......
......@@ -3,13 +3,11 @@
package utils
import (
"net/http"
"net/url"
)
import "github.com/segmentio/analytics-go"
const (
DIAGNOSTIC_URL = "https://d7zmvsa9e04kk.cloudfront.net"
SEGMENT_KEY = "ua1qQtmgOZWIM23YjD842tQAsN7Ydi5X"
PROP_DIAGNOSTIC_ID = "id"
PROP_DIAGNOSTIC_CATEGORY = "c"
......@@ -22,16 +20,156 @@ const (
PROP_DIAGNOSTIC_TEAM_COUNT = "tc"
PROP_DIAGNOSTIC_ACTIVE_USER_COUNT = "auc"
PROP_DIAGNOSTIC_UNIT_TESTS = "ut"
TRACK_CONFIG_SERVICE = "service"
TRACK_CONFIG_TEAM = "team"
TRACK_CONFIG_SQL = "sql"
TRACK_CONFIG_LOG = "log"
TRACK_CONFIG_FILE = "file"
TRACK_CONFIG_RATE = "rate"
TRACK_CONFIG_EMAIL = "email"
TRACK_CONFIG_PRIVACY = "privacy"
TRACK_CONFIG_OAUTH = "oauth"
TRACK_CONFIG_LDAP = "ldap"
TRACK_CONFIG_COMPLIANCE = "compliance"
TRACK_CONFIG_LOCALIZATION = "localization"
TRACK_CONFIG_SAML = "saml"
TRACK_LICENSE = "license"
TRACK_ACTIVITY = "activity"
TRACK_VERSION = "version"
)
func SendDiagnostic(values url.Values) {
if *Cfg.ServiceSettings.EnableSecurityFixAlert {
var client *analytics.Client
func SendGeneralDiagnostics() {
if *Cfg.LogSettings.EnableDiagnostics {
initDiagnostics()
trackConfig()
trackLicense()
}
}
func initDiagnostics() {
if client == nil {
client = analytics.New(SEGMENT_KEY)
client.Identify(&analytics.Identify{
UserId: CfgDiagnosticId,
})
}
}
func SendDiagnostic(event string, properties map[string]interface{}) {
client.Track(&analytics.Track{
Event: event,
UserId: CfgDiagnosticId,
Properties: properties,
})
}
func trackConfig() {
SendDiagnostic(TRACK_CONFIG_SERVICE, map[string]interface{}{
"web_server_mode": *Cfg.ServiceSettings.WebserverMode,
"enable_security_fix_alert": *Cfg.ServiceSettings.EnableSecurityFixAlert,
"enable_insecure_outgoing_connections": *Cfg.ServiceSettings.EnableInsecureOutgoingConnections,
"enable_incoming_webhooks": Cfg.ServiceSettings.EnableIncomingWebhooks,
"enable_outgoing_webhooks": Cfg.ServiceSettings.EnableOutgoingWebhooks,
"enable_commands": *Cfg.ServiceSettings.EnableCommands,
"enable_only_admin_integrations": *Cfg.ServiceSettings.EnableOnlyAdminIntegrations,
"enable_post_username_override": Cfg.ServiceSettings.EnablePostUsernameOverride,
"enable_post_icon_override": Cfg.ServiceSettings.EnablePostIconOverride,
"enable_custom_emoji": *Cfg.ServiceSettings.EnableCustomEmoji,
"restrict_custom_emoji_creation": *Cfg.ServiceSettings.RestrictCustomEmojiCreation,
"enable_testing": Cfg.ServiceSettings.EnableTesting,
"enable_developer": *Cfg.ServiceSettings.EnableDeveloper,
})
res, err := http.Get(DIAGNOSTIC_URL + "/i?" + values.Encode())
if err != nil {
return
}
SendDiagnostic(TRACK_CONFIG_TEAM, map[string]interface{}{
"enable_user_creation": Cfg.TeamSettings.EnableUserCreation,
"enable_team_creation": Cfg.TeamSettings.EnableTeamCreation,
"restrict_team_names": *Cfg.TeamSettings.RestrictTeamNames,
"restrict_team_invite": *Cfg.TeamSettings.RestrictTeamInvite,
"restrict_public_channel_management": *Cfg.TeamSettings.RestrictPublicChannelManagement,
"restrict_private_channel_management": *Cfg.TeamSettings.RestrictPrivateChannelManagement,
"enable_open_server": *Cfg.TeamSettings.EnableOpenServer,
"enable_custom_brand": *Cfg.TeamSettings.EnableCustomBrand,
})
SendDiagnostic(TRACK_CONFIG_SQL, map[string]interface{}{
"driver_name": Cfg.SqlSettings.DriverName,
})
SendDiagnostic(TRACK_CONFIG_LOG, map[string]interface{}{
"enable_console": Cfg.LogSettings.EnableConsole,
"console_level": Cfg.LogSettings.ConsoleLevel,
"enable_file": Cfg.LogSettings.EnableFile,
"file_level": Cfg.LogSettings.FileLevel,
"enable_webhook_debugging": Cfg.LogSettings.EnableWebhookDebugging,
})
SendDiagnostic(TRACK_CONFIG_FILE, map[string]interface{}{
"enable_public_links": Cfg.FileSettings.EnablePublicLink,
})
SendDiagnostic(TRACK_CONFIG_RATE, map[string]interface{}{
"enable_rate_limiter": Cfg.RateLimitSettings.EnableRateLimiter,
"vary_by_remote_address": Cfg.RateLimitSettings.VaryByRemoteAddr,
})
SendDiagnostic(TRACK_CONFIG_EMAIL, map[string]interface{}{
"enable_sign_up_with_email": Cfg.EmailSettings.EnableSignUpWithEmail,
"enable_sign_in_with_email": *Cfg.EmailSettings.EnableSignInWithEmail,
"enable_sign_in_with_username": *Cfg.EmailSettings.EnableSignInWithUsername,
"require_email_verification": Cfg.EmailSettings.RequireEmailVerification,
"send_email_notifications": Cfg.EmailSettings.SendEmailNotifications,
"connection_security": Cfg.EmailSettings.ConnectionSecurity,
"send_push_notifications": *Cfg.EmailSettings.SendPushNotifications,
"push_notification_contents": *Cfg.EmailSettings.PushNotificationContents,
})
SendDiagnostic(TRACK_CONFIG_PRIVACY, map[string]interface{}{
"show_email_address": Cfg.PrivacySettings.ShowEmailAddress,
"show_full_name": Cfg.PrivacySettings.ShowFullName,
})
SendDiagnostic(TRACK_CONFIG_OAUTH, map[string]interface{}{
"gitlab": Cfg.GitLabSettings.Enable,
"google": Cfg.GoogleSettings.Enable,
"office365": Cfg.Office365Settings.Enable,
})
SendDiagnostic(TRACK_CONFIG_LDAP, map[string]interface{}{
"enable": *Cfg.LdapSettings.Enable,
"connection_security": *Cfg.LdapSettings.ConnectionSecurity,
"skip_certificate_verification": *Cfg.LdapSettings.SkipCertificateVerification,
})
SendDiagnostic(TRACK_CONFIG_COMPLIANCE, map[string]interface{}{
"enable": *Cfg.ComplianceSettings.Enable,
"enable_daily": *Cfg.ComplianceSettings.EnableDaily,
})
SendDiagnostic(TRACK_CONFIG_LOCALIZATION, map[string]interface{}{
"default_server_locale": *Cfg.LocalizationSettings.DefaultServerLocale,
"default_client_locale": *Cfg.LocalizationSettings.DefaultClientLocale,
"available_locales": *Cfg.LocalizationSettings.AvailableLocales,
})
SendDiagnostic(TRACK_CONFIG_SAML, map[string]interface{}{
"enable": *Cfg.SamlSettings.Enable,
})
}
res.Body.Close()
func trackLicense() {
if IsLicensed {
SendDiagnostic(TRACK_LICENSE, map[string]interface{}{
"name": License.Customer.Name,
"company": License.Customer.Company,
"issued": License.IssuedAt,
"start": License.StartsAt,
"expire": License.ExpiresAt,
"users": *License.Features.Users,
"features": License.Features.ToMap(),
})
}
}
go-strftime
===========
go implementation of strftime
\ No newline at end of file
// go implementation of strftime
package strftime
import (
"strings"
"time"
)
// taken from time/format.go
var conversion = map[rune]string {
/*stdLongMonth */ 'B':"January",
/*stdMonth */ 'b': "Jan",
// stdNumMonth */ 'm': "1",
/*stdZeroMonth */ 'm': "01",
/*stdLongWeekDay */ 'A': "Monday",
/*stdWeekDay */ 'a': "Mon",
// stdDay */ 'd': "2",
// stdUnderDay */ 'd': "_2",
/*stdZeroDay */ 'd': "02",
/*stdHour */ 'H': "15",
// stdHour12 */ 'I': "3",
/*stdZeroHour12 */ 'I': "03",
// stdMinute */ 'M': "4",
/*stdZeroMinute */ 'M': "04",
// stdSecond */ 'S': "5",
/*stdZeroSecond */ 'S': "05",
/*stdLongYear */ 'Y': "2006",
/*stdYear */ 'y': "06",
/*stdPM */ 'p': "PM",
// stdpm */ 'p': "pm",
/*stdTZ */ 'Z': "MST",
// stdISO8601TZ */ 'z': "Z0700", // prints Z for UTC
// stdISO8601ColonTZ */ 'z': "Z07:00", // prints Z for UTC
/*stdNumTZ */ 'z': "-0700", // always numeric
// stdNumShortTZ */ 'b': "-07", // always numeric
// stdNumColonTZ */ 'b': "-07:00", // always numeric
}
// This is an alternative to time.Format because no one knows
// what date 040305 is supposed to create when used as a 'layout' string
// this takes standard strftime format options. For a complete list
// of format options see http://strftime.org/
func Format(format string, t time.Time) string {
retval := make([]byte, 0, len(format))
for i, ni := 0, 0; i < len(format); i = ni + 2 {
ni = strings.IndexByte(format[i:], '%')
if ni < 0 {
ni = len(format)
} else {
ni += i
}
retval = append(retval, []byte(format[i:ni])...)
if ni + 1 < len(format) {
c := format[ni + 1]
if c == '%' {
retval = append(retval, '%')
} else {
if layoutCmd, ok := conversion[rune(c)]; ok {
retval = append(retval, []byte(t.Format(layoutCmd))...)
} else {
retval = append(retval, '%', c)
}
}
} else {
if ni < len(format) {
retval = append(retval, '%')
}
}
}
return string(retval)
}
package strftime
import (
"time"
"fmt"
"testing"
)
func ExampleFormat() {
t := time.Unix(1340244776, 0)
utc, _ := time.LoadLocation("UTC")
t = t.In(utc)
fmt.Println(Format("%Y-%m-%d %H:%M:%S", t))
// Output:
// 2012-06-21 02:12:56
}
func TestNoLeadingPercentSign(t *testing.T) {
tm := time.Unix(1340244776, 0)
utc, _ := time.LoadLocation("UTC")
tm = tm.In(utc)
result := Format("aaabbb0123456789%Y", tm)
if result != "aaabbb01234567892012" {
t.Logf("%s != %s", result, "aaabbb01234567892012")
t.Fail()
}
}
func TestUnsupported(t *testing.T) {
tm := time.Unix(1340244776, 0)
utc, _ := time.LoadLocation("UTC")
tm = tm.In(utc)
result := Format("%0%1%%%2", tm)
if result != "%0%1%%2" {
t.Logf("%s != %s", result, "%0%1%%2")
t.Fail()
}
}
{
"ImportPath": "github.com/segmentio/analytics-go",
"GoVersion": "go1.4.2",
"Packages": [
"./..."
],
"Deps": [
{
"ImportPath": "github.com/jehiah/go-strftime",
"Rev": "834e15c05a45371503440cc195bbd05c9a0968d9"
},
{
"ImportPath": "github.com/xtgo/uuid",
"Rev": "a0b114877d4caeffbd7f87e3757c17fce570fea7"
}
]
}
This directory tree is generated automatically by godep.
Please do not edit.
See https://github.com/tools/godep for more information.
# Compiled Object files, Static and Dynamic libs (Shared Objects)
*.o
*.a
*.so
# Folders
_obj
_test
# Architecture specific extensions/prefixes
*.[568vq]
[568vq].out
*.cgo1.go
*.cgo2.c
_cgo_defun.c
_cgo_gotypes.go
_cgo_export.*
_testmain.go
*.exe