Commit Graph

66 Commits (730c4f74dbf7ebc74e43bc9b34cc83e2a644f78b)

Author SHA1 Message Date
Andrew Morgan 7736e247b7
Send Application Service Events (#477)
* Prevent sql scanning into nil value in accounts_table

Signed-off-by: Andrew Morgan <andrewm@matrix.org>

* Remove uneccessary logging, null checking

* Don't forget to set the localpart

* Simplify error checking

* Store And Send Application Service Events

* Modify INSTALL.md and dendrite-config.yaml for the new appservice database
* Correct all instances of casing on 'application service' to align with
spec
* Store incoming events that an app service is interested in in the
database to be later read by transaction workers.
* Retrieve these events from transaction workers, one per AS.
* Minimal transaction ID data is stored as well to recover after
server failure.
* Send events to AS and exponentially backoff on failure.

Signed-off-by: Andrew Morgan <andrewm@matrix.org>

* Finish my own sentences.

* Fix up database interaction

* Change to event-based AS sending

* Reduce cyclomatic complexity

* Appease the errcheck gods

* Delete by int ID instead of string.

This was causing some events to not be deleted, as < an eventID doesn't
really make much sense.

* Check if there are more events to send before sleeping

* Send same transaction if last send attempt failed

* Don't backoff on non-200s, tight send loop, 1 event query

* Remove tight send loop. Fix events not being deleted

* Additionally order by event id, track main.go

* Return the last txnID, which our events are using

* Remove old main.go file

* Prevent duplicate events from being sent...

* Strip event content if it doesn't contain anything

Signed-off-by: Andrew Morgan <andrewm@matrix.org>

* Update gomatrixserverlib and use Unsigned AS event prop

* Fixes

* Fix sync server comment
* Remove unnecessary printlns
* Use logrus Fields
* Worker state methods
* Remove sillyness

* Fix up event filtering

* Handle transaction event limit in loop

* Switch to using a sequence for transaction IDs

* Don't verify self-signed AS certificates

* Fix logging

* Use gmsl.Event instead of AS-only event in transactions

Also clear up the logic on lookupStateEvents a little bit.

* Change invalid_txn_id to global (for efficiency)

* Use a bool for EventsReady instead of an int
2018-07-05 09:34:59 -07:00
Andrew Morgan a76e31f1b3
Remove dependency errors (#507) 2018-06-18 05:44:25 -07:00
Andrew Morgan 241b1b5ace
Update gomatrixserverlib version (#476)
Signed-off-by: Andrew Morgan <andrewm@matrix.org>
2018-06-01 17:42:55 +01:00
Andrew Morgan c238048599
Update Prometheus metrics tracking (#459)
Signed-off-by: Andrew Morgan <andrewm@matrix.org>
2018-05-23 15:42:08 +01:00
Andrew Morgan f5cc5bd9bb
Revert "Add token generation using go macaroon (#437)" (#448)
This reverts commit afeab7b2d4.

Signed-off-by: Andrew Morgan <andrewm@matrix.org>
2018-05-22 11:29:19 +01:00
Anant Prakash afeab7b2d4 Add token generation using go macaroon (#437)
* Add Go macaroon library

Signed-off-by: Anant Prakash <anantprakashjsr@gmail.com>

* Add macaroon generation and serialization, for login token.

Signed-off-by: Anant Prakash <anantprakashjsr@gmail.com>

* Remove copyright, trim empty lines

* Make Serialize functions private

* Fix typos
2018-05-22 10:13:58 +01:00
Tristan Claverie 58c10c6d54 Enable debug logs using a configuration parameter and put all logs in a single file (#379) 2018-04-10 13:21:20 +01:00
Andrew Morgan fa362ecef2 Load Application Service Configuration Files (#377)
Signed-off-by: Andrew Morgan (https://amorgan.xyz) <andrew@amorgan.xyz>
2017-12-19 17:00:44 +00:00
Richard van der Hoff e2f88f3889 gb vendor update github.com/matrix-org/gomatrixserverlib 2017-12-12 10:26:25 +00:00
Richard van der Hoff c3cb6f8767 Update gometalinter, and disable gas (#371)
* Update gometalinter

* Disable gas linter

According to the gas github page:

> Gas is still in alpha and accepting feedback from early adopters. We do not
> consider it production ready at this time.

Generally it seems to shout about a lot of things which aren't very errory,
like executing subprocesses with anything other than a hardcoded commandline,
and creating directories with anything other than 700 perms.
2017-12-08 19:13:17 +00:00
Richard van der Hoff 0786318a04
Bump gomatrixserverlib (#353)
* Bump gomatrixserverlib

Mostly because I want to use Erik's go-faster jsoning.

* Update KeyDB for new KeyFetcher API

we now need to implement FetcherName.

* Attempt to fix integ tests

CanonicalJSON doesn't like the empty string, apparently, and anyway
canonicalising it is pointless.

* More integ test fix
2017-11-27 12:05:14 +00:00
Erik Johnston 8599a36fa6
Use a Postgres database rather than Memory for Naffka (#337)
* Update naffka dep

* User Postgres database rather than Memory for Naffka
2017-11-16 17:35:28 +00:00
Richard van der Hoff 4124ce2ac0
Store keys rather than json in the keydatabase (#330)
* bump gomatrixserverlib

(changes to KeyFetcher and KeyDatabase interfaces)

* Store keys rather than json in the keydatabase

Rather than storing the raw JSON returned from a /keys/v1/query call in the
table, store the key itself.

This makes keydb.Database implement the updated KeyDatabase interface.
2017-11-15 17:46:16 +00:00
Richard van der Hoff 8fff0e887c Update gometalinter config (#331)
* Update gometalinter config

gometalinter now uses `maligned` instead of `aligncheck`
(https://github.com/alecthomas/gometalinter/pull/367), so we need to update our
config accordingly.

* Update gometalinter

* Disable gotype linter

gotype does not seem to play nicely with the gb vendor directory. In
particular, it wants each of our dependencies to be built and installed (see
https://github.com/golang/go/issues/10969), but (empirically) it will not
accept them being installed in `pkg` but insists on them being in `vendor/pkg`.

This presents a problem because `gb build` builds the packages into `pkg`
(which doesn't seem entirely unreasonable since `.` comes before `vendor` in
`$GOPATH`). `go install github.com/x/y` does install in `vendor/pkg` but
requires us to know the name of each package.

The general conclusion of https://github.com/alecthomas/gometalinter/issues/91
seems to have been that the easiest thing to do is to disable `gotype` for now.

* Fix `unparam` lint

* Fix goshadow lint
2017-11-15 10:25:48 +00:00
Erik Johnston 32a2b3a5c0 Update vendor libraries and dendrite to s/Sirupsen/sirupsen/ (#304) (#305) 2017-10-13 14:57:54 +01:00
Erik Johnston 464fb5de1c Update vendor libraries and dendrite to s/Sirupsen/sirupsen/ (#304) 2017-10-13 14:42:57 +01:00
Erik Johnston bcf58fad84 gb vendor update github.com/matrix-org/gomatrix 2017-10-03 13:05:57 +01:00
Richard van der Hoff 831a76ae91 gb vendor updates
gb vendor update github.com/matrix-org/gomatrixserverlib
gb vendor update github.com/matrix-org/util
2017-10-02 10:33:43 +01:00
Richard van der Hoff b7687310fe gb vendor update github.com/matrix-org/gomatrixserverlib 2017-09-29 15:14:17 +01:00
Mark Haines 4a0b24c7f4 gb vendor fetch github.com/jaegertracing/jaeger 2017-09-28 15:03:21 +01:00
Mark Haines 88dde65efc gb vendor fetch github.com/opentracing/go-opentracing 2017-09-28 13:51:56 +01:00
Richard van der Hoff f7aa96fc9e gb vendor update github.com/matrix-org/gomatrixserverlib 2017-09-26 12:54:39 +01:00
Erik Johnston 8dabca0f07 Implement shared secret registration (#257)
* Implement shared secret registration

* Use HexString from gomatrixserverlib

* Correctly check username validility
2017-09-22 16:13:19 +01:00
Mark Haines ce019738ff Pass a context when downloading remote media (#251) 2017-09-21 16:20:10 +01:00
Mark Haines fbc4477be0 Use a read-only snapshot transaction for calculating sync responses (#236)
* Use a read-only snapshot transaction for calculating sync responses

* gb vendor update github.com/lib/pq
2017-09-19 16:22:02 +01:00
Erik Johnston 08b9940dde Make media repo use error rather than jsonErrorResponse (#235)
* Make media repo use error rather than jsonErrorResponse

* Update comments

* gb vendor github.com/pkg/errors

* Fixup error formats
2017-09-19 11:40:21 +01:00
Mark Haines 029e71828a Add context.Context to the federation client (#225)
* Add context.Context to the federation client

* gb vendor update github.com/matrix-org/gomatrixserverlib
2017-09-13 11:03:41 +01:00
Brendan Abolivier 086683459f Implement federation route PUT /exchange_third_party_invite (#224)
* Add comment

* gb vendor update github.com/matrix-org/gomatrixserverlib

* Add handler for the exchange_third_party_invite endpoint

* Doc

* Use SendEvents to send the invite to the roomserver

* Add missing error check

* Add checks
2017-09-12 17:15:13 +01:00
Mark Haines 6cb9d900b9 Fix response to federation /invite to match the format expected by synapse (#221)
* Fix response to /invite to match the format expected by synapse

* gb vendor update github.com/matrix-org/gomatrixserverlib

* Use gomatrixserverlib.RespInvite

* gb vendor update github.com/matrix-org/gomatrixserverlib
2017-09-11 18:07:12 +01:00
Brendan Abolivier 4d1d503d43 Processing of pending invites on 3PID binding (#218)
* Add missing file headers

* Move the ID server's signatures verification to common

* Allow verification without specifying a server name

* Add third-party structs to membership events content

* Add processing of 3PID onbind requests

* Use reference for third party invite data

* Fix return arguments order

* Revert "Move the ID server's signatures verification to common"

This reverts commit 93442010316ce71a77ac58ffd3613754ce8fe969.

* Revert "Allow verification without specifying a server name"

This reverts commit fd27afbf82eac50fe9f7b83b26cfce3c66d530d2.

* Remove checks that are already occurring in gomatrixserverlib

* Change return type of createInviteFrom3PIDInvite

* Add doc, add checks in fillDisplayName

* Use MakeFedAPI

* Invert condition

* Use AuthEvents to retrieve the 3PID invite

* Update comment

* Remove unused parameter

* gb vendor update github.com/matrix-org/gomatrixserverlib
2017-09-08 15:17:12 +01:00
Erik Johnston a26d7c2899 gb vendor fetch github.com/alecthomas/gometalinter 2017-09-05 17:09:50 +01:00
Mark Haines 0d894e3da5 gb vendor fetch github.com/matrix-org/naffka 2017-08-15 16:06:09 +01:00
Mark Haines a904380e1b gb vendor update github.com/matrix-org/gomatrixserverlib 2017-07-24 13:42:55 +01:00
Mark Haines 69c29172c3 Use utility methods from gomatrixserverlib. (#152)
* Use utility methods from gomatrixserverlib, rather than reimplementing them

* Return string rather than pointer to string

* Update gomatrixserverlib
2017-07-07 14:11:32 +01:00
Mark Haines 7cbdab30f4 Add a database for storing the server keys (#137)
* Add a database for storing the server keys

* Tweak wording, and comment on the resolution of the timestamp

* Update gomatrixserverlib
2017-06-09 18:07:34 +01:00
Robert Swain 2d202cec07 mediaapi: Add thumbnail support (#132)
* vendor: Add bimg image processing library

bimg is MIT licensed. It depends on the C library libvips which is LGPL
v2.1+ licensed. libvips must be installed separately.

* mediaapi: Add YAML config file support

* mediaapi: Add thumbnail support

* mediaapi: Add missing thumbnail files

* travis: Add ppa and install libvips-dev

* travis: Another ppa and install libvips-dev attempt

* travis: Add sudo: required for sudo apt* usage

* mediaapi/thumbnailer: Make comparison code more readable

* mediaapi: Simplify logging of thumbnail properties

* mediaapi/thumbnailer: Rename metrics to fitness

Metrics is used in the context of monitoring with Prometheus so renaming
to avoid confusion.

* mediaapi/thumbnailer: Use math.Inf() for max aspect and size

* mediaapi/thumbnailer: Limit number of parallel generators

Fall back to selecting from already-/pre-generated thumbnails or serving
the original.

* mediaapi/thumbnailer: Split bimg code into separate file

* vendor: Add github.com/nfnt/resize pure go image scaler

* mediaapi/thumbnailer: Add nfnt/resize thumbnailer

* travis: Don't install libvips-dev via ppa

* mediaapi: Add notes to README about resizers

* mediaapi: Elaborate on scaling libs in README
2017-06-07 01:12:49 +02:00
Robert Swain 0fff33928e vendor: Bump gomatrixserverlib to pull in CreateMediaDownloadRequest 2017-06-01 17:56:30 +02:00
Mark Haines 84ad4ff9f6 Start implementing /join for room aliases for rooms the server is not in. (#115)
* Start implementing the join room API

* Hacks to get join room working

* Make the TLS fingerprint configurable

* Fix the client API proxy to handle '#' correctly

* Return a 200 OK response

* Write the join event along with current state to the room server

* Comment on the error handling

* Fix typos

* Fix tab

* Add TODO for moving authEventIDs to gomatrixserverlib

* Comment on why we ignore the key ID argument for local keys

* Avoid 'preceeded'

* Neaten the control flow

* Neaten the control flow even more

* Return pointers

* Rename err to lastErr for the loop
2017-05-25 16:08:28 +01:00
Mark Haines e6835660b0 Update gomatrixserverlib 2017-05-24 15:05:42 +01:00
Mark Haines e756b3b0eb Update gomatrixserverlib 2017-05-22 18:11:30 +01:00
Mark Haines aa179d451c Update version of gomatrixserverlib (#111) 2017-05-19 10:46:17 +01:00
Kegsay 9d4d18ae7f Add AccountDatabase for storing user accounts (#110)
Including the ability to add new accounts with a user/password and
select accounts using a user/password. Uses bcrypt to hash passwords.
2017-05-19 10:27:03 +01:00
Robert Swain 042d636e34 Update github.com/matrix-org/util for request context fix for logging 2017-05-10 15:43:58 +02:00
Mark Haines a56f609b74 Update gomatrixserverlib (#90) 2017-05-05 17:43:42 +01:00
Mark Haines 0309035aad Update gomatrixserverlib (#89) 2017-05-05 16:19:48 +01:00
Kegsay 2fadb9e409 Make it possible to point Riot at Dendrite (#74) 2017-04-20 17:11:53 +01:00
Kegsay 53ec4a255b Calculate the complete /sync response (#69) 2017-04-13 16:56:46 +01:00
Kegan Dougal 7ab0436689 Add yaml dependency 2017-03-27 13:05:46 +01:00
Kegsay 346f5d6334 Use gomatrixserverlib.StateKeyTuple and helper functions for auth (#44)
* Update gomatrixserverib dep

* Use helper functions when creating rooms

* Use gomatrixserverlib.StateKeyTuple
2017-03-17 16:28:15 +00:00
Mark Haines 812fdbe22a gb vendor update github.com/matrix-org/gomatrixserverlib 2017-03-17 13:10:25 +00:00