Commit Graph

772 Commits (3cb04e80042104d14ccfa162b3e40a5b08819eac)

Author SHA1 Message Date
Abhinav Krishna C K ec38783192
maintenance: Fix matrix-org#896 use %w format verb to wrap errors (#916)
* maintenance: Fix matrix-org#896 use %w format verb to wrap errors

* In Go version 1.13 a new formatting verb introduced for fmt.Errorf
  %w https://blog.golang.org/go1.13-errors

* update %s to %w to wrap errors.

* Update all instances of error type to use %w

Signed-off-by: Abhinav Krishna C K <me@abhy.me>

Co-authored-by: Kegsay <kegan@matrix.org>
2020-03-18 12:48:51 +00:00
Prateek Sachan c019ad7086
Log errors from rows.Close (#920)
* Log errors from rows.Close

* fixed imports

* Added contextual messages

* fixed review changes
2020-03-18 10:17:18 +00:00
Neil Alexander c2bd0b97b3
Get room versions from database (#918)
* Retrieve room version where known in roomserver

* Get room versions in alias code

* Increase gocyclothreshold to 13, since we hit that number a lot

* Remove gocyclo nolint from StoreEvent

* Update interface to get room version from room ID instead of NID

* Remove new API

* Fixed this query for SQLite but not for Postgres
2020-03-17 18:00:10 +00:00
Kegsay 1467cc10d8
bugfix: Fix a bug which caused failures to join rooms over federation (#917)
* bugfix: Fix a bug which caused failures to join rooms over federation

The cause of this was the semantics of `/send_join`'s `auth_chain` response.

Previously, we would only send back the auth chain *for the join event* and
not the entire room state. However, we would then try to check that the
room state is valid, and then be missing auth events.

Now, we send back the entire auth chain for all room state in `/send_join`.
The spec needs to be clarified that this is what the chain should be.

* refactor: split out grabbing state to reduce cyclo complexity
2020-03-17 17:18:48 +00:00
Kegsay 4af8323df3
bugfix: Fix a bug which caused prev_content not to be sent to clients (#919)
I don't know how this ever passed QA... also fix a missing rows.Close()
2020-03-17 16:45:40 +00:00
Neil Alexander a66c701b29
Add room version to room create request, persist in storage (#915)
* Add room version into createRoomReq

* Extract room version from m.room.create event when persisting

* Reduce cyclomatic complexity

* Update whitelist, gomatrixserverlib, tweaks to roomserver

* Update sytest-whitelist again
2020-03-17 15:12:01 +00:00
Neil Alexander aebf347a79
Implement gomatrixserverlib.HeaderedEvent in roomserver Kafka output (#914)
* Use Event.Headered

* Use HeaderedEvent in roomserver kafka output

* Fix syncserver-integration-tests

* Update producers to roomserver inputs

* Update gomatrixserverlib

* Update gomatrixserverlib

* Update gomatrixserverlib

* Update gomatrixserverlib

* Update gomatrixserverlib

* Update gomatrixserverlib
2020-03-17 11:01:25 +00:00
Kegsay 9f74a8798e
bugfix: Fix #908 by setting the correct state after the event (#913)
* bugfix: Fix #908 by setting the correct state after the event

Previously, this would only happen if the state already existed
previously!

* Structured logging
2020-03-16 17:51:58 +00:00
Neil Alexander acb505b717
Implement gomatrixserverlib.HeaderedEvent in roomserver query API (#912)
* Implement gomatrixserverlib.HeaderedEvent, which should allow us to store room version headers along with the event across API boundaries and consumers/producers, and intercept unmarshalling to get the event structure right

* Add federationsender to previous
2020-03-16 17:29:52 +00:00
Neil Alexander 452f393dd7
Update room version descriptors, add error handling (#906)
* Update room version descriptors, add error handling

* Fix database queries

* Drop Get from version package

* Fix database wrapping, add comments for version descriptions

* Don't set default room_version value in SQL
2020-03-16 16:05:29 +00:00
Kegsay dfd8b93d93
Federation: Add event_auth endpoint (#905)
* Federation: Add event_auth endpoint

Signed-off-by: Sudhanshu Jaiswal <sudsjaiswal@gmail.com>

* Update metric names

Signed-off-by: Sudhanshu Jaiswal <sudsjaiswal@gmail.com>

* Rebase

* Add comment from @APwhitehat

Co-authored-by: Sudhanshu Jaiswal <sudsjaiswal@gmail.com>
2020-03-14 20:08:54 +00:00
Kegan Dougal 7353de7dbb Make P2P use JSServer 2020-03-13 10:15:01 +00:00
Kegsay 8bc5084d8d
p2p: Use JSServer for comms rather than GoJsConn (#888)
* p2p: Use JSServer for comms rather than GoJsConn

This has several benefits:
 - it fixes a bug whereby you could not transmit >4k bytes to/from JS/Go land.
 - it more clearly exposes the interface point between Go and JS: a single
   global function call.
 - it presents a nicer API shape than the previous `net.Conn`.
 - it doesn't needlessly 'stream' data which is already sitting in-memory.

This is currently only active for local CS API traffic, another PR will
add Federation P2P support.

* Typo
2020-03-11 12:18:37 +00:00
Kegsay d71b72816d
p2p: use separate topics! (#886)
Without it, setting a display name fails and /sync wedges forever.
2020-03-10 18:10:10 +00:00
Kegsay 09dfd4839a
p2p: Use a single ed25519 key for event signing/verifying and p2p peer IDs (#885)
* Pass in the private key seed to NewP2PLocalNode to use the same key for p2p

* Extract the ed25519 key from the server name and use it for event checks

This allows HSes which no longer exist to still have events verified for
them. This means that new members can join rooms that has missing servers.

* Remove no-op ipfs uri which we don't use anymore
2020-03-10 14:13:15 +00:00
Neil Alexander 176f722d53 Update .gitignore 2020-03-10 11:42:40 +00:00
Kegsay cdc1157785
Improve logging when sending events (#883)
We have some failing sytests on sqlite but it's very difficult to debug
due to lack of useful logging. This adds a log line for when a new event
is sent (incl. logging the event ID) as well as adding a user_id field
for all contextual logs so we know who initiated certain actions.
2020-03-09 14:37:51 +00:00
Kegsay c31cb02271
bugfix: Fix a race condition when creating guest accounts (#882)
* bugfix: Fix a race condition when creating guest accounts

It was possible to both select the same next numeric ID and then both
attempt to INSERT this into the table. This would cause a UNIQUE violation
which then presented itself as an error in sqlite because it does not
implement `common.IsUniqueConstraintViolationErr`.

The fix here is NOT to implement `common.IsUniqueConstraintViolationErr`
otherwise the 2 users would get the SAME guest account. Instead, all of
these operations should be done inside a transaction. This is what this
PR does.

* Update postgres

* Typo

* Actually use the txn when creating accounts

* bugfix for database is locked on guest reg
2020-03-06 18:00:07 +00:00
Neil Alexander 6a1111c3d4
Try to recursively find auth events (to a point) if they are missing (#881)
* Try to recursively find auth events (to a point) if they are missing

* Remove recursion limit for now and other review fixes

* Simplify error handling for recursion

* Pass room version 1 only to MakeJoin until room version support comes later
2020-03-06 16:58:10 +00:00
Kegan Dougal 87283e9de7 bugfix: fix sytest 155 by actually returning depth+1 and not 0 2020-03-06 14:31:12 +00:00
Kegsay a97b8eafd4
Add peer-to-peer support into Dendrite via libp2p and fetch (#880)
* Use a fork of pq which supports userCurrent on wasm

* Use sqlite3_js driver when running in JS

* Add cmd/dendritejs to pull in sqlite3_js driver for wasm only

* Update to latest go-sqlite-js version

* Replace prometheus with a stub. sigh

* Hard-code a config and don't use opentracing

* Latest go-sqlite3-js version

* Generate a key for now

* Listen for fetch traffic rather than HTTP

* Latest hacks for js

* libp2p support

* More libp2p

* Fork gjson to allow us to enforce auth checks as before

Previously, all events would come down redacted because the hash
checks would fail. They would fail because sjson.DeleteBytes didn't
remove keys not used for hashing. This didn't work because of a build
tag which included a file which no-oped the index returned.

See https://github.com/tidwall/gjson/issues/157

When it's resolved, let's go back to mainline.

* Use gjson@1.6.0 as it fixes https://github.com/tidwall/gjson/issues/157

* Use latest gomatrixserverlib for sig checks

* Fix a bug which could cause exclude_from_sync to not be set

Caused when sending events over federation.

* Use query variadic to make lookups actually work!

* Latest gomatrixserverlib

* Add notes on getting p2p up and running

Partly so I don't forget myself!

* refactor: Move p2p specific stuff to cmd/dendritejs

This is important or else the normal build of dendrite will fail
because the p2p libraries depend on syscall/js which doesn't work
on normal builds.

Also, clean up main.go to read a bit better.

* Update ho-http-js-libp2p to return errors from RoundTrip

* Add an LRU cache around the key DB

We actually need this for P2P because otherwise we can *segfault*
with things like: "runtime: unexpected return pc for runtime.handleEvent"
where the event is a `syscall/js` event, caused by spamming sql.js
caused by "Checking event signatures for 14 events of room state" which
hammers the key DB repeatedly in quick succession.

Using a cache fixes this, though the underlying cause is probably a bug
in the version of Go I'm on (1.13.7)

* breaking: Add Tracing.Enabled to toggle whether we do opentracing

Defaults to false, which is why this is a breaking change. We need
this flag because WASM builds cannot do opentracing.

* Start adding conditional builds for wasm to handle lib/pq

The general idea here is to have the wasm build have a `NewXXXDatabase`
that doesn't import any postgres package and hence we never import
`lib/pq`, which doesn't work under WASM (undefined `userCurrent`).

* Remove lib/pq for wasm for syncapi

* Add conditional building to remaining storage APIs

* Update build script to set env vars correctly for dendritejs

* sqlite bug fixes

* Docs

* Add a no-op main for dendritejs when not building under wasm

* Use the real prometheus, even for WASM

Instead, the dendrite-sw.js must mock out `process.pid` and
`fs.stat` - which must invoke the callback with an error (e.g `EINVAL`)
in order for it to work:

```
    global.process = {
        pid: 1,
    };
    global.fs.stat = function(path, cb) {
        cb({
            code: "EINVAL",
        });
    }
```

* Linting
2020-03-06 10:23:55 +00:00
Neil Alexander 0cda3c52d0 Don't fail hard if backward topology position fails, just use 1 instead 2020-03-02 18:01:24 +00:00
Neil Alexander 59a1f4b8ed
Remove httputil.LogThenError so that the line numbers are reported properly - make error reporting slightly more useful (#879) 2020-03-02 16:20:44 +00:00
Neil Alexander 72565f2eeb
Fix bug in devices endpoint (#877) 2020-02-28 15:06:16 +00:00
Neil Alexander 6460b3725d
Make sure PDUs and EDUs in transaction don't marshal to null (#876) 2020-02-28 14:54:51 +00:00
Kegan Dougal 420ee15433 Kick CI 2020-02-28 14:46:03 +00:00
Neil Alexander 0352f250b8 Fix dockerfile 2020-02-20 13:54:50 +00:00
Kegsay 5caae6f3a0
sqlite: fixes from sytest (#872)
* bugfix: fix panic on new invite events from sytest

I'm unsure why the previous code didn't work, but it's
clearer, quicker and easier to read the `LastInsertID()` way.
Previously, the code would panic as the SELECT would fail
to find the last inserted row ID.

* sqlite: Fix UNIQUE violations and close more cursors

- Add missing `defer rows.Close()`
- Do not have the state block NID as a PRIMARY KEY else it breaks for blocks
  with >1 state event in them. Instead, rejig the queries so we can still
  have monotonically increasing integers without using AUTOINCREMENT (which
  mandates PRIMARY KEY).

* sqlite: Add missing variadic function

* Use LastInsertId because empirically it works over the SELECT form (though I don't know why that is)

* sqlite: Fix invite table by using the global stream pos rather than one specific to invites

If we don't use the global, clients don't get notified about any invites
because the position is too low.

* linting: shadowing

* sqlite: do not use last rowid, we already know the stream pos!

* sqlite: Fix account data table in syncapi by commiting insert txns!

* sqlite: Fix failing federation invite

Was failing with 'database is locked' due to multiple write txns
being taken out.

* sqlite: Ensure we return exactly the number of events found in the database

Previously we would return exactly the number of *requested* events, which
meant that several zero-initialised events would bubble through the system,
failing at JSON serialisation time.

* sqlite: let's just ignore the problem for now....

* linting
2020-02-20 09:28:03 +00:00
Neil Alexander 3dabf4d4ed
More SQLite (#871)
* SQLite support for appservice

* SQLite support for mediaapi

* Copyright notices

* SQLite for public rooms API (although with some slight differences in behaviour)

* Lazy match aliases, add TODOs
2020-02-14 14:12:33 +00:00
Andrew Morgan 409fec2a48
Add whitelist/blacklist maintenance complainer to buildkite annotations (#870) 2020-02-14 13:07:14 +00:00
Kegsay b6ea1bc67a
Support sqlite in addition to postgres (#869)
* Move current work into single branch

* Initial massaging of clientapi etc (not working yet)

* Interfaces for accounts/devices databases

* Duplicate postgres package for sqlite3 (no changes made to it yet)

* Some keydb, accountdb, devicedb, common partition fixes, some more syncapi tweaking

* Fix accounts DB, device DB

* Update naffka dependency for SQLite

* Naffka SQLite

* Update naffka to latest master

* SQLite support for federationsender

* Mostly not-bad support for SQLite in syncapi (although there are problems where lots of events get classed incorrectly as backward extremities, probably because of IN/ANY clauses that are badly supported)

* Update Dockerfile -> Go 1.13.7, add build-base (as gcc and friends are needed for SQLite)

* Implement GET endpoints for account_data in clientapi

* Nuke filtering for now...

* Revert "Implement GET endpoints for account_data in clientapi"

This reverts commit 4d80dff4583d278620d9b3ed437e9fcd8d4674ee.

* Implement GET endpoints for account_data in clientapi (#861)

* Implement GET endpoints for account_data in clientapi

* Fix accountDB parameter

* Remove fmt.Println

* Fix insertAccountData SQLite query

* Fix accountDB storage interfaces

* Add empty push rules into account data on account creation (#862)

* Put SaveAccountData into the right function this time

* Not sure if roomserver is better or worse now

* sqlite work

* Allow empty last sent ID for the first event

* sqlite: room creation works

* Support sending messages

* Nuke fmt.println

* Move QueryVariadic etc into common, other device fixes

* Fix some linter issues

* Fix bugs

* Fix some linting errors

* Fix errcheck lint errors

* Make naffka use postgres as fallback, fix couple of compile errors

* What on earth happened to the /rooms/{roomID}/send/{eventType} routing

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2020-02-13 17:27:33 +00:00
Thibaut CHARLES 6942ee1de0
Upgrade gomatrixserverlib dependency (#808)
* Upgrade gomatrixserverlib dependency

Signed-off-by: Thibaut CHARLES cromfr@gmail.com

* Added missing passing sytest

Signed-off-by: Thibaut CHARLES cromfr@gmail.com

* Fix login using identifier key

Not a full fix, it only really supports logging in with
the localpart of an mxid.

Signed-off-by: Serra Allgood <serra@allgood.dev>

* Replace deprecated prometheus.InstrumentHandler and unsafe time.Ticker

* goimports

* re-add temporarily missing deps?

* Refactor InstrumentHandlerCounter definition

* URL decode args

* Return server names  (#833)

* Remove unnecessary map->array processing

* Return server names in room federation directory query

* Knock off a TODO

* Fix /send_join and /send_leave (#821)

Fix the /send_join and /send_leave endpoints, so that they use the v2 endpoints as mandated by MSC1802. Also comment out the SyTest tests that are failing because of lack of support for the v1 endpoints.

* Refuse /send_join without m.room.create (#824)

Signed-off-by: Abhishek Kumar <abhishekkumar2718@gmail.com>

* AS should use the v1 endpoint, rather than r0 (#827)

* docker: Passthrough parameters to dendrite-monolith-server

* Fix copy & paste error (#812)

* Use gomatrixserverlib.Transaction instead of local type (#590) (#811)

* Move files back if linting fails (#810)

* replaced gometalinter description with golangci-lint (#837)

* Amend syncapi SQL queries to return missing columns (#840)

* This commit updates a couple of the syncapi SQL queries to return additional columns that are required/expected by rowsToStreamEvents in output_room_events_table.go.

It's not exactly clear to me yet what transaction_id and session_id do, but these being added n #367 results in state events breaking the /sync endpoint.

This is a temporary fix. We need to come up with a better solution.

* gomatrix to gomatrixserverlib on some weird line change

* Tweaks from @babolivier review comments

* Implement storage interfaces (#841)

* Implement interfaces for federationsender storage

* Implement interfaces for mediaapi storage

* Implement interfaces for publicroomsapi storage

* Implement interfaces for roomserver storage

* Implement interfaces for syncapi storage

* Implement interfaces for keydb storage

* common.PartitionStorer in publicroomsapi interface

* Update copyright notices

* make cmd directory path absolute in build.sh (#830)

* Resync testfile with current sytest pass/fail (#832)

* Resync testfile with current sytest pass/fail

* Add displayname test

* Fall back to postgres when database connection string parsing fails (#842)

* Fall back to postgres when parsing the database connection string for a URI schema fails

* Fix behaviour so that it really tries postgres when URL parsing fails and it complains about unknown schema if it succeeds

* Fix #842

* Fix #842 - again...

* Federation fixes (#845)

* Update gomatrixserverlib to p2p commit 92c0338, other tweaks

* Update gomatrixserverlib to p2p commit e5dcc65

* Rewrite getAuthChain

* Update gomatrixserverlib in go.mod/go.sum

* Correct a couple of package refs for updated gmsl/gomatrix

* Update gomatrixserverlib ref in go.mod/go.sum

* Update getAuthChain comments following @babolivier review

* Add a Sytest blacklist file (#849)

* Add more passing tests to the testfile, add test blacklist file (#848)

* CS API: Support for /messages, fixes for /sync (#847)

* Merge forward

* Tidy up a bit

* TODO: What to do with NextBatch here?

* Replace SyncPosition with PaginationToken throughout syncapi

* Fix PaginationTokens

* Fix lint errors

* Add a couple of missing functions into the syncapi external storage interface

* Some updates based on review comments from @babolivier

* Some updates based on review comments from @babolivier

* argh whitespacing

* Fix opentracing span

* Remove dead code

* Don't overshadow err (fix lint issue)

* Handle extremities after inserting event into topology

* Try insert event topology as ON CONFLICT DO NOTHING

* Prevent OOB error in addRoomDeltaToResponse

* Thwarted by gocyclo again

* Fix NewPaginationTokenFromString, define unit test for it

* Update pagination token test

* Update sytest-whitelist

* Hopefully fix some of the sync batch tokens

* Remove extraneous sync position func

* Revert to topology tokens in addRoomDeltaToResponse etc

* Fix typo

* Remove prevPDUPos as dead now that backwardTopologyPos is used instead

* Fix selectEventsWithEventIDsSQL

* Update sytest-blacklist

* Update sytest-whitelist

* Some fixes for #847 (#850)

* Fix a couple of cases where backfilling events we already had causes panics, hopefully fix ordering of events, update GMSL dependency for backfill URL fixes

* Remove commented out lines from output_room_events_table schema

* Wire up publicroomsapi for roomserver events (#851)

* Wire up publicroomsapi to roomserver events

* Remove parameter that was incorrectly brought over from p2p work

* nolint containsBackwardExtremity for now

* Store our own keys in the keydb (#853)

* Store our own keys in the keydb

The DirectKeyFetcher makes the assumption that you can always reach the key/v2/server endpoint of any server, including our own. We previously haven't bothered to store our own keys in the keydb so this would mean we end up making key requests to ourselves.

In the libp2p world as an example, self-dialling is not possible, therefore this would render it impossible to get our own keys.

This commit adds our own keys into the keydb so that we don't create unnecessarily (and maybe impossible) requests.

* Use golang.org/x/crypto/ed25519 instead of crypto/ed25519 for pre-Go 1.13

* More sync fixes (#854)

* Further sync tweaks

* Remove unnecessary blank line

* getBackwardTopologyPos always returns a usable value

* Revert order fixing

* Implement GET endpoints for account_data in clientapi (#861)

* Implement GET endpoints for account_data in clientapi

* Fix accountDB parameter

* Remove fmt.Println

* Add empty push rules into account data on account creation (#862)

* Handle kind=guest query parameter on /register (#860)

* Handle kind=guest query parameter on /register

* Reorganized imports

* Pass device_id as nil

* Added tests to systest-whitelist

* Update sytest-whitelist

* Blacklist 'displayname updates affect room member events' (#859)

* Room version abstractions (#865)

* Rough first pass at adding room version abstractions

* Define newer room versions

* Update room version metadata

* Fix roomserver/versions

* Try to fix whitespace in roomsSchema

* Implement room version capabilities in CS API (#866)

* Add wiring for querying the roomserver for the default room version

* Try to implement /capabilities for room versions

* Update copyright notices

* Update sytests, add /capabilities endpoint into CS API

* Update sytest-whitelist

* Add GetDefaultRoomVersion

* Fix cases where state package was shadowed

* Fix version formatting

* Update Dockerfile to Go 1.13.6

* oh yes types I remember

* And fix the default too

* Update documentation for Go 1.13 (#867)

* Pass cfg by reference around the codebase (#819)

* Pass cfg by reference around the codebase

* Merge branch 'master' into pass-cfg-by-ref

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>

* Implement missing device management features (#835)

* Implement missing device management features

Signed-off-by: Till Faelligen <tfaelligen@gmail.com>

* Add a little more documentation

* Undo changes

* Use non-anonymous struct to decode devices list

* Update sytest-whitelist

* Update sytest-whitelist

* Update sytest-blacklist

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>

* Adding sslmode: disable to sytest server config (#813)

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>

* Fix AppService bind addrs in test (#805)

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>

* Always defer *sql.Rows.Close and consult with Err (#844)

* Always defer *sql.Rows.Close and consult with Err

database/sql.Rows.Next() makes sure to call Close only after exhausting
result rows which would NOT happen when returning early from a bad Scan.
Close being idempotent makes it a great candidate to get always deferred
regardless of what happens later on the result set.

This change also makes sure call Err() after exhausting Next() and
propagate non-nil results from it as the documentation advises.

Closes #764

Signed-off-by: Kiril Vladimiroff <kiril@vladimiroff.org>

* Override named result parameters in last returns

Signed-off-by: Kiril Vladimiroff <kiril@vladimiroff.org>

* Do the same over new changes that got merged

Signed-off-by: Kiril Vladimiroff <kiril@vladimiroff.org>

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>

* Clean up

Co-authored-by: Serra Allgood <serra@allgood.dev>
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
Co-authored-by: Brendan Abolivier <github@brendanabolivier.com>
Co-authored-by: Abhishek Kumar <31231064+abhishekkumar2718@users.noreply.github.com>
Co-authored-by: Will Hunt <will@half-shot.uk>
Co-authored-by: S7evinK <tfaelligen@gmail.com>
Co-authored-by: Arshpreet <30545756+arsh-7@users.noreply.github.com>
Co-authored-by: Prateek Sachan <42961174+prateek2211@users.noreply.github.com>
Co-authored-by: Behouba Manassé <behouba@gmail.com>
Co-authored-by: aditsachde <23707194+aditsachde@users.noreply.github.com>
Co-authored-by: Kiril Vladimiroff <kiril@vladimiroff.org>
2020-02-11 15:46:51 +00:00
Kiril Vladimiroff d5dbe546e4
Always defer *sql.Rows.Close and consult with Err (#844)
* Always defer *sql.Rows.Close and consult with Err

database/sql.Rows.Next() makes sure to call Close only after exhausting
result rows which would NOT happen when returning early from a bad Scan.
Close being idempotent makes it a great candidate to get always deferred
regardless of what happens later on the result set.

This change also makes sure call Err() after exhausting Next() and
propagate non-nil results from it as the documentation advises.

Closes #764

Signed-off-by: Kiril Vladimiroff <kiril@vladimiroff.org>

* Override named result parameters in last returns

Signed-off-by: Kiril Vladimiroff <kiril@vladimiroff.org>

* Do the same over new changes that got merged

Signed-off-by: Kiril Vladimiroff <kiril@vladimiroff.org>

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2020-02-11 14:12:21 +00:00
aditsachde d45f869cdd
Fix AppService bind addrs in test (#805)
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2020-02-11 13:53:54 +00:00
Thibaut CHARLES 880545bfa8
Adding sslmode: disable to sytest server config (#813)
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2020-02-11 13:53:00 +00:00
S7evinK 3dfafd4824
Implement missing device management features (#835)
* Implement missing device management features

Signed-off-by: Till Faelligen <tfaelligen@gmail.com>

* Add a little more documentation

* Undo changes

* Use non-anonymous struct to decode devices list

* Update sytest-whitelist

* Update sytest-whitelist

* Update sytest-blacklist

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2020-02-11 12:13:38 +00:00
Behouba Manassé 9937c05bea
Pass cfg by reference around the codebase (#819)
* Pass cfg by reference around the codebase

* Merge branch 'master' into pass-cfg-by-ref

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2020-02-11 11:18:12 +00:00
Neil Alexander b72d7eb0cf
Update documentation for Go 1.13 (#867) 2020-02-06 11:54:26 +00:00
Neil Alexander c20109a573
Implement room version capabilities in CS API (#866)
* Add wiring for querying the roomserver for the default room version

* Try to implement /capabilities for room versions

* Update copyright notices

* Update sytests, add /capabilities endpoint into CS API

* Update sytest-whitelist

* Add GetDefaultRoomVersion

* Fix cases where state package was shadowed

* Fix version formatting

* Update Dockerfile to Go 1.13.6

* oh yes types I remember

* And fix the default too
2020-02-05 18:06:39 +00:00
Neil Alexander 880d8ae024
Room version abstractions (#865)
* Rough first pass at adding room version abstractions

* Define newer room versions

* Update room version metadata

* Fix roomserver/versions

* Try to fix whitespace in roomsSchema
2020-02-05 16:25:58 +00:00
Andrew Morgan 4da2630904
Blacklist 'displayname updates affect room member events' (#859) 2020-02-01 23:19:20 +00:00
Neil Alexander ed457049fb Update sytest-whitelist 2020-01-30 11:29:52 +00:00
Prateek Sachan 4113599f61
Handle kind=guest query parameter on /register (#860)
* Handle kind=guest query parameter on /register

* Reorganized imports

* Pass device_id as nil

* Added tests to systest-whitelist
2020-01-30 11:25:57 +00:00
Neil Alexander 68ee11d96c
Add empty push rules into account data on account creation (#862) 2020-01-30 11:20:44 +00:00
Neil Alexander 6885c10083
Implement GET endpoints for account_data in clientapi (#861)
* Implement GET endpoints for account_data in clientapi

* Fix accountDB parameter

* Remove fmt.Println
2020-01-29 17:53:05 +00:00
Neil Alexander 054f5383c4
More sync fixes (#854)
* Further sync tweaks

* Remove unnecessary blank line

* getBackwardTopologyPos always returns a usable value

* Revert order fixing
2020-01-25 14:23:16 +00:00
Neil Alexander 2cb7c91c5d
Store our own keys in the keydb (#853)
* Store our own keys in the keydb

The DirectKeyFetcher makes the assumption that you can always reach the key/v2/server endpoint of any server, including our own. We previously haven't bothered to store our own keys in the keydb so this would mean we end up making key requests to ourselves.

In the libp2p world as an example, self-dialling is not possible, therefore this would render it impossible to get our own keys.

This commit adds our own keys into the keydb so that we don't create unnecessarily (and maybe impossible) requests.

* Use golang.org/x/crypto/ed25519 instead of crypto/ed25519 for pre-Go 1.13
2020-01-25 14:12:52 +00:00
Neil Alexander 37d117f2b7
Wire up publicroomsapi for roomserver events (#851)
* Wire up publicroomsapi to roomserver events

* Remove parameter that was incorrectly brought over from p2p work

* nolint containsBackwardExtremity for now
2020-01-24 17:11:20 +00:00
Neil Alexander dece31f41e
Some fixes for #847 (#850)
* Fix a couple of cases where backfilling events we already had causes panics, hopefully fix ordering of events, update GMSL dependency for backfill URL fixes

* Remove commented out lines from output_room_events_table schema
2020-01-24 11:40:27 +00:00
Neil Alexander 49f760a30b
CS API: Support for /messages, fixes for /sync (#847)
* Merge forward

* Tidy up a bit

* TODO: What to do with NextBatch here?

* Replace SyncPosition with PaginationToken throughout syncapi

* Fix PaginationTokens

* Fix lint errors

* Add a couple of missing functions into the syncapi external storage interface

* Some updates based on review comments from @babolivier

* Some updates based on review comments from @babolivier

* argh whitespacing

* Fix opentracing span

* Remove dead code

* Don't overshadow err (fix lint issue)

* Handle extremities after inserting event into topology

* Try insert event topology as ON CONFLICT DO NOTHING

* Prevent OOB error in addRoomDeltaToResponse

* Thwarted by gocyclo again

* Fix NewPaginationTokenFromString, define unit test for it

* Update pagination token test

* Update sytest-whitelist

* Hopefully fix some of the sync batch tokens

* Remove extraneous sync position func

* Revert to topology tokens in addRoomDeltaToResponse etc

* Fix typo

* Remove prevPDUPos as dead now that backwardTopologyPos is used instead

* Fix selectEventsWithEventIDsSQL

* Update sytest-blacklist

* Update sytest-whitelist
2020-01-23 17:51:10 +00:00