Commit Graph

22 Commits (30e9353fdd4cc2a3c3e5bd6de139ca0e31dae7bb)

Author SHA1 Message Date
Kegsay a1b7e4ef3f
log less for failed key querys, add counters for incoming pdus/edus (#1801)
* log less for failed key querys, add counters for incoming pdus/edus

* use labels

* Blacklist flakey test

* Fix metrics
2021-03-23 11:33:36 +00:00
Neil Alexander 81312b8a78
Return the current OTK count on an empty upload request (#1774)
* Always return OTK counts

* Fix parameter ordering

* Send IDs over to keyserver internal API

* Review comments

* Fix syntax error

* Fix panic, hopefully

* Require user ID to be set

* Fix user API call
2021-03-02 11:40:20 +00:00
Neil Alexander 68d92fd3b4
Update sytest-whitelist/blacklist 2020-12-04 14:09:21 +00:00
Neil Alexander 3034d8e805
Update sytest lists to blacklist flaky 'Forgotten room messages cannot be paginated' test 2020-11-16 10:51:55 +00:00
Neil Alexander c7bf122a26
Update sytest lists 2020-10-19 15:38:42 +01:00
Neil Alexander 2e71d2708f
Resolve state after event against current room state when determining latest state changes (#1479)
* Resolve state after event against current room state when determining latest state changes

* Update sytest-whitelist

* Update sytest-whitelist, blacklist
2020-10-05 17:47:08 +01:00
Neil Alexander a06c18bb56
Soft-fail (#1364)
* Initial work on soft-fail

* Fix state block retrieval

* Copy-pasta QueryLatestEventsAndState code

* Fix state lookup

* Clean up

* Fix up failing sytest

* Linting

* Update previous events SQLite insert query

* Update SQLite InsertPreviousEvent properly

* Hopefully fix the event references updates

Co-authored-by: Kegan Dougal <kegan@matrix.org>
2020-09-21 14:55:46 +01:00
Kegsay 18231f25b4
Implement rejected events (#1426)
* WIP Event rejection

* Still send back errors for rejected events

Instead, discard them at the federationapi /send layer rather than
re-implementing checks at the clientapi/PerformJoin layer.

* Implement rejected events

Critically, rejected events CAN cause state resolution to happen
as it can merge forks in the DAG. This is fine, _provided_ we
do not add the rejected event when performing state resolution,
which is what this PR does. It also fixes the error handling
when NotAllowed happens, as we were checking too early and needlessly
handling NotAllowed in more than one place.

* Update test to match reality

* Modify InputRoomEvents to no longer return an error

Errors do not serialise across HTTP boundaries in polylith mode,
so instead set fields on the InputRoomEventsResponse. Add `Err()`
function to make the API shape basically the same.

* Remove redundant returns; linting

* Update blacklist
2020-09-16 13:00:52 +01:00
Kegsay 9dd2ed7f65
Implement key uploads (#1202)
* Add storage layer for postgres/sqlite

* Return OTK counts when inserting new keys

* Hook up the key DB and make a test pass

* Convert postgres queries to be sqlite queries

* Blacklist test due to requiring rejected events

* Unbreak tests

* Update blacklist
2020-07-15 12:02:34 +01:00
Kegan Dougal 3797c38ec8 Blacklist a test 2020-07-03 14:18:58 +01:00
Kegsay a06d0921c9
Make same membership transitions 403, add sytests (#1161)
* Make same membership transitions 403, add sytests

* Update blacklist
2020-06-24 18:19:14 +01:00
Kegsay 02565c37aa
/send auth errors are silent (#1149)
* /send auth errors are silent

* Fix test
2020-06-23 10:31:17 +01:00
Neil Alexander a5d822004d
Send-to-device support (#1072)
* Groundwork for send-to-device messaging

* Update sample config

* Add unstable routing for now

* Send to device consumer in sync API

* Start the send-to-device consumer

* fix indentation in dendrite-config.yaml

* Create send-to-device database tables, other tweaks

* Add some logic for send-to-device messages, add them into sync stream

* Handle incoming send-to-device messages, count them with EDU stream pos

* Undo changes to test

* pq.Array

* Fix sync

* Logging

* Fix a couple of transaction things, fix client API

* Add send-to-device test, hopefully fix bugs

* Comments

* Refactor a bit

* Fix schema

* Fix queries

* Debug logging

* Fix storing and retrieving of send-to-device messages

* Try to avoid database locks

* Update sync position

* Use latest sync position

* Jiggle about sync a bit

* Fix tests

* Break out the retrieval from the update/delete behaviour

* Comments

* nolint on getResponseWithPDUsForCompleteSync

* Try to line up sync tokens again

* Implement wildcard

* Add all send-to-device tests to whitelist, what could possibly go wrong?

* Only care about wildcard when targeted locally

* Deduplicate transactions

* Handle tokens properly, return immediately if waiting send-to-device messages

* Fix sync

* Update sytest-whitelist

* Fix copyright notice (need to do more of this)

* Comments, copyrights

* Return errors from Do, fix dendritejs

* Review comments

* Comments

* Constructor for TransactionWriter

* defletions

* Update gomatrixserverlib, sytest-blacklist
2020-06-01 17:50:19 +01:00
Neil Alexander 895a72b6ee
Move /room/{roomID}/state endpoints into client API (#606) (#962)
* Move /room/{roomID}/state endpoints into client API (#606)

* Update sytest-whitelist

* Blacklist tests which rely on endpoints we don't implement
2020-04-14 18:36:08 +01:00
Neil Alexander dacee648f7
Federation for v3/v4 rooms (#954)
* Update gomatrixserverlib

* Default to room version 4

* Update gomatrixserverlib

* Limit prev_events and auth_events

* Fix auth_events, prev_events

* Fix linter issues

* Update gomatrixserverlib

* Fix getState

* Update sytest-whitelist

* Squashed commit of the following:

commit 067b875063
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Fri Apr 3 14:29:06 2020 +0100

    Invites v2 endpoint (#952)

    * Start converting v1 invite endpoint to v2

    * Update gomatrixserverlib

    * Early federationsender code for sending invites

    * Sending invites sorta happens now

    * Populate invite request with stripped state

    * Remodel a bit, don't reflect received invites

    * Handle invite_room_state

    * Handle room versions a bit better

    * Update gomatrixserverlib

    * Tweak order in destinationQueue.next

    * Revert check in processMessage

    * Tweak federation sender destination queue code a bit

    * Add comments

commit 955244c092
Author: Ben B <benne@klimlive.de>
Date:   Fri Apr 3 12:40:50 2020 +0200

    use custom http client instead of the http DefaultClient (#823)

    This commit replaces the default client from the http lib with a custom one.
    The previously used default client doesn't come with a timeout. This could cause
    unwanted locks.
    That solution chosen here creates a http client in the base component dendrite
    with a constant timeout of 30 seconds. If it should be necessary to overwrite
    this, we could include the timeout in the dendrite configuration.
    Here it would be a good idea to extend the type "Address" by a timeout and
    create an http client for each service.

    Closes #820

    Signed-off-by: Benedikt Bongartz <benne@klimlive.de>

    Co-authored-by: Kegsay <kegan@matrix.org>

* Update sytest-whitelist, sytest-blacklist

* Update go.mod/go.sum

* Add some error wrapping for debug

* Add a NOTSPEC to common/events.go

* Perform state resolution at send_join

* Set default room version to v2 again

* Tweak GetCapabilities

* Add comments to ResolveConflictsAdhoc

* Update sytest-blacklist

* go mod tidy

* Update sytest-whitelist, sytest-blacklist

* Update versions

* Updates from review comments

* Update sytest-blacklist, sytest-whitelist

* Check room versions compatible at make_join, add some comments, update gomatrixserverlib, other tweaks

* Set default room version back to v2

* Update gomatrixserverlib, sytest-whitelist
2020-04-09 15:46:06 +01:00
Neil Alexander 05e1ae8745
Further room version wiring (#936)
* Room version 2 by default, other wiring updates, update gomatrixserverlib

* Fix nil pointer exception

* Fix some more nil pointer exceptions hopefully

* Update gomatrixserverlib

* Send all room versions when joining, not just stable ones

* Remove room version cquery

* Get room version when getting events from the roomserver database

* Reset default back to room version 2

* Don't generate event IDs unless needed

* Revert "Remove room version cquery"

This reverts commit a170d5873360dd059614460acc8b21ab2cda9767.

* Query room version in federation API, client API as needed

* Improvements to make_join send_join dance

* Make room server producers use headered events

* Lint tweaks

* Update gomatrixserverlib

* Versioned SendJoin

* Query room version in syncapi backfill

* Handle transaction marshalling/unmarshalling within Dendrite

* Sorta fix federation (kinda)

* whoops commit federation API too

* Use NewEventFromTrustedJSON when getting events from the database

* Update gomatrixserverlib

* Strip headers on federationapi endpoints

* Fix bug in clientapi profile room version query

* Update gomatrixserverlib

* Return more useful error if room version query doesn't find the room

* Update gomatrixserverlib

* Update gomatrixserverlib

* Maybe fix federation

* Fix formatting directive

* Update sytest whitelist and blacklist

* Temporarily disable room versions 3 and 4 until gmsl is fixed

* Fix count of EDUs in logging

* Update gomatrixserverlib

* Update gomatrixserverlib

* Update gomatrixserverlib

* Rely on EventBuilder in gmsl to generate the event IDs for us

* Some review comments fixed

* Move function out of common and into gmsl

* Comment in federationsender destinationqueue

* Update gomatrixserverlib
2020-03-27 16:28:22 +00:00
S7evinK 3dfafd4824
Implement missing device management features (#835)
* Implement missing device management features

Signed-off-by: Till Faelligen <tfaelligen@gmail.com>

* Add a little more documentation

* Undo changes

* Use non-anonymous struct to decode devices list

* Update sytest-whitelist

* Update sytest-whitelist

* Update sytest-blacklist

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2020-02-11 12:13:38 +00:00
Neil Alexander c20109a573
Implement room version capabilities in CS API (#866)
* Add wiring for querying the roomserver for the default room version

* Try to implement /capabilities for room versions

* Update copyright notices

* Update sytests, add /capabilities endpoint into CS API

* Update sytest-whitelist

* Add GetDefaultRoomVersion

* Fix cases where state package was shadowed

* Fix version formatting

* Update Dockerfile to Go 1.13.6

* oh yes types I remember

* And fix the default too
2020-02-05 18:06:39 +00:00
Andrew Morgan 4da2630904
Blacklist 'displayname updates affect room member events' (#859) 2020-02-01 23:19:20 +00:00
Neil Alexander 49f760a30b
CS API: Support for /messages, fixes for /sync (#847)
* Merge forward

* Tidy up a bit

* TODO: What to do with NextBatch here?

* Replace SyncPosition with PaginationToken throughout syncapi

* Fix PaginationTokens

* Fix lint errors

* Add a couple of missing functions into the syncapi external storage interface

* Some updates based on review comments from @babolivier

* Some updates based on review comments from @babolivier

* argh whitespacing

* Fix opentracing span

* Remove dead code

* Don't overshadow err (fix lint issue)

* Handle extremities after inserting event into topology

* Try insert event topology as ON CONFLICT DO NOTHING

* Prevent OOB error in addRoomDeltaToResponse

* Thwarted by gocyclo again

* Fix NewPaginationTokenFromString, define unit test for it

* Update pagination token test

* Update sytest-whitelist

* Hopefully fix some of the sync batch tokens

* Remove extraneous sync position func

* Revert to topology tokens in addRoomDeltaToResponse etc

* Fix typo

* Remove prevPDUPos as dead now that backwardTopologyPos is used instead

* Fix selectEventsWithEventIDsSQL

* Update sytest-blacklist

* Update sytest-whitelist
2020-01-23 17:51:10 +00:00
Andrew Morgan 43ecf8d1f9
Add more passing tests to the testfile, add test blacklist file (#848) 2020-01-22 16:11:40 +00:00
Andrew Morgan 0f8d6cc1c1
Add a Sytest blacklist file (#849) 2020-01-22 13:31:22 +00:00