char/dendrite
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
dendrite/syncapi
History
Kegsay a97b8eafd4
Add peer-to-peer support into Dendrite via libp2p and fetch (#880) 
* Use a fork of pq which supports userCurrent on wasm

* Use sqlite3_js driver when running in JS

* Add cmd/dendritejs to pull in sqlite3_js driver for wasm only

* Update to latest go-sqlite-js version

* Replace prometheus with a stub. sigh

* Hard-code a config and don't use opentracing

* Latest go-sqlite3-js version

* Generate a key for now

* Listen for fetch traffic rather than HTTP

* Latest hacks for js

* libp2p support

* More libp2p

* Fork gjson to allow us to enforce auth checks as before

Previously, all events would come down redacted because the hash
checks would fail. They would fail because sjson.DeleteBytes didn't
remove keys not used for hashing. This didn't work because of a build
tag which included a file which no-oped the index returned.

See https://github.com/tidwall/gjson/issues/157

When it's resolved, let's go back to mainline.

* Use gjson@1.6.0 as it fixes https://github.com/tidwall/gjson/issues/157

* Use latest gomatrixserverlib for sig checks

* Fix a bug which could cause exclude_from_sync to not be set

Caused when sending events over federation.

* Use query variadic to make lookups actually work!

* Latest gomatrixserverlib

* Add notes on getting p2p up and running

Partly so I don't forget myself!

* refactor: Move p2p specific stuff to cmd/dendritejs

This is important or else the normal build of dendrite will fail
because the p2p libraries depend on syscall/js which doesn't work
on normal builds.

Also, clean up main.go to read a bit better.

* Update ho-http-js-libp2p to return errors from RoundTrip

* Add an LRU cache around the key DB

We actually need this for P2P because otherwise we can *segfault*
with things like: "runtime: unexpected return pc for runtime.handleEvent"
where the event is a `syscall/js` event, caused by spamming sql.js
caused by "Checking event signatures for 14 events of room state" which
hammers the key DB repeatedly in quick succession.

Using a cache fixes this, though the underlying cause is probably a bug
in the version of Go I'm on (1.13.7)

* breaking: Add Tracing.Enabled to toggle whether we do opentracing

Defaults to false, which is why this is a breaking change. We need
this flag because WASM builds cannot do opentracing.

* Start adding conditional builds for wasm to handle lib/pq

The general idea here is to have the wasm build have a `NewXXXDatabase`
that doesn't import any postgres package and hence we never import
`lib/pq`, which doesn't work under WASM (undefined `userCurrent`).

* Remove lib/pq for wasm for syncapi

* Add conditional building to remaining storage APIs

* Update build script to set env vars correctly for dendritejs

* sqlite bug fixes

* Docs

* Add a no-op main for dendritejs when not building under wasm

* Use the real prometheus, even for WASM

Instead, the dendrite-sw.js must mock out `process.pid` and
`fs.stat` - which must invoke the callback with an error (e.g `EINVAL`)
in order for it to work:

```
    global.process = {
        pid: 1,
    };
    global.fs.stat = function(path, cb) {
        cb({
            code: "EINVAL",
        });
    }
```

* Linting
2020-03-06 10:23:55 +00:00
..
api Upgrade gomatrixserverlib dependency (#808) 2020-02-11 15:46:51 +00:00
consumers sqlite: fixes from sytest (#872) 2020-02-20 09:28:03 +00:00
routing Remove httputil.LogThenError so that the line numbers are reported properly - make error reporting slightly more useful (#879) 2020-03-02 16:20:44 +00:00
storage Add peer-to-peer support into Dendrite via libp2p and fetch (#880) 2020-03-06 10:23:55 +00:00
sync Remove httputil.LogThenError so that the line numbers are reported properly - make error reporting slightly more useful (#879) 2020-03-02 16:20:44 +00:00
types CS API: Support for /messages, fixes for /sync (#847) 2020-01-23 17:51:10 +00:00
README.md use go module for dependencies (#594) 2019-05-21 21:56:55 +01:00
syncapi.go Support sqlite in addition to postgres (#869) 2020-02-13 17:27:33 +00:00

README.md

Sync API Server

This server is responsible for servicing /sync requests. It gets its data from the room server output log. Currently, the sync server will:

  • Return a valid /sync response for the user represented by the provided access_token.
  • Return a "complete sync" if no since value is provided, and return a valid next_batch token. This contains all rooms the user has been invited to or has joined. For joined rooms, this includes the complete current room state and the most recent 20 (hard-coded) events in the timeline.
  • For "incremental syncs" (a since value is provided), as you get invited to, join, or leave rooms they will be reflected correctly in the /sync response.
  • For very large state deltas, the state section of a room is correctly populated with the state of the room at the start of the timeline.
  • When you join a room, the /sync which transitions your client to be "joined" will include the complete current room state as per the specification.
  • Only wake up user streams it needs to wake up.
  • Honours the timeout query parameter value.

Internals

When the server gets a /sync request, it needs to:

  • Work out which rooms to return to the client.
  • For each room, work out which events to return to the client.

The logic for working out which rooms is based on Synapse:

  1. Get the CURRENT joined room list for this user.
  2. Get membership list changes for this user between the provided stream position and now.
  3. For each room which has membership list changes:
    • Check if the room is 'newly joined' (insufficient to just check for a join event because we allow dupe joins). If it is, then we need to send the full room state down (and 'limited' is always true).
    • Check if user is still CURRENTLY invited to the room. If so, add room to 'invited' block.
    • Check if the user is CURRENTLY left/banned. If so, add room to 'archived' block.
  4. Add joined rooms (joined room list)

For each room, the /sync response returns the most recent timeline events and the state of the room at the start of the timeline. The logic for working out which events is not based entirely on Synapse code, as it is known broken with respect to working out room state. In order to know which events to return, the server needs to calculate room state at various points in the history of the room. For example, imagine a room with the following 15 events (letters are state events (updated via '), numbers are timeline events):

index     0  1  2  3  4  5  6  7  8   9  10  11  12  13    14     15   (1-based indexing as StreamPosition(0) represents no event)
timeline    [A, B, C, D, 1, 2, 3, D', 4, D'', 5, B', D''', D'''', 6]

The current state of this room is: [A, B', C, D''''].

If this room was requested with ?since=14&limit=5 then 1 timeline event would be returned, the most recent one:

    15
   [ 6 ]

If this room was requested with ?since=9&limit=5 then 5 timeline events would be returned, the most recent ones:

    11 12  13    14     15
   [5, B', D''', D'''', 6]

The state of the room at the START of the timeline can be represented in 2 ways:

  • The full_state from index 0 : [A, B, C, D''] (aka the state between 0-11 exclusive)
  • A partial state from index 9 : [D''] (aka the state between 9-11 exclusive)

Servers advance state events (e.g from D' to D'') based on the state conflict resolution algorithm. You might think that you could advance the current state by just updating the entry for the (event type, state_key) tuple for each state event, but this state can diverge from the state calculated using the state conflict resolution algorithm. For example, if there are two "simultaneous" updates to the same state key, that is two updates at the same depth in the event graph, then the final result of the state conflict resolution algorithm might not match the order the events appear in the timeline.

The correct advancement for state events is represented by the AddsStateEventIDs and RemovesStateEventIDs that are in OutputRoomEvents from the room server.

This version of the sync server uses very simple indexing to calculate room state at various points. This is inefficient when a very old since value is provided, or the full_state is requested, as the state delta becomes very large. This is mitigated slightly with indexes, but better data structures could be used in the future.

Known Issues

  • m.room.history_visibility is not honoured: it is always treated as "shared".
  • All ephemeral events are not implemented (presence, typing, receipts).
  • Account data (both user and room) is not implemented.
  • to_device messages are not implemented.
  • Back-pagination via prev_batch is not implemented.
  • The limited flag can lie.
  • Filters are not honoured or implemented. The limit for each room is hard-coded to 20.
  • The full_state query parameter is not implemented.
  • The set_presence query parameter is not implemented.
  • "Ignored" users are not ignored.
  • Redacted events are still sent to clients.
  • Invites over federation (if it existed) won't work as they aren't "real" events and so won't be in the right tables.
  • invite_state is not implemented (for similar reasons to the above point).
  • The current implementation scales badly when a very old since token is provided.
  • The entire current room state can be re-sent to the client if they send a duplicate "join" event which should be a no-op.