Charlotte's custom fork of the Dendrite matrix homeserver
 
 
 
 
Go to file
Mark Haines 9a8a8aedcb Add a query API to the roomserver for getting the latest events in a room. (#23)
* Start implementing a query API for go using net/rpc

* Use a conventional JSON POST API rather than go net/rpc

net/rpc doesn't automatically handle reconnecting and we have better
logging and metrics infrastructure for monitoring HTTP apis.

* Implement the query API and add it to the integration tests

* Increase the timeout, travis seems to be a bit slow

* Clarify that state events are the things that are not returned if they are not requested

* Add utility function for converting arrays of numeric event IDs to pq Int64Arrays

* Warn people against requesting empty state keys by accident
2017-03-06 14:29:39 +00:00
hooks A kafkaesque room event consumer for the roomserver. (#1) 2017-02-03 13:52:32 +00:00
src/github.com/matrix-org/dendrite Add a query API to the roomserver for getting the latest events in a room. (#23) 2017-03-06 14:29:39 +00:00
vendor Add a query API to the roomserver for getting the latest events in a room. (#23) 2017-03-06 14:29:39 +00:00
.gitignore A kafkaesque room event consumer for the roomserver. (#1) 2017-02-03 13:52:32 +00:00
.travis.yml Add integration tests for roomserver to travis (#21) 2017-03-01 14:55:27 +00:00
README.md Update README.md (#22) 2017-03-01 09:39:05 +00:00
WIRING.md Add README 2017-01-20 18:25:51 +00:00
travis-install-kafka.sh Add integration tests for roomserver to travis (#21) 2017-03-01 14:55:27 +00:00
travis-test.sh Add integration tests for roomserver to travis (#21) 2017-03-01 14:55:27 +00:00

README.md

Dendrite Build Status

Dendrite will be a matrix homeserver written in go.

Design

Log Based Architecture

Decomposition and Decoupling

A matrix homeserver can be built around append-only event logs built from the messages, receipts, presence, typing notifications, device messages and other events sent by users on the homeservers or by other homeservers.

The server would then decompose into two categories: writers that add new entries to the logs and readers that read those entries.

The event logs then serve to decouple the two components, the writers and readers need only agree on the format of the entries in the event log. This format could be largely derived from the wire format of the events used in the client and federation protocols:

 C-S API   +---------+    Event Log    +---------+   C-S API
---------> |         |+  (e.g. kafka)  |         |+ --------->
           | Writers || =============> | Readers ||
---------> |         ||                |         || --------->
 S-S API   +---------+|                +---------+|   S-S API
            +---------+                 +---------+

However the way matrix handles state events in a room creates a few complications for this model.

  1. Writers require the room state at an event to check if it is allowed.
  2. Readers require the room state at an event to determine the users and servers that are allowed to see the event.
  3. A client can query the current state of the room from a reader.

The writers and readers cannot extract the necessary information directly from the event logs because it would take too long to extract the information as the state is built up by collecting individual state events from the event history.

The writers and readers therefore need access to something that stores copies of the event state in a form that can be efficiently queried. One possibility would be for the readers and writers to maintain copies of the current state in local databases. A second possibility would be to add a dedicated component that maintained the state of the room and exposed an API that the readers and writers could query to get the state. The second has the advantage that the state is calculated and stored in a single location.

 C-S API   +---------+    Log   +--------+   Log   +---------+   C-S API
---------> |         |+ ======> |        | ======> |         |+ --------->
           | Writers ||         |  Room  |         | Readers ||
---------> |         || <------ | Server | ------> |         || --------->
 S-S API   +---------+|  Query  |        |  Query  +---------+|  S-S API
            +---------+         +--------+          +---------+

The room server can annotate the events it logs to the readers with room state so that the readers can avoid querying the room server unnecessarily.

This architecture can be extended to cover most of the APIs.

TODO

  • gomatrixlib
    • Canonical JSON.
    • Signed JSON.
    • Event hashing.
    • Event signing.
    • Federation server discovery.
    • Federation key lookup.
    • Federation request signing.
    • Event authentication.
    • Event visibility.
    • State resolution.
    • Third party invites authentication.
  • Room Server
    • Inputing new events from logs.
    • Inputing backfilled events from logs.
    • Outputing events and current state to logs.
    • Querying state at an event.
    • Querying current forward extremities and state.
    • Querying message history.
    • Exporting/importing messages from other servers.
    • Other Room Server stuff.
  • Client Room Send
    • Handling /client/r0/room/... HTTP PUTs
    • Talk to remote servers for joins to remote servers.
    • Outputting new events to logs.
    • Updating the last active time in presence.
    • Other Client Room Send stuff.
  • Client Sync
    • Inputing new room events and state from the logs.
    • Handling /client/r0/sync HTTP GETs
    • Outputing whether the client is syncing to the logs.
    • Other Client Sync Stuff.
  • Other Components.