char
/
dendrite
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
dendrite/WIRING.md

230 lines
13 KiB
Markdown
Raw Normal View History

Add README
2017-01-20 18:25:51 +00:00
# Wiring
The diagram is incomplete. The following things aren't shown on the diagram:
Small cleanup to WIRING.md
2018-05-22 13:12:14 +00:00
* Device Messages
* User Profiles
* Notification Counts
* Sending federation.
* Querying federation.
* Other things that aren't shown on the diagram.
Add README
2017-01-20 18:25:51 +00:00
Diagram:
W -> Writer
S -> Server/Store/Service/Something/Stuff
R -> Reader
+---+ +---+ +---+
+----------| W | +----------| S | +--------| R |
| +---+ | Receipts +---+ | Client +---+
| Federation |>=========================================>| Server |>=====================>| Sync |
Add notes on what message sending will look like in dendrite (#45)
2017-03-20 16:46:07 +00:00
| Receiver | | | | |
Add README
2017-01-20 18:25:51 +00:00
| | +---+ | | | |
| | +--------| W | | | | |
| | | Client +---+ | | | |
| | | Receipt |>=====>| | | |
| | | Updater | | | | |
| | +----------+ | | | |
| | | | | |
| | +---+ +---+ | | +---+ | |
| | +------------| W | +------| S | | | +--------| R | | |
| | | Federation +---+ | Room +---+ | | | Client +---+ | |
| | | Backfill |>=====>| Server |>=====>| |>=====>| Push | | |
| | +--------------+ | | +------------+ | | | |
| | | | | | | |
| | | |>==========================>| | | |
| | | | +----------+ | |
Document internal APIs and Application Services component (#499) * Document internal APIs, app services in WIRING.md * Add application services component to WIRING diagram * Fix typo and clarify what mode internal APIs use HTTP
2018-07-06 09:33:53 +00:00
| | | | +---+ | |
| | | | +-------------| R | | |
| | | |>=====>| Application +---+ | |
| | | | | Services | | |
| | | | +--------------+ | |
Add README
2017-01-20 18:25:51 +00:00
| | | | +---+ | |
| | | | +--------| R | | |
| | | | | Client +---+ | |
| |>========================>| |>==========================>| Search | | |
| | | | | | | |
| | | | +----------+ | |
| | | | | |
| | | |>==========================================>| |
| | | | | |
| | +---+ | | +---+ | |
| | +--------| W | | | +----------| S | | |
| | | Client +---+ | | | Presence +---+ | |
Add notes on what message sending will look like in dendrite (#45)
2017-03-20 16:46:07 +00:00
| | | API |>=====>| |>=====>| Server |>=====================>| |
| | | /send | +--------+ | | | |
Add README
2017-01-20 18:25:51 +00:00
| | | | | | | |
| | | |>======================>| |<=====================<| |
| | +----------+ | | | |
| | | | | |
| | +---+ | | | |
| | +--------| W | | | | |
| | | Client +---+ | | | |
| | | Presence |>=====>| | | |
| | | Setter | | | | |
| | +----------+ | | | |
| | | | | |
| | | | | |
| |>=========================================>| | | |
| | +------------+ | |
| | | |
| | +---+ | |
| | +----------| S | | |
| | | Typing +---+ | |
| |>=========================================>| Server |>=====================>| |
+------------+ | | +----------+
+---+ | |
+--------| W | | |
| Client +---+ | |
| Typing |>=====>| |
| Setter | | |
+----------+ +------------+
# Component Descriptions
Many of the components are logical rather than physical. For example it is
possible that all of the client API writers will end up being glued together
and always deployed as a single unit.
Outbound federation requests will probably need to be funnelled through a
choke-point to implement ratelimiting and backoff correctly.
## Federation Send
* Handles `/federation/v1/send/` requests.
* Fetches missing ``prev_events`` from the remote server if needed.
* Fetches missing room state from the remote server if needed.
* Checks signatures on remote events, downloading keys if needed.
* Queries information needed to process events from the Room Server.
* Writes room events to logs.
* Writes presence updates to logs.
* Writes receipt updates to logs.
* Writes typing updates to logs.
* Writes other updates to logs.
Add notes on what message sending will look like in dendrite (#45)
2017-03-20 16:46:07 +00:00
## Client API /send
Add README
2017-01-20 18:25:51 +00:00
* Handles puts to `/client/v1/rooms/` that create room events.
* Queries information needed to process events from the Room Server.
* Talks to remote servers if needed for joins and invites.
* Writes room event pdus.
* Writes presence updates to logs.
## Client Presence Setter
Small cleanup to WIRING.md
2018-05-22 13:12:14 +00:00
* Handles puts to the [client API presence paths](https://matrix.org/docs/spec/client_server/unstable.html#id41).
Add README
2017-01-20 18:25:51 +00:00
* Writes presence updates to logs.
## Client Typing Setter
Small cleanup to WIRING.md
2018-05-22 13:12:14 +00:00
* Handles puts to the [client API typing paths](https://matrix.org/docs/spec/client_server/unstable.html#id32).
Add README
2017-01-20 18:25:51 +00:00
* Writes typing updates to logs.
## Client Receipt Updater
Small cleanup to WIRING.md
2018-05-22 13:12:14 +00:00
* Handles puts to the [client API receipt paths](https://matrix.org/docs/spec/client_server/unstable.html#id36).
* Writes receipt updates to logs.
Add README
2017-01-20 18:25:51 +00:00
## Federation Backfill
* Backfills events from other servers
* Writes the resulting room events to logs.
* Is a different component from the room server itself cause it'll
be easier if the room server component isn't making outbound HTTP requests
to remote servers
## Room Server
* Reads new and backfilled room events from the logs written by FS, FB and CRS.
* Tracks the current state of the room and the state at each event.
* Probably does auth checks on the incoming events.
* Handles state resolution as part of working out the current state and the
Small cleanup to WIRING.md
2018-05-22 13:12:14 +00:00
state at each event.
Add README
2017-01-20 18:25:51 +00:00
* Writes updates to the current state and new events to logs.
* Shards by room ID.
## Receipt Server
* Reads new updates to receipts from the logs written by the FS and CRU.
* Somehow learns enough information from the room server to workout how the
current receipt markers move with each update.
* Writes the new marker positions to logs
* Shards by room ID?
* It may be impossible to implement without folding it into the Room Server
forever coupling the components together.
## Typing Server
* Reads new updates to typing from the logs written by the FS and CTS.
* Updates the current list of people typing in a room.
* Writes the current list of people typing in a room to the logs.
* Shards by room ID?
## Presence Server
* Reads the current state of the rooms from the logs to track the intersection
of room membership between users.
Add misspell and gofmt simplify to the pre-commit hooks (#138)
2017-06-12 17:30:47 +00:00
* Reads updates to presence from the logs written by the FS and the CPS.
Add README
2017-01-20 18:25:51 +00:00
* Reads when clients sync from the logs from the Client Sync.
* Tracks any timers for users.
* Writes the changes to presence state to the logs.
* Shards by user ID somehow?
## Client Sync
* Handle /client/v2/sync requests.
* Reads new events and the current state of the rooms from logs written by the Room Server.
* Reads new receipts positions from the logs written by the Receipts Server.
* Reads changes to presence from the logs written by the Presence Server.
* Reads changes to typing from the logs written by the Typing Server.
* Writes when a client starts and stops syncing to the logs.
## Client Search
* Handle whatever the client API path for event search is?
* Reads new events and the current state of the rooms from logs writeen by the Room Server.
* Maintains a full text search index of somekind.
## Client Push
* Pushes unread messages to remote push servers.
* Reads new events and the current state of the rooms from logs writeen by the Room Server.
* Reads the position of the read marker from the Receipts Server.
* Makes outbound HTTP hits to the push server for the client device.
Document internal APIs and Application Services component (#499) * Document internal APIs, app services in WIRING.md * Add application services component to WIRING diagram * Fix typo and clarify what mode internal APIs use HTTP
2018-07-06 09:33:53 +00:00
## Application Service
* Receives events from the Room Server.
* Filters events and sends them to each registered application service.
* Runs a separate goroutine for each application service.
# Internal Component API
Some dendrite components use internal APIs to communicate information back
and forth between each other. There are two implementations of each API, one
that uses HTTP requests and one that does not. The HTTP implementation is
used in multi-process mode, so processes on separate computers may still
communicate, whereas in single-process or Monolith mode, the direct
implementation is used. HTTP is preferred here to kafka streams as it allows
for request responses.
Running `dendrite-monolith-server` will set up direct connections between
components, whereas running each individual component (which are only run in
multi-process mode) will set up HTTP-based connections.
The functions that make HTTP requests to internal APIs of a component are
located in `/<component name>/api/<name>.go`, named according to what
functionality they cover. Each of these requests are handled in `/<component
name>/<name>/<name>.go`.
As an example, the `appservices` component allows other Dendrite components
to query external application services via its internal API. A component
would call the desired function in `/appservices/api/query.go`. In
multi-process mode, this would send an internal HTTP request, which would
be handled by a function in `/appservices/query/query.go`. In single-process
mode, no internal HTTP request occurs, instead functions are simply called
directly, thus requiring no changes on the calling component's end.