char
/
dendrite
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Update README/docs (#1199) 

* Update README

* Update docs
main
Kegsay 2020-07-14 12:58:00 +01:00 committed by GitHub
parent 72b3160776
commit 37db60f4d4
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
3 changed files with 122 additions and 139 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

68
README.md
View File

@ -1,22 +1,68 @@
# Dendrite [![Build Status](https://badge.buildkite.com/4be40938ab19f2bbc4a6c6724517353ee3ec1422e279faf374.svg?branch=master)](https://buildkite.com/matrix-dot-org/dendrite) [![Dendrite Dev on Matrix](https://img.shields.io/matrix/dendrite-dev:matrix.org.svg?label=%23dendrite-dev%3Amatrix.org&logo=matrix&server_fqdn=matrix.org)](https://matrix.to/#/#dendrite-dev:matrix.org) [![Dendrite on Matrix](https://img.shields.io/matrix/dendrite:matrix.org.svg?label=%23dendrite%3Amatrix.org&logo=matrix&server_fqdn=matrix.org)](https://matrix.to/#/#dendrite:matrix.org) # Dendrite [![Build Status](https://badge.buildkite.com/4be40938ab19f2bbc4a6c6724517353ee3ec1422e279faf374.svg?branch=master)](https://buildkite.com/matrix-dot-org/dendrite) [![Dendrite Dev on Matrix](https://img.shields.io/matrix/dendrite-dev:matrix.org.svg?label=%23dendrite-dev%3Amatrix.org&logo=matrix&server_fqdn=matrix.org)](https://matrix.to/#/#dendrite-dev:matrix.org) [![Dendrite on Matrix](https://img.shields.io/matrix/dendrite:matrix.org.svg?label=%23dendrite%3Amatrix.org&logo=matrix&server_fqdn=matrix.org)](https://matrix.to/#/#dendrite:matrix.org)
Dendrite will be a second-generation Matrix homeserver written in Go. Dendrite is a second-generation Matrix homeserver written in Go. It is not recommended to use Dendrite as
a production homeserver at this time as there is no stable release. An overview of the design can be found
in [DESIGN.md](docs/DESIGN.md).
It's still very much a work in progress, but installation instructions can be # Quick start
found in [INSTALL.md](docs/INSTALL.md). It is not recommended to use Dendrite as a
production homeserver at this time. Requires Go 1.13+ and SQLite3 (Postgres is also supported):
```bash
$ git clone https://github.com/matrix-org/dendrite
$ cd dendrite
# generate self-signed certificate and an event signing key for federation
$ go build ./cmd/generate-keys
$ ./generate-keys --private-key matrix_key.pem --tls-cert server.crt --tls-key server.key
# Copy and modify the config file:
# you'll need to set a server name and paths to the keys at the very least, along with setting
# up the database filenames
$ cp dendrite-config.yaml dendrite.yaml
# build and run the server
$ go build ./cmd/dendrite-monolith-server
$ ./dendrite-monolith-server --tls-cert server.crt --tls-key server.key --config dendrite.yaml
```
Then point your favourite Matrix client at `http://localhost:8008`. For full installation information, see
[INSTALL.md](docs/INSTALL.md). For running in Docker, see [build/docker](build/docker).
# Progress
We use a script called Are We Synapse Yet which checks Sytest compliance rates. Sytest is a black-box homeserver
test rig with around 900 tests. The script works out how many of these tests are passing on Dendrite and it
updates with CI. As of July 2020 we're at around 48% CS API coverage and 50% Federation coverage, though check
CI for the latest numbers. In practice, this means you can communicate locally and via federation with Synapse
servers such as matrix.org reasonably well. There's a long list of features that are not implemented, notably:
- Receipts
- Push
- Search and Context
- User Directory
- Presence
- Guests
- E2E keys and device lists
We are prioritising features that will benefit single-user homeservers first (e.g Receipts, E2E) rather
than features that massive deployments may be interested in (User Directory, OpenID, Guests, Admin APIs, AS API).
This means Dendrite supports amongst others:
- Core room functionality (creating rooms, invites, auth rules)
- Federation in rooms v1-v6
- Backfilling locally and via federation
- Accounts, Profiles and Devices
- Published room lists
- Typing
- Media APIs
- Redaction
- Tagging
An overview of the design can be found in [DESIGN.md](docs/DESIGN.md).
# Contributing # Contributing
Everyone is welcome to help out and contribute! See Everyone is welcome to help out and contribute! See
[CONTRIBUTING.md](docs/CONTRIBUTING.md) to get started! [CONTRIBUTING.md](docs/CONTRIBUTING.md) to get started!
Please note that, as of February 2020, Dendrite now only targets Go 1.13 or
later. Please ensure that you are using at least Go 1.13 when developing for
Dendrite.
# Discussion # Discussion
For questions about Dendrite we have a dedicated room on Matrix For questions about Dendrite we have a dedicated room on Matrix
@ -24,8 +70,4 @@ For questions about Dendrite we have a dedicated room on Matrix
discussion should happen in discussion should happen in
[#dendrite-dev:matrix.org](https://matrix.to/#/#dendrite-dev:matrix.org). [#dendrite-dev:matrix.org](https://matrix.to/#/#dendrite-dev:matrix.org).
# Progress
There's plenty still to do to make Dendrite usable! We're tracking progress in a
[project board](https://github.com/matrix-org/dendrite/projects/2).

42
docs/CODE_STYLE.md
View File

@ -1,12 +1,7 @@
# Code Style # Code Style
We follow the standard Go style using goimports, but with a few extra In addition to standard Go code style (`gofmt`, `goimports`), we use `golangci-lint`
considerations. to run a number of linters, the exact list can be found under linters in [.golangci.yml](.golangci.yml).
## Linters
We use `golangci-lint` to run a number of linters, the exact list can be found
under linters in [.golangci.yml](.golangci.yml).
[Installation](https://github.com/golangci/golangci-lint#install) and [Editor [Installation](https://github.com/golangci/golangci-lint#install) and [Editor
Integration](https://github.com/golangci/golangci-lint#editor-integration) for Integration](https://github.com/golangci/golangci-lint#editor-integration) for
it can be found in the readme of golangci-lint. it can be found in the readme of golangci-lint.
@ -22,37 +17,10 @@ The linters can be run using [scripts/find-lint.sh](scripts/find-lint.sh)
[scripts/build-test-lint.sh](scripts/build-test-lint.sh). [scripts/build-test-lint.sh](scripts/build-test-lint.sh).
## HTTP Error Handling ## Labels
Unfortunately, converting errors into HTTP responses with the correct status
code and message can be done in a number of ways in golang:
1. Having functions return `JSONResponse` directly, which can then either set
it to an error response or a `200 OK`.
2. Have the HTTP handler try and cast error values to types that are handled
differently.
3. Have the HTTP handler call functions whose errors can only be interpreted
one way, for example if a `validate(...)` call returns an error then handler
knows to respond with a `400 Bad Request`.
We attempt to always use option #3, as it more naturally fits with the way that
golang generally does error handling. In particular, option #1 effectively
requires reinventing a new error handling scheme just for HTTP handlers.
## Line length
We strive for a line length of roughly 80 characters, though less than 100 is
acceptable if necessary. Longer lines are fine if there is nothing of interest
after the first 80-100 characters (e.g. long string literals).
## TODOs and FIXMEs
The majority of TODOs and FIXMEs should have an associated tracking issue on
github. These can be added just before merging of the PR to master, and the
issue number should be added to the comment, e.g. `// TODO(#324): ...`
In addition to `TODO` and `FIXME` we also use `NOTSPEC` to identify deviations
from the Matrix specification.
## Logging ## Logging

151
docs/INSTALL.md
View File

@ -3,20 +3,21 @@
Dendrite can be run in one of two configurations: Dendrite can be run in one of two configurations:
* **Polylith mode**: A cluster of individual components, dealing with different * **Polylith mode**: A cluster of individual components, dealing with different
aspects of the Matrix protocol (see [WIRING.md](WIRING.md)). Components communicate with each other using internal HTTP APIs and [Apache Kafka](https://kafka.apache.org). This will almost certainly be the preferred model aspects of the Matrix protocol (see [WIRING.md](WIRING-Current.md)). Components communicate
for large-scale deployments. with each other using internal HTTP APIs and [Apache Kafka](https://kafka.apache.org).
This will almost certainly be the preferred model for large-scale deployments.
* **Monolith mode**: All components run in the same process. In this mode, * **Monolith mode**: All components run in the same process. In this mode,
Kafka is completely optional and can instead be replaced with an in-process Kafka is completely optional and can instead be replaced with an in-process
lightweight implementation called [Naffka](https://github.com/matrix-org/naffka). This will usually be the preferred model for low-volume, low-user lightweight implementation called [Naffka](https://github.com/matrix-org/naffka). This
or experimental deployments. will usually be the preferred model for low-volume, low-user or experimental deployments.
Regardless of whether you are running in polylith or monolith mode, each Dendrite component that requires storage has its own database. Both Postgres Regardless of whether you are running in polylith or monolith mode, each Dendrite component that
and SQLite are supported and can be mixed-and-matched across components as requires storage has its own database. Both Postgres and SQLite are supported and can be
needed in the configuration file. mixed-and-matched across components as needed in the configuration file.
Be advised that Dendrite is still developmental and it's not recommended for Be advised that Dendrite is still in development and it's not recommended for
use in production environments yet! use in production environments just yet!
## Requirements ## Requirements
@ -119,16 +120,10 @@ Assuming that Postgres 9.5 (or later) is installed:
Each Dendrite server requires unique server keys. Each Dendrite server requires unique server keys.
Generate the self-signed SSL certificate for federation: Generate the self-signed SSL certificate for federation and the server signing key:
```bash ```bash
test -f server.key || openssl req -x509 -newkey rsa:4096 -keyout server.key -out server.crt -days 3650 -nodes -subj /CN=localhost ./bin/generate-keys --private-key matrix_key.pem --tls-cert server.crt --tls-key server.key
```
Generate the server signing key:
```
test -f matrix_key.pem || ./bin/generate-keys -private-key matrix_key.pem
``` ```
### Configuration file ### Configuration file
@ -152,7 +147,8 @@ public keys for dead homeservers from somewhere else.
## Starting a monolith server ## Starting a monolith server
It is possible to use Naffka as an in-process replacement to Kafka when using It is possible to use Naffka as an in-process replacement to Kafka when using
the monolith server. To do this, set `use_naffka: true` in your `dendrite.yaml` configuration and uncomment the relevant Naffka line in the `database` section. the monolith server. To do this, set `use_naffka: true` in your `dendrite.yaml`
configuration and uncomment the relevant Naffka line in the `database` section.
Be sure to update the database username and password if needed. Be sure to update the database username and password if needed.
The monolith server can be started as shown below. By default it listens for The monolith server can be started as shown below. By default it listens for
@ -166,60 +162,7 @@ as shown below, it will also listen for HTTPS connections on port 8448.
## Starting a polylith deployment ## Starting a polylith deployment
The following contains scripts which will run all the required processes in order to point a Matrix client at Dendrite. Conceptually, you are wiring together to form the following diagram: The following contains scripts which will run all the required processes in order to point a Matrix client at Dendrite.
```
/media +---------------------------+
+----------->+------------->| dendrite-media-api-server |
^ ^ +---------------------------+
| | :7774
| |
| |
| |
| |
| |
| |
| |
| |
| | /sync +--------------------------+
| | +--------->| dendrite-sync-api-server |<================++
| | | +--------------------------+ ||
| | | :7773 | ^^ ||
Matrix +------------------+ | | | || client_data ||
Clients --->| client-api-proxy |-------+ +<-----------+ ++=============++ ||
+------------------+ | | | || ||
:8008 | | CS API +----------------------------+ || ||
| +--------->| dendrite-client-api-server |==++ ||
| | +----------------------------+ ||
| | :7771 | ||
| | | ||
| +<-----------+ ||
| | ||
| | ||
| | +----------------------+ room_event ||
| +---------->| dendrite-room-server |===============++
| | +----------------------+ ||
| | :7770 ||
| | ++==========================++
| +<------------+ ||
| | | VV
| | +-----------------------------------+ Matrix
| | | dendrite-federation-sender-server |------------> Servers
| | +-----------------------------------+
| | :7776
| |
+---------->+ +<-----------+
| |
Matrix +----------------------+ SS API +--------------------------------+
Servers --->| federation-api-proxy |--------->| dendrite-federation-api-server |
+----------------------+ +--------------------------------+
:8448 :7772
A --> B = HTTP requests (A = client, B = server)
A ==> B = Kafka (A = producer, B = consumer)
```
### Client proxy ### Client proxy
@ -248,21 +191,13 @@ to support federation.
### Client API server ### Client API server
This is what implements message sending. Clients talk to this via the proxy in This is what implements CS API endpoints. Clients talk to this via the proxy in
order to send messages. order to send messages, create and join rooms, etc.
```bash ```bash
./bin/dendrite-client-api-server --config=dendrite.yaml ./bin/dendrite-client-api-server --config=dendrite.yaml
``` ```
### Room server
This is what implements the room DAG. Clients do not talk to this.
```bash
./bin/dendrite-room-server --config=dendrite.yaml
```
### Sync server ### Sync server
This is what implements `/sync` requests. Clients talk to this via the proxy This is what implements `/sync` requests. Clients talk to this via the proxy
@ -283,7 +218,7 @@ order to upload and retrieve media.
### Federation API server ### Federation API server
This implements federation requests. Servers talk to this via the proxy in This implements the federation API. Servers talk to this via the proxy in
order to send transactions. This is only required if you want to support order to send transactions. This is only required if you want to support
federation. federation.
@ -291,7 +226,30 @@ federation.
./bin/dendrite-federation-api-server --config dendrite.yaml ./bin/dendrite-federation-api-server --config dendrite.yaml
``` ```
### Federation sender ### Internal components
This refers to components that are not directly spoken to by clients. They are only
contacted by other components. This includes the following components.
#### Room server
This is what implements the room DAG. Clients do not talk to this.
```bash
./bin/dendrite-room-server --config=dendrite.yaml
```
#### Current state server
This tracks the current state of rooms which various components need to know. For example,
`/publicRooms` implemented by client API asks this server for the room names, joined member
counts, etc.
```bash
./bin/dendrite-current-state-server --config=dendrite.yaml
```
#### Federation sender
This sends events from our users to other servers. This is only required if This sends events from our users to other servers. This is only required if
you want to support federation. you want to support federation.
@ -300,7 +258,7 @@ you want to support federation.
./bin/dendrite-federation-sender-server --config dendrite.yaml ./bin/dendrite-federation-sender-server --config dendrite.yaml
``` ```
### Appservice server #### Appservice server
This sends events from the network to [application This sends events from the network to [application
services](https://matrix.org/docs/spec/application_service/unstable.html) services](https://matrix.org/docs/spec/application_service/unstable.html)
@ -311,16 +269,31 @@ application services on your homeserver.
./bin/dendrite-appservice-server --config dendrite.yaml ./bin/dendrite-appservice-server --config dendrite.yaml
``` ```
### Key server #### Key server
This manages end-to-end encryption keys (or rather, it will do when it's This manages end-to-end encryption keys for users.
finished).
```bash ```bash
./bin/dendrite-key-server --config dendrite.yaml ./bin/dendrite-key-server --config dendrite.yaml
``` ```
### User server #### Server Key server
This manages signing keys for servers.
```bash
./bin/dendrite-server-key-api-server --config dendrite.yaml
```
#### EDU server
This manages processing EDUs such as typing, send-to-device events and presence. Clients do not talk to
```bash
./bin/dendrite-edu-server --config dendrite.yaml
```
#### User server
This manages user accounts, device access tokens and user account data, This manages user accounts, device access tokens and user account data,
amongst other things. amongst other things.