Commit Graph

51 Commits (a49d06138ec3bb35c3f5ee6e97c3c8ff0c198720)

Author SHA1 Message Date
Neil Alexander 1002e87b60
Pinecone P2P demo (#1856)
* Pinecone demo

* Enable multicast, fix HTTP routing

* Fix multicast import

* Fix build

* Update Pinecone demo

* Fix the keys

* Tweaks

* Pinecone room directory support (early)

* Fix gobind-pinecone

* Add pinecone listener

* Fix public key value

* Use AuthenticatedConnect for dial

* Fix gobind-pinecone

* Stop panics

* Give fsAPI to keyserver

* Pinecone demo fixes

* Update gobind build scripts

* Account creation

* Tweaks

* Setup tweaks

* API tweaks

* API tweaks

* API tweaks

* Port mutex

* Re-enable multicast

* Add ReadCopy

* Update quic-go, fixes

* Shutdowns fixed for iOS

* Update build script

* Add WebSocket support

* Bug fixes

* Netconn context

* Fix WebSocket connectivity

* Fixes to gobind API

* Strip frameworks

* Configurability updates

* Update go.mod

* Update go.mod/go.sum

* Update go.mod/go.sum

* Update go.mod/go.sum

* Try to stay connected tto static peer

* Update gobind-pinecone

* Update go.mod/go.sum

* Test uTP+TLS

* Use HTTP/2

* Don't use HTTP/2

* Update go.mod/go.sum

* Attempt to reconnect to the static peer if it drops

* Stay connected to static peers more stickily

* Retry room directory lookups if they fail

* NewQUIC -> NewSessions

* Storage updates

* Don't return immediately when there's nothing to sync

* Updates

* Try to reconnect to static peer more

* Update go.mod/go.sum

* Require Go 1.14

* Update go.mod/go.sum

* Update go.mod/go.sum
2021-05-06 12:00:42 +01:00
Marco Kundt d1496793b9
fix database names to reflect renaming (#1636)
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2021-02-17 15:20:06 +00:00
Matthew Hodgson 0571d395b5
Peeking over federation via MSC2444 (#1391)
* a very very WIP first cut of peeking via MSC2753.

doesn't yet compile or work.
needs to actually add the peeking block into the sync response.
checking in now before it gets any bigger, and to gather any initial feedback on the vague shape of it.

* make PeekingDeviceSet private

* add server_name param

* blind stab at adding a `peek` section to /sync

* make it build

* make it launch

* add peeking to getResponseWithPDUsForCompleteSync

* cancel any peeks when we join a room

* spell out how to runoutside of docker if you want speed

* fix SQL

* remove unnecessary txn for SelectPeeks

* fix s/join/peek/ cargocult fail

* HACK: Track goroutine IDs to determine when we write by the wrong thread

To use: set `DENDRITE_TRACE_SQL=1` then grep for `unsafe`

* Track partition offsets and only log unsafe for non-selects

* Put redactions in the writer goroutine

* Update filters on writer goroutine

* wrap peek storage in goid hack

* use exclusive writer, and MarkPeeksAsOld more efficiently

* don't log ascii in binary at sql trace...

* strip out empty roomd deltas

* re-add txn to SelectPeeks

* re-add accidentally deleted field

* reject peeks for non-worldreadable rooms

* move perform_peek

* fix package

* correctly refactor perform_peek

* WIP of implementing MSC2444

* typo

* Revert "Merge branch 'kegan/HACK-goid-sqlite-db-is-locked' into matthew/peeking"

This reverts commit 3cebd8dbfbccdf82b7930b7b6eda92095ca6ef41, reversing
changes made to ed4b3a58a7855acc43530693cc855b439edf9c7c.

* (almost) make it build

* clean up bad merge

* support SendEventWithState with optional event

* fix build & lint

* fix build & lint

* reinstate federated peeks in the roomserver (doh)

* fix sql thinko

* todo for authenticating state returned by /peek

* support returning current state from QueryStateAndAuthChain

* handle SS /peek

* reimplement SS /peek to prod the RS to tell the FS about the peek

* rename RemotePeeks as OutboundPeeks

* rename remote_peeks_table as outbound_peeks_table

* add perform_handle_remote_peek.go

* flesh out federation doc

* add inbound peeks table and hook it up

* rename ambiguous RemotePeek as InboundPeek

* rename FSAPI's PerformPeek as PerformOutboundPeek

* setup inbound peeks db correctly

* fix api.SendEventWithState with no event

* track latestevent on /peek

* go fmt

* document the peek send stream race better

* fix SendEventWithRewrite not to bail if handed a non-state event

* add fixme

* switch SS /peek to use SendEventWithRewrite

* fix comment

* use reverse topo ordering to find latest extrem

* support postgres for federated peeking

* go fmt

* back out bogus go.mod change

* Fix performOutboundPeekUsingServer

* Fix getAuthChain -> GetAuthChain

* Fix build issues

* Fix build again

* Fix getAuthChain -> GetAuthChain

* Don't repeat outbound peeks for the same room ID to the same servers

* Fix lint

* Don't omitempty to appease sytest

Co-authored-by: Kegan Dougal <kegan@matrix.org>
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2021-01-22 14:55:08 +00:00
rany ef24ea2678
Add madvdontneed=1 to decrease RAM consumption for systemd service (#1690)
* Add madvdontneed=1 to decrease RAM consumption for systemd service

* Update monolith-example.service

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2021-01-14 10:31:45 +00:00
Neil Alexander d7824ed5b1
Update INSTALL.md (#1637) 2020-12-14 09:42:27 +00:00
Neil Alexander b794873fa0
Update INSTALL.md 2020-12-12 20:55:34 +00:00
bodqhrohro 45dec20593
Update links to golangci-lint documentation (#1617)
Signed-off-by: Bohdan Horbeshko <bodqhrohro@gmail.com>
2020-12-07 11:23:01 +00:00
Neil Alexander 8f5393db5f
Update FAQ 2020-12-07 10:20:19 +00:00
TR_SLimey c052edafdd
Fix broken image in CONTRIBUTING.md (#1612) 2020-12-04 14:10:01 +00:00
Petter Jakub Økland 19b1d40d64
Hiawatha reverse proxy sample for monolith and fixes to polylith sample (#1609)
* Created polylith reverse proxy sample for Hiawatha

* Create monolith-sample.conf

* Added timeout to proxies and binding explanation.

* Fixed typo.

* Consistency with regards to polylith-sample.conf.
2020-12-03 13:52:00 +00:00
Neil Alexander 52905ffb82
Update PROFILING.md 2020-12-03 13:49:34 +00:00
Neil Alexander 253b05ccde
Update PROFILING.md 2020-12-03 13:48:02 +00:00
Neil Alexander 246866a131
Add PROFILING.md 2020-12-03 13:46:56 +00:00
Neil Alexander eef8f88092
Update FAQ.md 2020-12-03 13:28:41 +00:00
Neil Alexander a4bf9921ad
Update FAQ.md 2020-12-03 13:27:48 +00:00
Neil Alexander 71327b8efa
Add FAQ.md 2020-12-03 13:22:32 +00:00
bodqhrohro c667a19775
Assign CREATEDB permission to the test user (#1591)
During the tests databases get recreated, and this fails despite of the
user being the owner of a dropped database. Maybe related to certain
PostgreSQL version.

Signed-off-by: Bohdan Horbeshko <bodqhrohro@gmail.com>
2020-12-01 16:59:47 +00:00
TR_SLimey 3bcb003248
Add IPv6 setting to nginx conf (#1582)
* Added IPv6 setting to nginx conf

https://matrix.to/#/!yomrOFwgFXzmeMAbzX:matrix.org/$sIkSxL002BKottY1Fop-bkusuyCJ1Y0zIFzqSWDYRrc?via=matrix.org&via=privacytools.io&via=dendrite.link

* Add IPv6 support to polylith nginx conf also
2020-11-18 10:33:17 +00:00
Petter Jakub Økland fe339c8b98
Created polylith reverse proxy sample for Hiawatha (#1571) 2020-10-28 16:37:56 +00:00
Neil Alexander 0af35bec1a
Update documentation (#1569)
* Update dendrite-config.yaml

* Update README.md

* Update INSTALL.md

* Update INSTALL.md
2020-10-26 22:09:13 +00:00
Adit Sachde b66b2eab32
Document dendrite server key format (#1520)
Co-authored-by: Kegsay <kegan@matrix.org>
2020-10-22 12:21:31 +01:00
Neil Alexander 39c7a8915c
Multi-personality polylith binary (#1552)
* Initial work oon multipersonality binary

* Remove old binaries

* Monolith and polylith binaries

* Better logging

* dendrite-poly-multi

* Fix path

* Copyright notices etc

* Tweaks

* Update Docker, INSTALL.md

* Take first argument if flags package doesn't find any args

* Postgres 9.6 or later, fix some more Docker stuff

* Don't create unnecessary e2ekey DB

* Run go mod tidy
2020-10-20 16:11:24 +01:00
Neil Alexander a71360d099
Update INSTALL.md (#1549) 2020-10-20 11:22:46 +01:00
Pieter 0804594a61
Add systemd example service to docs (#1512)
Signed-off-by: Pieter Hollander <git@pieterhollander.nl>
2020-10-12 10:06:56 +01:00
Kegsay 009401ad4d
Version 0.1.0
Beta README (#1466)

* Beta docs

* More tweaks

* More docs

* Update README.md (#1497)

* Call out missing features

* Add CHANGES

* Call out CHANGES

* Update INSTALL.md

* Update README.md

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2020-10-08 17:47:43 +01:00
Kegsay bf7e85848b
Rename serverkeyapi to signingkeyserver (#1492)
* Rename serverkeyapi to signingkeyserver

We use "api" for public facing stuff and "server" for internal stuff.
As the server key API is internal only, we call it 'signing key server',
which also clarifies the type of key (as opposed to TLS keys, E2E keys, etc)

* Convert docker/scripts to use signing-key-server

* Rename missed bits
2020-10-07 16:23:18 +01:00
S7evinK 3bd66ff196
Route several paths to sync_api (#1473) 2020-10-02 22:35:54 +01:00
Neil Alexander 43b3c4a2fc
Fix bugs in nginx sample configs 2020-10-01 14:41:08 +01:00
Neil Alexander cb4b93b16c
Add nginx sample reverse proxy configs (#1458)
* Add nginx sample reverse proxy configs

* Add line breaks at end of files
2020-10-01 14:33:48 +01:00
bn4t 4ff7ac7b65
Fix a small typo (#1448)
Signed-off-by: Benjamin Nater <me@bn4t.me>
2020-09-29 10:07:23 +01:00
Matthew Hodgson 39507bacc3
Peeking via MSC2753 (#1370)
Initial implementation of MSC2753, as tested by https://github.com/matrix-org/sytest/pull/944.
Doesn't yet handle unpeeks, peeked EDUs, or history viz changing during a peek - these will follow.
https://github.com/matrix-org/dendrite/pull/1370 has full details.
2020-09-10 14:39:18 +01:00
Kegsay c992f4f1f4
Remove current state server (#1405)
* Remove current state server

Closes #1365 #1272 #1357

* Remove current state server from scripts/docs
2020-09-07 14:47:59 +01:00
Kegsay 421b6b2313
Add OpenTracing documentation (#1345)
* Add OpenTracing documentation

Closes #274

* Reshuffle

* Update setup.md
2020-08-27 10:24:56 +01:00
Kegsay 3b14119aff
Update README/CONTRIBUTING/sytest for AWSY issues (#1333)
* Update README/CONTRIBUTING/sytest for AWSY issues

* More docs
2020-08-25 08:11:41 +01:00
oliverpool 466d795375
Fix scripts path in documentation (#1294)
Broken by 24d8df664c

Signed-off-by: Olivier Charvin <git@olivier.pfad.fr>
2020-08-24 18:13:43 +01:00
Creed Haymond f3c482b078
Adding the e2ekey database to the create_db script and docs (#1206)
Signed-off-by: Creed Haymond <creedasaurus@gmail.com>
2020-07-20 09:18:48 +01:00
Kegsay 37db60f4d4
Update README/docs (#1199)
* Update README

* Update docs
2020-07-14 12:58:00 +01:00
Kegsay 9c1f38621c
Remove publicroomsapi (#1176)
* Remove all of publicroomsapi

* Remove references to publicroomsapi

* Remove doc references to publicroomsapi
2020-07-02 17:11:33 +01:00
Neil Alexander 72444e4a4f
User API polylith component (#1151)
* User API polylith component

* Add docker-pull.sh
2020-06-19 09:37:19 +01:00
Kegsay 83391da0e0
Make syncapi use userapi (#1136)
* Make syncapi use userapi

* Unbreak things

* Fix tests

* Lint
2020-06-16 17:05:38 +01:00
Kegsay 0ba1245a46
Current wiring (#1125)
* Current wiring

* Add ServerKeyAPI lines
2020-06-15 10:13:57 +01:00
Kegan Dougal 267a4d1823 Update p2p docs 2020-05-27 09:39:26 +01:00
Neil Alexander 3cb04e8004
Update INSTALL.md, move docs (#1034)
* Update INSTALL.md

* Move some things

* Clean up

* Move some more things

* Don't build all the things for the monolith

* Update INSTALL.md

* Nuke hooks
2020-05-14 16:49:18 +01:00
Alex Chen d359851708
Update guides for running sytest (#958)
* Update guides for running sytest

Bring the details for running sytest against postgres up to date; prefer docker over manual setup

Signed-off-by: Alex Chen <minecnly@gmail.com>

* Better flags

Signed-off-by: Alex Chen <minecnly@gmail.com>
2020-04-12 13:48:24 +01:00
Thibaut CHARLES 880545bfa8
Adding sslmode: disable to sytest server config (#813)
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2020-02-11 13:53:00 +00:00
Andrew Morgan 0f8d6cc1c1
Add a Sytest blacklist file (#849) 2020-01-22 13:31:22 +00:00
Andrew Morgan 76040bfa87
Add CI information to CONTRIBUTING.md (#778)
Add information about how the continuous integration is set up in Dendrite and how to run the tests locally so that people don't need to wait around for things to churn.
2019-08-07 11:46:36 +01:00
Alex Chen 7792f12e6f Fix testfile path hardcoded in show-expected-fail-tests.sh (#719)
Signed-off-by: Alex Chen <minecnly@gmail.com>
2019-06-25 11:50:19 +01:00
Alex Chen adaec7c8e7 Refine config and docs for sytest (#714)
This PR adds the "passed but expected fail" tests to testfile, as well as instructions on finding which tests to add for new PRs.
2019-06-21 16:49:37 +01:00
Andrew Morgan 1667a4b09f
Add SyTest support to CI (#704)
Adds SyTest CI through CircleCI.

Also need to add all the tests that currently pass to testfile, but I don't think that's a hard blocker for merging.
2019-05-24 15:00:03 +01:00