* Initial work on persistent queues
* Update index for event ID and server name
* Put things into database (postgres for now)
* Duplicate postgres code into sqlite for now just to stop build errors, will fix SQLite soon
* Fix table name
* Fix index
* Fix table name
* Use RETURNING because LastInsertID is not supported by postgres
* Use functions
* Marshal headered event
* Don't error on now rows
* Don't block if there are PDUs waiting
* Try to tidy up JSON
* Debug logging
* Fix query, use transactions in postgres
* Clean up
* Rehydrate more opportunistically
* Fix SQLite
* remove unused types
* Review comments
* Shuffle things around a bit
* Clean up transaction properly
* Don't send empty transactions
* Reduce unnecessary retries
* Count PDUs to make more resilient
* Don't stop when there is work to be done
* Try to limit wakeups
* well this is tedious
* Fix race in incomplete transactions
* Thread safety on transaction ID/count
* Remove membership table from account DB
And make code which needs that data use the currentstate server
* Unbreak tests; use a membership enum for space
* BREAKING: Make eduserver/appservice use userapi
This is a breaking change because this PR restructures how the AS API
tracks its position in Kafka streams. Previously, it used the account DB
to store partition offsets. However, this is also being used by `clientapi`
for the same purpose, which is bad (each component needs to store offsets
independently or else you might lose messages across restarts). This PR
changes this behaviour to now store partition offsets in the `appservice`
database.
This means that:
- Upon restart, the `appservice` component will attempt to replay all
room events from the beginning of time.
- An additional table will be created in the appservice database, which
in and of itself is backwards compatible.
* Return ErrorConflict
* Make userapi responsible for checking access tokens
There's still plenty of dependencies on account/device DBs, but this
is a start. This is a breaking change as it adds a required config
value `listen.user_api`.
* Cleanup
* Review comments and test fix
* Fix rooms v3 url paths for good - with tests
- Add a test rig around `federationapi` to test routing.
- Use `JSONVerifier` over `KeyRing` so we can stub things out more easily.
- Add `test.NopJSONVerifier` which verifies nothing.
- Add `base.BaseMux` which is the original `mux.Router` used to spawn public/internal routers.
- Listen on `base.BaseMux` and not the default serve mux as it cleans paths which we don't want.
- Factor out `ListenAndServe` to `test.ListenAndServe` and add flag for listening on TLS.
* Fix comments
* Linting
* Remove clientapi producers which aren't actually producers
They are actually just convenience wrappers around the internal APIs
for roomserver/eduserver. Move their logic to their respective `api`
packages and call them directly.
* Remove TODO
* unbreak ygg
Neil Alexander
Kegsay