{"action":"created","issue":{"url":"https://api.github.com/repos/matrix-org/synapse/issues/1633","repository_url":"https://api.github.com/repos/matrix-org/synapse","labels_url":"https://api.github.com/repos/matrix-org/synapse/issues/1633/labels{/name}","comments_url":"https://api.github.com/repos/matrix-org/synapse/issues/1633/comments","events_url":"https://api.github.com/repos/matrix-org/synapse/issues/1633/events","html_url":"https://github.com/matrix-org/synapse/pull/1633","id":190499071,"number":1633,"title":"Aggregations (experimental)","user":{"login":"pik","id":10236920,"avatar_url":"https://avatars.githubusercontent.com/u/10236920?v=3","gravatar_id":"","url":"https://api.github.com/users/pik","html_url":"https://github.com/pik","followers_url":"https://api.github.com/users/pik/followers","following_url":"https://api.github.com/users/pik/following{/other_user}","gists_url":"https://api.github.com/users/pik/gists{/gist_id}","starred_url":"https://api.github.com/users/pik/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pik/subscriptions","organizations_url":"https://api.github.com/users/pik/orgs","repos_url":"https://api.github.com/users/pik/repos","events_url":"https://api.github.com/users/pik/events{/privacy}","received_events_url":"https://api.github.com/users/pik/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":5,"created_at":"2016-11-19T14:27:34Z","updated_at":"2016-11-19T14:27:35Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/matrix-org/synapse/pulls/1633","html_url":"https://github.com/matrix-org/synapse/pull/1633","diff_url":"https://github.com/matrix-org/synapse/pull/1633.diff","patch_url":"https://github.com/matrix-org/synapse/pull/1633.patch"},"body":"Matrix is currently missing an implementation/spec for processing Aggregation type events.\r\n\r\nThis includes some functionality necessary and expected by users from a chat client, such as being able to edit a message or add a reaction a to a message sent by another user. \r\n\r\nWhile it's possible to implement said functionality as direct and hard-coded endpoints, this would have a number of drawbacks: \r\n- Specs are hard to alter without breaking things, the more exact the standard, the less maleable to future needs\r\n- Specing hard-coded suppport for a function such as 'message edit' does not provide anything for users wishing to have richer client experiences (emoji reactions)\r\n- Specing hard-coded support for a particular kind of aggregation is restrictive and still provides nothing for users of Matrix/Synapse as a federate-distributed message log rather than as a chat client.\r\n\r\nThis is a stab at an open-ended aggregation approach (only works with postgresql backend atm because of different semantics between sqlite3 and postgresql JSON). Currently the following is functional: \r\n- A room creator can POST an `aggregation_spec` to `/room/(?P<room_id>[^/]+)/aggregation$`\r\n- Synapse will process aggregation type events `'m.room._aggregation'` in the background and fill in information for the `target_id` in the `aggregation_entries` table.\r\n- `append` and `replace` type operations are supported e.g. emoticon support to a room could be added by the room creator POSTing the following:\r\n```python\r\nemoticon_aggregation_spec = {\r\n    'constraints': [],\r\n    'aggregation_event_name': 'm.room._aggregation.emoticon',\r\n    'aggregation_field_names': ['emoticon'],\r\n    'aggregation_type': 'append',\r\n    'aggregation_event_schema': {\r\n        'type': 'object',\r\n        'properties': {\r\n            'emoticon': { 'type': 'string' },\r\n            'msgtype': { 'type': 'string' },\r\n            'target_id':  { 'type': 'string' }\r\n        },\r\n        'required': ['emoticon', 'msgtype', 'target_id'],\r\n        'additionalProperties': False\r\n    }\r\n}\r\n```\r\nThis in turn would cause Synapse to run background_updates which will process `'m.room._aggregation.emoticon'` type events and append them to an array in the `aggregate_entries` table e.g.:\r\n\r\n```\r\ntarget_id        | $14794019940aHDNb:pik-test\r\nroom_id          | !QKscJgkpWveOZhvGME:pik-test\r\nevent_name       | m.room._aggregation.emoticon\r\nlatest_event_id  | $14794147870yHlCA:pik-test\r\naggregation_data | [\"{\\\"emoticon\\\": \\\"::smile::\\\", \\\"event_id\\\": \\\"$14794020481Zmmku:pik-test\\\", \\\"sender\\\": \\\"@pik:pik-test\\\"}\", \"{\\\"emoticon\\\": \\\"::flowers::\\\", \\\"event_id\\\": \\\"$14794022480fdPPD:pik-test\\\", \\\"sender\\\": \\\"@pik:pik-test\\\"}\"]\r\n```\r\n\r\nWhen retrieving events for a client it would than be possible for Synapse to OUTER JOIN on the aggregation_entries table and send a single bulk entry for each `target_id`.\r\n\r\n#### TODO\r\n##### Update Synapse Events Endpoint\r\nImplement event retreiving API endpoints which OUTER JOIN on aggregation_entries. There are a number of approaches to this depending on whether 'aggregated' events are pruned from the `events` table or not in the database. While the client when receiving a set of bulk events for a `target_id` should always receive a `latest_event_id` used in the aggregation and would know to ignore events prior to this, it would be preferable to not send those events at all to the client after they have been aggregated. This should mean either filtering the retrieved event array (e.g comparing to a cache of already known aggregated values) or pruning the aggregated events from the db table entirely. \r\n\r\n##### Emoticon / Edit support in vector-web (riot) client.\r\nThis is a little bit complicated because the client should support both receiving bulked (aggregated entries) and singular events (for singular events it should imitate the server aggregation strategy).\r\n\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/matrix-org/synapse/issues/comments/261716868","html_url":"https://github.com/matrix-org/synapse/pull/1633#issuecomment-261716868","issue_url":"https://api.github.com/repos/matrix-org/synapse/issues/1633","id":261716868,"user":{"login":"matrixbot","id":10127976,"avatar_url":"https://avatars.githubusercontent.com/u/10127976?v=3","gravatar_id":"","url":"https://api.github.com/users/matrixbot","html_url":"https://github.com/matrixbot","followers_url":"https://api.github.com/users/matrixbot/followers","following_url":"https://api.github.com/users/matrixbot/following{/other_user}","gists_url":"https://api.github.com/users/matrixbot/gists{/gist_id}","starred_url":"https://api.github.com/users/matrixbot/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/matrixbot/subscriptions","organizations_url":"https://api.github.com/users/matrixbot/orgs","repos_url":"https://api.github.com/users/matrixbot/repos","events_url":"https://api.github.com/users/matrixbot/events{/privacy}","received_events_url":"https://api.github.com/users/matrixbot/received_events","type":"User","site_admin":false},"created_at":"2016-11-19T14:27:35Z","updated_at":"2016-11-19T14:27:35Z","body":"Can one of the admins verify this patch?"}}
{"actor":{"display_login":"matrixbot"}}