{"action":"created","issue":{"url":"https://api.github.com/repos/Particular/ServiceControl/issues/694","repository_url":"https://api.github.com/repos/Particular/ServiceControl","labels_url":"https://api.github.com/repos/Particular/ServiceControl/issues/694/labels{/name}","comments_url":"https://api.github.com/repos/Particular/ServiceControl/issues/694/comments","events_url":"https://api.github.com/repos/Particular/ServiceControl/issues/694/events","html_url":"https://github.com/Particular/ServiceControl/pull/694","id":140829319,"number":694,"title":"Add a retention policy for error messages","user":{"login":"johnsimons","id":122651,"avatar_url":"https://avatars.githubusercontent.com/u/122651?v=3","gravatar_id":"","url":"https://api.github.com/users/johnsimons","html_url":"https://github.com/johnsimons","followers_url":"https://api.github.com/users/johnsimons/followers","following_url":"https://api.github.com/users/johnsimons/following{/other_user}","gists_url":"https://api.github.com/users/johnsimons/gists{/gist_id}","starred_url":"https://api.github.com/users/johnsimons/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/johnsimons/subscriptions","organizations_url":"https://api.github.com/users/johnsimons/orgs","repos_url":"https://api.github.com/users/johnsimons/repos","events_url":"https://api.github.com/users/johnsimons/events{/privacy}","received_events_url":"https://api.github.com/users/johnsimons/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/Particular/ServiceControl/labels/Critical","name":"Critical","color":"e11d21"},{"url":"https://api.github.com/repos/Particular/ServiceControl/labels/State:%20In%20Progress","name":"State: In Progress","color":"ededed"},{"url":"https://api.github.com/repos/Particular/ServiceControl/labels/Type:%20Bug","name":"Type: Bug","color":"e10c02"}],"state":"open","locked":false,"assignee":{"login":"johnsimons","id":122651,"avatar_url":"https://avatars.githubusercontent.com/u/122651?v=3","gravatar_id":"","url":"https://api.github.com/users/johnsimons","html_url":"https://github.com/johnsimons","followers_url":"https://api.github.com/users/johnsimons/followers","following_url":"https://api.github.com/users/johnsimons/following{/other_user}","gists_url":"https://api.github.com/users/johnsimons/gists{/gist_id}","starred_url":"https://api.github.com/users/johnsimons/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/johnsimons/subscriptions","organizations_url":"https://api.github.com/users/johnsimons/orgs","repos_url":"https://api.github.com/users/johnsimons/repos","events_url":"https://api.github.com/users/johnsimons/events{/privacy}","received_events_url":"https://api.github.com/users/johnsimons/received_events","type":"User","site_admin":false},"milestone":{"url":"https://api.github.com/repos/Particular/ServiceControl/milestones/52","html_url":"https://github.com/Particular/ServiceControl/milestones/1.13.0","labels_url":"https://api.github.com/repos/Particular/ServiceControl/milestones/52/labels","id":1663009,"number":52,"title":"1.13.0","description":"","creator":{"login":"gbiellem","id":2351700,"avatar_url":"https://avatars.githubusercontent.com/u/2351700?v=3","gravatar_id":"","url":"https://api.github.com/users/gbiellem","html_url":"https://github.com/gbiellem","followers_url":"https://api.github.com/users/gbiellem/followers","following_url":"https://api.github.com/users/gbiellem/following{/other_user}","gists_url":"https://api.github.com/users/gbiellem/gists{/gist_id}","starred_url":"https://api.github.com/users/gbiellem/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gbiellem/subscriptions","organizations_url":"https://api.github.com/users/gbiellem/orgs","repos_url":"https://api.github.com/users/gbiellem/repos","events_url":"https://api.github.com/users/gbiellem/events{/privacy}","received_events_url":"https://api.github.com/users/gbiellem/received_events","type":"User","site_admin":false},"open_issues":1,"closed_issues":0,"state":"open","created_at":"2016-03-24T00:14:17Z","updated_at":"2016-03-24T00:14:42Z","due_on":null,"closed_at":null},"comments":12,"created_at":"2016-03-14T23:49:42Z","updated_at":"2016-03-24T00:15:52Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/Particular/ServiceControl/pulls/694","html_url":"https://github.com/Particular/ServiceControl/pull/694","diff_url":"https://github.com/Particular/ServiceControl/pull/694.diff","patch_url":"https://github.com/Particular/ServiceControl/pull/694.patch"},"body":"## Symptoms\r\nServiceControl currently only has a retention policy for audit messages.\r\nError messages are currently kept forever.\r\nAt worse this causes ServiceControl to eventually run out of disk space and bring down the process.\r\n\r\n## Who's affected\r\nAll users of ServiceControl that are currently running v1.11.1.\r\n\r\n## Plan of attack\r\n\r\n#### Service Control Management per instance \r\n- [ ] New field (`ServiceControl/ErrorRetentionPeriod`) to capture retention for the unresolved error messages, this field range has to be between 10-45 days, so minimum would be 10 days and maximum 45 days;\r\n- [ ] New field (`ServiceControl/AuditRetentionPeriod`) to capture retention for the audits, this field range is between minutes-days, so minimum would be 1 hour and maximum 365 days;\r\n\r\n#### Service Control  \r\n- Api Changes:\r\n  - [x] Modify /api/errors to include `LastModified` datetime in results and allow to sort and filter on that same new field.  \r\n  Example of usage `\r\n  http://localhost:33333/api/errors/?sort=modified&modified=2016-01-25T15:38:35.6767764Z...2016-01-25T15:38:36.6767764Z&status=archived`.  \r\n  Date formatting for the query parameters must follow the [ISO8601 standard](http://en.wikipedia.org/wiki/ISO_8601), [here is an example on how to do it in javascript](http://www.w3schools.com/jsref/jsref_toisostring.asp).\r\n  - [x] We are also including #703\r\n  - [x] Unarchive by date range and by array of ids.  \r\n  Example of usage by date range:\r\n  ```\r\n  PATCH http://localhost:33333/api/errors/2016-01-25T15:38:35.6767764Z...2016-01-25T15:38:36.6767764Z/unarchive\r\n  ```\r\n  Example of usage by ids:\r\n  ```\r\n  PATCH http://localhost:33333/api/errors/unarchive\r\n  ```\r\n  ```json\r\n  [ \r\n \"e989a17f-4cf5-79f4-6399-2bddd3084412\",\r\n \"5feb1b8c-c171-abfe-afad-f863b7681d09\"\r\n  ]\r\n  ```\r\n  - [x] Get current cool-off settings  \r\n  Example of usage:\r\n  ```\r\n  GET http://localhost:33333/api/configuration\r\n  ```\r\n  ```json\r\n{\r\n    \"host\": {\r\n        \"service_name\": \"Particular.ServiceControl\",\r\n        \"raven_db_path\": \"C:\\\\ProgramData\\\\Particular\\\\ServiceControl\\\\localhost-33333\",\r\n        \"logging\": {\r\n            \"log_path\": \"C:\\\\Users\\\\john\\\\AppData\\\\Local\\\\Particular\\\\ServiceControl\\\\logs\",\r\n            \"logging_level\": \"Warn\",\r\n            \"raven_db_log_level\": \"Warn\"\r\n        }\r\n    },\r\n    \"data_retention\": {\r\n        \"audit_retention_period\": \"01:00:00\",\r\n        \"error_retention_period\": \"02:00:00\"\r\n    },\r\n    \"performance_tunning\": {\r\n        \"maximum_message_throughput_per_second\": 350,\r\n        \"max_body_size_to_store\": 102400,\r\n        \"http_default_connection_limit\": 100,\r\n        \"external_integrations_dispatching_batch_size\": 100,\r\n        \"expiration_process_batch_size\": 32756,\r\n        \"expiration_process_timer_in_seconds\": 600\r\n    },\r\n    \"transport\": {\r\n        \"transport_type\": \"NServiceBus.MsmqTransport, NServiceBus.Core, Version=5.0.0.0, Culture=neutral, PublicKeyToken=9fc386479f8a226c\",\r\n        \"audit_log_queue\": \"audit.log\",\r\n        \"audit_queue\": \"audit\",\r\n        \"error_log_queue\": \"error.log\",\r\n        \"error_queue\": \"error\",\r\n        \"forward_audit_messages\": false,\r\n        \"forward_error_messages\": false\r\n    },\r\n    \"plugins\": {\r\n        \"heartbeat_grace_period\": \"00:00:40\"\r\n    }\r\n}\r\n  ```\r\n  - [x] SignalR messages for service pulse, when messages are unarchived\r\n  - [x] Schedule job to do deletion of unresolved error messages that are older than the cool-off period\r\n- [x] Rename existing `ServiceControl/HoursToKeepMessagesBeforeExpiring` to `ServiceControl/AuditRetentionPeriod`\r\n\r\n#### Update doco\r\n- [x] http://docs.particular.net/servicecontrol/creating-config-file\r\n- [x] http://docs.particular.net/servicecontrol/how-purge-expired-data\r\n- [x] http://docs.particular.net/servicecontrol/installation-silent\r\n\r\nOngoing PR - https://github.com/Particular/docs.particular.net/pull/1318"},"comment":{"url":"https://api.github.com/repos/Particular/ServiceControl/issues/comments/200592693","html_url":"https://github.com/Particular/ServiceControl/pull/694#issuecomment-200592693","issue_url":"https://api.github.com/repos/Particular/ServiceControl/issues/694","id":200592693,"user":{"login":"gbiellem","id":2351700,"avatar_url":"https://avatars.githubusercontent.com/u/2351700?v=3","gravatar_id":"","url":"https://api.github.com/users/gbiellem","html_url":"https://github.com/gbiellem","followers_url":"https://api.github.com/users/gbiellem/followers","following_url":"https://api.github.com/users/gbiellem/following{/other_user}","gists_url":"https://api.github.com/users/gbiellem/gists{/gist_id}","starred_url":"https://api.github.com/users/gbiellem/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gbiellem/subscriptions","organizations_url":"https://api.github.com/users/gbiellem/orgs","repos_url":"https://api.github.com/users/gbiellem/repos","events_url":"https://api.github.com/users/gbiellem/events{/privacy}","received_events_url":"https://api.github.com/users/gbiellem/received_events","type":"User","site_admin":false},"created_at":"2016-03-24T00:15:52Z","updated_at":"2016-03-24T00:15:52Z","body":"Bumped this to 1.13 so we can get the SQLTransport patch out now"}}