{"action":"created","issue":{"url":"https://api.github.com/repos/JaidenAshmore/java-dynamic-sqs-listener/issues/1","repository_url":"https://api.github.com/repos/JaidenAshmore/java-dynamic-sqs-listener","labels_url":"https://api.github.com/repos/JaidenAshmore/java-dynamic-sqs-listener/issues/1/labels{/name}","comments_url":"https://api.github.com/repos/JaidenAshmore/java-dynamic-sqs-listener/issues/1/comments","events_url":"https://api.github.com/repos/JaidenAshmore/java-dynamic-sqs-listener/issues/1/events","html_url":"https://github.com/JaidenAshmore/java-dynamic-sqs-listener/pull/1","id":373754772,"node_id":"MDExOlB1bGxSZXF1ZXN0MjI1NjA5OTA4","number":1,"title":"NONE: Added a core implementation with the basic architecture for this tool","user":{"login":"JaidenAshmore","id":1059637,"node_id":"MDQ6VXNlcjEwNTk2Mzc=","avatar_url":"https://avatars3.githubusercontent.com/u/1059637?v=4","gravatar_id":"","url":"https://api.github.com/users/JaidenAshmore","html_url":"https://github.com/JaidenAshmore","followers_url":"https://api.github.com/users/JaidenAshmore/followers","following_url":"https://api.github.com/users/JaidenAshmore/following{/other_user}","gists_url":"https://api.github.com/users/JaidenAshmore/gists{/gist_id}","starred_url":"https://api.github.com/users/JaidenAshmore/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/JaidenAshmore/subscriptions","organizations_url":"https://api.github.com/users/JaidenAshmore/orgs","repos_url":"https://api.github.com/users/JaidenAshmore/repos","events_url":"https://api.github.com/users/JaidenAshmore/events{/privacy}","received_events_url":"https://api.github.com/users/JaidenAshmore/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2018-10-25T03:06:14Z","updated_at":"2018-11-12T18:35:15Z","closed_at":null,"author_association":"OWNER","pull_request":{"url":"https://api.github.com/repos/JaidenAshmore/java-dynamic-sqs-listener/pulls/1","html_url":"https://github.com/JaidenAshmore/java-dynamic-sqs-listener/pull/1","diff_url":"https://github.com/JaidenAshmore/java-dynamic-sqs-listener/pull/1.diff","patch_url":"https://github.com/JaidenAshmore/java-dynamic-sqs-listener/pull/1.patch"},"body":"This adds the core architecture for the SQS Listener which should hopefully provide the ability to easily configure and make it dynamic. Still got work to do but first stage.\r\n\r\n### How to review?\r\n1. Start at the root README.md to get a basic understanding of the architecture.\r\n1. Read the API interfaces for futher information about the architecture\r\n1. Download the code and test out one of the core examples, see /Users/jashmore/dev/java-dynamic-sqs-listener/examples/java-dynamic-sqs-listener-core-examples/src/main/java/com/jashmore/sqs/examples/ConcurrentBrokerExample.java which will run with a local elasticmq server.\r\n1. Look at the core implementations.\r\n\r\n### What is weird?\r\n- The PrefetchingMessageRetriever has a weird way using a blocking queue to cache the prefetched messages, it works but is a little bit unintuitive.\r\n- IndividualMessageRetriever has a weird way to handle timeouts for retrieving a message where tries to get a message every 20 seconds and will try to get the message in the provided timeout as close as possible but isn't great.\r\n- I only just really learnt about interrupting threads so the way I implemented stopping the brokers changed a lot during the implementation, this may still be weird.\r\n- Testing concurrent code is far more difficult.\r\n\r\n### Next up\r\n- Adding the Spring Starter wrapper around this. I had originally been building this at the same time but that just caused problems with the design of the API and trying to do too much at once. Now that I have the core implementation I can then create any specific classes that are needed for spring in the spring starter and not have that pollute the core implement. I had begun working this when I realised how I handle shutting down the listeners wasn't optimal so had to go and rewrite the shutting down implementation using interrupts. See https://github.com/JaidenAshmore/java-dynamic-sqs-listener/pull/3 for what I am working on in regards to this\r\n- Add a batching message retriever that is in between the prefetching and individual message retriever. This will try and reduce the overhead of the individual message retriever that requests a message for every request but doesn't prefetch any messages and therefore won't have any messages visibility timing out before they are even processed. When a message is requested it will wait a certain period for other threads to request messages and then send a single request for multiple messages. Therefore if 7 threads are all wanting messages it will send a single call requesting for 7 messages.\r\n- Experiment with RxJava and see if it makes the broker easier to implement with this. This will be kept outside of core so that consumers will only pull in the RxJava implementation when they need it. We don't want Core polluted."},"comment":{"url":"https://api.github.com/repos/JaidenAshmore/java-dynamic-sqs-listener/issues/comments/437985149","html_url":"https://github.com/JaidenAshmore/java-dynamic-sqs-listener/pull/1#issuecomment-437985149","issue_url":"https://api.github.com/repos/JaidenAshmore/java-dynamic-sqs-listener/issues/1","id":437985149,"node_id":"MDEyOklzc3VlQ29tbWVudDQzNzk4NTE0OQ==","user":{"login":"JaidenAshmore","id":1059637,"node_id":"MDQ6VXNlcjEwNTk2Mzc=","avatar_url":"https://avatars3.githubusercontent.com/u/1059637?v=4","gravatar_id":"","url":"https://api.github.com/users/JaidenAshmore","html_url":"https://github.com/JaidenAshmore","followers_url":"https://api.github.com/users/JaidenAshmore/followers","following_url":"https://api.github.com/users/JaidenAshmore/following{/other_user}","gists_url":"https://api.github.com/users/JaidenAshmore/gists{/gist_id}","starred_url":"https://api.github.com/users/JaidenAshmore/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/JaidenAshmore/subscriptions","organizations_url":"https://api.github.com/users/JaidenAshmore/orgs","repos_url":"https://api.github.com/users/JaidenAshmore/repos","events_url":"https://api.github.com/users/JaidenAshmore/events{/privacy}","received_events_url":"https://api.github.com/users/JaidenAshmore/received_events","type":"User","site_admin":false},"created_at":"2018-11-12T18:35:15Z","updated_at":"2018-11-12T18:35:15Z","author_association":"OWNER","body":"\r\n> So are you ok w/ the package name? Imagine if this takes off and people start using it - is \"com.jashmore.sqs\" what you want in it? I am not saying its good or bad - just came to mind.\r\n\r\n\r\nYeah that was definitely something I have been considering and yes I would probably want to change it but I have no idea what I would change it to.\r\n\r\n"}}
{"actor":{"display_login":"JaidenAshmore"}}