{"action":"created","issue":{"url":"https://api.github.com/repos/EclecticIQ/cabby/issues/34","repository_url":"https://api.github.com/repos/EclecticIQ/cabby","labels_url":"https://api.github.com/repos/EclecticIQ/cabby/issues/34/labels{/name}","comments_url":"https://api.github.com/repos/EclecticIQ/cabby/issues/34/comments","events_url":"https://api.github.com/repos/EclecticIQ/cabby/issues/34/events","html_url":"https://github.com/EclecticIQ/cabby/issues/34","id":149885868,"number":34,"title":"Unable to Read Proxied Content with Taxii-Poll","user":{"login":"nickwallen","id":2475409,"avatar_url":"https://avatars.githubusercontent.com/u/2475409?v=3","gravatar_id":"","url":"https://api.github.com/users/nickwallen","html_url":"https://github.com/nickwallen","followers_url":"https://api.github.com/users/nickwallen/followers","following_url":"https://api.github.com/users/nickwallen/following{/other_user}","gists_url":"https://api.github.com/users/nickwallen/gists{/gist_id}","starred_url":"https://api.github.com/users/nickwallen/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nickwallen/subscriptions","organizations_url":"https://api.github.com/users/nickwallen/orgs","repos_url":"https://api.github.com/users/nickwallen/repos","events_url":"https://api.github.com/users/nickwallen/events{/privacy}","received_events_url":"https://api.github.com/users/nickwallen/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2016-04-20T21:04:57Z","updated_at":"2016-04-25T14:54:47Z","closed_at":null,"body":"### Goal \r\nI am trying to use `taxii-proxy` to fetch some data from hailataxi.com and push into my own local collection.  \r\n\r\n### Problem\r\nBased on the size of the database file, it appears this is happening.  But when I try to use `taxii-poll` on my local collection it does not return any data.  Why is this?\r\n\r\n1. The size of the data file indicates how much data roughly is in my collection.\r\n  ```\r\n  (opentaxii-venv)[root@source opentaxii]# ls -l data\r\n  total 1804\r\n  -rw-r--r--. 1 root root 1838080 Apr 20 20:50 data.db\r\n  -rw-r--r--. 1 root root    3072 Apr 20 20:48 auth.db\r\n  ```\r\n\r\n2. Now proxy the data from hailataxii.com\r\n  ```\r\n  (opentaxii-venv)[root@source opentaxii]# taxii-proxy --poll-path http://hailataxii.com/taxii-data             \r\n    --poll-collection guest.phishtank_com             \r\n    --inbox-path http://localhost:9000/services/inbox             \r\n    --inbox-collection hailataxii-phishtank             \r\n    --binding urn:stix.mitre.org:xml:1.1             \r\n    --inbox-username guest             \r\n    --inbox-password guest             \r\n    --begin 2016-04-20 \r\n  2016-04-20 20:51:53,139 INFO: Sending Poll_Request to http://hailataxii.com/taxii-data\r\n  2016-04-20 20:51:53,146 INFO: Starting new HTTP connection (1): hailataxii.com\r\n  2016-04-20 20:51:53,624 INFO: Response received for Poll_Request from http://hailataxii.com/taxii-data\r\n  2016-04-20 20:51:53,800 INFO: Sending Inbox_Message to http://localhost:9000/services/inbox\r\n  ...\r\n  ```\r\n\r\n3. Check the size of the data file again.  Its bigger!  We must have successfully received data.\r\n  ```\r\n  (opentaxii-venv)[root@source opentaxii]# ls -l data\r\n  total 2736\r\n  -rw-r--r--. 1 root root 2791424 Apr 20 20:51 data.db\r\n  -rw-r--r--. 1 root root    3072 Apr 20 20:48 auth.db\r\n  ```\r\n\r\n4. But there is no data.  Why is that?\r\n  ```\r\n  (opentaxii-venv)[root@source opentaxii]# taxii-poll --discovery http://localhost:9000/services/discovery -c hailataxii-phishtank\r\n  2016-04-20 20:55:01,873 INFO: Polling using data binding: ALL\r\n  2016-04-20 20:55:01,874 INFO: Sending Discovery_Request to http://localhost:9000/services/discovery\r\n  2016-04-20 20:55:01,880 INFO: Starting new HTTP connection (1): localhost\r\n  2016-04-20 20:55:01,886 INFO: Response received for Discovery_Request from http://localhost:9000/services/discovery\r\n  2016-04-20 20:55:01,888 INFO: 3 services discovered\r\n  2016-04-20 20:55:01,888 INFO: Sending Poll_Request to http://localhost:9000/services/poll\r\n  2016-04-20 20:55:01,890 INFO: Starting new HTTP connection (1): localhost\r\n  2016-04-20 20:55:01,898 INFO: Response received for Poll_Request from http://localhost:9000/services/poll\r\n  2016-04-20 20:55:01,899 INFO: 0 blocks polled\r\n  ```\r\n\r\n### Configuration\r\n\r\nservices.yml\r\n```\r\n---\r\n#\r\n# discovery: used by a TAXII Client to discover available TAXII Service\r\n#\r\n- id: discovery\r\n  type: discovery\r\n  address: /services/discovery\r\n  description: Discovery service\r\n  advertised_services:\r\n    - inbox\r\n    - collection\r\n    - poll\r\n  protocol_bindings:\r\n    - urn:taxii.mitre.org:protocol:http:1.0\r\n\r\n#\r\n# inbox: used by a TAXII Client to push information to a TAXII Server\r\n#\r\n- id: inbox\r\n  type: inbox\r\n  address: /services/inbox\r\n  description: Inbox for hailataxii.com feed\r\n  destination_collection_required: yes\r\n  accept_all_content: no\r\n  authentication_required: yes\r\n  protocol_bindings:\r\n    - urn:taxii.mitre.org:protocol:http:1.0\r\n\r\n#\r\n# collection_management: used by a TAXII Client to request information about\r\n# available data collections or request a subscription.\r\n#\r\n- id: collection\r\n  type: collection_management\r\n  address: /services/collection\r\n  description: Collection management service for hailataxii.com feed\r\n  protocol_bindings:\r\n    - urn:taxii.mitre.org:protocol:http:1.0\r\n\r\n#\r\n# poll: used by a TAXII Client to request information from a TAXII Server\r\n#\r\n- id: poll\r\n  type: poll\r\n  address: /services/poll\r\n  description: Poll for hailataxii.com feed\r\n  subscription_required: no\r\n  authentication_required: no\r\n  protocol_bindings:\r\n    - urn:taxii.mitre.org:protocol:http:1.0\r\n```\r\n\r\ncollections.yml\r\n```\r\n---\r\n- name: hailataxii-phishtank\r\n  available: true\r\n  accept_all_content: true\r\n  supported_content:\r\n    - urn:stix.mitre.org:xml:1.0\r\n    - urn:stix.mitre.org:xml:1.1\r\n  service_ids:\r\n    #- discovery\r\n    - inbox\r\n    - collection\r\n    - poll\r\n```"},"comment":{"url":"https://api.github.com/repos/EclecticIQ/cabby/issues/comments/214374029","html_url":"https://github.com/EclecticIQ/cabby/issues/34#issuecomment-214374029","issue_url":"https://api.github.com/repos/EclecticIQ/cabby/issues/34","id":214374029,"user":{"login":"nickwallen","id":2475409,"avatar_url":"https://avatars.githubusercontent.com/u/2475409?v=3","gravatar_id":"","url":"https://api.github.com/users/nickwallen","html_url":"https://github.com/nickwallen","followers_url":"https://api.github.com/users/nickwallen/followers","following_url":"https://api.github.com/users/nickwallen/following{/other_user}","gists_url":"https://api.github.com/users/nickwallen/gists{/gist_id}","starred_url":"https://api.github.com/users/nickwallen/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nickwallen/subscriptions","organizations_url":"https://api.github.com/users/nickwallen/orgs","repos_url":"https://api.github.com/users/nickwallen/repos","events_url":"https://api.github.com/users/nickwallen/events{/privacy}","received_events_url":"https://api.github.com/users/nickwallen/received_events","type":"User","site_admin":false},"created_at":"2016-04-25T14:54:46Z","updated_at":"2016-04-25T14:54:46Z","body":"Is `save_raw_inbox_message` documented anywhere?  I don't understand what that is supposed to do."}}