{"action":"created","issue":{"url":"https://api.github.com/repos/petabyte-research/redflags/issues/42","repository_url":"https://api.github.com/repos/petabyte-research/redflags","labels_url":"https://api.github.com/repos/petabyte-research/redflags/issues/42/labels{/name}","comments_url":"https://api.github.com/repos/petabyte-research/redflags/issues/42/comments","events_url":"https://api.github.com/repos/petabyte-research/redflags/issues/42/events","html_url":"https://github.com/petabyte-research/redflags/issues/42","id":182228969,"number":42,"title":"Engine pobiera tylko 500 zamówień z MP API","user":{"login":"KrzysztofMadejski","id":849975,"avatar_url":"https://avatars.githubusercontent.com/u/849975?v=3","gravatar_id":"","url":"https://api.github.com/users/KrzysztofMadejski","html_url":"https://github.com/KrzysztofMadejski","followers_url":"https://api.github.com/users/KrzysztofMadejski/followers","following_url":"https://api.github.com/users/KrzysztofMadejski/following{/other_user}","gists_url":"https://api.github.com/users/KrzysztofMadejski/gists{/gist_id}","starred_url":"https://api.github.com/users/KrzysztofMadejski/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/KrzysztofMadejski/subscriptions","organizations_url":"https://api.github.com/users/KrzysztofMadejski/orgs","repos_url":"https://api.github.com/users/KrzysztofMadejski/repos","events_url":"https://api.github.com/users/KrzysztofMadejski/events{/privacy}","received_events_url":"https://api.github.com/users/KrzysztofMadejski/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":{"login":"dzierzy","id":286593,"avatar_url":"https://avatars.githubusercontent.com/u/286593?v=3","gravatar_id":"","url":"https://api.github.com/users/dzierzy","html_url":"https://github.com/dzierzy","followers_url":"https://api.github.com/users/dzierzy/followers","following_url":"https://api.github.com/users/dzierzy/following{/other_user}","gists_url":"https://api.github.com/users/dzierzy/gists{/gist_id}","starred_url":"https://api.github.com/users/dzierzy/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dzierzy/subscriptions","organizations_url":"https://api.github.com/users/dzierzy/orgs","repos_url":"https://api.github.com/users/dzierzy/repos","events_url":"https://api.github.com/users/dzierzy/events{/privacy}","received_events_url":"https://api.github.com/users/dzierzy/received_events","type":"User","site_admin":false},"assignees":[{"login":"dzierzy","id":286593,"avatar_url":"https://avatars.githubusercontent.com/u/286593?v=3","gravatar_id":"","url":"https://api.github.com/users/dzierzy","html_url":"https://github.com/dzierzy","followers_url":"https://api.github.com/users/dzierzy/followers","following_url":"https://api.github.com/users/dzierzy/following{/other_user}","gists_url":"https://api.github.com/users/dzierzy/gists{/gist_id}","starred_url":"https://api.github.com/users/dzierzy/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dzierzy/subscriptions","organizations_url":"https://api.github.com/users/dzierzy/orgs","repos_url":"https://api.github.com/users/dzierzy/repos","events_url":"https://api.github.com/users/dzierzy/events{/privacy}","received_events_url":"https://api.github.com/users/dzierzy/received_events","type":"User","site_admin":false}],"milestone":{"url":"https://api.github.com/repos/petabyte-research/redflags/milestones/1","html_url":"https://github.com/petabyte-research/redflags/milestones/Scaling-PL","labels_url":"https://api.github.com/repos/petabyte-research/redflags/milestones/1/labels","id":1802331,"number":1,"title":"Scaling-PL","description":"Scaling RedFlags to Polish context","creator":{"login":"KrzysztofMadejski","id":849975,"avatar_url":"https://avatars.githubusercontent.com/u/849975?v=3","gravatar_id":"","url":"https://api.github.com/users/KrzysztofMadejski","html_url":"https://github.com/KrzysztofMadejski","followers_url":"https://api.github.com/users/KrzysztofMadejski/followers","following_url":"https://api.github.com/users/KrzysztofMadejski/following{/other_user}","gists_url":"https://api.github.com/users/KrzysztofMadejski/gists{/gist_id}","starred_url":"https://api.github.com/users/KrzysztofMadejski/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/KrzysztofMadejski/subscriptions","organizations_url":"https://api.github.com/users/KrzysztofMadejski/orgs","repos_url":"https://api.github.com/users/KrzysztofMadejski/repos","events_url":"https://api.github.com/users/KrzysztofMadejski/events{/privacy}","received_events_url":"https://api.github.com/users/KrzysztofMadejski/received_events","type":"User","site_admin":false},"open_issues":6,"closed_issues":9,"state":"open","created_at":"2016-06-01T13:31:30Z","updated_at":"2016-10-11T12:18:20Z","due_on":"2016-07-30T22:00:00Z","closed_at":null},"comments":5,"created_at":"2016-10-11T10:28:12Z","updated_at":"2016-10-11T18:44:46Z","closed_at":null,"body":"Wyciąg z logów:\r\n```\r\n2016-09-29 12:31:14,996 [INFO ] [     rf-engine-0] h.p.redflags.engine.epforgpl.EPFScope    > loading procurements. Page number: 1\r\n2016-09-29 12:31:14,997 [INFO ] [     rf-engine-0] h.p.redflags.engine.epforgpl.EPFData     > fetching procurement list\r\n2016-09-29 12:31:14,997 [INFO ] [     rf-engine-0] h.p.redflags.engine.epforgpl.Connector   > fetching data from zamowienia_publiczne/?limit=10000&page=1\r\n2016-09-29 12:31:15,340 [INFO ] [            main] h.p.redflags.engine.RedflagsEngineApp    > Started RedflagsEngineApp in 5.092 seconds (JVM running for 5.967)\r\n2016-09-29 12:31:17,368 [INFO ] [     rf-engine-0] h.p.redflags.engine.epforgpl.EPFScope    > processing notice 2884665, 1 procurement in row\r\n.....\r\n2016-09-29 12:31:17,508 [INFO ] [     rf-engine-0] h.p.redflags.engine.epforgpl.EPFScope    > processing notice 2871817, 500 procurement in row\r\n2016-09-29 12:31:17,508 [INFO ] [     rf-engine-0] h.p.redflags.engine.epforgpl.EPFScope    > loading procurements. Page number: 2\r\n2016-09-29 12:31:17,757 [INFO ] [ pool-2-thread-1] h.p.redflags.engine.epforgpl.Connector   > fetching data from zamowienia_publiczne_zamawiajacy/1458\r\n... tu już treść zamówień\r\n2016-09-29 12:37:18,805 [DEBUG] [ pool-2-thread-1] h.p.redflags.engine.gear.GearTrain       > Processing of notice 2871817 finished\r\n2016-09-29 12:37:18,805 [INFO ] [     rf-engine-0] h.p.r.engine.gear.export.FlagExporter    > Nothing to export\r\n2016-09-29 12:37:18,805 [INFO ] [     rf-engine-0] h.p.r.engine.gear.export.FlagExporter    > Nothing to export\r\n2016-09-29 12:37:18,806 [DEBUG] [     rf-engine-0] h.p.r.engine.gear.export.MySQLExporter   > Pushing CSV into MySQL: te_organization_20160929\r\n2016-09-29 12:37:18,823 [DEBUG] [     rf-engine-0] h.p.r.engine.gear.export.MySQLExporter   > Pushing CSV into MySQL: te_relationdescriptor_20160929\r\n2016-09-29 12:37:18,857 [DEBUG] [     rf-engine-0] h.p.r.engine.gear.export.MySQLExporter   > Pushing CSV into MySQL: te_objofthecontract_20160929\r\n2016-09-29 12:37:18,880 [DEBUG] [     rf-engine-0] h.p.r.engine.gear.export.MySQLExporter   > Pushing CSV into MySQL: te_datatype_20160929\r\n2016-09-29 12:37:18,885 [DEBUG] [     rf-engine-0] h.p.r.engine.gear.export.MySQLExporter   > Pushing CSV into MySQL: te_cpv_20160929\r\n2016-09-29 12:37:18,886 [DEBUG] [     rf-engine-0] h.p.r.engine.gear.export.MySQLExporter   > Pushing CSV into MySQL: te_notice_20160929\r\n2016-09-29 12:37:18,898 [DEBUG] [     rf-engine-0] h.p.r.engine.gear.export.MySQLExporter   > Pushing CSV into MySQL: te_duration_20160929\r\n2016-09-29 12:37:18,907 [DEBUG] [     rf-engine-0] h.p.r.engine.gear.export.MySQLExporter   > Pushing CSV into MySQL: te_procedure_20160929\r\n2016-09-29 12:37:18,919 [DEBUG] [     rf-engine-0] h.p.r.engine.gear.export.MySQLExporter   > Pushing CSV into MySQL: te_address_20160929\r\n2016-09-29 12:37:18,949 [DEBUG] [     rf-engine-0] h.p.r.engine.gear.export.MySQLExporter   > Pushing CSV into MySQL: te_contractingauthority_20160929\r\n2016-09-29 12:37:18,961 [DEBUG] [     rf-engine-0] h.p.r.engine.gear.export.MySQLExporter   > Pushing CSV into MySQL: te_data_20160929\r\n2016-09-29 12:37:18,977 [DEBUG] [     rf-engine-0] h.p.r.engine.gear.export.MySQLExporter   > Pushing CSV into MySQL: te_flag_20160929\r\n2016-09-29 12:37:18,987 [INFO ] [     rf-engine-0] h.p.r.engine.gear.export.MySQLExporter   > Exported 500 notices into MySQL\r\n2016-09-29 12:37:18,987 [INFO ] [     rf-engine-0] h.p.r.engine.boot.RedflagsEngineSession  > Finished Redflags engine session with scope hu.petabyte.redflags.engine.epforgpl.EPFScope@687a4832 (HEAP: 55M)\r\n2016-09-29 12:37:18,988 [INFO ] [     rf-engine-0] h.p.r.engine.boot.RedflagsEngineSession  > Processed 500 notices in 6 minutes 4 seconds 949 milliseconds\r\n2016-09-29 12:37:18,990 [INFO ] [    main-destroy] h.p.r.engine.boot.RedflagsEngineBoot     > *** REDFLAGS ENGINE - Stopping\r\n```\r\n\r\nDlaczego więcej stron nie pobrał?\r\nhttps://api-v3.mojepanstwo.pl/dane/zamowienia_publiczne/?limit=10000&page=1 pobieraa tylko 500 obiektów, bo taki jest limit ustawiony maksymalny, może to trzeba poprawić w skrypcie, bo pewnie myśli że skoro chciał 10000 a dostał 500 to już więcej nie ma\r\n\r\na https://api-v3.mojepanstwo.pl/dane/zamowienia_publiczne/?limit=10000&page=2&_type=objects dalej dane zwraca..\r\n"},"comment":{"url":"https://api.github.com/repos/petabyte-research/redflags/issues/comments/253007596","html_url":"https://github.com/petabyte-research/redflags/issues/42#issuecomment-253007596","issue_url":"https://api.github.com/repos/petabyte-research/redflags/issues/42","id":253007596,"user":{"login":"KrzysztofMadejski","id":849975,"avatar_url":"https://avatars.githubusercontent.com/u/849975?v=3","gravatar_id":"","url":"https://api.github.com/users/KrzysztofMadejski","html_url":"https://github.com/KrzysztofMadejski","followers_url":"https://api.github.com/users/KrzysztofMadejski/followers","following_url":"https://api.github.com/users/KrzysztofMadejski/following{/other_user}","gists_url":"https://api.github.com/users/KrzysztofMadejski/gists{/gist_id}","starred_url":"https://api.github.com/users/KrzysztofMadejski/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/KrzysztofMadejski/subscriptions","organizations_url":"https://api.github.com/users/KrzysztofMadejski/orgs","repos_url":"https://api.github.com/users/KrzysztofMadejski/repos","events_url":"https://api.github.com/users/KrzysztofMadejski/events{/privacy}","received_events_url":"https://api.github.com/users/KrzysztofMadejski/received_events","type":"User","site_admin":false},"created_at":"2016-10-11T18:44:46Z","updated_at":"2016-10-11T18:44:46Z","body":"Idzie!"}}
{"actor":{"display_login":"KrzysztofMadejski"}}