{"action":"created","issue":{"url":"https://api.github.com/repos/grafana/metrictank/issues/913","repository_url":"https://api.github.com/repos/grafana/metrictank","labels_url":"https://api.github.com/repos/grafana/metrictank/issues/913/labels{/name}","comments_url":"https://api.github.com/repos/grafana/metrictank/issues/913/comments","events_url":"https://api.github.com/repos/grafana/metrictank/issues/913/events","html_url":"https://github.com/grafana/metrictank/issues/913","id":323964190,"number":913,"title":"interval change resulting in unequal length series triggering panic in mergeSeries","user":{"login":"Dieterbe","id":20774,"avatar_url":"https://avatars2.githubusercontent.com/u/20774?v=4","gravatar_id":"","url":"https://api.github.com/users/Dieterbe","html_url":"https://github.com/Dieterbe","followers_url":"https://api.github.com/users/Dieterbe/followers","following_url":"https://api.github.com/users/Dieterbe/following{/other_user}","gists_url":"https://api.github.com/users/Dieterbe/gists{/gist_id}","starred_url":"https://api.github.com/users/Dieterbe/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Dieterbe/subscriptions","organizations_url":"https://api.github.com/users/Dieterbe/orgs","repos_url":"https://api.github.com/users/Dieterbe/repos","events_url":"https://api.github.com/users/Dieterbe/events{/privacy}","received_events_url":"https://api.github.com/users/Dieterbe/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2018-05-17T10:30:22Z","updated_at":"2018-05-17T10:41:30Z","closed_at":null,"author_association":"CONTRIBUTOR","body":"similar to #761 but different.\r\n\r\nI can reproduce like so:\r\n```\r\n./launch.sh docker-dev-custom-cfg-kafka\r\n# run this for a minute or two\r\nfakemetrics feed --period 30s --flush 30s --gnet-addr http://localhost:9000/metrics --gnet-key not_very_secret_key --mpo 30 --orgs 1\r\n# then run this. will probably need to run several minutes before can repro\r\nfakemetrics feed --period 120s --flush 120s --gnet-addr http://localhost:9000/metrics --gnet-key not_very_secret_key --mpo 120 --orgs 1\r\n```\r\n\r\nwhile running the 3rd command, issue this query in grafana:\r\n`some.id.of.a.metric.1*` with timerange now-5min to now\r\nthis will make MT panic.\r\ni added some debug info to MT like so (note, the comments andprint statements related to from/to were re diagnosing a different problem):\r\n```\r\ndiff --git a/api/dataprocessor.go b/api/dataprocessor.go\r\nindex 5311e4d7..89f137cd 100644\r\n--- a/api/dataprocessor.go\r\n+++ b/api/dataprocessor.go\r\n@@ -9,6 +9,7 @@ import (\r\n \t\"sync\"\r\n \t\"time\"\r\n \r\n+\t\"github.com/davecgh/go-spew/spew\"\r\n \t\"github.com/grafana/metrictank/api/models\"\r\n \t\"github.com/grafana/metrictank/consolidation\"\r\n \t\"github.com/grafana/metrictank/mdata\"\r\n@@ -138,6 +139,7 @@ func divide(pointsA, pointsB []schema.Point) []schema.Point {\r\n }\r\n \r\n func (s *Server) getTargets(ctx context.Context, reqs []models.Req) ([]models.Series, error) {\r\n+\t// from/to of the req here is same as plan, except if tweaked by function processing\r\n \t// split reqs into local and remote.\r\n \tlocalReqs := make([]models.Req, 0)\r\n \tremoteReqs := make(map[string][]models.Req)\r\n@@ -197,6 +199,8 @@ func (s *Server) getTargets(ctx context.Context, reqs []models.Req) ([]models.Se\r\n \treturn out, nil\r\n }\r\n \r\n+// getTargetsRemote issues the requests on other nodes\r\n+// it's nothing more than a thin network wrapper around getTargetsLocal of a peer.\r\n func (s *Server) getTargetsRemote(ctx context.Context, remoteReqs map[string][]models.Req) ([]models.Series, error) {\r\n \tresponses := make(chan getTargetsResp, len(remoteReqs))\r\n \trCtx, cancel := context.WithCancel(ctx)\r\n@@ -609,6 +613,10 @@ func mergeSeries(in []models.Series) []models.Series {\r\n \t\t\tlog.Debug(\"DP mergeSeries: %s has multiple series.\", series[0].Target)\r\n \t\t\tfor i := range series[0].Datapoints {\r\n \t\t\t\tfor j := 0; j < len(series); j++ {\r\n+\t\t\t\t\tif len(series[j].Datapoints) <= i {\r\n+\t\t\t\t\t\tspew.Dump(series)\r\n+\t\t\t\t\t\tpanic(\"uneven length detected\")\r\n+\t\t\t\t\t}\r\n \t\t\t\t\tif !math.IsNaN(series[j].Datapoints[i].Val) {\r\n \t\t\t\t\t\tseries[0].Datapoints[i].Val = series[j].Datapoints[i].Val\r\n \t\t\t\t\t\tbreak\r\n@@ -659,9 +667,14 @@ func newRequestContext(ctx context.Context, req *models.Req, consolidator consol\r\n \t// `from`   1..60 needs data    1..60   -> always adjust `from` to previous boundary+1 (here 1)\r\n \t// `to`  181..240 needs data 121..180   -> always adjust `to`   to previous boundary+1 (here 181)\r\n \r\n+\tfmt.Println(\"Consol is\", consolidator)\r\n+\r\n \tif consolidator == consolidation.None {\r\n+\t\tfmt.Println(\"consol None so:\")\r\n \t\trc.From = prevBoundary(req.From, req.ArchInterval) + 1\r\n \t\trc.To = prevBoundary(req.To, req.ArchInterval) + 1\r\n+\t\tfmt.Println(\"from\", req.From, \"->\", rc.From)\r\n+\t\tfmt.Println(\"to  \", req.To, \"->\", rc.To)\r\n \t\trc.AMKey = schema.AMKey{MKey: req.MKey}\r\n \t} else {\r\n \t\trc.From = req.From\r\ndiff --git a/api/graphite.go b/api/graphite.go\r\nindex db690537..0403bc0b 100644\r\n--- a/api/graphite.go\r\n+++ b/api/graphite.go\r\n@@ -674,6 +674,7 @@ func (s *Server) metricsDeleteRemote(ctx context.Context, orgId uint32, query st\r\n // note if you do something like sum(foo.*) and all of those metrics happen to be on another node,\r\n // we will collect all the indidividual series from the peer, and then sum here. that could be optimized\r\n func (s *Server) executePlan(ctx context.Context, orgId uint32, plan expr.Plan) ([]models.Series, error) {\r\n+\t// plan from/to are basically what user provided (but +1)\r\n \r\n \tminFrom := uint32(math.MaxUint32)\r\n \tvar maxTo uint32\r\n```\r\n\r\nand got the following output:\r\n(while fakemetrics was flushing at xx:15s every 2min)\r\n```\r\nmetrictank_1    | [Macaron] 2018-05-17 10:23:12: Started POST /render for 172.19.0.1, 172.19.0.1, 172.19.0.2\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526552293 -> 1526552281\r\nmetrictank_1    | to   1526552593 -> 1526552521\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526552293 -> 1526552281\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526552293 -> 1526552281\r\nmetrictank_1    | to   1526552593 -> 1526552521\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526552293 -> 1526552281\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526552293 -> 1526552281\r\nmetrictank_1    | to   1526552593 -> 1526552521\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526552293 -> 1526552281\r\nmetrictank_1    | to   1526552593 -> 1526552521\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526552293 -> 1526552281\r\nmetrictank_1    | to   1526552593 -> 1526552521\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526552293 -> 1526552281\r\nmetrictank_1    | to   1526552593 -> 1526552521\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526552293 -> 1526552281\r\nmetrictank_1    | to   1526552593 -> 1526552521\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526552293 -> 1526552281\r\nmetrictank_1    | to   1526552593 -> 1526552581\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526552293 -> 1526552281\r\nmetrictank_1    | to   1526552593 -> 1526552581\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526552293 -> 1526552281\r\nmetrictank_1    | to   1526552593 -> 1526552521\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526552293 -> 1526552281\r\nmetrictank_1    | to   1526552593 -> 1526552581\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526552293 -> 1526552281\r\nmetrictank_1    | to   1526552593 -> 1526552521\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526552293 -> 1526552281\r\nmetrictank_1    | to   1526552593 -> 1526552521\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526552293 -> 1526552281\r\nmetrictank_1    | to   1526552593 -> 1526552521\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526552293 -> 1526552281\r\nmetrictank_1    | to   1526552593 -> 1526552521\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526552293 -> 1526552281\r\nmetrictank_1    | to   1526552593 -> 1526552521\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526552293 -> 1526552281\r\nmetrictank_1    | to   1526552593 -> 1526552521\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | to   1526552593 -> 1526552581\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | from 1526552293 -> 1526552281\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526552293 -> 1526552281\r\nmetrictank_1    | to   1526552593 -> 1526552521\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526552293 -> 1526552281\r\nmetrictank_1    | to   1526552593 -> 1526552581\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526552293 -> 1526552281\r\nmetrictank_1    | to   1526552593 -> 1526552521\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526552293 -> 1526552281\r\nmetrictank_1    | to   1526552593 -> 1526552521\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526552293 -> 1526552281\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | from 1526552293 -> 1526552281\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | to   1526552593 -> 1526552521\r\nmetrictank_1    | to   1526552593 -> 1526552521\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526552293 -> 1526552281\r\nmetrictank_1    | to   1526552593 -> 1526552581\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | from 1526552293 -> 1526552281\r\nmetrictank_1    | from 1526552293 -> 1526552281\r\nmetrictank_1    | from 1526552293 -> 1526552281\r\nmetrictank_1    | to   1526552593 -> 1526552521\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526552293 -> 1526552281\r\nmetrictank_1    | to   1526552593 -> 1526552521\r\nmetrictank_1    | to   1526552593 -> 1526552521\r\nmetrictank_1    | from 1526552293 -> 1526552281\r\nmetrictank_1    | to   1526552593 -> 1526552521\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526552293 -> 1526552281\r\nmetrictank_1    | to   1526552593 -> 1526552521\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526552293 -> 1526552281\r\nmetrictank_1    | to   1526552593 -> 1526552521\r\nmetrictank_1    | from 1526552293 -> 1526552281\r\nmetrictank_1    | to   1526552593 -> 1526552521\r\nmetrictank_1    | from 1526552293 -> 1526552281\r\nmetrictank_1    | to   1526552593 -> 1526552521\r\nmetrictank_1    | from 1526552293 -> 1526552281\r\nmetrictank_1    | to   1526552593 -> 1526552521\r\nmetrictank_1    | to   1526552593 -> 1526552581\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526552293 -> 1526552281\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | to   1526552593 -> 1526552521\r\nmetrictank_1    | to   1526552593 -> 1526552521\r\nmetrictank_1    | to   1526552593 -> 1526552521\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526552293 -> 1526552281\r\nmetrictank_1    | to   1526552593 -> 1526552581\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526552293 -> 1526552281\r\nmetrictank_1    | to   1526552593 -> 1526552521\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526552293 -> 1526552281\r\nmetrictank_1    | to   1526552593 -> 1526552581\r\nmetrictank_1    | from 1526552293 -> 1526552281\r\nmetrictank_1    | to   1526552593 -> 1526552581\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526552293 -> 1526552281\r\nmetrictank_1    | to   1526552593 -> 1526552581\r\nmetrictank_1    | ([]models.Series) (len=2 cap=2) {\r\nmetrictank_1    |  (models.Series) {\r\nmetrictank_1    |   Target: (string) (len=37) \"some.id.of.a.metric.12;id=12;some=tag\",\r\nmetrictank_1    |   Datapoints: ([]schema.Point) (len=3 cap=10) {\r\nmetrictank_1    |    (schema.Point) {\r\nmetrictank_1    |     Val: (float64) 7.418930526012729,\r\nmetrictank_1    |     Ts: (uint32) 1526552400\r\nmetrictank_1    |    },\r\nmetrictank_1    |    (schema.Point) {\r\nmetrictank_1    |     Val: (float64) 9.763679531881163,\r\nmetrictank_1    |     Ts: (uint32) 1526552520\r\nmetrictank_1    |    },\r\nmetrictank_1    |    (schema.Point) {\r\nmetrictank_1    |     Val: (float64) NaN,\r\nmetrictank_1    |     Ts: (uint32) 1526552640\r\nmetrictank_1    |    }\r\nmetrictank_1    |   },\r\nmetrictank_1    |   Tags: (map[string]string) <nil>,\r\nmetrictank_1    |   Interval: (uint32) 120,\r\nmetrictank_1    |   QueryPatt: (string) (len=22) \"some.id.of.a.metric.1*\",\r\nmetrictank_1    |   QueryFrom: (uint32) 1526552293,\r\nmetrictank_1    |   QueryTo: (uint32) 1526552593,\r\nmetrictank_1    |   QueryCons: (consolidation.Consolidator) NoneConsolidator,\r\nmetrictank_1    |   Consolidator: (consolidation.Consolidator) AverageConsolidator\r\nmetrictank_1    |  },\r\nmetrictank_1    |  (models.Series) {\r\nmetrictank_1    |   Target: (string) (len=37) \"some.id.of.a.metric.12;id=12;some=tag\",\r\nmetrictank_1    |   Datapoints: ([]schema.Point) (len=2 cap=2) {\r\nmetrictank_1    |    (schema.Point) {\r\nmetrictank_1    |     Val: (float64) NaN,\r\nmetrictank_1    |     Ts: (uint32) 1526552400\r\nmetrictank_1    |    },\r\nmetrictank_1    |    (schema.Point) {\r\nmetrictank_1    |     Val: (float64) 9.763679531881163,\r\nmetrictank_1    |     Ts: (uint32) 1526552520\r\nmetrictank_1    |    }\r\nmetrictank_1    |   },\r\nmetrictank_1    |   Tags: (map[string]string) <nil>,\r\nmetrictank_1    |   Interval: (uint32) 120,\r\nmetrictank_1    |   QueryPatt: (string) (len=22) \"some.id.of.a.metric.1*\",\r\nmetrictank_1    |   QueryFrom: (uint32) 1526552293,\r\nmetrictank_1    |   QueryTo: (uint32) 1526552593,\r\nmetrictank_1    |   QueryCons: (consolidation.Consolidator) NoneConsolidator,\r\nmetrictank_1    |   Consolidator: (consolidation.Consolidator) AverageConsolidator\r\nmetrictank_1    |  }\r\nmetrictank_1    | }\r\nmetrictank_1    | [Macaron] PANIC: uneven length detected\r\nmetrictank_1    | /usr/lib/go/src/runtime/panic.go:502 (0x42a7a8)\r\ngrafana_1       | t=2018-05-17T10:23:12+0000 lvl=eror msg=\"Request Completed\" logger=context userId=1 orgId=1 uname=admin method=POST path=/api/datasources/proxy/3/render status=500 remote_addr=172.19.0.1 time_ms=12 size=5459 referer=\"http://localhost:3000/d/zFTgAw7mk/fake-metrics-data?orgId=1&from=now-5m&to=now\"\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/api/dataprocessor.go:618 (0xa7554f)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/api/graphite.go:772 (0xa7d7f4)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/api/graphite.go:278 (0xa78705)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/api/routes.go:56 (0xa92284)\r\nmetrictank_1    | /usr/lib/go/src/runtime/asm_amd64.s:576 (0x457b34)\r\nmetrictank_1    | /usr/lib/go/src/reflect/value.go:447 (0x4b5008)\r\nmetrictank_1    | /usr/lib/go/src/reflect/value.go:308 (0x4b4583)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/vendor/github.com/go-macaron/inject/inject.go:177 (0x9d75b4)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/vendor/github.com/go-macaron/inject/inject.go:137 (0x9d6f39)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/vendor/gopkg.in/macaron.v1/context.go:113 (0x9f225d)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/vendor/gopkg.in/macaron.v1/context.go:104 (0x9f219e)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/api/middleware/tracer.go:64 (0xa0e99a)\r\nmetrictank_1    | /usr/lib/go/src/runtime/asm_amd64.s:573 (0x45796a)\r\nmetrictank_1    | /usr/lib/go/src/reflect/value.go:447 (0x4b5008)\r\nmetrictank_1    | /usr/lib/go/src/reflect/value.go:308 (0x4b4583)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/vendor/github.com/go-macaron/inject/inject.go:177 (0x9d75b4)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/vendor/github.com/go-macaron/inject/inject.go:137 (0x9d6f39)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/vendor/gopkg.in/macaron.v1/context.go:113 (0x9f225d)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/vendor/gopkg.in/macaron.v1/context.go:104 (0x9f219e)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/api/middleware/stats.go:72 (0xa0e1d8)\r\nmetrictank_1    | /usr/lib/go/src/runtime/asm_amd64.s:573 (0x45796a)\r\nmetrictank_1    | /usr/lib/go/src/reflect/value.go:447 (0x4b5008)\r\nmetrictank_1    | /usr/lib/go/src/reflect/value.go:308 (0x4b4583)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/vendor/github.com/go-macaron/inject/inject.go:177 (0x9d75b4)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/vendor/github.com/go-macaron/inject/inject.go:137 (0x9d6f39)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/vendor/gopkg.in/macaron.v1/context.go:113 (0x9f225d)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/vendor/gopkg.in/macaron.v1/context.go:104 (0x9f219e)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/vendor/github.com/raintank/gziper/gzip.go:73 (0xa69bb7)\r\nmetrictank_1    | /usr/lib/go/src/runtime/asm_amd64.s:573 (0x45796a)\r\nmetrictank_1    | /usr/lib/go/src/reflect/value.go:447 (0x4b5008)\r\nmetrictank_1    | /usr/lib/go/src/reflect/value.go:308 (0x4b4583)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/vendor/github.com/go-macaron/inject/inject.go:177 (0x9d75b4)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/vendor/github.com/go-macaron/inject/inject.go:137 (0x9d6f39)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/vendor/gopkg.in/macaron.v1/context.go:113 (0x9f225d)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/vendor/gopkg.in/macaron.v1/context.go:104 (0x9f219e)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/vendor/gopkg.in/macaron.v1/recovery.go:161 (0xa0218a)\r\nmetrictank_1    | /usr/lib/go/src/runtime/asm_amd64.s:573 (0x45796a)\r\nmetrictank_1    | /usr/lib/go/src/reflect/value.go:447 (0x4b5008)\r\nmetrictank_1    | /usr/lib/go/src/reflect/value.go:308 (0x4b4583)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/vendor/github.com/go-macaron/inject/inject.go:177 (0x9d75b4)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/vendor/github.com/go-macaron/inject/inject.go:137 (0x9d6f39)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/vendor/gopkg.in/macaron.v1/context.go:113 (0x9f225d)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/vendor/gopkg.in/macaron.v1/context.go:104 (0x9f219e)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/vendor/gopkg.in/macaron.v1/logger.go:43 (0xa014d0)\r\nmetrictank_1    | /usr/lib/go/src/runtime/asm_amd64.s:573 (0x45796a)\r\nmetrictank_1    | /usr/lib/go/src/reflect/value.go:447 (0x4b5008)\r\nmetrictank_1    | /usr/lib/go/src/reflect/value.go:308 (0x4b4583)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/vendor/github.com/go-macaron/inject/inject.go:177 (0x9d75b4)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/vendor/github.com/go-macaron/inject/inject.go:137 (0x9d6f39)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/vendor/gopkg.in/macaron.v1/context.go:113 (0x9f225d)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/vendor/gopkg.in/macaron.v1/router.go:184 (0xa03331)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/vendor/gopkg.in/macaron.v1/router.go:286 (0x9fce1b)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/vendor/gopkg.in/macaron.v1/macaron.go:177 (0x9f6074)\r\nmetrictank_1    | /usr/lib/go/src/net/http/server.go:2694 (0x6d67cb)\r\nmetrictank_1    | /usr/lib/go/src/net/http/server.go:1830 (0x6d2ad0)\r\nmetrictank_1    | /usr/lib/go/src/runtime/asm_amd64.s:2361 (0x459fb0)\r\nmetrictank_1    | [Macaron] 2018-05-17 10:23:12: Completed /render 500 Internal Server Error in 3.655613ms\r\n```"},"comment":{"url":"https://api.github.com/repos/grafana/metrictank/issues/comments/389823986","html_url":"https://github.com/grafana/metrictank/issues/913#issuecomment-389823986","issue_url":"https://api.github.com/repos/grafana/metrictank/issues/913","id":389823986,"user":{"login":"Dieterbe","id":20774,"avatar_url":"https://avatars2.githubusercontent.com/u/20774?v=4","gravatar_id":"","url":"https://api.github.com/users/Dieterbe","html_url":"https://github.com/Dieterbe","followers_url":"https://api.github.com/users/Dieterbe/followers","following_url":"https://api.github.com/users/Dieterbe/following{/other_user}","gists_url":"https://api.github.com/users/Dieterbe/gists{/gist_id}","starred_url":"https://api.github.com/users/Dieterbe/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Dieterbe/subscriptions","organizations_url":"https://api.github.com/users/Dieterbe/orgs","repos_url":"https://api.github.com/users/Dieterbe/repos","events_url":"https://api.github.com/users/Dieterbe/events{/privacy}","received_events_url":"https://api.github.com/users/Dieterbe/received_events","type":"User","site_admin":false},"created_at":"2018-05-17T10:41:30Z","updated_at":"2018-05-17T10:41:30Z","author_association":"CONTRIBUTOR","body":"I was also able to trigger the similar error by querying a long time range, then I got this:\r\n```\r\nmetrictank_1    | [Macaron] 2018-05-17 10:39:57: Started POST /render for 172.19.0.1, 172.19.0.1, 172.19.0.2\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526551449 -> 1526551441\r\nmetrictank_1    | to   1526552947 -> 1526552881\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526551449 -> 1526551441\r\nmetrictank_1    | to   1526552947 -> 1526552881\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526551449 -> 1526551441\r\nmetrictank_1    | to   1526552947 -> 1526552941\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526551449 -> 1526551441\r\nmetrictank_1    | to   1526552947 -> 1526552881\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526551449 -> 1526551441\r\nmetrictank_1    | to   1526552947 -> 1526552881\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526551449 -> 1526551441\r\nmetrictank_1    | to   1526552947 -> 1526552881\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526551449 -> 1526551441\r\nmetrictank_1    | to   1526552947 -> 1526552881\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526551449 -> 1526551441\r\nmetrictank_1    | to   1526552947 -> 1526552881\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526551449 -> 1526551441\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526551449 -> 1526551441\r\nmetrictank_1    | to   1526552947 -> 1526552881\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526551449 -> 1526551441\r\nmetrictank_1    | to   1526552947 -> 1526552881\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526551449 -> 1526551441\r\nmetrictank_1    | to   1526552947 -> 1526552881\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526551449 -> 1526551441\r\nmetrictank_1    | to   1526552947 -> 1526552941\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526551449 -> 1526551441\r\nmetrictank_1    | to   1526552947 -> 1526552941\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526551449 -> 1526551441\r\nmetrictank_1    | to   1526552947 -> 1526552881\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526551449 -> 1526551441\r\nmetrictank_1    | to   1526552947 -> 1526552881\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526551449 -> 1526551441\r\nmetrictank_1    | to   1526552947 -> 1526552881\r\nmetrictank_1    | to   1526552947 -> 1526552941\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526551449 -> 1526551441\r\nmetrictank_1    | to   1526552947 -> 1526552881\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526551449 -> 1526551441\r\nmetrictank_1    | to   1526552947 -> 1526552941\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526551449 -> 1526551441\r\nmetrictank_1    | to   1526552947 -> 1526552941\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526551449 -> 1526551441\r\nmetrictank_1    | to   1526552947 -> 1526552941\r\nmetrictank_1    | from 1526551449 -> 1526551441\r\nmetrictank_1    | to   1526552947 -> 1526552941\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526551449 -> 1526551441\r\nmetrictank_1    | to   1526552947 -> 1526552881\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526551449 -> 1526551441\r\nmetrictank_1    | to   1526552947 -> 1526552881\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526551449 -> 1526551441\r\nmetrictank_1    | to   1526552947 -> 1526552881\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526551449 -> 1526551441\r\nmetrictank_1    | to   1526552947 -> 1526552881\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526551449 -> 1526551441\r\nmetrictank_1    | to   1526552947 -> 1526552941\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526551449 -> 1526551441\r\nmetrictank_1    | to   1526552947 -> 1526552881\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526551449 -> 1526551441\r\nmetrictank_1    | to   1526552947 -> 1526552881\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526551449 -> 1526551441\r\nmetrictank_1    | to   1526552947 -> 1526552941\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526551449 -> 1526551441\r\nmetrictank_1    | to   1526552947 -> 1526552881\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526551449 -> 1526551441\r\nmetrictank_1    | to   1526552947 -> 1526552881\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526551449 -> 1526551441\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526551449 -> 1526551441\r\nmetrictank_1    | to   1526552947 -> 1526552881\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526551449 -> 1526551441\r\nmetrictank_1    | to   1526552947 -> 1526552881\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526551449 -> 1526551441\r\nmetrictank_1    | to   1526552947 -> 1526552881\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526551449 -> 1526551441\r\nmetrictank_1    | to   1526552947 -> 1526552881\r\nmetrictank_1    | to   1526552947 -> 1526552881\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526551449 -> 1526551441\r\nmetrictank_1    | to   1526552947 -> 1526552881\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526551449 -> 1526551441\r\nmetrictank_1    | to   1526552947 -> 1526552881\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526551449 -> 1526551441\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | Consol is NoneConsolidator\r\ngrafana_1       | t=2018-05-17T10:39:57+0000 lvl=eror msg=\"Request Completed\" logger=context userId=1 orgId=1 uname=admin method=POST path=/api/datasources/proxy/3/render status=500 remote_addr=172.19.0.1 time_ms=10 size=5459 referer=\"http://localhost:3000/d/zFTgAw7mk/fake-metrics-data?orgId=1&from=1526551508908&to=1526552886281&panelId=1&fullscreen&edit\"\r\nmetrictank_1    | to   1526552947 -> 1526552881\r\nmetrictank_1    | Consol is NoneConsolidator\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526551449 -> 1526551441\r\nmetrictank_1    | to   1526552947 -> 1526552881\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526551449 -> 1526551441\r\nmetrictank_1    | to   1526552947 -> 1526552941\r\nmetrictank_1    | consol None so:\r\nmetrictank_1    | from 1526551449 -> 1526551441\r\nmetrictank_1    | to   1526552947 -> 1526552881\r\nmetrictank_1    | ([]models.Series) (len=2 cap=2) {\r\nmetrictank_1    |  (models.Series) {\r\nmetrictank_1    |   Target: (string) (len=37) \"some.id.of.a.metric.11;id=11;some=tag\",\r\nmetrictank_1    |   Datapoints: ([]schema.Point) (len=13 cap=50) {\r\nmetrictank_1    |    (schema.Point) {\r\nmetrictank_1    |     Val: (float64) 6.78719139816364,\r\nmetrictank_1    |     Ts: (uint32) 1526551560\r\nmetrictank_1    |    },\r\nmetrictank_1    |    (schema.Point) {\r\nmetrictank_1    |     Val: (float64) 6.65660304638063,\r\nmetrictank_1    |     Ts: (uint32) 1526551680\r\nmetrictank_1    |    },\r\nmetrictank_1    |    (schema.Point) {\r\nmetrictank_1    |     Val: (float64) NaN,\r\nmetrictank_1    |     Ts: (uint32) 1526551800\r\nmetrictank_1    |    },\r\nmetrictank_1    |    (schema.Point) {\r\nmetrictank_1    |     Val: (float64) NaN,\r\nmetrictank_1    |     Ts: (uint32) 1526551920\r\nmetrictank_1    |    },\r\nmetrictank_1    |    (schema.Point) {\r\nmetrictank_1    |     Val: (float64) NaN,\r\nmetrictank_1    |     Ts: (uint32) 1526552040\r\nmetrictank_1    |    },\r\nmetrictank_1    |    (schema.Point) {\r\nmetrictank_1    |     Val: (float64) NaN,\r\nmetrictank_1    |     Ts: (uint32) 1526552160\r\nmetrictank_1    |    },\r\nmetrictank_1    |    (schema.Point) {\r\nmetrictank_1    |     Val: (float64) NaN,\r\nmetrictank_1    |     Ts: (uint32) 1526552280\r\nmetrictank_1    |    },\r\nmetrictank_1    |    (schema.Point) {\r\nmetrictank_1    |     Val: (float64) 3.3391286938078175,\r\nmetrictank_1    |     Ts: (uint32) 1526552400\r\nmetrictank_1    |    },\r\nmetrictank_1    |    (schema.Point) {\r\nmetrictank_1    |     Val: (float64) 5.66733891352272,\r\nmetrictank_1    |     Ts: (uint32) 1526552520\r\nmetrictank_1    |    },\r\nmetrictank_1    |    (schema.Point) {\r\nmetrictank_1    |     Val: (float64) 7.632219501689304,\r\nmetrictank_1    |     Ts: (uint32) 1526552640\r\nmetrictank_1    |    },\r\nmetrictank_1    |    (schema.Point) {\r\nmetrictank_1    |     Val: (float64) 0.7563181472237022,\r\nmetrictank_1    |     Ts: (uint32) 1526552760\r\nmetrictank_1    |    },\r\nmetrictank_1    |    (schema.Point) {\r\nmetrictank_1    |     Val: (float64) 2.8292831316353944,\r\nmetrictank_1    |     Ts: (uint32) 1526552880\r\nmetrictank_1    |    },\r\nmetrictank_1    |    (schema.Point) {\r\nmetrictank_1    |     Val: (float64) NaN,\r\nmetrictank_1    |     Ts: (uint32) 1526553000\r\nmetrictank_1    |    }\r\nmetrictank_1    |   },\r\nmetrictank_1    |   Tags: (map[string]string) <nil>,\r\nmetrictank_1    |   Interval: (uint32) 120,\r\nmetrictank_1    |   QueryPatt: (string) (len=22) \"some.id.of.a.metric.1*\",\r\nmetrictank_1    |   QueryFrom: (uint32) 1526551449,\r\nmetrictank_1    |   QueryTo: (uint32) 1526552947,\r\nmetrictank_1    |   QueryCons: (consolidation.Consolidator) NoneConsolidator,\r\nmetrictank_1    |   Consolidator: (consolidation.Consolidator) AverageConsolidator\r\nmetrictank_1    |  },\r\nmetrictank_1    |  (models.Series) {\r\nmetrictank_1    |   Target: (string) (len=37) \"some.id.of.a.metric.11;id=11;some=tag\",\r\nmetrictank_1    |   Datapoints: ([]schema.Point) (len=12 cap=12) {\r\nmetrictank_1    |    (schema.Point) {\r\nmetrictank_1    |     Val: (float64) NaN,\r\nmetrictank_1    |     Ts: (uint32) 1526551560\r\nmetrictank_1    |    },\r\nmetrictank_1    |    (schema.Point) {\r\nmetrictank_1    |     Val: (float64) NaN,\r\nmetrictank_1    |     Ts: (uint32) 1526551680\r\nmetrictank_1    |    },\r\nmetrictank_1    |    (schema.Point) {\r\nmetrictank_1    |     Val: (float64) NaN,\r\nmetrictank_1    |     Ts: (uint32) 1526551800\r\nmetrictank_1    |    },\r\nmetrictank_1    |    (schema.Point) {\r\nmetrictank_1    |     Val: (float64) NaN,\r\nmetrictank_1    |     Ts: (uint32) 1526551920\r\nmetrictank_1    |    },\r\nmetrictank_1    |    (schema.Point) {\r\nmetrictank_1    |     Val: (float64) NaN,\r\nmetrictank_1    |     Ts: (uint32) 1526552040\r\nmetrictank_1    |    },\r\nmetrictank_1    |    (schema.Point) {\r\nmetrictank_1    |     Val: (float64) NaN,\r\nmetrictank_1    |     Ts: (uint32) 1526552160\r\nmetrictank_1    |    },\r\nmetrictank_1    |    (schema.Point) {\r\nmetrictank_1    |     Val: (float64) NaN,\r\nmetrictank_1    |     Ts: (uint32) 1526552280\r\nmetrictank_1    |    },\r\nmetrictank_1    |    (schema.Point) {\r\nmetrictank_1    |     Val: (float64) NaN,\r\nmetrictank_1    |     Ts: (uint32) 1526552400\r\nmetrictank_1    |    },\r\nmetrictank_1    |    (schema.Point) {\r\nmetrictank_1    |     Val: (float64) 5.66733891352272,\r\nmetrictank_1    |     Ts: (uint32) 1526552520\r\nmetrictank_1    |    },\r\nmetrictank_1    |    (schema.Point) {\r\nmetrictank_1    |     Val: (float64) 7.632219501689304,\r\nmetrictank_1    |     Ts: (uint32) 1526552640\r\nmetrictank_1    |    },\r\nmetrictank_1    |    (schema.Point) {\r\nmetrictank_1    |     Val: (float64) 0.7563181472237022,\r\nmetrictank_1    |     Ts: (uint32) 1526552760\r\nmetrictank_1    |    },\r\nmetrictank_1    |    (schema.Point) {\r\nmetrictank_1    |     Val: (float64) 2.8292831316353944,\r\nmetrictank_1    |     Ts: (uint32) 1526552880\r\nmetrictank_1    |    }\r\nmetrictank_1    |   },\r\nmetrictank_1    |   Tags: (map[string]string) <nil>,\r\nmetrictank_1    |   Interval: (uint32) 120,\r\nmetrictank_1    |   QueryPatt: (string) (len=22) \"some.id.of.a.metric.1*\",\r\nmetrictank_1    |   QueryFrom: (uint32) 1526551449,\r\nmetrictank_1    |   QueryTo: (uint32) 1526552947,\r\nmetrictank_1    |   QueryCons: (consolidation.Consolidator) NoneConsolidator,\r\nmetrictank_1    |   Consolidator: (consolidation.Consolidator) AverageConsolidator\r\nmetrictank_1    |  }\r\nmetrictank_1    | }\r\nmetrictank_1    | [Macaron] PANIC: uneven length detected\r\nmetrictank_1    | /usr/lib/go/src/runtime/panic.go:502 (0x42a7a8)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/api/dataprocessor.go:618 (0xa7554f)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/api/graphite.go:772 (0xa7d7f4)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/api/graphite.go:278 (0xa78705)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/api/routes.go:56 (0xa92284)\r\nmetrictank_1    | /usr/lib/go/src/runtime/asm_amd64.s:576 (0x457b34)\r\nmetrictank_1    | /usr/lib/go/src/reflect/value.go:447 (0x4b5008)\r\nmetrictank_1    | /usr/lib/go/src/reflect/value.go:308 (0x4b4583)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/vendor/github.com/go-macaron/inject/inject.go:177 (0x9d75b4)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/vendor/github.com/go-macaron/inject/inject.go:137 (0x9d6f39)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/vendor/gopkg.in/macaron.v1/context.go:113 (0x9f225d)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/vendor/gopkg.in/macaron.v1/context.go:104 (0x9f219e)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/api/middleware/tracer.go:64 (0xa0e99a)\r\nmetrictank_1    | /usr/lib/go/src/runtime/asm_amd64.s:573 (0x45796a)\r\nmetrictank_1    | /usr/lib/go/src/reflect/value.go:447 (0x4b5008)\r\nmetrictank_1    | /usr/lib/go/src/reflect/value.go:308 (0x4b4583)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/vendor/github.com/go-macaron/inject/inject.go:177 (0x9d75b4)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/vendor/github.com/go-macaron/inject/inject.go:137 (0x9d6f39)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/vendor/gopkg.in/macaron.v1/context.go:113 (0x9f225d)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/vendor/gopkg.in/macaron.v1/context.go:104 (0x9f219e)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/api/middleware/stats.go:72 (0xa0e1d8)\r\nmetrictank_1    | /usr/lib/go/src/runtime/asm_amd64.s:573 (0x45796a)\r\nmetrictank_1    | /usr/lib/go/src/reflect/value.go:447 (0x4b5008)\r\nmetrictank_1    | /usr/lib/go/src/reflect/value.go:308 (0x4b4583)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/vendor/github.com/go-macaron/inject/inject.go:177 (0x9d75b4)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/vendor/github.com/go-macaron/inject/inject.go:137 (0x9d6f39)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/vendor/gopkg.in/macaron.v1/context.go:113 (0x9f225d)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/vendor/gopkg.in/macaron.v1/context.go:104 (0x9f219e)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/vendor/github.com/raintank/gziper/gzip.go:73 (0xa69bb7)\r\nmetrictank_1    | /usr/lib/go/src/runtime/asm_amd64.s:573 (0x45796a)\r\nmetrictank_1    | /usr/lib/go/src/reflect/value.go:447 (0x4b5008)\r\nmetrictank_1    | /usr/lib/go/src/reflect/value.go:308 (0x4b4583)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/vendor/github.com/go-macaron/inject/inject.go:177 (0x9d75b4)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/vendor/github.com/go-macaron/inject/inject.go:137 (0x9d6f39)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/vendor/gopkg.in/macaron.v1/context.go:113 (0x9f225d)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/vendor/gopkg.in/macaron.v1/context.go:104 (0x9f219e)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/vendor/gopkg.in/macaron.v1/recovery.go:161 (0xa0218a)\r\nmetrictank_1    | /usr/lib/go/src/runtime/asm_amd64.s:573 (0x45796a)\r\nmetrictank_1    | /usr/lib/go/src/reflect/value.go:447 (0x4b5008)\r\nmetrictank_1    | /usr/lib/go/src/reflect/value.go:308 (0x4b4583)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/vendor/github.com/go-macaron/inject/inject.go:177 (0x9d75b4)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/vendor/github.com/go-macaron/inject/inject.go:137 (0x9d6f39)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/vendor/gopkg.in/macaron.v1/context.go:113 (0x9f225d)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/vendor/gopkg.in/macaron.v1/context.go:104 (0x9f219e)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/vendor/gopkg.in/macaron.v1/logger.go:43 (0xa014d0)\r\nmetrictank_1    | /usr/lib/go/src/runtime/asm_amd64.s:573 (0x45796a)\r\nmetrictank_1    | /usr/lib/go/src/reflect/value.go:447 (0x4b5008)\r\nmetrictank_1    | /usr/lib/go/src/reflect/value.go:308 (0x4b4583)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/vendor/github.com/go-macaron/inject/inject.go:177 (0x9d75b4)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/vendor/github.com/go-macaron/inject/inject.go:137 (0x9d6f39)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/vendor/gopkg.in/macaron.v1/context.go:113 (0x9f225d)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/vendor/gopkg.in/macaron.v1/router.go:184 (0xa03331)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/vendor/gopkg.in/macaron.v1/router.go:286 (0x9fce1b)\r\nmetrictank_1    | /home/dieter/go/src/github.com/grafana/metrictank/vendor/gopkg.in/macaron.v1/macaron.go:177 (0x9f6074)\r\nmetrictank_1    | /usr/lib/go/src/net/http/server.go:2694 (0x6d67cb)\r\nmetrictank_1    | /usr/lib/go/src/net/http/server.go:1830 (0x6d2ad0)\r\nmetrictank_1    | /usr/lib/go/src/runtime/asm_amd64.s:2361 (0x459fb0)\r\nmetrictank_1    | [Macaron] 2018-05-17 10:39:57: Completed /render 500 Internal Server Error in 9.183073ms\r\n```"}}
{"actor":{"display_login":"Dieterbe"}}