{"action":"created","issue":{"url":"https://api.github.com/repos/dantezhao/data-group/issues/30","repository_url":"https://api.github.com/repos/dantezhao/data-group","labels_url":"https://api.github.com/repos/dantezhao/data-group/issues/30/labels{/name}","comments_url":"https://api.github.com/repos/dantezhao/data-group/issues/30/comments","events_url":"https://api.github.com/repos/dantezhao/data-group/issues/30/events","html_url":"https://github.com/dantezhao/data-group/issues/30","id":310294280,"number":30,"title":"【问题】 app数据采集可能会存在数据上报延时，倾向于采用数据的生成的时间还是服务器收到的时间？","user":{"login":"detectivelcy","id":26423413,"avatar_url":"https://avatars1.githubusercontent.com/u/26423413?v=4","gravatar_id":"","url":"https://api.github.com/users/detectivelcy","html_url":"https://github.com/detectivelcy","followers_url":"https://api.github.com/users/detectivelcy/followers","following_url":"https://api.github.com/users/detectivelcy/following{/other_user}","gists_url":"https://api.github.com/users/detectivelcy/gists{/gist_id}","starred_url":"https://api.github.com/users/detectivelcy/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/detectivelcy/subscriptions","organizations_url":"https://api.github.com/users/detectivelcy/orgs","repos_url":"https://api.github.com/users/detectivelcy/repos","events_url":"https://api.github.com/users/detectivelcy/events{/privacy}","received_events_url":"https://api.github.com/users/detectivelcy/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2018-04-01T04:52:24Z","updated_at":"2018-04-11T11:48:54Z","closed_at":null,"author_association":"NONE","body":"【场景】\r\n行业原因，车开进地库了，就没有4G信号了，然后车主可能一周后再出车，此时数据才上来。导致app数据采集可能会存在数据上报延，数据会存在两个时间，数据生成的时间、服务器收到的时间。我们在建立ODS的时候，表的分区一般会采用天来做分区。\r\n【目前想到的解决方案】\r\n无论按服务器时间还是数据生成时间，因为dw是定期执行的，数据延迟到达的话无论如何都无法避免。此时只能针对特定业务来处理，和业务确认一个合理的最大延迟时间。超过最大延迟的丢弃。合理时间内的dw按数据生成时间来做，然后某些在dw上汇总统计操作再定期重跑。"},"comment":{"url":"https://api.github.com/repos/dantezhao/data-group/issues/comments/380424108","html_url":"https://github.com/dantezhao/data-group/issues/30#issuecomment-380424108","issue_url":"https://api.github.com/repos/dantezhao/data-group/issues/30","id":380424108,"user":{"login":"qinglww","id":38284293,"avatar_url":"https://avatars3.githubusercontent.com/u/38284293?v=4","gravatar_id":"","url":"https://api.github.com/users/qinglww","html_url":"https://github.com/qinglww","followers_url":"https://api.github.com/users/qinglww/followers","following_url":"https://api.github.com/users/qinglww/following{/other_user}","gists_url":"https://api.github.com/users/qinglww/gists{/gist_id}","starred_url":"https://api.github.com/users/qinglww/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/qinglww/subscriptions","organizations_url":"https://api.github.com/users/qinglww/orgs","repos_url":"https://api.github.com/users/qinglww/repos","events_url":"https://api.github.com/users/qinglww/events{/privacy}","received_events_url":"https://api.github.com/users/qinglww/received_events","type":"User","site_admin":false},"created_at":"2018-04-11T11:48:54Z","updated_at":"2018-04-11T11:48:54Z","author_association":"NONE","body":"按服务器收到时间分区，按数据生成时间这个维度来进行统计，不冲突；就是如果不是每天统计全量的话，历史统计得重新计算。"}}
{"actor":{"display_login":"qinglww"}}