{"push_id":3011729117,"size":11,"distinct_size":1,"ref":"refs/heads/CARBONDATA-3038_RefactorConfiguration","head":"db377ce7c568b3d0ba9ea9d31239364842b7ae3c","before":"d7dcd08ba96c634bdce7b1fcc559f340a74f84f3","commits":[{"sha":"10b393808e91344b017ba3e946b28217c2dd9757","author":{"name":"Indhumathi27","email":"64e95f1c0d1c3e5c2c9f9570b887e7e0a1ee8ec3@gmail.com"},"message":"[CARBONDATA-3042] Column Schema objects are present in Driver and Executor even after dropping table\n\nProblem:\nColumn Schema objects are present in Driver and Executor even after dropping table.\n\nSolution:\nIn Driver: After dropping table, remove entry of tableInfo from CarbonMetaDataInstance.\nIn Executor: Remove usage of CarbonMetaDataInstance object and instead pass CarbonTable Object itself\n\nThis closes #2852","distinct":false,"url":"https://api.github.com/repos/xubo245/carbondata/commits/10b393808e91344b017ba3e946b28217c2dd9757"},{"sha":"d27f20523161e342f44bb1bfaea7772de784c57d","author":{"name":"Zhang Zhichao","email":"ceea95473ad9845f773ec09d24a301e65b415f55@qq.com"},"message":"[HOTFIX] Remove unuse javax.servlet jar from cli module\n\nRemove unuse javax.servlet-2.5 jar from cli module\n\nThis closes #2881","distinct":false,"url":"https://api.github.com/repos/xubo245/carbondata/commits/d27f20523161e342f44bb1bfaea7772de784c57d"},{"sha":"3e0693b9e9cecf5660ec8669f63f178fde15deab","author":{"name":"xubo245","email":"b6d8a9432a54bd02c734f964202960918b2bb0bf@huawei.com"},"message":"[CARBONDATA-3000] Provide C++ interface for writing carbon data in CSDK\n\n1.suport string, short, int, long, double, float, array<string>, boolean data type\n2.provide builder, build, write, close interface\n3.support write data to S3\n4. add log for SDK and CSDK\n\nThis closes #2837","distinct":false,"url":"https://api.github.com/repos/xubo245/carbondata/commits/3e0693b9e9cecf5660ec8669f63f178fde15deab"},{"sha":"43ffe79f3574a6cef926987045d4e8759f8f8b95","author":{"name":"Manhua","email":"9342ca58f48b7865f769c68504b4d3060a00abf5@qq.com"},"message":"[CARBONDATA-3058] Fix some exception coding in data loading\n\nThis closes #2879","distinct":false,"url":"https://api.github.com/repos/xubo245/carbondata/commits/43ffe79f3574a6cef926987045d4e8759f8f8b95"},{"sha":"0e39abf81b0c7279abbecb9a4a387e447054a29a","author":{"name":"xuchuanyin","email":"ea7e5e16c33472e7ca342c18fae68881a3d50cda@hust.edu.cn"},"message":"[CARBONDATA-3051] Fix bugs in unclosed streams\n\nFailed to run CarbonReaderTests in windows env. This is due to two\nproblems in carbondata:\n\nProblem 1: While encountering problems, we should close all the streams instead of\nonly close current one.\n\nProblem 2: Current FileReader is not singleton nor the fileCache\nvariable is a class variable, so it can only provide cache function if\nwe pass the object among the methods. After reading version info, the\nfileReader cannot be used by other methods, so we must close it.\n\nThis closes #2871","distinct":false,"url":"https://api.github.com/repos/xubo245/carbondata/commits/0e39abf81b0c7279abbecb9a4a387e447054a29a"},{"sha":"bcf3e0fd595f612ee33a8ee2d9aa6197998f626e","author":{"name":"ajantha-bhat","email":"283c3d84fb3be3fbb2c689671922b0c4ff69adf9@gmail.com"},"message":"[CARBONDATA-3054] Fix Dictionary file cannot be read in S3a with CarbonDictionaryDecoder.doConsume() codeGen\n\nproblem: In S3a environment, when queried the data which has dictionary\nfiles, Dictionary file cannot be read in S3a with CarbonDictionaryDecoder.doConsume() codeGen even though file is present.\n\ncause: CarbonDictionaryDecoder.doConsume() codeGen doesn't set hadoop conf in thread local variable, only doExecute() sets it.\nHence, when getDictionaryWrapper() called from doConsume() codeGen, AbstractDictionaryCache.getDictionaryMetaCarbonFile() returns false for fileExists() operation.\n\nsolution:\nIn CarbonDictionaryDecoder.doConsume() codeGen, set hadoop conf in thread local variable\n\nThis closes #2876","distinct":false,"url":"https://api.github.com/repos/xubo245/carbondata/commits/bcf3e0fd595f612ee33a8ee2d9aa6197998f626e"},{"sha":"94a4f8314068ffd4c0743907752b58879578749b","author":{"name":"ajantha-bhat","email":"283c3d84fb3be3fbb2c689671922b0c4ff69adf9@gmail.com"},"message":"[HOTFIX-compatibility] Handle Lazy loading with inverted index for ColumnarVectorWrapperDirectWithInvertedIndex\n\nProblem:\nCreate a store with 1.4 code with inverted index and read it with vector filling (latest master code).\nbelow exception will be thrown from AbstractCarbonColumnarVector.\nUnsupportedOperationException(\"Not allowed from here \" + getClass().getName());\n\ncause:\nwhen the lazy loading with an inverted index,\ngetBlockDataType() was not implemented for ColumnarVectorWrapperDirectWithInvertedIndex.\n\nSo, Added implementation.\n\nThis closes #2870","distinct":false,"url":"https://api.github.com/repos/xubo245/carbondata/commits/94a4f8314068ffd4c0743907752b58879578749b"},{"sha":"6258447849f0fb0934b6a0e728d99d62223a2854","author":{"name":"akashrn5","email":"e895d8ea0ba28b59e83fc930aa49fb689668ab5c@gmail.com"},"message":"[CARBONDATA-3025]handle passing spark appname for partition table and file format\n\nChanges in this PR\n\n1.Dataload with partion table file format fails, as the appname is not in carbonproperties in executor.\n2.This PR sets the spark appname in carbon properties which will be written to carbondata footer.\n3.the appname is set in hadoop conf and then set in carbonproperties in executor from getting the same from the conf\ninstead of hardcoding the spark property, get from exposed API appName\n\nThis closes #2861","distinct":false,"url":"https://api.github.com/repos/xubo245/carbondata/commits/6258447849f0fb0934b6a0e728d99d62223a2854"},{"sha":"82eec10cb482e88df914657240a42a8dff8b1c51","author":{"name":"manishgupta88","email":"4820017a5fb7db80256b43a7ddce74c044934b5e@gmail.com"},"message":"[CARBONDATA-3052] Improve drop table performance by reducing the namenode RPC calls during physical deletion of files\n\nProblem\nCurrent drop table command takes more than 1 minute to delete 3000 files during drop table operation from HDFS\n\nAnalysis\nEven though we are using HDFS file system we are explicitly we are recursively iterating through the table folders and deleting each file. For each file deletion and file listing one rpc call is made to namenode. To delete 3000 files 3000 rpc calls are made to namenode for file deletion and few more rpc calls for file listing in each folder.\n\nSolution\nHDFS provides an API for deleting all folders and files recursively for a given path in a single RPC call. Use that API and improve the drop table operation performance.\n\nResult: After these code changes drop table operation time to delete 3000 files from HDFS has reduced from 1 minute to ~2 sec.\n\nThis closes #2868","distinct":false,"url":"https://api.github.com/repos/xubo245/carbondata/commits/82eec10cb482e88df914657240a42a8dff8b1c51"},{"sha":"26d1cdd9cac47cf4caa41611413fe5d18e896d17","author":{"name":"xuchuanyin","email":"ea7e5e16c33472e7ca342c18fae68881a3d50cda@hust.edu.cn"},"message":"[CARBONDATA-3053][Cli] Fix bugs for carbon-cli found in windows env\n\nTests run failed in windows env due to:\n\n1.unclosed file stream causing failure in clearing files in windows\n2.the line separator is not n in windows\nIn this commit, We fix the bugs.\n\nThis closes #2874","distinct":false,"url":"https://api.github.com/repos/xubo245/carbondata/commits/26d1cdd9cac47cf4caa41611413fe5d18e896d17"},{"sha":"db377ce7c568b3d0ba9ea9d31239364842b7ae3c","author":{"name":"xubo245","email":"b6d8a9432a54bd02c734f964202960918b2bb0bf@huawei.com"},"message":" [CARBONDATA-3038] Refactor dynamic configuration\n    1.add dynamicConfigurable for CarbonProperty annotation\n    2.optimize the properties: remove some warning","distinct":true,"url":"https://api.github.com/repos/xubo245/carbondata/commits/db377ce7c568b3d0ba9ea9d31239364842b7ae3c"}]}
{"actor":{"display_login":"xubo245"}}