{"push_id":1174995471,"size":22,"distinct_size":22,"ref":"refs/heads/fkey_triggers","head":"4dd31c29953d046d376a638c428ce87973cfb15e","before":"0f082ed04636cf0901c9329c36e77f84fc2c007e","commits":[{"sha":"753a53268484b648b8c5cfffb188bbd7aad7505f","author":{"name":"Adam Lee","email":"6bef8114f791610f325e21bf98cab17795dfd86b@pivotal.io"},"message":"s3ext: refactor s3wrapper and s3downloader codes\n\nrefactor, optimize and rename them to gpreader and s3reader.","distinct":true,"url":"https://api.github.com/repos/jimmyyih/gpdb/commits/753a53268484b648b8c5cfffb188bbd7aad7505f"},{"sha":"122a8dfa2556e7b7bf45f705173158f49398e421","author":{"name":"Adam Lee","email":"6bef8114f791610f325e21bf98cab17795dfd86b@pivotal.io"},"message":"s3ext: add a bucket reader class\n\nAlso introducing google mock for unit tests.","distinct":true,"url":"https://api.github.com/repos/jimmyyih/gpdb/commits/122a8dfa2556e7b7bf45f705173158f49398e421"},{"sha":"5b8a3003b1a6162f97d62532ce26dcf179d3007c","author":{"name":"Adam Lee","email":"6bef8114f791610f325e21bf98cab17795dfd86b@pivotal.io"},"message":"s3ext: add s3interface and s3restful classes\n\ns3interface is to handle all s3 restful api wrapper, like listBucket()\nand fetchData(), s3restful is to handle HTTP GET, and PUT in future.","distinct":true,"url":"https://api.github.com/repos/jimmyyih/gpdb/commits/5b8a3003b1a6162f97d62532ce26dcf179d3007c"},{"sha":"023134e42dd5483cce1af36a96e636412e9335b6","author":{"name":"Haozhou Wang","email":"3db33d1580d60d5cbe6e372f4f9bcfb5a940ee17@pivotal.io"},"message":"s3ext: refactor gzip downloader\n\n* change code style to avoid AllowShortFunctionsOnASingleLine\n\n* add a few cases for uncompression\n\n* gzip reader phase 2: read() with the buffer <= size of chuck\n\n* gzip reader phase 3: read() with the size of chuck < internal read buffer\n\n* gzip reader phase 4: read() with the buffer >> size of chuck\n\nSigned-off-by: Kuien Liu <kliu@pivotal.io>","distinct":true,"url":"https://api.github.com/repos/jimmyyih/gpdb/commits/023134e42dd5483cce1af36a96e636412e9335b6"},{"sha":"524dcd0fc23fb2aaadccacb64127a505526db5c0","author":{"name":"Adam Lee","email":"6bef8114f791610f325e21bf98cab17795dfd86b@pivotal.io"},"message":"s3ext: refactor s3key_reader codes\n\nSeparate key downloading and decompressing functions, make it more\nflexible and robust.\n\nSigned-off-by: Adam Lee <ali@pivotal.io>\nSigned-off-by: Peifeng Qiu <pqiu@pivotal.io>","distinct":true,"url":"https://api.github.com/repos/jimmyyih/gpdb/commits/524dcd0fc23fb2aaadccacb64127a505526db5c0"},{"sha":"a0b7b0290a7c4f3517338161fe77aceb482b61fb","author":{"name":"Adam Lee","email":"6bef8114f791610f325e21bf98cab17795dfd86b@pivotal.io"},"message":"s3ext: add a compression type detection function\n\nAlso retire DEBUG_S3_CURL, make it configurable.\n\nSigned-off-by: Adam Lee <ali@pivotal.io>\nSigned-off-by: Haozhou Wang <hawang@pivotal.io>","distinct":true,"url":"https://api.github.com/repos/jimmyyih/gpdb/commits/a0b7b0290a7c4f3517338161fe77aceb482b61fb"},{"sha":"a82f47ca52463f33ab0f5447542849a77745cb9c","author":{"name":"Peifeng Qiu","email":"d8721495590703eed183a852facfb23f79ea3f7b@pivotal.io"},"message":"s3ext: remove dead codes before refactoring gpreader\n\n1. add gpcommon.h to keep 'GpId' data structures;\n2. remove s3reader.h because s3key_reader is ready;\n3. move two pthread related functions to gpreader.h\n   because they are invoked by gpreader only.\n4. move XMLParserCallback and XMLInfo to s3uploader.cpp\n   because they are deprecated already for readers.\n\nSigned-off-by: Kuien Liu <kliu@pivotal.io>","distinct":true,"url":"https://api.github.com/repos/jimmyyih/gpdb/commits/a82f47ca52463f33ab0f5447542849a77745cb9c"},{"sha":"74cb7d1110a8c441bfd7b2d35b5e036649eeefd9","author":{"name":"Adam Lee","email":"6bef8114f791610f325e21bf98cab17795dfd86b@pivotal.io"},"message":"s3ext: enhance responses' error handling\n\nAlso fix s3conf test cases' issue exposed by this change.\n\nSigned-off-by: Adam Lee <ali@pivotal.io>\nSigned-off-by: Haozhou Wang <hawang@pivotal.io>","distinct":true,"url":"https://api.github.com/repos/jimmyyih/gpdb/commits/74cb7d1110a8c441bfd7b2d35b5e036649eeefd9"},{"sha":"33aab67d8480d7f0fb1855db5569e2f353cc8beb","author":{"name":"Yandong Yao","email":"34842a647946a8f3a831061fc45255ca4c97b654@pivotal.io"},"message":"s3ext: support running a subset of unit test cases\n\nRun all cases:\n    $ make test\n\nRun a subset cases using pattern match, run all tests in S3KeyReaderTest suite:\n    $ make test gtest_filter=S3KeyReaderTest.*\n\nRun a subset cases using pattern match, run all MTRead tests:\n    $ make test gtest_filter=S3KeyReaderTest.MTReadWith*","distinct":true,"url":"https://api.github.com/repos/jimmyyih/gpdb/commits/33aab67d8480d7f0fb1855db5569e2f353cc8beb"},{"sha":"3d134b341352e77aeb6d48dfddeed794532e95fd","author":{"name":"Peifeng Qiu","email":"d8721495590703eed183a852facfb23f79ea3f7b@pivotal.io"},"message":"s3ext: fix a race condition and a exception bug\n\n== race condition\n\nWithout this fix, ChunkBuffer::read() might be waiting for the signal\nfrom ChunkBuffer::fill(), which has already been terminated in some\ncertain cases.\n\n== exception bug\n\nWhen a FAIL response from KeyReader is given, an exception\nwill be throw from S3Service::fetchData(), and this exception\nwill crash the downloader threads and process (core dumped).\n\nWe replace exception with error return, and it will be handled\nby the caller.\n\nSigned-off-by: Adam Lee <ali@pivotal.io>\nSigned-off-by: Kuien Liu <kliu@pivotal.io>\nSigned-off-by: Haozhou Wang <hawang@pivotal.io>","distinct":true,"url":"https://api.github.com/repos/jimmyyih/gpdb/commits/3d134b341352e77aeb6d48dfddeed794532e95fd"},{"sha":"16ea34ffe4b897e36e867b7389254d58056dac5e","author":{"name":"Adam Lee","email":"6bef8114f791610f325e21bf98cab17795dfd86b@pivotal.io"},"message":"s3ext: refactor Makefiles\n\nPut Makefile in each subdirectory, instead of using one Makefile at top\ndirectory to do all the things, and extract the common parts.\n\nSigned-off-by: Adam Lee <ali@pivotal.io>\nSigned-off-by: Haozhou Wang <hawang@pivotal.io>","distinct":true,"url":"https://api.github.com/repos/jimmyyih/gpdb/commits/16ea34ffe4b897e36e867b7389254d58056dac5e"},{"sha":"9cbc4d18e3e4631cf236a0c5f4c6faba05c02d00","author":{"name":"Peifeng Qiu","email":"d8721495590703eed183a852facfb23f79ea3f7b@pivotal.io"},"message":"s3ext: refactor gpreader\n\n1. write the GPReader class to connect all readers\n\n2. add unit tests on Open, Construct, Close, Read,\n   multi-thread Read, and failed response handling\n\n3. and a few cleanup\n  3.1. clean source codes in s3ext\n  3.2. refactor debug switch method\n  3.3. move mockRESTfulService to mock_classes.h\n  3.4. reset QueryCancelPending to FALSE after tests\n\nSigned-off-by: Kuien Liu <kliu@pivotal.io>","distinct":true,"url":"https://api.github.com/repos/jimmyyih/gpdb/commits/9cbc4d18e3e4631cf236a0c5f4c6faba05c02d00"},{"sha":"b6f57d4d54578dc1e6d315913d587f9dbe9c1e1b","author":{"name":"Peifeng Qiu","email":"d8721495590703eed183a852facfb23f79ea3f7b@pivotal.io"},"message":"s3ext: fix C++ 98 compliance issues\n\nThe clang in mac does not check the -std=c++98 flag strictly.\nHence when compiling on the other platform like redhat 7 (using gcc),\nthe source code may not be compiled correctly. The compiling errors have\nbeen fixed and the tests have been passed on redhat7 using c++98.\n\n1. std::vector<T>::shrink_to_fit is added in C++11.\n2. need to include <stdexcept> to use std::runtime_error.\n3. In C++98, if a class has reference member, then it can't be copy assigned by default,\n   we need to implement operator= explicitly.\n4. stoi is added in C++11.\n\nSigned-off-by: Haozhou Wang <hawang@pivotal.io>","distinct":true,"url":"https://api.github.com/repos/jimmyyih/gpdb/commits/b6f57d4d54578dc1e6d315913d587f9dbe9c1e1b"},{"sha":"87dac3b04f6a2efa73c103603d6c95ebcb34a486","author":{"name":"Adam Lee","email":"6bef8114f791610f325e21bf98cab17795dfd86b@pivotal.io"},"message":"s3ext: connect all parts and make the library work\n\nTwo major problems this commit fixes:\ncredential should be copied to keep the data.\nkeyReader must be reset before every new file downloading.\n\nSigned-off-by: Kuien Liu <kliu@pivotal.io>","distinct":true,"url":"https://api.github.com/repos/jimmyyih/gpdb/commits/87dac3b04f6a2efa73c103603d6c95ebcb34a486"},{"sha":"8fc342dc083a7e62e4c36cd80a38a9eb7e25658d","author":{"name":"Peifeng Qiu","email":"d8721495590703eed183a852facfb23f79ea3f7b@pivotal.io"},"message":"s3ext: link decompression reader into s3 module\n\n1. add s3common_reader to detect the format of file\n   if the file is compressed call decompressReader\n   otherwise call S3KeyReader directly.\n2. add checkCompressionType() into S3Interface.\n3. fix a bug if the file size is less than S3_MAGIC_BYTES_NUM\n   and add a unit test\n4. fix a bug if chunkSize is zero.\n5. add unit tests\n6. rename uncompress* to decompress*\n\nSigned-off-by: Haozhou Wang <hawang@pivotal.io>","distinct":true,"url":"https://api.github.com/repos/jimmyyih/gpdb/commits/8fc342dc083a7e62e4c36cd80a38a9eb7e25658d"},{"sha":"2e1646a9175d1e37dda8a0e01ae53f8c3c1fe210","author":{"name":"Kuien Liu","email":"8aa9d0feec3348a732554f419630c7811b86d1bf@pivotal.io"},"message":"s3ext: refactor gpcheckcloud codes\n\nUsing new GPReader, and make it clean.\n\nSigned-off-by: Kuien Liu <kliu@pivotal.io>","distinct":true,"url":"https://api.github.com/repos/jimmyyih/gpdb/commits/2e1646a9175d1e37dda8a0e01ae53f8c3c1fe210"},{"sha":"d0fe2b6614de898d4798d23eb63ce316fb324ebc","author":{"name":"Peifeng Qiu","email":"d8721495590703eed183a852facfb23f79ea3f7b@pivotal.io"},"message":"s3ext: refactor error message handling\n\n1. add global error message variable to report to gpdb console.\n2. add try/catch block around fetchData() call to catch error message.\n3. refactor shared error mechanism between ChunkBuffers.\n4. fix unit tests.\n\nAlso fixed a memleak thanks to Kuien.\n\nSigned-off-by: Haozhou Wang <hawang@pivotal.io>\nSigned-off-by: Kuien Liu <kliu@pivotal.io>","distinct":true,"url":"https://api.github.com/repos/jimmyyih/gpdb/commits/d0fe2b6614de898d4798d23eb63ce316fb324ebc"},{"sha":"e44564253e6ff8365d96f26d18ef33869a362a02","author":{"name":"Dave Cramer","email":"f839a1a93476788f85c53bacfce95f076e8b9950@gmail.com"},"message":"added required packages (#880)","distinct":true,"url":"https://api.github.com/repos/jimmyyih/gpdb/commits/e44564253e6ff8365d96f26d18ef33869a362a02"},{"sha":"c34226540f0ef53d2b53680bed8e9838c1bdeff9","author":{"name":"Zak Auerbach","email":"b333d81d0c6cd6ea68d8615830ff7232d03ef35f@zaksoup.com"},"message":"Create dockerfile and utility scripts for docker image (#773)\n\n* Add section to readme with info on how to consume docker image\r\n* Docker image automatically starts demo cluster and sources relevant env files","distinct":true,"url":"https://api.github.com/repos/jimmyyih/gpdb/commits/c34226540f0ef53d2b53680bed8e9838c1bdeff9"},{"sha":"c0e1f00c2532d1e2ef8d3b409dc8fee901a7cfe2","author":{"name":"George Caragea","email":"444cdcba9315288b658abdc7d4c4b00b8db650d6@pivotal.io"},"message":"Allocate target's tuple table slot in PortalHeapMemory during split partition\n\n    When we execute an ALTER TABLE statement that splits default partition into a \"New\" and a new Default partition, we perform the following 5 steps:\n\n     * copy the tuples located in the default partition into a temporary table,\n     * drop default partition,\n     * create the “New“ partition,\n     * create the new DEFAULT partition, and\n     * copy all tuples from the temporary table into the newly created partitions.\n\n    The last step is executed in split_rows, where for each tuple in the temporary table,\n\n     * we determine the target partition/table,\n     * construct the slot of the target table (if is NULL), and\n     * store the tuple into the target table.\n\n    Currently, we allocate the tuple table slot of the target table into a per-tuple memory context. If the size of that memory context exceeds 50KB, then we reset it. This will cause issues, since it will free target table's slot and at the next iteration (copy the next tuple) we will try to access the attributes of a freed (and not NULL) slot. In addition, the target table slot lifespan is much longer than an individual tuple. So it is not correct to allocate the slot in a per-tuple memory context.\n\n    To solve this issue, we allocate target's tuple table slot in PortalHeapMemory. This is the CurrentMemoryContext that is used in split_rows before we start copying tuples. PortalHeapMemory is used already for storing the tuple table slot of the temporary table and the ResultRelInfo of the target tables. PortalHeapMemory is not freed while we copy tuples. After copying all tuples to the new partitions, we drop target tables' slots.\nSigned-off-by: Nikos Armenatzoglou <nikos.armenatzoglou@gmail.com>","distinct":true,"url":"https://api.github.com/repos/jimmyyih/gpdb/commits/c0e1f00c2532d1e2ef8d3b409dc8fee901a7cfe2"}]}
{"actor":{"display_login":"jimmyyih"}}