{"push_id":2985883675,"size":57,"distinct_size":3,"ref":"refs/heads/SNAP-2350","head":"0f44181c5ab0bdc1384b56abb59c955cdc5f74fc","before":"a44e01cf8c7ec6ac638e1e575d2fc1b498b07446","commits":[{"sha":"cc8f1139478a4ff57b7afa5debbae4154a436609","author":{"name":"Lizy Geogy","email":"4ecfdc122c7b3cdff94242d9137d40fc3e8e241a@users.noreply.github.com"},"message":"Additional changes to documentation version 1.0.2 (#1142)\n\nAdditional Changes to Documentation.","distinct":false,"url":"https://api.github.com/repos/SnappyDataInc/snappydata/commits/cc8f1139478a4ff57b7afa5debbae4154a436609"},{"sha":"b1f62362243608b57e8255b88a2575c4ae5ad614","author":{"name":"Lizy","email":"182b97bb98dc8f5dd1f827c86518dc4b0c6fae10@snappydata.io"},"message":"* Adding SnappyData version below the logo image.\n* css changes for logo","distinct":false,"url":"https://api.github.com/repos/SnappyDataInc/snappydata/commits/b1f62362243608b57e8255b88a2575c4ae5ad614"},{"sha":"9f6ff79ed68996ea42fd03a6605566ec566119ab","author":{"name":"Lizy","email":"182b97bb98dc8f5dd1f827c86518dc4b0c6fae10@snappydata.io"},"message":"Changes to the style.css to include the logo.","distinct":false,"url":"https://api.github.com/repos/SnappyDataInc/snappydata/commits/9f6ff79ed68996ea42fd03a6605566ec566119ab"},{"sha":"572e719f2c10da2ae3fa4dd2f69914562aa8adaf","author":{"name":"ahshahid","email":"de188d6da5b3d5d1e701aa98d9731587c9ab9d23@snappydata.io"},"message":"added test for policy behaviour on views","distinct":false,"url":"https://api.github.com/repos/SnappyDataInc/snappydata/commits/572e719f2c10da2ae3fa4dd2f69914562aa8adaf"},{"sha":"2ccdfbe45e0cc8713342904767491818bd015368","author":{"name":"vatsal mevada","email":"9a7d060472fab36dba0f4c32176db118dbd63641@live.com"},"message":"Snap 2487 (#1143)\n\n* SNAP-2487 - Properly fetching the current active snappy streaming context and closing it\r\n\r\n* SNAP-2487 - Properly fetching the current active snappy streaming context and closing it when \"streaming.batch_interval\" is not passed as part of configuration","distinct":false,"url":"https://api.github.com/repos/SnappyDataInc/snappydata/commits/2ccdfbe45e0cc8713342904767491818bd015368"},{"sha":"15adc499e3cce86c072b5c0240eaafbf8c62b29b","author":{"name":"Abhijeet Boje","email":"d120ae92c0d3b543835810cb7c0a4f8f6a16f8bc@users.noreply.github.com"},"message":"disabling local hash joins (#1145)","distinct":false,"url":"https://api.github.com/repos/SnappyDataInc/snappydata/commits/15adc499e3cce86c072b5c0240eaafbf8c62b29b"},{"sha":"0ded4dd784bb3f0e6b999b99a0f2bedc38d8c043","author":{"name":"Abhijeet","email":"40ff32ba171c0bbe7f0858c23c95b45a8554b4ca@snappydata.io"},"message":"Reverting a merge done incorrectly.","distinct":false,"url":"https://api.github.com/repos/SnappyDataInc/snappydata/commits/0ded4dd784bb3f0e6b999b99a0f2bedc38d8c043"},{"sha":"41dbee30a381e7570b3121873b0dda06abc9923c","author":{"name":"Lizy Geogy","email":"4ecfdc122c7b3cdff94242d9137d40fc3e8e241a@users.noreply.github.com"},"message":"Docv1.0.2 (#1147)\n\n* Updates to formatting and other doc fixes\r\n\r\n* Update topic Create Table for default behavior of USING option.\r\n\r\n* Edited Using SnappyData Cloudbuilder > Quick Start Steps > Deploying SnappyData Cloud Cluster on AWS using Scripts for including the link for  EC2 scripts release page. Edited the \"Here\" link in Programming Guide. Minor spell edits for the term YARN.\r\n\r\n* minor edits for typos.\r\n\r\n* Edited Install on  premise section for all items in Multiple Host Installation.# Please enter the commit message for your changes. Lines starting\r\n\r\n* Updates to SnappyData Pulse content and addition of corresponding images.\r\n\r\n* >>>>>>> origin/master\r\nAdd chapter for Kubernetes. Update the mkdocs.yml file to include index entry for Getting Started with SnappyData on Kubernetes chapter\r\n\r\n* Some edits to the Kubernetes chapter content.\r\n\r\n* Corrected minor error and added content for troubleshooting member startup issue due to missing disk stores.\r\n\r\n* Incorporated review comments for SnappyData Pulse.\r\n\r\n* Review comments incorporation for Kubernetes documentation.\r\n\r\n* Minor changes to Kubernetes documentation content and corresponding changes to yml file.\r\n\r\n* changes done to the publish-site.sh  for serving the published content.\r\n\r\n* Including changes suggested by Sumedh in Best Practices > Operator System Setting > OS cache Size.\r\n\r\n* Including updates to Troubleshooting > Member Startup Problem section.\r\n\r\n* Minor edits to Troubleshooting > Member Startup Problems section.\r\n\r\n* Review comment from Sumedh incorporated.\r\n\r\n* Based on Pierce's email, created a subsection for configuring secondary lead and provided a link to it from the best practices section. Added content in Configuring the cluster> Configuring Secondary Lead section. Provided link to this chapter in Best Practices>HA considerations>Lead section\r\n\r\n* Incorporated review comments and included content for Accessing Logs and Configuring Log level.\r\n\r\n* Final review comment incorporation and edits for SnappyData Pulse.\r\n\r\n* Updates done for SNAP-2454, SNAP-227-, SNAP-2256\r\n\r\n* For SNAP 2454 Removed links of Create Synonym and Drop Synonym as suggested by Sonal/Shirish\r\n\r\n* Changes/corrections to Configuring the Cluster > Configuring Data Servers section suggested by Rishi.\r\n\r\n* Review incorporation for Kubernetes content from the table top review with Amogh and Shirish.\r\n\r\n* Minor edits for SnappyData Pulse content. Added updated image for pulse.\r\n\r\n* Minor edits to Install On- Premise section.\r\n\r\n* Edited some grammatical errors.\r\n\r\n* Added content in troubleshooting section for Resolving Catalog Inconsistency Issues.\r\n\r\n* Adding catalog_inconsistency.md.\r\n\r\n* Added Securing SnappyData Pulse UI connection section under Configuring the cluster section. Added link to the same in Managing and Monitoring section.\r\n\r\n* Incorporate review comments for SnappyData Pulse UI Secure connection and Troubleshooting section.\r\n\r\n* Minor link correction.\r\n\r\n* Added content in Best Practices section for Code Generation and Tokenization. Add the property spark.sql.codegen.cacheSize in the common  list of properties and list of properties for leads.\r\n\r\n* Corrected minor code error in the using_the_spark_shell_and_spark-submit.md document.\r\n\r\n* Created draft for Row Level Security (RLS) topic.\r\n\r\n* Change release version of SnappyData to 1.0.2.\r\n\r\n Please enter the commit message for your changes. Lines starting\r\n\r\n* Add content for GemFire Connector. Minor edits to connector pages and RLS content.\r\n\r\n* added -client-port property into Locator property list and Server property list as suggested by Asif in Doc_Tasks.\r\n\r\n* Review comment incorporation for SNAP-2454\r\n\r\n* Added appropriate links to \"Apache Zeppelin Binary Package\" download for version 0.7.2 and 0.7.3.\r\n\r\n* Some editorial updates to the content.\r\n\r\n* Incorporate review comments from Asif and editorial comments.\r\n\r\n* Updated content in Create Schema as suggested by Amogh.\r\n\r\n* Removed the spaces after -heap-size property in an example as suggested by Nikhil.\r\n\r\n* Converted the SnappyData Community Edition features vs Enterprise edition features into a tabular format as suggested by Sudhir/Pierce.\r\n\r\n* Update kubernetes.md\r\n\r\nMinor corrections. One action item for Lizy.\r\n\r\n* Update monitoring.md\r\n\r\nMinor edits ...\r\n\r\n* Moved the Securing SnappyData Pulse UI connection and SSL Set-up for Client-Server topics to the Managing Security section as suggested by Jags.\r\n\r\n*  Minor editorial updates.\r\n\r\n* Updated details about Port in Create SnappyData DSN from ODBC Data Sources 64-bit/32-bit topic as suggested by Jags.\r\n\r\n* Archiving previous version SnappyData Release Notes.\r\n\r\n* Added a note in System requirements topic, in Memory regarding Enterprise edition and OSS memory requirements to start a cluster. As suggested by Amogh.\r\n\r\n* Updates to Upgrade doc for upgrading to 1.0.2.\r\n\r\n* Following changes done as suggested by Jags and further content suggestions by Amogh:\r\nRemoved the title Download and Install and changed it to \"Provisioning SnappyData\"\r\nUpdated the entries under Getting Started and Provisioning SnappyData sections in the TOC.\r\nContent in Using Cloudbuilder moved into AWS section within \"Provisioning SnappyData\".\r\nProvisioning SnappyData>AWS section edited to accomodate Deployment via Cloubuilder, EC2 Scripts, and AWS Management Console.\r\nEdited/Corrected the links from other chapters to \"Using SnappyData Cloudbuilder\" chapter.\r\n\r\n* Update mkdocs.yml\r\n\r\nChanging section title from DB connectors to 'working with external data sources'\r\n\r\n* Update connector.md\r\n\r\n@lizygeogy This section needs a lot more attention. The whole 'how to' section was a placeholder for misc items that really need to be moved around and/or linked to. This 'data sourcing' section is very important. Made some basic changes. But, keep a note that this is something we need to clean up.  Can you please check if I have the correct syntax for linking to other sub-sections?\r\n\r\n* Update connector.md\r\n\r\nminor fix ...\r\n\r\n* Edits to Working with External Data Sources and the YML file.\r\n\r\n* minor edits to yml file.\r\n\r\n* Review comment incorporation for Kubernetes.\r\n\r\n* Review comments incorporation for cloudbuilder and EC2 scripts in AWS section from Amogh.\r\n\r\n* Updates for  Specifying Encrypted Passwords in Conf Files or in Client Connections.\r\n\r\n* Updates to yml file for entry for Specifying encrypted passwords.\r\n\r\n*  Incorporate review comments for Specifying Encrypted Passwords topic.\r\n\r\n* Incorporate review comments for Upgrade Instructions section.\r\n\r\n* Incorporated review comments based on the testing for Deployment on Machines without SSH Passwordless.\r\n\r\n* Removed a para from the description as suggested by Amogh.\r\n\r\n* Added section for Specifying configuration properties using environment variables.\r\n\r\n* Incorporated comments suggested by Amogh\r\n\r\n* Updating known issues section.\r\n\r\n* Include new features for Enterprise and OSS.\r\n\r\n* Add -J-Dsnappydata.RESTRICT_TABLE_CREATION property\r\n\r\n* Added section on Adding restrictions in default schema.\r\n\r\n* Adding Release Notes for 1.0.2 version.\r\n\r\n* Add disclaimer for Enterprise only. Add links to new topics in Managing Security.\r\n\r\n* SNAP-2488. Remove the references of TRIGGER command.\r\n\r\n* Adding chaptes for Deploy statements in Reference guide > sql reference section.\r\n\r\n* Added topic for validate-disk-store in the Reference Guide Command line utilities section.\r\n\r\n* Add topic for Deployment of Third party connectors as well as linking topics.\r\n\r\n* Corrected Broken links\r\n\r\n* Fix minor errors.\r\n\r\n* edit CSS files to correct the readthedocs format.\r\n\r\n* minor edits for publishing to readthedocs.\r\n\r\n* Testing for readthedocs.\r\n\r\n* Update breadcrumbs.html\r\n\r\n* readthedocs update for publishing.\r\n\r\n* Updates to Troubleshooting section for SNAP-1397. Added validate-missing-diskstore and list-missing-diskstore to the yml file.\r\n\r\n* Added topic in best practices for ODBC/JDBC clients as suggested by Sumedh.\r\n\r\n* Minor edits to the yml file.\r\n\r\n* Add RLS enable property.\r\n\r\n* Add SnappyData Quick Start content. Minor edits to Provisioning SnappyData section.\r\n\r\n* Connector Chapter review comment incorporation.\r\n\r\n* Removing instances of \"experimental phrases\" Kubernetes documentation.\r\n\r\n* Add points in system requirements section as suggested by Sumedh.\r\n\r\n* Commenting out SnappyData QuickStart Guide.\r\n\r\n* Correcting minor doc errors.","distinct":false,"url":"https://api.github.com/repos/SnappyDataInc/snappydata/commits/41dbee30a381e7570b3121873b0dda06abc9923c"},{"sha":"dd27f1cd9933f79f584e2f953faae7cb43e854fa","author":{"name":"Lizy","email":"182b97bb98dc8f5dd1f827c86518dc4b0c6fae10@snappydata.io"},"message":"Added an echo statement to remind the user to run gradle docs task.","distinct":false,"url":"https://api.github.com/repos/SnappyDataInc/snappydata/commits/dd27f1cd9933f79f584e2f953faae7cb43e854fa"},{"sha":"929747f11577ef6f7211d70d527632afa12d164e","author":{"name":"Sumedh Wale","email":"279f59017abb899f68fc989745adf86c9a5d21fc@snappydata.io"},"message":"[SNAP-2511] enable lazy initialization in SortMerge and Hash joins (#1146)\n\nThis enables lazy initialisation of build-side for both SortMergeJoin and SnappyHashJoin.\r\nAdditionally optimized handling for the case of zero-size build-side map in SnappyHashJoin has\r\nbeen added. Handling the same for outer-joins with SortMergeJoin has been added on the Spark side\r\nand will not happen for smart connector mode because adding the same would require copying\r\nof most of the code from Spark's SortMergeJoinExec. This will be added in future when SortMergeJoin\r\nas a whole is optimized.\r\n\r\n## Changes proposed in this pull request\r\n\r\n- extend SortExec with SnappySortExec having a lazily initialized iterator\r\n  that will avoid pre-sorting the build side if stream side is empty;\r\n  also added a mapPartitionsPreserveInternal implicit for the iterator that\r\n  preserves that preferred locations like mapPartitionsPreserve but also\r\n  avoid closure cleaning like Spark's mapPartitionsInternal\r\n- add an OptimizeSortPlans rule that will replace the RHS of SortMergeJoin\r\n  with above SnappySortExec to enable the same\r\n- made the hash map initialization in hash joins to be lazy where \"initMap\" on\r\n  the build side is not done upfront in doProduce rather checked and done in\r\n  first iteration in doConsume\r\n- also optimize hash join for the case when build side is zero to continue\r\n  iteration without calculating hash or doing lookup (for the case of\r\n      non-inner joins, the hash/lookup is skipped but key columns may\r\n      still be materialized)\r\n- correct CodegenSparkFallback to also handle CompileException and walk down\r\n  exception causes for all cases including for java.lang.Error\r\n- don't fail \"NonRecursivePlans\" on first nested call rather after a few\r\n  iterations to handle a few cases where it does get invoked twice\r\n- never fallback to store side \"raw\" CREATE TABLE execution in case of\r\n  USING/OPTIONS keywords that are only used by Spark side; this fixes\r\n  syntax error messages that use the documented SnappyData syntax\r\n  (and not the old rowstore mode like syntax)\r\n- allow for explicit cast to INTERVAL type to allow for easy conversions like:\r\n  cast (('interval ' || col || ' secs') as interval)\r\n- updated instructions for Intellij setup in build-instructions.md","distinct":false,"url":"https://api.github.com/repos/SnappyDataInc/snappydata/commits/929747f11577ef6f7211d70d527632afa12d164e"},{"sha":"29c784bcde0cc44c89d70bc72bf6186c89b55d60","author":{"name":"Sumedh Wale","email":"279f59017abb899f68fc989745adf86c9a5d21fc@snappydata.io"},"message":"[SNAP-2555] clear non-default configs for internal hive meta-store client (#1154)\n\n- adding a new \"SnappyHiveConf\" that extends HiveConf to clear all non-default configuration\r\n  (via reflection for few items) from the internal hive meta-store client\r\n- preserve log-level from initialized SparkContext when booting store in local mode\r\n  if not set explicitly at store level (using \"snappydata.store.log-level\" property)\r\n- correct scalastyle warnings/errors in some files","distinct":false,"url":"https://api.github.com/repos/SnappyDataInc/snappydata/commits/29c784bcde0cc44c89d70bc72bf6186c89b55d60"},{"sha":"2fc086d68dea5db1c596c94fbea9f266e6ae57a5","author":{"name":"Sumedh Wale","email":"279f59017abb899f68fc989745adf86c9a5d21fc@snappydata.io"},"message":"update store link","distinct":false,"url":"https://api.github.com/repos/SnappyDataInc/snappydata/commits/2fc086d68dea5db1c596c94fbea9f266e6ae57a5"},{"sha":"3dab9b998c308a8a12bdc76f3e5c9c99c3916ab1","author":{"name":"Jags Ramnarayan","email":"252be170f699a6628522ec41ec4e333dc5205967@users.noreply.github.com"},"message":"Update getting_started_with_your_spark_distribution.md\n\nMinor edits....","distinct":false,"url":"https://api.github.com/repos/SnappyDataInc/snappydata/commits/3dab9b998c308a8a12bdc76f3e5c9c99c3916ab1"},{"sha":"b84254da195f59efca9dd6f004c7f265939e0c35","author":{"name":"Sumedh Wale","email":"279f59017abb899f68fc989745adf86c9a5d21fc@snappydata.io"},"message":"Enable default off-heap memory-size when possible\n\n- setting default memory-size so that total of heap+off-heap is 75% of total RAM (excluding swap)\n- also account for any leads started on the node (passed using snappydata.numLeadsOnNode property\n    and used by GemFireCache during its memory-size configuration)\n- disable any default setting if there are multiple servers configured on the same host;\n  there still exists a corner condition if two different host/IPs in the configuration refer\n  to the same physical host and that can cause overallocation but for these cases users should\n  know what they are doing and set memory-size explicitly\n- also avoid overriding log-level with log4j internal default unless it has been explicitly\n  set to WARN or higher programmatically (e.g. in a spark-shell or smart connector environment)","distinct":false,"url":"https://api.github.com/repos/SnappyDataInc/snappydata/commits/b84254da195f59efca9dd6f004c7f265939e0c35"},{"sha":"858436ac30f3a0f00f712636e9b8ddd26bf27966","author":{"name":"Sumedh Wale","email":"279f59017abb899f68fc989745adf86c9a5d21fc@snappydata.io"},"message":"Mark numLeadsOnNode as negative if heap-size/memory-size has been configured explicitly\n\nFor the case when there are leads running on the same node as servers, and have\ntheir heap-size/memory-size explicitly configured, then skip off-heap auto-configuration\non the servers since it will be difficult to determine the correct numbers","distinct":false,"url":"https://api.github.com/repos/SnappyDataInc/snappydata/commits/858436ac30f3a0f00f712636e9b8ddd26bf27966"},{"sha":"fc8ea5c6731090c677de6c8099052e37b274cb47","author":{"name":"Trilok Khairnar","email":"f14697e8cb54649e5fefa37861daf7a19b57ce19@users.noreply.github.com"},"message":"SNAP-2447: Set seed of random number used in query parameterization for reproducibility (#1149)\n\nPass an additional parameter randomSeed in PerfRun.conf for TPC-H . Initialize a random number in TPCH_Queries with this seed for repeatable experiments (with exact same parameter values).","distinct":false,"url":"https://api.github.com/repos/SnappyDataInc/snappydata/commits/fc8ea5c6731090c677de6c8099052e37b274cb47"},{"sha":"1d060275caddd9a7ca25fe9affb379610112d29e","author":{"name":"Swati Sawant","email":"2d3382aec74f6fbaea504966b0db28354a4c3fe3@snappydata.io"},"message":"Framework changes for starting the cluster with user specified confs,generate the long running test artifacts and new tests to measure the throughput of analytical load  (#1141)\n\n- Added changes for starting the cluster with user-specified confs in the framework and generate the long-running test artifacts in case of user-specified confs for starting the cluster.\r\n- Modified the concurrency tests to use user-specified confs while starting the cluster\r\n- Modified the concurrency tests to use local log directories for all members along with spark.local.dir pointing to the local directory for each of the members.\r\n- Added tests with a different combination of threads executing point lookup and analytical queries concurrently.\r\n- Removed extra spaces from the Query Strings\r\n- Added local.confs to be used while running the concurrency tests\r\n- Added support for passing user-specified configuration parameters in order to connect to the cluster started using user confs\r\n- Added concurrency tests to measure the throughput of analytical load using the varying number of threads and TPCH schema\r\n- Added support to execute TPCH Q15 through concurrency test.\r\n- Removed unnecessary VMArgs in the conf file\r\n- Modified the TPCH q15 temp query to use the add_months function\r\n- Fixed a minor issue in deleteSnappyArtifacts in case of smart Connector mode. Added a fix to delete the file only if it exists.\r\n- Changes to create a temp view revenue once per session and use it throughout till session termination in case of tpch schema in concurrency tests\r\n- Parameterized the warmUpTime for concurrent queries execution","distinct":false,"url":"https://api.github.com/repos/SnappyDataInc/snappydata/commits/1d060275caddd9a7ca25fe9affb379610112d29e"},{"sha":"f870cd7f24f9978d4ead16ca13cad54ad0b23e46","author":{"name":"kneeraj","email":"e036a94f6f76953066f908f219021d93a9581386@snappydata.io"},"message":"SnappyData specific GfxdDataSerializable types were not getting registered (#1144)\n\nwhen validate tool was getting launched through SnapyUtilLauncher.\r\nThis is because the StoreCallback was not getting initialized when only the validate or any other tool runs from the launcher. So the registry of GfxdDataSerializables which happens as part of static intialization of the StoreCallback was not happening.","distinct":false,"url":"https://api.github.com/repos/SnappyDataInc/snappydata/commits/f870cd7f24f9978d4ead16ca13cad54ad0b23e46"},{"sha":"56420fb3fc18156828d226b3512d81089f285b18","author":{"name":"Sumedh Wale","email":"279f59017abb899f68fc989745adf86c9a5d21fc@snappydata.io"},"message":"Fixing some issues in startup scripts\n\n- moved snippet to get AWS public host name inside \"snappy\" script since it must\n  be done on the actual server rather than where \"snappy-start-all.sh\" has been invoked\n- use hostname-for-clients to set the SPARK_PUBLIC_DNS so that both JDBC/ODBC and WebUI point\n  to the same address if required; any explicit SPARK_PUBLIC_DNS is ignored because it will\n  be common for the cluster and will not work with multiple leads\n- likewise use bind-address to set SPARK_LOCAL_IP on every node instead of reading the global\n  environment variable","distinct":false,"url":"https://api.github.com/repos/SnappyDataInc/snappydata/commits/56420fb3fc18156828d226b3512d81089f285b18"},{"sha":"bcc7177f7b12f64b0f14485960ed67f41a003879","author":{"name":"kneeraj","email":"e036a94f6f76953066f908f219021d93a9581386@snappydata.io"},"message":"Snap 2566 (#1162)\n\nOffset argument to Lead and Lag window functions were not being marked as foldable\r\ncausing analysis error. Added them in Foldable functions list.","distinct":false,"url":"https://api.github.com/repos/SnappyDataInc/snappydata/commits/bcc7177f7b12f64b0f14485960ed67f41a003879"}]}
{"actor":{"display_login":"spillai14"}}