{"push_id":2841684435,"size":1,"distinct_size":1,"ref":"refs/heads/salesforce_and_sftp_operators_with_s3","head":"bd17d7be753a790c9ef4fa0d2c0e71f85e849e44","before":"fd6d9c0f98a8577ec86bbb6d1decabbcad4769b1","commits":[{"sha":"bd17d7be753a790c9ef4fa0d2c0e71f85e849e44","author":{"name":"wayne.morris","email":"15a82fbbf53da25943ef7e3f608db5b555feab31@modmed.com"},"message":"[AIRFLOW-2993] Added sftp_to_s3 and s3_to_sftp operators (#3828)\n\nAddition of s3_to_sftp and sftp_to_s3 operators.\n\n[AIRFLOW-2900] Show code for packaged DAGs (#3749)\n\n[AIRFLOW-2983] Add prev_ds_nodash and next_ds_nodash macro (#3821)\n\n[AIRFLOW-2989] Add param to set bootDiskType in Dataproc Op (#3825)\n\nAdd param to set bootDiskType for master and\nworker nodes in `DataprocClusterCreateOperator`\n\n[AIRFLOW-2974] Extended Databricks hook with clusters operation (#3817)\n\nAdd hooks for:\n- cluster start,\n- restart,\n- terminate.\nAdd unit tests for the added hooks.\nAdd hooks for cluster start, restart and terminate.\nAdd unit tests for the added hooks.\nAdd cluster_id variable for performing cluster operation tests.\n\n[AIRFLOW-2993] Fix Docstrings for Operators (#3828)\n\nAddition of s3_to_sftp and sftp_to_s3 operators.\n\nAdd 'steps' into template_fields in EmrAddSteps\n\nRendering templates which are in steps is especially useful if you\nwant to pass execution time as one of the paramaters of a step in\nan EMR cluster. All fields in template_fields will get rendered.\n\n[AIRFLOW-2993] Addition of s3_to_sftp and sftp_to_s3 operators.\n\nAdd 'steps' into template_fields in EmrAddSteps\n\nRendering templates which are in steps is especially useful if you\nwant to pass execution time as one of the paramaters of a step in\nan EMR cluster. All fields in template_fields will get rendered.\n\n[AIRFLOW-1762] Implement key_file support in ssh_hook create_tunnel\n\nSwitched to using sshtunnel package instead of\npopen approach\n\nCloses #3473 from NielsZeilemaker/ssh_hook\n\nAddition of s3_to_sftp and sftp_to_s3 operators.\n\n[AIRFLOW-2993] sftp_to_s3 and s3_to_sftp Operators (#3828)\n\nAddition of s3_to_sftp and sftp_to_s3 operators.\n\nAdd 'steps' into template_fields in EmrAddSteps\n\nRendering templates which are in steps is especially useful if you\nwant to pass execution time as one of the paramaters of a step in\nan EMR cluster. All fields in template_fields will get rendered.\n\n[AIRFLOW-2993] Addition of s3_to_sftp and sftp_to_s3 operators.\n\nAdd 'steps' into template_fields in EmrAddSteps\n\nRendering templates which are in steps is especially useful if you\nwant to pass execution time as one of the paramaters of a step in\nan EMR cluster. All fields in template_fields will get rendered.\n\n[AIRFLOW-1762] Implement key_file support in ssh_hook create_tunnel\n\nSwitched to using sshtunnel package instead of\npopen approach\n\nCloses #3473 from NielsZeilemaker/ssh_hook\n\nAddition of s3_to_sftp and sftp_to_s3 operators.\n\n[AIRFLOW-2974] Extended Databricks hook with clusters operation (#3817)\n\nAdd hooks for:\n- cluster start,\n- restart,\n- terminate.\nAdd unit tests for the added hooks.\nAdd hooks for cluster start, restart and terminate.\nAdd unit tests for the added hooks.\nAdd cluster_id variable for performing cluster operation tests.\n\n[AIRFLOW-XXX] Fix Docstrings for Operators (#3820)\n\nAddition of s3_to_sftp and sftp_to_s3 operators.\n\nAdd 'steps' into template_fields in EmrAddSteps\n\nRendering templates which are in steps is especially useful if you\nwant to pass execution time as one of the paramaters of a step in\nan EMR cluster. All fields in template_fields will get rendered.\n\n[AIRFLOW-2993] Addition of s3_to_sftp and sftp_to_s3 operators.\n\nAdd 'steps' into template_fields in EmrAddSteps\n\nRendering templates which are in steps is especially useful if you\nwant to pass execution time as one of the paramaters of a step in\nan EMR cluster. All fields in template_fields will get rendered.\n\n[AIRFLOW-1762] Implement key_file support in ssh_hook create_tunnel\n\nSwitched to using sshtunnel package instead of\npopen approach\n\nCloses #3473 from NielsZeilemaker/ssh_hook\n\nAddition of s3_to_sftp and sftp_to_s3 operators.\n\n[AIRFLOW-XXX] Fix Docstrings for Operators (#3820)\n\nAddition of s3_to_sftp and sftp_to_s3 operators.\n\nAdd 'steps' into template_fields in EmrAddSteps\n\nRendering templates which are in steps is especially useful if you\nwant to pass execution time as one of the paramaters of a step in\nan EMR cluster. All fields in template_fields will get rendered.\n\n[AIRFLOW-2993] Addition of s3_to_sftp and sftp_to_s3 operators.\n\nAdd 'steps' into template_fields in EmrAddSteps\n\nRendering templates which are in steps is especially useful if you\nwant to pass execution time as one of the paramaters of a step in\nan EMR cluster. All fields in template_fields will get rendered.\n\n[AIRFLOW-1762] Implement key_file support in ssh_hook create_tunnel\n\nSwitched to using sshtunnel package instead of\npopen approach\n\nCloses #3473 from NielsZeilemaker/ssh_hook\n\n[AIRFLOW-2900] Show code for packaged DAGs (#3749)\n\n[AIRFLOW-2983] Add prev_ds_nodash and next_ds_nodash macro (#3821)\n\n[AIRFLOW-2989] Add param to set bootDiskType in Dataproc Op (#3825)\n\nAdd param to set bootDiskType for master and\nworker nodes in `DataprocClusterCreateOperator`\n\n[AIRFLOW-2974] Extended Databricks hook with clusters operation (#3817)\n\nAdd hooks for:\n- cluster start,\n- restart,\n- terminate.\nAdd unit tests for the added hooks.\nAdd hooks for cluster start, restart and terminate.\nAdd unit tests for the added hooks.\nAdd cluster_id variable for performing cluster operation tests.\n\n[AIRFLOW-2993] Fix Docstrings for Operators (#3828)\n\nAddition of s3_to_sftp and sftp_to_s3 operators.\n\nAdd 'steps' into template_fields in EmrAddSteps\n\nRendering templates which are in steps is especially useful if you\nwant to pass execution time as one of the paramaters of a step in\nan EMR cluster. All fields in template_fields will get rendered.\n\n[AIRFLOW-2993] Addition of s3_to_sftp and sftp_to_s3 operators.\n\nAdd 'steps' into template_fields in EmrAddSteps\n\nRendering templates which are in steps is especially useful if you\nwant to pass execution time as one of the paramaters of a step in\nan EMR cluster. All fields in template_fields will get rendered.\n\n[AIRFLOW-1762] Implement key_file support in ssh_hook create_tunnel\n\nSwitched to using sshtunnel package instead of\npopen approach\n\nCloses #3473 from NielsZeilemaker/ssh_hook\n\nAddition of s3_to_sftp and sftp_to_s3 operators.\n\n[AIRFLOW-2993] sftp_to_s3 and s3_to_sftp Operators (#3828)\n\nAddition of s3_to_sftp and sftp_to_s3 operators.\n\nAdd 'steps' into template_fields in EmrAddSteps\n\nRendering templates which are in steps is especially useful if you\nwant to pass execution time as one of the paramaters of a step in\nan EMR cluster. All fields in template_fields will get rendered.\n\n[AIRFLOW-2993] Addition of s3_to_sftp and sftp_to_s3 operators.\n\nAdd 'steps' into template_fields in EmrAddSteps\n\nRendering templates which are in steps is especially useful if you\nwant to pass execution time as one of the paramaters of a step in\nan EMR cluster. All fields in template_fields will get rendered.\n\n[AIRFLOW-1762] Implement key_file support in ssh_hook create_tunnel\n\nSwitched to using sshtunnel package instead of\npopen approach\n\nCloses #3473 from NielsZeilemaker/ssh_hook\n\nAddition of s3_to_sftp and sftp_to_s3 operators.\n\n[AIRFLOW-2974] Extended Databricks hook with clusters operation (#3817)\n\nAdd hooks for:\n- cluster start,\n- restart,\n- terminate.\nAdd unit tests for the added hooks.\nAdd hooks for cluster start, restart and terminate.\nAdd unit tests for the added hooks.\nAdd cluster_id variable for performing cluster operation tests.\n\n[AIRFLOW-XXX] Fix Docstrings for Operators (#3820)\n\nAddition of s3_to_sftp and sftp_to_s3 operators.\n\nAdd 'steps' into template_fields in EmrAddSteps\n\nRendering templates which are in steps is especially useful if you\nwant to pass execution time as one of the paramaters of a step in\nan EMR cluster. All fields in template_fields will get rendered.\n\n[AIRFLOW-2993] Addition of s3_to_sftp and sftp_to_s3 operators.\n\nAdd 'steps' into template_fields in EmrAddSteps\n\nRendering templates which are in steps is especially useful if you\nwant to pass execution time as one of the paramaters of a step in\nan EMR cluster. All fields in template_fields will get rendered.\n\n[AIRFLOW-1762] Implement key_file support in ssh_hook create_tunnel\n\nSwitched to using sshtunnel package instead of\npopen approach\n\nCloses #3473 from NielsZeilemaker/ssh_hook\n\nAddition of s3_to_sftp and sftp_to_s3 operators.\n\n[AIRFLOW-XXX] Fix Docstrings for Operators (#3820)\n\nAddition of s3_to_sftp and sftp_to_s3 operators.\n\nAdd 'steps' into template_fields in EmrAddSteps\n\nRendering templates which are in steps is especially useful if you\nwant to pass execution time as one of the paramaters of a step in\nan EMR cluster. All fields in template_fields will get rendered.\n\n[AIRFLOW-2993] Addition of s3_to_sftp and sftp_to_s3 operators.\n\nAdd 'steps' into template_fields in EmrAddSteps\n\nRendering templates which are in steps is especially useful if you\nwant to pass execution time as one of the paramaters of a step in\nan EMR cluster. All fields in template_fields will get rendered.\n\n[AIRFLOW-1762] Implement key_file support in ssh_hook create_tunnel\n\nSwitched to using sshtunnel package instead of\npopen approach\n\nCloses #3473 from NielsZeilemaker/ssh_hook\n\nAddition of s3_to_sftp and sftp_to_s3 operators.","distinct":true,"url":"https://api.github.com/repos/modmed/incubator-airflow/commits/bd17d7be753a790c9ef4fa0d2c0e71f85e849e44"}]}
{"actor":{"display_login":"wmorris75"}}