Recurse through a subdirectory on the master and copy said subdirectory
    over to the specified path.

    name
        The directory to set the recursion in

    source
        The source directory, this directory is located on the salt master file
        server and is specified with the salt:// protocol. If the directory is
        located on the master in the directory named spam, and is called eggs,
        the source string is salt://spam/eggs

    keep_source : True
        Set to ``False`` to discard the cached copy of the source file once the
        state completes. This can be useful for larger files to keep them from
        taking up space in minion cache. However, keep in mind that discarding
        the source file will result in the state needing to re-download the
        source file if the state is run again.

        .. versionadded:: 2017.7.3

    clean
        Make sure that only files that are set up by salt and required by this
        function are kept. If this option is set then everything in this
        directory will be deleted unless it is required.

    require
        Require other resources such as packages or files

    user
        The user to own the directory. This defaults to the user salt is
        running as on the minion

    group
        The group ownership set for the directory. This defaults to the group
        salt is running as on the minion. On Windows, this is ignored

    dir_mode
        The permissions mode to set on any directories created.

        The default mode for new files and directories corresponds umask of salt
        process. For existing files and directories it's not enforced.

        .. note::
            This option is **not** supported on Windows.

    file_mode
        The permissions mode to set on any files created.

        The default mode for new files and directories corresponds umask of salt
        process. For existing files and directories it's not enforced.

        .. note::
            This option is **not** supported on Windows.

        .. versionchanged:: 2016.11.0
            This option can be set to ``keep``, and Salt will keep the mode
            from the Salt fileserver. This is only supported when the
            ``source`` URL begins with ``salt://``, or for files local to the
            minion. Because the ``source`` option cannot be used with any of
            the ``contents`` options, setting the ``mode`` to ``keep`` is also
            incompatible with the ``contents`` options.

    sym_mode
        The permissions mode to set on any symlink created.

        The default mode for new files and directories corresponds umask of salt
        process. For existing files and directories it's not enforced.

        .. note::
            This option is **not** supported on Windows.

    template
        If this setting is applied, the named templating engine will be used to
        render the downloaded file. The following templates are supported:

        - :mod:`cheetah<salt.renderers.cheetah>`
        - :mod:`genshi<salt.renderers.genshi>`
        - :mod:`jinja<salt.renderers.jinja>`
        - :mod:`mako<salt.renderers.mako>`
        - :mod:`py<salt.renderers.py>`
        - :mod:`wempy<salt.renderers.wempy>`

        .. note::

            The template option is required when recursively applying templates.

    replace : True
        If set to ``False`` and the file already exists, the file will not be
        modified even if changes would otherwise be made. Permissions and
        ownership will still be enforced, however.

    context
        Overrides default context variables passed to the template.

    defaults
        Default context passed to the template.

    include_empty
        Set this to True if empty directories should also be created
        (default is False)

    backup
        Overrides the default backup mode for all replaced files. See
        :ref:`backup_mode documentation <file-state-backups>` for more details.

    include_pat
        When copying, include only this pattern, or list of patterns, from the
        source. Default is glob match; if prefixed with 'E@', then regexp match.
        Example:

        .. code-block:: text

          - include_pat: hello*       :: glob matches 'hello01', 'hello02'
                                         ... but not 'otherhello'
          - include_pat: E@hello      :: regexp matches 'otherhello',
                                         'hello01' ...

        .. versionchanged:: Neon

            List patterns are now supported

        .. code-block:: text

            - include_pat:
                - hello01
                - hello02

    exclude_pat
        Exclude this pattern, or list of patterns, from the source when copying.
        If both ``include_pat`` and ``exclude_pat`` are supplied, then it will apply
        conditions cumulatively. i.e. first select based on ``include_pat``, and
        then within that result apply ``exclude_pat``.

        Also, when ``clean=True``, exclude this pattern from the removal list
        and preserve in the destination. For example:

        .. code-block:: text

            # glob matches APPDATA.01, APPDATA.02,.. for exclusion
            - exclude_pat: APPDATA*
            # regexp matches APPDATA or TEMPDATA for exclusion
            - exclude_pat: E@(APPDATA)|(TEMPDATA)

        .. versionchanged:: Neon
            List patterns are now supported

        .. code-block:: text

            - exclude_pat:
                - APPDATA.01
                - APPDATA.02

    maxdepth
        When copying, only copy paths which are of depth `maxdepth` from the
        source path.
        Example:

        .. code-block:: text

          - maxdepth: 0      :: Only include files located in the source
                                directory
          - maxdepth: 1      :: Only include files located in the source
                                or immediate subdirectories

    keep_symlinks
        Keep symlinks when copying from the source. This option will cause
        the copy operation to terminate at the symlink. If desire behavior
        similar to rsync, then set this to True.

    force_symlinks
        Force symlink creation. This option will force the symlink creation.
        If a file or directory is obstructing symlink creation it will be
        recursively removed so that symlink creation can proceed. This
        option is usually not needed except in special circumstances.

    win_owner : None
        The owner of the symlink and directories if ``makedirs`` is True. If
        this is not passed, ``user`` will be used. If ``user`` is not passed,
        the account under which Salt is running will be used.

        .. versionadded:: 2017.7.7

    win_perms : None
        A dictionary containing permissions to grant

        .. versionadded:: 2017.7.7

    win_deny_perms : None
        A dictionary containing permissions to deny

        .. versionadded:: 2017.7.7

    win_inheritance : None
        True to inherit permissions from parent, otherwise False

        .. versionadded:: 2017.7.7

def recurse(name,
            source,
            keep_source=True,
            clean=False,
            require=None,
            user=None,
            group=None,
            dir_mode=None,
            file_mode=None,
            sym_mode=None,
            template=None,
            context=None,
            replace=True,
            defaults=None,
            include_empty=False,
            backup='',
            include_pat=None,
            exclude_pat=None,
            maxdepth=None,
            keep_symlinks=False,
            force_symlinks=False,
            win_owner=None,
            win_perms=None,
            win_deny_perms=None,
            win_inheritance=True,
            **kwargs):
    '''
    Recurse through a subdirectory on the master and copy said subdirectory
    over to the specified path.

    name
        The directory to set the recursion in

    source
        The source directory, this directory is located on the salt master file
        server and is specified with the salt:// protocol. If the directory is
        located on the master in the directory named spam, and is called eggs,
        the source string is salt://spam/eggs

    keep_source : True
        Set to ``False`` to discard the cached copy of the source file once the
        state completes. This can be useful for larger files to keep them from
        taking up space in minion cache. However, keep in mind that discarding
        the source file will result in the state needing to re-download the
        source file if the state is run again.

        .. versionadded:: 2017.7.3

    clean
        Make sure that only files that are set up by salt and required by this
        function are kept. If this option is set then everything in this
        directory will be deleted unless it is required.

    require
        Require other resources such as packages or files

    user
        The user to own the directory. This defaults to the user salt is
        running as on the minion

    group
        The group ownership set for the directory. This defaults to the group
        salt is running as on the minion. On Windows, this is ignored

    dir_mode
        The permissions mode to set on any directories created.

        The default mode for new files and directories corresponds umask of salt
        process. For existing files and directories it's not enforced.

        .. note::
            This option is **not** supported on Windows.

    file_mode
        The permissions mode to set on any files created.

        The default mode for new files and directories corresponds umask of salt
        process. For existing files and directories it's not enforced.

        .. note::
            This option is **not** supported on Windows.

        .. versionchanged:: 2016.11.0
            This option can be set to ``keep``, and Salt will keep the mode
            from the Salt fileserver. This is only supported when the
            ``source`` URL begins with ``salt://``, or for files local to the
            minion. Because the ``source`` option cannot be used with any of
            the ``contents`` options, setting the ``mode`` to ``keep`` is also
            incompatible with the ``contents`` options.

    sym_mode
        The permissions mode to set on any symlink created.

        The default mode for new files and directories corresponds umask of salt
        process. For existing files and directories it's not enforced.

        .. note::
            This option is **not** supported on Windows.

    template
        If this setting is applied, the named templating engine will be used to
        render the downloaded file. The following templates are supported:

        - :mod:`cheetah<salt.renderers.cheetah>`
        - :mod:`genshi<salt.renderers.genshi>`
        - :mod:`jinja<salt.renderers.jinja>`
        - :mod:`mako<salt.renderers.mako>`
        - :mod:`py<salt.renderers.py>`
        - :mod:`wempy<salt.renderers.wempy>`

        .. note::

            The template option is required when recursively applying templates.

    replace : True
        If set to ``False`` and the file already exists, the file will not be
        modified even if changes would otherwise be made. Permissions and
        ownership will still be enforced, however.

    context
        Overrides default context variables passed to the template.

    defaults
        Default context passed to the template.

    include_empty
        Set this to True if empty directories should also be created
        (default is False)

    backup
        Overrides the default backup mode for all replaced files. See
        :ref:`backup_mode documentation <file-state-backups>` for more details.

    include_pat
        When copying, include only this pattern, or list of patterns, from the
        source. Default is glob match; if prefixed with 'E@', then regexp match.
        Example:

        .. code-block:: text

          - include_pat: hello*       :: glob matches 'hello01', 'hello02'
                                         ... but not 'otherhello'
          - include_pat: E@hello      :: regexp matches 'otherhello',
                                         'hello01' ...

        .. versionchanged:: Neon

            List patterns are now supported

        .. code-block:: text

            - include_pat:
                - hello01
                - hello02

    exclude_pat
        Exclude this pattern, or list of patterns, from the source when copying.
        If both ``include_pat`` and ``exclude_pat`` are supplied, then it will apply
        conditions cumulatively. i.e. first select based on ``include_pat``, and
        then within that result apply ``exclude_pat``.

        Also, when ``clean=True``, exclude this pattern from the removal list
        and preserve in the destination. For example:

        .. code-block:: text

            # glob matches APPDATA.01, APPDATA.02,.. for exclusion
            - exclude_pat: APPDATA*
            # regexp matches APPDATA or TEMPDATA for exclusion
            - exclude_pat: E@(APPDATA)|(TEMPDATA)

        .. versionchanged:: Neon
            List patterns are now supported

        .. code-block:: text

            - exclude_pat:
                - APPDATA.01
                - APPDATA.02

    maxdepth
        When copying, only copy paths which are of depth `maxdepth` from the
        source path.
        Example:

        .. code-block:: text

          - maxdepth: 0      :: Only include files located in the source
                                directory
          - maxdepth: 1      :: Only include files located in the source
                                or immediate subdirectories

    keep_symlinks
        Keep symlinks when copying from the source. This option will cause
        the copy operation to terminate at the symlink. If desire behavior
        similar to rsync, then set this to True.

    force_symlinks
        Force symlink creation. This option will force the symlink creation.
        If a file or directory is obstructing symlink creation it will be
        recursively removed so that symlink creation can proceed. This
        option is usually not needed except in special circumstances.

    win_owner : None
        The owner of the symlink and directories if ``makedirs`` is True. If
        this is not passed, ``user`` will be used. If ``user`` is not passed,
        the account under which Salt is running will be used.

        .. versionadded:: 2017.7.7

    win_perms : None
        A dictionary containing permissions to grant

        .. versionadded:: 2017.7.7

    win_deny_perms : None
        A dictionary containing permissions to deny

        .. versionadded:: 2017.7.7

    win_inheritance : None
        True to inherit permissions from parent, otherwise False

        .. versionadded:: 2017.7.7
    '''
    if 'env' in kwargs:
        # "env" is not supported; Use "saltenv".
        kwargs.pop('env')

    name = os.path.expanduser(sdecode(name))

    user = _test_owner(kwargs, user=user)
    if salt.utils.platform.is_windows():
        if group is not None:
            log.warning(
                'The group argument for %s has been ignored as this '
                'is a Windows system.', name
            )
        group = user
    ret = {
        'name': name,
        'changes': {},
        'result': True,
        'comment': {}  # { path: [comment, ...] }
    }

    if 'mode' in kwargs:
        ret['result'] = False
        ret['comment'] = (
            '\'mode\' is not allowed in \'file.recurse\'. Please use '
            '\'file_mode\' and \'dir_mode\'.'
        )
        return ret

    if any([x is not None for x in (dir_mode, file_mode, sym_mode)]) \
            and salt.utils.platform.is_windows():
        return _error(ret, 'mode management is not supported on Windows')

    # Make sure that leading zeros stripped by YAML loader are added back
    dir_mode = salt.utils.files.normalize_mode(dir_mode)

    try:
        keep_mode = file_mode.lower() == 'keep'
        if keep_mode:
            # We're not hard-coding the mode, so set it to None
            file_mode = None
    except AttributeError:
        keep_mode = False

    file_mode = salt.utils.files.normalize_mode(file_mode)

    u_check = _check_user(user, group)
    if u_check:
        # The specified user or group do not exist
        return _error(ret, u_check)
    if not os.path.isabs(name):
        return _error(
            ret, 'Specified file {0} is not an absolute path'.format(name))

    # expand source into source_list
    source_list = _validate_str_list(source)

    for idx, val in enumerate(source_list):
        source_list[idx] = val.rstrip('/')

    for precheck in source_list:
        if not precheck.startswith('salt://'):
            return _error(ret, ('Invalid source \'{0}\' '
                                '(must be a salt:// URI)'.format(precheck)))

    # Select the first source in source_list that exists
    try:
        source, source_hash = __salt__['file.source_list'](source_list, '', __env__)
    except CommandExecutionError as exc:
        ret['result'] = False
        ret['comment'] = 'Recurse failed: {0}'.format(exc)
        return ret

    # Check source path relative to fileserver root, make sure it is a
    # directory
    srcpath, senv = salt.utils.url.parse(source)
    if senv is None:
        senv = __env__
    master_dirs = __salt__['cp.list_master_dirs'](saltenv=senv)
    if srcpath not in master_dirs \
            and not any((x for x in master_dirs
                         if x.startswith(srcpath + '/'))):
        ret['result'] = False
        ret['comment'] = (
            'The directory \'{0}\' does not exist on the salt fileserver '
            'in saltenv \'{1}\''.format(srcpath, senv)
        )
        return ret

    # Verify the target directory
    if not os.path.isdir(name):
        if os.path.exists(name):
            # it is not a dir, but it exists - fail out
            return _error(
                ret, 'The path {0} exists and is not a directory'.format(name))
        if not __opts__['test']:
            if salt.utils.platform.is_windows():
                win_owner = win_owner if win_owner else user
                __salt__['file.makedirs_perms'](path=name,
                                                owner=win_owner,
                                                grant_perms=win_perms,
                                                deny_perms=win_deny_perms,
                                                inheritance=win_inheritance)
            else:
                __salt__['file.makedirs_perms'](name=name,
                                                user=user,
                                                group=group,
                                                mode=dir_mode)

    def add_comment(path, comment):
        comments = ret['comment'].setdefault(path, [])
        if isinstance(comment, six.string_types):
            comments.append(comment)
        else:
            comments.extend(comment)

    def merge_ret(path, _ret):
        # Use the most "negative" result code (out of True, None, False)
        if _ret['result'] is False or ret['result'] is True:
            ret['result'] = _ret['result']

        # Only include comments about files that changed
        if _ret['result'] is not True and _ret['comment']:
            add_comment(path, _ret['comment'])

        if _ret['changes']:
            ret['changes'][path] = _ret['changes']

    def manage_file(path, source, replace):
        if clean and os.path.exists(path) and os.path.isdir(path) and replace:
            _ret = {'name': name, 'changes': {}, 'result': True, 'comment': ''}
            if __opts__['test']:
                _ret['comment'] = 'Replacing directory {0} with a ' \
                                  'file'.format(path)
                _ret['result'] = None
                merge_ret(path, _ret)
                return
            else:
                __salt__['file.remove'](path)
                _ret['changes'] = {'diff': 'Replaced directory with a '
                                           'new file'}
                merge_ret(path, _ret)

        # Conflicts can occur if some kwargs are passed in here
        pass_kwargs = {}
        faults = ['mode', 'makedirs']
        for key in kwargs:
            if key not in faults:
                pass_kwargs[key] = kwargs[key]

        _ret = managed(
            path,
            source=source,
            keep_source=keep_source,
            user=user,
            group=group,
            mode='keep' if keep_mode else file_mode,
            attrs=None,
            template=template,
            makedirs=True,
            replace=replace,
            context=context,
            defaults=defaults,
            backup=backup,
            **pass_kwargs)
        merge_ret(path, _ret)

    def manage_directory(path):
        if os.path.basename(path) == '..':
            return
        if clean and os.path.exists(path) and not os.path.isdir(path):
            _ret = {'name': name, 'changes': {}, 'result': True, 'comment': ''}
            if __opts__['test']:
                _ret['comment'] = 'Replacing {0} with a directory'.format(path)
                _ret['result'] = None
                merge_ret(path, _ret)
                return
            else:
                __salt__['file.remove'](path)
                _ret['changes'] = {'diff': 'Replaced file with a directory'}
                merge_ret(path, _ret)

        _ret = directory(
            path,
            user=user,
            group=group,
            recurse=[],
            dir_mode=dir_mode,
            file_mode=None,
            makedirs=True,
            clean=False,
            require=None)
        merge_ret(path, _ret)

    mng_files, mng_dirs, mng_symlinks, keep = _gen_recurse_managed_files(
        name,
        source,
        keep_symlinks,
        include_pat,
        exclude_pat,
        maxdepth,
        include_empty)

    for srelpath, ltarget in mng_symlinks:
        _ret = symlink(os.path.join(name, srelpath),
                       ltarget,
                       makedirs=True,
                       force=force_symlinks,
                       user=user,
                       group=group,
                       mode=sym_mode)
        if not _ret:
            continue
        merge_ret(os.path.join(name, srelpath), _ret)
    for dirname in mng_dirs:
        manage_directory(dirname)
    for dest, src in mng_files:
        manage_file(dest, src, replace)

    if clean:
        # TODO: Use directory(clean=True) instead
        keep.update(_gen_keep_files(name, require))
        removed = _clean_dir(name, list(keep), exclude_pat)
        if removed:
            if __opts__['test']:
                if ret['result']:
                    ret['result'] = None
                add_comment('removed', removed)
            else:
                ret['changes']['removed'] = removed

    # Flatten comments until salt command line client learns
    # to display structured comments in a readable fashion
    ret['comment'] = '\n'.join('\n#### {0} ####\n{1}'.format(
        k, v if isinstance(v, six.string_types) else '\n'.join(v)
    ) for (k, v) in six.iteritems(ret['comment'])).strip()

    if not ret['comment']:
        ret['comment'] = 'Recursively updated {0}'.format(name)

    if not ret['changes'] and ret['result']:
        ret['comment'] = 'The directory {0} is in the correct state'.format(
            name
        )

    return ret