REF: catch more specific exceptions in Series.__setitem__ 
- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] whatsnew entry

In a few cases we can avoid some com.is_bool_indexer or lib.infer_dtype checks
cc @mroeschke 

```
lsof: WARNING: can't stat() nsfs file system /run/docker/netns/e6f7fa210b0b
      Output information may be incomplete.
lsof: WARNING: can't stat() overlay file system /var/lib/docker/overlay2/f8dcbf9270f81fe4daa26f1802aa3e8855accbf4a344b2a7df4f0e97bde6c398/merged
      Output information may be incomplete.
lsof: WARNING: can't stat() nsfs file system /run/docker/netns/634339963c8a
      Output information may be incomplete.
lsof: WARNING: can't stat() overlay file system /var/lib/docker/overlay2/5a39676f965bf508d62143609ce3c0cdc9f8534fb9c6ac8e3eb5a37fb6895649/merged
      Output information may be incomplete.

[...]

E           AssertionError: Caused unexpected warning(s): [('ResourceWarning', ResourceWarning('unclosed file <_io.BufferedRandom name=16>'), '/usr/share/miniconda/envs/pandas-dev/lib/python3.8/inspect.py', 744), ('ResourceWarning', ResourceWarning('unclosed file <_io.BufferedRandom name=17>'), '/usr/share/miniconda/envs/pandas-dev/lib/python3.8/inspect.py', 744), ('ResourceWarning', ResourceWarning('unclosed file <_io.BufferedRandom name=18>'), '/usr/share/miniconda/envs/pandas-dev/lib/python3.8/inspect.py', 744), ('ResourceWarning', ResourceWarning('unclosed file <_io.BufferedRandom name=20>'), '/usr/share/miniconda/envs/pandas-dev/lib/python3.8/inspect.py', 744), ('ResourceWarning', ResourceWarning('unclosed file <_io.BufferedRandom name=19>'), '/usr/share/miniconda/envs/pandas-dev/lib/python3.8/inspect.py', 744), ('ResourceWarning', ResourceWarning('unclosed file <_io.BufferedRandom name=21>'), '/usr/share/miniconda/envs/pandas-dev/lib/python3.8/inspect.py', 744), ('ResourceWarning', ResourceWarning('unclosed file <_io.BufferedRandom name=23>'), '/usr/share/miniconda/envs/pandas-dev/lib/python3.8/inspect.py', 744), ('ResourceWarning', ResourceWarning('unclosed file <_io.BufferedRandom name=22>'), '/usr/share/miniconda/envs/pandas-dev/lib/python3.8/inspect.py', 744)]

``` > cc @mroeschke
> 
> ```
> lsof: WARNING: can't stat() nsfs file system /run/docker/netns/e6f7fa210b0b
>       Output information may be incomplete.
> lsof: WARNING: can't stat() overlay file system /var/lib/docker/overlay2/f8dcbf9270f81fe4daa26f1802aa3e8855accbf4a344b2a7df4f0e97bde6c398/merged
>       Output information may be incomplete.
> lsof: WARNING: can't stat() nsfs file system /run/docker/netns/634339963c8a
>       Output information may be incomplete.
> lsof: WARNING: can't stat() overlay file system /var/lib/docker/overlay2/5a39676f965bf508d62143609ce3c0cdc9f8534fb9c6ac8e3eb5a37fb6895649/merged
>       Output information may be incomplete.
> 
> [...]
> 
> E           AssertionError: Caused unexpected warning(s): [('ResourceWarning', ResourceWarning('unclosed file <_io.BufferedRandom name=16>'), '/usr/share/miniconda/envs/pandas-dev/lib/python3.8/inspect.py', 744), ('ResourceWarning', ResourceWarning('unclosed file <_io.BufferedRandom name=17>'), '/usr/share/miniconda/envs/pandas-dev/lib/python3.8/inspect.py', 744), ('ResourceWarning', ResourceWarning('unclosed file <_io.BufferedRandom name=18>'), '/usr/share/miniconda/envs/pandas-dev/lib/python3.8/inspect.py', 744), ('ResourceWarning', ResourceWarning('unclosed file <_io.BufferedRandom name=20>'), '/usr/share/miniconda/envs/pandas-dev/lib/python3.8/inspect.py', 744), ('ResourceWarning', ResourceWarning('unclosed file <_io.BufferedRandom name=19>'), '/usr/share/miniconda/envs/pandas-dev/lib/python3.8/inspect.py', 744), ('ResourceWarning', ResourceWarning('unclosed file <_io.BufferedRandom name=21>'), '/usr/share/miniconda/envs/pandas-dev/lib/python3.8/inspect.py', 744), ('ResourceWarning', ResourceWarning('unclosed file <_io.BufferedRandom name=23>'), '/usr/share/miniconda/envs/pandas-dev/lib/python3.8/inspect.py', 744), ('ResourceWarning', ResourceWarning('unclosed file <_io.BufferedRandom name=22>'), '/usr/share/miniconda/envs/pandas-dev/lib/python3.8/inspect.py', 744)]
> ```

Thanks I'll look into it. The saga continues...