<p dir="auto">IsolationForest is implemented with a bagging of ExtraTreeRegressor and the author of extrmely randomized trees propose  a way to generate random (binary) splits by selecting a random subset of<br>
their possible values.<br>
Can IsolationForest handle with categorical attributes in sklearn？</p>
<p dir="auto">Ref.<br>
<a href="https://orbi.uliege.be/bitstream/2268/9357/1/geurts-mlj-advance.pdf" rel="nofollow">https://orbi.uliege.be/bitstream/2268/9357/1/geurts-mlj-advance.pdf</a></p>
<p dir="auto">I am planning on submitting several PRs in an attempt to merge <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="97688031" data-permission-text="Title is private" data-url="https://github.com/scikit-learn/scikit-learn/issues/5041" data-hovercard-type="pull_request" data-hovercard-url="/scikit-learn/scikit-learn/pull/5041/hovercard" href="https://github.com/scikit-learn/scikit-learn/pull/5041">#5041</a> in slowly, with the ultimate goal being a clean implementation of multithreaded decision tree building so that Gradient Boosting can be faster. With one of the main concepts merged (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="104484815" data-permission-text="Title is private" data-url="https://github.com/scikit-learn/scikit-learn/issues/5203" data-hovercard-type="pull_request" data-hovercard-url="/scikit-learn/scikit-learn/pull/5203/hovercard" href="https://github.com/scikit-learn/scikit-learn/pull/5203">#5203</a>), here is a list of separate PRs which I'd like to merge in the near future.</p>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> Reorganize _tree.pyx into several files (see PR <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="105451553" data-permission-text="Title is private" data-url="https://github.com/scikit-learn/scikit-learn/issues/5230" data-hovercard-type="pull_request" data-hovercard-url="/scikit-learn/scikit-learn/pull/5230/hovercard" href="https://github.com/scikit-learn/scikit-learn/pull/5230">#5230</a>)[merged]</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> Add proxy impurity improvement methods to both Gini and entropy (see PR <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="105609300" data-permission-text="Title is private" data-url="https://github.com/scikit-learn/scikit-learn/issues/5233" data-hovercard-type="pull_request" data-hovercard-url="/scikit-learn/scikit-learn/pull/5233/hovercard" href="https://github.com/scikit-learn/scikit-learn/pull/5233">#5233</a>)[closed]</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> Reevaluate constant feature caching [closed]</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> Support sparse data for gradient boosting (see PR <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="106017928" data-permission-text="Title is private" data-url="https://github.com/scikit-learn/scikit-learn/issues/5252" data-hovercard-type="pull_request" data-hovercard-url="/scikit-learn/scikit-learn/pull/5252/hovercard" href="https://github.com/scikit-learn/scikit-learn/pull/5252">#5252</a>)</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> Add caching of computation between different split levels to avoid recomputation</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> Ensure feature importance converge in ensemble (see PR <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="106159677" data-permission-text="Title is private" data-url="https://github.com/scikit-learn/scikit-learn/issues/5261" data-hovercard-type="pull_request" data-hovercard-url="/scikit-learn/scikit-learn/pull/5261/hovercard" href="https://github.com/scikit-learn/scikit-learn/pull/5261">#5261</a>)</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> Add tests to ensure the correctness of impurity values, wrt hand-computed values on toy data.</li>
</ul>
<p dir="auto">Longer range goals which I'd like to work towards (but have no clear plan as of right now) are the following:</p>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> Add an approximate splitter</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> Add multithreading support for single decision trees</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> Add a partial fit method for tree building</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> Support categorical variables</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> Support missing values</li>
</ul>
<p dir="auto">At this point, it will be clearer to me what specific changes to Splitter, Criteria, and TreeBuilder need to be added to make multithreading a possibility. <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/glouppe/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/glouppe">@glouppe</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/arjoly/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/arjoly">@arjoly</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/GaelVaroquaux/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/GaelVaroquaux">@GaelVaroquaux</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/pprett/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/pprett">@pprett</a> if you have any comments, I'd love to hear them.</p>