<p>You should use a different algorithm. There has been a lot of research on how to speed up feature selection. The RFE's computational complexity is prohibitive for a large set of features. You should consider using appoaches for high dimentional data, such as <strong>FBED</strong> (Forward-Backward-Early-Dropping), <strong>OMP</strong> (Orthogonal-Matching-Pursuit), <strong>SES</strong> (Statistically-Equivalent-Signatures), <strong>LASSO</strong> etc.</p>

<p>Fbed <a href="https://arxiv.org/abs/1705.10770" rel="nofollow noreferrer">https://arxiv.org/abs/1705.10770</a></p>

<p>OMP <a href="https://arxiv.org/abs/2004.00281" rel="nofollow noreferrer">https://arxiv.org/abs/2004.00281</a></p>

<p>SES <a href="https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-018-2023-7" rel="nofollow noreferrer">https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-018-2023-7</a></p>

<p>LASSO <a href="https://ieeexplore.ieee.org/document/7887916" rel="nofollow noreferrer">https://ieeexplore.ieee.org/document/7887916</a></p>
How can I speed up Recursive Feature Elimination on 6,100,000 Features?
<p>I am trying to obtain a ranking of features from a rather large set of features (~6,100,000) in sklearn. Here's the code I have thus far:</p>

<pre><code>train, test = train_test_split(rows, test_size=0.2, random_state=310)
train, val = train_test_split(train, test_size=0.25, random_state=310)
train_target = [i[-1] for i in train]

svc = SVC(verbose=5, random_state=310, kernel='linear')
svc.fit([i[1:-1] for i in train], train_target)

model=svc
rfe = RFE(model, verbose=5, step=1, n_features_to_select=1)
rfe.fit([i[1:-1] for i in train], train_target)
rank = rfe.ranking_
</code></pre>

<p>Each training of the model takes ~10 minutes. for 6,100,000 features that means <em>decades</em> of computation time. Actually 115.9 years. Any better way to do this? I know rfe requires the results of the last elimination, but is there any way I can speed this through parallelizing up or obtain a ranking differently? I can use thousands of nodes (Thanks company I work for!) so any kind of parallelism would be awesome!</p>

<p>I do have the list coefficients of the linear SVM's hyperplane. Ordering those is easy enough, but the paper which this is being done for is going to be reviewed by a Stanford data science professor and he has a strong reservation against using non-ranking algorithms for ranking....and non-Stanford alums like me. :P</p>

<p>I can take a larger <code>step</code> but that would remove the ability to actually rank all features. rather I would rank groups of 100,000 or 10,000 features which isn't super helpful.</p>

<p>EDIT: nSV might be useful so I've included it below:</p>

<pre><code>obj = -163.983323, rho = -0.999801
nSV = 182, nBSV = 148
Total nSV = 182
</code></pre>