Hi reader, this is a small and simple guide to RNNs, I will discuss all the basic requirements that you need to get started with RNNs from under neath concepts to code implementation.I will attatch some more resource at the bottom of this post.

### Table of Content:

### What is RNNs?
The R in RNN,stands for recurrent which literally means repeating. Now you will think what is reapting here? Here reapting reffers to the reapting structurre of RNNs. There are different kind of data avilable, like normal tabular data with different features,image data,time series data, text data etc. From them time series data and text data are the data which has a sequence in it.Let me discuss the meaning of sequence for text data. It's very noticeable that every sentance has a sequence of words, and if you change the sequence of that sentance then the sentance wont has same meaning. Because of this reason we cant use MLPs on text data (there are other reasons too like large parameter size,size of i/p word and o/p word can be different etc.).
Now, a RNN have many structure as per the problem requirments like- one to one, one to many,many to one,many to many as fig 1.
<br>
<center></center>
<br>

But, here i will be discussing many to one network.But, keep in mind that a single RNN cell is same in every where, just they are placed in different ways to satisfy the problem requirments.

###  Many to One RNN: 
The structure of a many to many looks like fig2.Application of many to one networks are spam detection, sentiment analysis etc. Now lets dive deep and understand the bare bones of RNN.

<center></center>

Every vertical block with input and output arrows is called a RNN cell, at the beginning it uses a vector with zeros which is named here as $O_0$, this is only for showing the repeating structure of RNN. the circles in the blocks are activation functions.And if you check out carefully,you will observe that there are three type of weights.$W$ which are with input word vector, $W'$ is with RNN cell o/p and finally $W''$ is at the final RNN cell which helps to make the prediction.O/p of RNN cells are taken as an i/p for the next RNN cell,this is to keep the sequence information preserved. Now, $O_i$ is,

> \begin{equation}
 O_i = f(X_iW+O_iW')
 \end{equation}

And at the end, the output of the last RNN cell and the $W''$ are taken and passed through an activation function(here sigmoid is taken) to generate the prediction, $Y' = S(O_nW'')$. And rest of the things are taken care by backpropagation and gradient descent stuff, normal model training.
<br>
> Note, description of all symbols are given in the image itself.

## Utils


```python
import warnings
warnings.filterwarnings("ignore")                     #Ignoring unnecessory warnings

import numpy as np                                  #for large and multi-dimensional arrays
import pandas as pd                                 #for data manipulation and analysis
import nltk                                         #Natural language processing tool-kit

from nltk.corpus import stopwords                   #Stopwords corpus
from nltk.stem import PorterStemmer                 # Stemmer

from sklearn.feature_extraction.text import CountVectorizer          #For Bag of words
from sklearn.feature_extraction.text import TfidfVectorizer          #For TF-IDF
from gensim.models import Word2Vec                                   #For Word2Vec

from tensorflow.keras.layers import Embedding
from tensorflow.keras.preprocessing.sequence import pad_sequences
from tensorflow.keras.models import Sequential
from tensorflow.keras.preprocessing.text import one_hot
from tensorflow.keras.layers import LSTM
from tensorflow.keras.layers import Dropout
from tensorflow.keras.layers import Dense


```

## Loading Data Using Pandas:


```python
import pandas as pd
pd.pandas.set_option('display.max_columns',None)
pd.pandas.set_option('display.max_rows',None)
df = pd.read_csv('resume_data2.csv')
df.head()
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>Unnamed: 0</th>
      <th>cv_txt</th>
      <th>score</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>0</td>
      <td>Your Name November 13, 2019 Short description ...</td>
      <td>7</td>
    </tr>
    <tr>
      <th>1</th>
      <td>1</td>
      <td>Last Updated on 21st June 2019 Debarghya Das d...</td>
      <td>6</td>
    </tr>
    <tr>
      <th>2</th>
      <td>2</td>
      <td>DougHeffernan Passionate Driver EDUCATION Marc...</td>
      <td>9</td>
    </tr>
    <tr>
      <th>3</th>
      <td>3</td>
      <td>Jon Snow HOUSE Stark REALM Kingdom of The Nort...</td>
      <td>9</td>
    </tr>
    <tr>
      <th>4</th>
      <td>4</td>
      <td>Experience Sep/2015 Present Job 4 Employer, Co...</td>
      <td>0</td>
    </tr>
  </tbody>
</table>
</div>




```python
df.drop(['Unnamed: 0'],inplace=True,axis=1)
```

### Some Basic Exploratory Data Analysis:


```python
df.head()
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>cv_txt</th>
      <th>score</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>Your Name November 13, 2019 Short description ...</td>
      <td>7</td>
    </tr>
    <tr>
      <th>1</th>
      <td>Last Updated on 21st June 2019 Debarghya Das d...</td>
      <td>6</td>
    </tr>
    <tr>
      <th>2</th>
      <td>DougHeffernan Passionate Driver EDUCATION Marc...</td>
      <td>9</td>
    </tr>
    <tr>
      <th>3</th>
      <td>Jon Snow HOUSE Stark REALM Kingdom of The Nort...</td>
      <td>9</td>
    </tr>
    <tr>
      <th>4</th>
      <td>Experience Sep/2015 Present Job 4 Employer, Co...</td>
      <td>0</td>
    </tr>
  </tbody>
</table>
</div>




```python
df.shape
```




    (319, 2)




```python
df.isna().sum()
```




    cv_txt    0
    score     0
    dtype: int64



### Data Preprocessing:

Here, we are dropping the data points where `UseId`,`"ProfileName`,`Time`,`Text` are same and keeping the first data point.


```python
# df1 = df1.drop_duplicates(subset={"UserId","ProfileName","Time","Text"}, keep='first', inplace=False)
```

Here, we are keeping the values where `HelpfulnessNumerator` <= `HelpfulnessDenominator`.


```python
# df1 = df1[df1['HelpfulnessNumerator']<=df1['HelpfulnessDenominator']]
```

As we have taken this dataset as a NLP problem, so we are only considering the text column as the i/p data and `score` as the labels.


```python
# list1 = list(df['score'])
# list2 = list(df['cv_txt'])

# score_df = pd.DataFrame(list1,columns = ['score'])
# text_df = pd.DataFrame(list2,columns = ['cv_txt'])

# df2 = pd.concat([text_df,score_df],axis=1)
# df2.head()
```

Checking out some of the text which starts from 500 index and ends at 550 index.


```python
for i in range(1,3):
    print(df['cv_txt'].values[i])
    print('-----------------------------------------------------')
```

    Last Updated on 21st June 2019 Debarghya Das debarghyadas.com| fb.co/dd deedy@fb.com | 607.379.5733 | dd367@cornell.edu EDUCATION CORNELL UNIVERSITY MENG IN COMPUTER SCIENCE Dec 2014 | Ithaca, NY CORNELL UNIVERSITY BS IN COMPUTER SCIENCE May 2014 | Ithaca, NY College of Engineering Magna Cum Laude Cum. GPA: 3.83 / 4.0 Major GPA: 3.9 / 4.0 LA MARTINIERE FOR BOYS Grad. May 2011| Kolkata, India LINKS Facebook:// dd Github:// deedydas LinkedIn:// debarghyadas YouTube:// DeedyDash007 Twitter:// @debarghya_das Quora:// Debarghya-Das COURSEWORK GRADUATE Advanced Machine Learning Open Source Software Engineering Advanced Interactive Graphics Compilers + Practicum Cloud Computing Evolutionary Computation Defending Computer Networks Machine Learning UNDERGRADUATE Information Retrieval Operating Systems Artificial Intelligence + Practicum Functional Programming Computer Graphics + Practicum (Research Asst. & Teaching Asst 2x) Unix Tools and Scripting SKILLS PROGRAMMING Over 5000 lines: Java • Shell • Python • Javascript OCaml • Matlab • Rails • LATEX Over 1000 lines: C • C++ • CSS • PHP • Assembly Familiar: AS3 • iOS • Android • MySQL EXPERIENCE FACEBOOK | SOFTWARE ENGINEER Jan 2015 - Present | New York, NY COURSERA | KPCB FELLOW + SOFTWARE ENGINEERING INTERN June 2014 – Sep 2014 | Mountain View, CA • 52 out of 2500 applicants chosen to be a KPCB Fellow 2014. • Led and shipped Yoda - the admin interface for the new Phoenix platform. • Full-stack developer - Wrote and reviewed code for JS using Backbone, Jade, Stylus and Require and Scala using Play GOOGLE | SOFTWARE ENGINEERING INTERN May 2013 – Aug 2013 | Mountain View, CA • Worked on the YouTube Captions team, in Javascript and Python to plan, to design and develop the full stack to add and edit Automatic Speech Recognition captions. In production. • Created a backbone.js-like framework for the Captions editor. PHABRICATOR | OPEN SOURCE CONTRIBUTOR & TEAM LEADER Jan 2013 – May 2013 | Palo Alto, CA & Ithaca, NY • Phabricator is used daily by Facebook, Dropbox, Quora, Asana and more. • I created the Meme generator and more in PHP and Shell. • Led a team from MIT, Cornell, IC London and UHelsinki for the project. RESEARCH CORNELL ROBOT LEARNING LAB | RESEARCHER Jan 2014 – Jan 2015 | Ithaca, NY Worked with Ashesh Jain and Prof Ashutosh Saxena to create PlanIt, a tool which learns from large scale user preference feedback to plan robot trajectories in human environments. CORNELL PHONETICS LAB | HEAD UNDERGRADUATE RESEARCHER Mar 2012 – May 2013 | Ithaca, NY Led the development of QuickTongue, the first ever breakthrough tongue-controlled game with Prof Sam Tilsen to aid in Linguistics research. AWARDS 2014 top 52/2500 KPCB Engineering Fellow 2014 1st/50 Microsoft Coding Competition, Cornell 2013 National Jump Trading Challenge Finalist 2013 7th/120 CS 3410 Cache Race Bot Tournament 2012 2nd/150 CS 3110 Biannual Intra-Class Bot Tournament 2011 National Indian National Mathematics Olympiad (INMO) Finalist PUBLICATIONS [1] A. Jain, D. Das, and A. Saxena. Planit: A crowdsourcing approach for learning to plan paths from large scale preference feedback. Tech Report, ICRA, in press. [2] S. Tilsen, D. Das, and B. McKee. Real-time articulatory biofeedback with electromagnetic articulography. Linguistics Vanguard, in press. 
    -----------------------------------------------------
    DougHeffernan Passionate Driver EDUCATION March, 2005 Advanced Training | internal  International Parcel Service (IPS) • Completed the IPS aptitude test for aspiring managerial staff • Subsequently decided to keep pursuing his passion for driving and delivering instead 1985 - 1986 Junior College | no degree  Nassau Community College Coursework: College Preparatory English, Introduction to College Biology I, Contemporary Music, The History of Sports in America 1980 - 1984 High School Diploma | ⌀ 1.7 GPA  St. Gregory's High School, Queens, NY Honors Classes: Physical Education, running back (All-County) EXPERIENCE 1994 - now Truck Driver | Queens Area   International Parcel Service (IPS) • Delivered packages to costumers around New York City (mostly Queens) • Practiced solidarity as a union member during a driver strike in the year 2000 • Occasionally chosen to move specialty items such as live animals between NY Zoos • Company-wide record holder for number of days without an ”incident” - no complaints or broken packages • Was profiled in the company newsletter IP-Yes May, 1999 Interim Shift Supervisor | Queens Office   International Parcel Service (IPS) • Brief detour into white collar occupation when the team was in need • Provided temporary support to the office in the absense of supervisor O´Boyle (personal leave of absense) • Scheduled routes for drivers • Managed the budget • Contributed to the motivation of all drivers and loaders 1992 - 1993 Truck Loader | Packages & Cargo   International Parcel Service (IPS) • Loaded all packages securely and in a logical order for the drivers to deliver • Worked closely with the drivers to ensure maximum efficiency and safety • Briefly returned to loading duties due to an unfortunate test-taking incident in 2002 1988 - 1991 Security Guard | Nightclub & Bar  Sour Polly, Queens, NY • Providing rigorous access control of costumers • Making difficult and critical entry decisions • Keeping the customers safe and entertained Queens, NY, May 14th, 2007 February 9, 1965 � 3121 Aberdeen Road, Queens, NY  (718) 555-LOGS � doug.heffernan@ips-nyc.com �  Skills & Strengths Driving • Save and punctual delivery of packages to end consumers • Rudimentary vehicle maintenance task People Skills • Contributes significantly to collegial athmosphere BBQ • Your steaks, your bacons, your bratwursts • Definitely no tofu! Athletics • Tried out for Nassau Rebels semi-pro football team twice Caregiving • Providing home (basement) for father-in-law for years creative problem solver keeping it light ultimate team player modest leader in spirit dependable driver Eager to deepen knowledge about... • ...baloney-darts • ...getting back into shape • ...containing cranky old people Sports Teams 5/5 4/5 5/5 4/5 -6/5 4/5 Languages English Japanese Spanish French Interests & Activities Football (watching, occasionally playing) | Rock Music | Bruce Springsteen | Good Food Bowling Cooper´s Onion Ringers since 1999 Softball  board member from 1998-1999 
    -----------------------------------------------------



```python
# df3 = df2.head(50000)
# df3.head()
```

Here we have created a function which will mark the score from 1 to 3 as negative and 4 to 5 as positive. A we are going to do binary classification that's why we are keeping the labels to positive and negative.


```python
# def partition(x):
#     if x < 3:
#         return 'negative'
#     return 'positive'

# score_upd = df3['score']
# t = score_upd.map(partition)
# df3['score']=t

# df3.head()
```


```python
df.isna().sum()
```




    cv_txt    0
    score     0
    dtype: int64



Here, df_x is the i/p text data and df_y is the i/p lable.


```python
df_x = df['cv_txt']
df_y = df['score']
```


```python
stop_words = set(stopwords.words('english'))
len(stop_words) #finding stop words
```




    179



Here we are doing the real pre-processing,which are like- keeping only the alphabets,making all the alphabets lower,removing all the stop word.


```python
import re
from nltk.corpus import stopwords
from nltk.stem.porter import PorterStemmer
snow = nltk.stem.SnowballStemmer('english')

corpus = []
for i in range(0, len(df)):
    review = re.sub('[^a-zA-Z]', ' ', df['cv_txt'][i])
    review = review.lower()
    review = review.split()
    
    review = [snow.stem(word) for word in review if not word in stopwords.words('english')]
    review = ' '.join(review)
    corpus.append(review)
```


```python
corpus[1]
```




    'last updat st june debarghya das debarghyada com fb co dd deedi fb com dd cornel edu educ cornel univers meng comput scienc dec ithaca ny cornel univers bs comput scienc may ithaca ny colleg engin magna cum laud cum gpa major gpa la martinier boy grad may kolkata india link facebook dd github deedyda linkedin debarghyada youtub deedydash twitter debarghya das quora debarghya das coursework graduat advanc machin learn open sourc softwar engin advanc interact graphic compil practicum cloud comput evolutionari comput defend comput network machin learn undergradu inform retriev oper system artifici intellig practicum function program comput graphic practicum research asst teach asst x unix tool script skill program line java shell python javascript ocaml matlab rail latex line c c css php assembl familiar io android mysql experi facebook softwar engin jan present new york ny coursera kpcb fellow softwar engin intern june sep mountain view ca applic chosen kpcb fellow led ship yoda admin interfac new phoenix platform full stack develop wrote review code js use backbon jade stylus requir scala use play googl softwar engin intern may aug mountain view ca work youtub caption team javascript python plan design develop full stack add edit automat speech recognit caption product creat backbon js like framework caption editor phabric open sourc contributor team leader jan may palo alto ca ithaca ny phabric use daili facebook dropbox quora asana creat meme generat php shell led team mit cornel ic london uhelsinki project research cornel robot learn lab research jan jan ithaca ny work ashesh jain prof ashutosh saxena creat planit tool learn larg scale user prefer feedback plan robot trajectori human environ cornel phonet lab head undergradu research mar may ithaca ny led develop quicktongu first ever breakthrough tongu control game prof sam tilsen aid linguist research award top kpcb engin fellow st microsoft code competit cornel nation jump trade challeng finalist th cs cach race bot tournament nd cs biannual intra class bot tournament nation indian nation mathemat olympiad inmo finalist public jain das saxena planit crowdsourc approach learn plan path larg scale prefer feedback tech report icra press tilsen das b mckee real time articulatori biofeedback electromagnet articulographi linguist vanguard press'




```python
df_x = corpus
```


```python
type(df_x)
```




    list



This is an important step, here we are creating word vectors by doing one hot encoing, and we are only taking 5000 words as the dictionary.


```python
voc_size=5000
onehot_repr=[one_hot(words,voc_size)for words in corpus] 
type(onehot_repr)
```




    list



Padding is one of the most important part before feeding the data to the model. In simple way padding is a way to keep the input size same for all i/p text by adding zeros at the front. Here we are considering each i/p text corpus will be of 400 words. 


```python
sent_length=400
embedded_docs=pad_sequences(onehot_repr,padding='pre',maxlen=sent_length)
print(embedded_docs)
```

    [[   0    0    0 ... 1730 4180   32]
     [   0    0    0 ... 1055 2683 2402]
     [   0    0    0 ... 4087  530  616]
     ...
     [   0    0    0 ... 1551  244 4276]
     [1645 1314 1686 ...  244  244 4276]
     [   0    0    0 ... 1647  244 4276]]


This is the model build using tensorflow 2.0. Think about the whole model like this,

first its a sequencial model starting with a `Embedding` layer (wich is deep learning version of word to vec,or more intuitively WV is an example of `Embedding` and we are use it because we can be trained,it's not static like WV) and input size is 5000x40 and output size of 400x40.Now we created a dropout layer using `Dropout()` which will reduce the chance of overfitting. After that we are taking 100 layers of LSTM and feeding the output of Embedding leyer to each of the LSTM layer . Now, each of the `LSTM` layer will spit out a scaller value and then we will be stacking them out,because of that the output size after `LSTM` layer would be 100. Ten we againg perform a Dropout layer. At the end we have a single activation unit of `sigmoid` (line 8) which will take the 100 sized vector and give a prediction or in general words give a scaler value between 0 to 1. 

Now we are setting the optimixer as `adam` and evaluation matrics as `accurecy` and loss as `binary_crossentropy`(this is because we have taken this as a binary classification problem) using the `model.compile()` method.
At the 10th line we are printing the model architecture using `model.summary()` method.


```python
## Creating model
embedding_vector_features=40
model=Sequential()
model.add(Embedding(voc_size,embedding_vector_features,input_length=sent_length))
model.add(Dropout(0.3))
model.add(LSTM(100))
model.add(Dropout(0.3))
model.add(Dense(1,activation='sigmoid'))
model.compile(loss='binary_crossentropy',optimizer='adam',metrics=['accuracy'])
print(model.summary())
```

    Model: "sequential"
    _________________________________________________________________
    Layer (type)                 Output Shape              Param #   
    =================================================================
    embedding (Embedding)        (None, 400, 40)           200000    
    _________________________________________________________________
    dropout (Dropout)            (None, 400, 40)           0         
    _________________________________________________________________
    lstm (LSTM)                  (None, 100)               56400     
    _________________________________________________________________
    dropout_1 (Dropout)          (None, 100)               0         
    _________________________________________________________________
    dense (Dense)                (None, 1)                 101       
    =================================================================
    Total params: 256,501
    Trainable params: 256,501
    Non-trainable params: 0
    _________________________________________________________________
    None


Some preprocessing before feeding the data. We are doing lable encoding of the `score` column.


```python
# from sklearn.preprocessing import LabelEncoder
# encode = LabelEncoder()
# df_y2 = encode.fit_transform(df_y)
# type(df_y2)
```

This is a important part, where we car converting our data to nd arrays as we cant just input a pandas data frame.


```python
import numpy as np
X_final=np.array(embedded_docs)
y_final=np.array(df_y)
```

### Train Test Split:


```python
from sklearn.model_selection import train_test_split
X_train, X_test, Y_train, Y_test = train_test_split(X_final, y_final, test_size=0.2, random_state=42)
```


```python
#we are feeding the 
model.fit(X_train,Y_train,validation_data=(X_test,Y_test),epochs=100,batch_size=64)
```

    Epoch 1/100
    4/4 [==============================] - 2s 611ms/step - loss: -46.3091 - accuracy: 0.1020 - val_loss: -46.6244 - val_accuracy: 0.1094
    Epoch 2/100
    4/4 [==============================] - 2s 519ms/step - loss: -48.3868 - accuracy: 0.1020 - val_loss: -48.7969 - val_accuracy: 0.1094
    Epoch 3/100
    4/4 [==============================] - 2s 544ms/step - loss: -50.5598 - accuracy: 0.1020 - val_loss: -50.8603 - val_accuracy: 0.1094
    Epoch 4/100
    4/4 [==============================] - 2s 520ms/step - loss: -52.7137 - accuracy: 0.1020 - val_loss: -52.8727 - val_accuracy: 0.1094
    Epoch 5/100
    4/4 [==============================] - 3s 634ms/step - loss: -54.9985 - accuracy: 0.1020 - val_loss: -54.7651 - val_accuracy: 0.1094
    Epoch 6/100
    4/4 [==============================] - 2s 530ms/step - loss: -56.8089 - accuracy: 0.1020 - val_loss: -56.5599 - val_accuracy: 0.1094
    Epoch 7/100
    4/4 [==============================] - 2s 566ms/step - loss: -58.4549 - accuracy: 0.1020 - val_loss: -58.2650 - val_accuracy: 0.1094
    Epoch 8/100
    4/4 [==============================] - 3s 678ms/step - loss: -61.0506 - accuracy: 0.1020 - val_loss: -59.9366 - val_accuracy: 0.1094
    Epoch 9/100
    4/4 [==============================] - 2s 615ms/step - loss: -61.7713 - accuracy: 0.1020 - val_loss: -61.5751 - val_accuracy: 0.1094
    Epoch 10/100
    4/4 [==============================] - 2s 625ms/step - loss: -64.0513 - accuracy: 0.1020 - val_loss: -63.1913 - val_accuracy: 0.1094
    Epoch 11/100
    4/4 [==============================] - 3s 635ms/step - loss: -65.0146 - accuracy: 0.1020 - val_loss: -64.7642 - val_accuracy: 0.1094
    Epoch 12/100
    4/4 [==============================] - 3s 638ms/step - loss: -67.1630 - accuracy: 0.1020 - val_loss: -66.3328 - val_accuracy: 0.1094
    Epoch 13/100
    4/4 [==============================] - 2s 590ms/step - loss: -67.9706 - accuracy: 0.1020 - val_loss: -67.8515 - val_accuracy: 0.1094
    Epoch 14/100
    4/4 [==============================] - 3s 649ms/step - loss: -69.5245 - accuracy: 0.1020 - val_loss: -69.3874 - val_accuracy: 0.1094
    Epoch 15/100
    4/4 [==============================] - 2s 572ms/step - loss: -70.8560 - accuracy: 0.1020 - val_loss: -70.9155 - val_accuracy: 0.1094
    Epoch 16/100
    4/4 [==============================] - 2s 617ms/step - loss: -73.5927 - accuracy: 0.1020 - val_loss: -72.4228 - val_accuracy: 0.1094
    Epoch 17/100
    4/4 [==============================] - 2s 580ms/step - loss: -74.0576 - accuracy: 0.1020 - val_loss: -73.9384 - val_accuracy: 0.1094
    Epoch 18/100
    4/4 [==============================] - 3s 667ms/step - loss: -75.8880 - accuracy: 0.1020 - val_loss: -75.4372 - val_accuracy: 0.1094
    Epoch 19/100
    4/4 [==============================] - 2s 600ms/step - loss: -77.2198 - accuracy: 0.1020 - val_loss: -76.9385 - val_accuracy: 0.1094
    Epoch 20/100
    4/4 [==============================] - 2s 612ms/step - loss: -79.1817 - accuracy: 0.1020 - val_loss: -78.4453 - val_accuracy: 0.1094
    Epoch 21/100
    4/4 [==============================] - 2s 619ms/step - loss: -80.0703 - accuracy: 0.1020 - val_loss: -79.9599 - val_accuracy: 0.1094
    Epoch 22/100
    4/4 [==============================] - 2s 603ms/step - loss: -83.3171 - accuracy: 0.1020 - val_loss: -81.4534 - val_accuracy: 0.1094
    Epoch 23/100
    4/4 [==============================] - 3s 647ms/step - loss: -83.7021 - accuracy: 0.1020 - val_loss: -82.9289 - val_accuracy: 0.1094
    Epoch 24/100
    4/4 [==============================] - 3s 675ms/step - loss: -85.7571 - accuracy: 0.1020 - val_loss: -84.4067 - val_accuracy: 0.1094
    Epoch 25/100
    4/4 [==============================] - 3s 743ms/step - loss: -86.7934 - accuracy: 0.1020 - val_loss: -85.8781 - val_accuracy: 0.1094
    Epoch 26/100
    4/4 [==============================] - 2s 594ms/step - loss: -88.0754 - accuracy: 0.1020 - val_loss: -87.3284 - val_accuracy: 0.1094
    Epoch 27/100
    4/4 [==============================] - 2s 587ms/step - loss: -89.1287 - accuracy: 0.1020 - val_loss: -88.7803 - val_accuracy: 0.1094
    Epoch 28/100
    4/4 [==============================] - 2s 556ms/step - loss: -91.0331 - accuracy: 0.1020 - val_loss: -90.2300 - val_accuracy: 0.1094
    Epoch 29/100
    4/4 [==============================] - 2s 605ms/step - loss: -93.9298 - accuracy: 0.1020 - val_loss: -91.6666 - val_accuracy: 0.1094
    Epoch 30/100
    4/4 [==============================] - 2s 561ms/step - loss: -94.5145 - accuracy: 0.1020 - val_loss: -93.1043 - val_accuracy: 0.1094
    Epoch 31/100
    4/4 [==============================] - 2s 571ms/step - loss: -96.1667 - accuracy: 0.1020 - val_loss: -94.5534 - val_accuracy: 0.1094
    Epoch 32/100
    4/4 [==============================] - 2s 555ms/step - loss: -97.8732 - accuracy: 0.1020 - val_loss: -95.9910 - val_accuracy: 0.1094
    Epoch 33/100
    4/4 [==============================] - 2s 589ms/step - loss: -98.2464 - accuracy: 0.1020 - val_loss: -97.4123 - val_accuracy: 0.1094
    Epoch 34/100
    4/4 [==============================] - 2s 558ms/step - loss: -99.6337 - accuracy: 0.1020 - val_loss: -98.8414 - val_accuracy: 0.1094
    Epoch 35/100
    4/4 [==============================] - 2s 579ms/step - loss: -101.7342 - accuracy: 0.1020 - val_loss: -100.2700 - val_accuracy: 0.1094
    Epoch 36/100
    4/4 [==============================] - 2s 557ms/step - loss: -103.0547 - accuracy: 0.1020 - val_loss: -101.6741 - val_accuracy: 0.1094
    Epoch 37/100
    4/4 [==============================] - 2s 610ms/step - loss: -104.0765 - accuracy: 0.1020 - val_loss: -103.0903 - val_accuracy: 0.1094
    Epoch 38/100
    4/4 [==============================] - 3s 636ms/step - loss: -105.7448 - accuracy: 0.1020 - val_loss: -104.4975 - val_accuracy: 0.1094
    Epoch 39/100
    4/4 [==============================] - 2s 624ms/step - loss: -108.0765 - accuracy: 0.1020 - val_loss: -105.9063 - val_accuracy: 0.1094
    Epoch 40/100
    4/4 [==============================] - 2s 563ms/step - loss: -109.2088 - accuracy: 0.1020 - val_loss: -107.2975 - val_accuracy: 0.1094
    Epoch 41/100
    4/4 [==============================] - 3s 631ms/step - loss: -109.7319 - accuracy: 0.1020 - val_loss: -108.6897 - val_accuracy: 0.1094
    Epoch 42/100
    4/4 [==============================] - 2s 608ms/step - loss: -112.3104 - accuracy: 0.1020 - val_loss: -110.0871 - val_accuracy: 0.1094
    Epoch 43/100
    4/4 [==============================] - 2s 576ms/step - loss: -113.7392 - accuracy: 0.1020 - val_loss: -111.4866 - val_accuracy: 0.1094
    Epoch 44/100
    4/4 [==============================] - 2s 581ms/step - loss: -114.8638 - accuracy: 0.1020 - val_loss: -112.8802 - val_accuracy: 0.1094
    Epoch 45/100
    4/4 [==============================] - 2s 575ms/step - loss: -116.6323 - accuracy: 0.1020 - val_loss: -114.2715 - val_accuracy: 0.1094
    Epoch 46/100
    4/4 [==============================] - 2s 624ms/step - loss: -117.8433 - accuracy: 0.1020 - val_loss: -115.6760 - val_accuracy: 0.1094
    Epoch 47/100
    4/4 [==============================] - 2s 562ms/step - loss: -118.3289 - accuracy: 0.1020 - val_loss: -117.0730 - val_accuracy: 0.1094
    Epoch 48/100
    4/4 [==============================] - 2s 565ms/step - loss: -120.0476 - accuracy: 0.1020 - val_loss: -118.4560 - val_accuracy: 0.1094
    Epoch 49/100
    4/4 [==============================] - 2s 561ms/step - loss: -122.5570 - accuracy: 0.1020 - val_loss: -119.8456 - val_accuracy: 0.1094
    Epoch 50/100
    4/4 [==============================] - 3s 690ms/step - loss: -123.0122 - accuracy: 0.1020 - val_loss: -121.2344 - val_accuracy: 0.1094
    Epoch 51/100
    4/4 [==============================] - 2s 611ms/step - loss: -124.3600 - accuracy: 0.1020 - val_loss: -122.6060 - val_accuracy: 0.1094
    Epoch 52/100
    4/4 [==============================] - 3s 712ms/step - loss: -127.4976 - accuracy: 0.1020 - val_loss: -123.9900 - val_accuracy: 0.1094
    Epoch 53/100
    4/4 [==============================] - 3s 831ms/step - loss: -128.5877 - accuracy: 0.1020 - val_loss: -125.3777 - val_accuracy: 0.1094
    Epoch 54/100
    4/4 [==============================] - 3s 702ms/step - loss: -129.5526 - accuracy: 0.1020 - val_loss: -126.7619 - val_accuracy: 0.1094
    Epoch 55/100
    4/4 [==============================] - 3s 738ms/step - loss: -131.1774 - accuracy: 0.1020 - val_loss: -128.1428 - val_accuracy: 0.1094
    Epoch 56/100
    4/4 [==============================] - 3s 720ms/step - loss: -131.2066 - accuracy: 0.1020 - val_loss: -129.5187 - val_accuracy: 0.1094
    Epoch 57/100
    4/4 [==============================] - 2s 593ms/step - loss: -132.5866 - accuracy: 0.1020 - val_loss: -130.8970 - val_accuracy: 0.1094
    Epoch 58/100
    4/4 [==============================] - 2s 615ms/step - loss: -132.9069 - accuracy: 0.1020 - val_loss: -132.2606 - val_accuracy: 0.1094
    Epoch 59/100
    4/4 [==============================] - 3s 684ms/step - loss: -135.4856 - accuracy: 0.1020 - val_loss: -133.6174 - val_accuracy: 0.1094
    Epoch 60/100
    4/4 [==============================] - 3s 805ms/step - loss: -137.6893 - accuracy: 0.1020 - val_loss: -134.9874 - val_accuracy: 0.1094
    Epoch 61/100
    4/4 [==============================] - 2s 602ms/step - loss: -138.4461 - accuracy: 0.1020 - val_loss: -136.3597 - val_accuracy: 0.1094
    Epoch 62/100
    4/4 [==============================] - 2s 610ms/step - loss: -139.3759 - accuracy: 0.1020 - val_loss: -137.7312 - val_accuracy: 0.1094
    Epoch 63/100
    4/4 [==============================] - 2s 576ms/step - loss: -142.2977 - accuracy: 0.1020 - val_loss: -139.0827 - val_accuracy: 0.1094
    Epoch 64/100
    4/4 [==============================] - 2s 590ms/step - loss: -142.6428 - accuracy: 0.1020 - val_loss: -140.4498 - val_accuracy: 0.1094
    Epoch 65/100
    4/4 [==============================] - 3s 629ms/step - loss: -144.3045 - accuracy: 0.1020 - val_loss: -141.8036 - val_accuracy: 0.1094
    Epoch 66/100
    4/4 [==============================] - 2s 606ms/step - loss: -146.4097 - accuracy: 0.1020 - val_loss: -143.1669 - val_accuracy: 0.1094
    Epoch 67/100
    4/4 [==============================] - 3s 673ms/step - loss: -146.8904 - accuracy: 0.1020 - val_loss: -144.5316 - val_accuracy: 0.1094
    Epoch 68/100
    4/4 [==============================] - 2s 536ms/step - loss: -148.0528 - accuracy: 0.1020 - val_loss: -145.8961 - val_accuracy: 0.1094
    Epoch 69/100
    4/4 [==============================] - 2s 583ms/step - loss: -149.5098 - accuracy: 0.1020 - val_loss: -147.2458 - val_accuracy: 0.1094
    Epoch 70/100
    4/4 [==============================] - 2s 544ms/step - loss: -149.8831 - accuracy: 0.1020 - val_loss: -148.6012 - val_accuracy: 0.1094
    Epoch 71/100
    4/4 [==============================] - 2s 598ms/step - loss: -152.8379 - accuracy: 0.1020 - val_loss: -149.9671 - val_accuracy: 0.1094
    Epoch 72/100
    4/4 [==============================] - 2s 532ms/step - loss: -154.9726 - accuracy: 0.1020 - val_loss: -151.3173 - val_accuracy: 0.1094
    Epoch 73/100
    4/4 [==============================] - 2s 593ms/step - loss: -154.6913 - accuracy: 0.1020 - val_loss: -152.6729 - val_accuracy: 0.1094
    Epoch 74/100
    4/4 [==============================] - 2s 532ms/step - loss: -157.6419 - accuracy: 0.1020 - val_loss: -154.0298 - val_accuracy: 0.1094
    Epoch 75/100
    4/4 [==============================] - 2s 592ms/step - loss: -159.5061 - accuracy: 0.1020 - val_loss: -155.3839 - val_accuracy: 0.1094
    Epoch 76/100
    4/4 [==============================] - 2s 539ms/step - loss: -159.4568 - accuracy: 0.1020 - val_loss: -156.7413 - val_accuracy: 0.1094
    Epoch 77/100
    4/4 [==============================] - 2s 599ms/step - loss: -160.9652 - accuracy: 0.1020 - val_loss: -158.0995 - val_accuracy: 0.1094
    Epoch 78/100
    4/4 [==============================] - 2s 533ms/step - loss: -161.6451 - accuracy: 0.1020 - val_loss: -159.4579 - val_accuracy: 0.1094
    Epoch 79/100
    4/4 [==============================] - 2s 596ms/step - loss: -162.7414 - accuracy: 0.1020 - val_loss: -160.7985 - val_accuracy: 0.1094
    Epoch 80/100
    4/4 [==============================] - 2s 534ms/step - loss: -164.5095 - accuracy: 0.1020 - val_loss: -162.1350 - val_accuracy: 0.1094
    Epoch 81/100
    4/4 [==============================] - 2s 598ms/step - loss: -164.9468 - accuracy: 0.1020 - val_loss: -163.4789 - val_accuracy: 0.1094
    Epoch 82/100
    4/4 [==============================] - 2s 543ms/step - loss: -168.2708 - accuracy: 0.1020 - val_loss: -164.8335 - val_accuracy: 0.1094
    Epoch 83/100
    4/4 [==============================] - 2s 589ms/step - loss: -168.4128 - accuracy: 0.1020 - val_loss: -166.1789 - val_accuracy: 0.1094
    Epoch 84/100
    4/4 [==============================] - 2s 532ms/step - loss: -169.5794 - accuracy: 0.1020 - val_loss: -167.5129 - val_accuracy: 0.1094
    Epoch 85/100
    4/4 [==============================] - 2s 583ms/step - loss: -173.3778 - accuracy: 0.1020 - val_loss: -168.8726 - val_accuracy: 0.1094
    Epoch 86/100
    4/4 [==============================] - 2s 536ms/step - loss: -174.2135 - accuracy: 0.1020 - val_loss: -170.2151 - val_accuracy: 0.1094
    Epoch 87/100
    4/4 [==============================] - 3s 626ms/step - loss: -175.2147 - accuracy: 0.1020 - val_loss: -171.5490 - val_accuracy: 0.1094
    Epoch 88/100
    4/4 [==============================] - 2s 612ms/step - loss: -176.0010 - accuracy: 0.1020 - val_loss: -172.9023 - val_accuracy: 0.1094
    Epoch 89/100
    4/4 [==============================] - 3s 724ms/step - loss: -176.3309 - accuracy: 0.1020 - val_loss: -174.2505 - val_accuracy: 0.1094
    Epoch 90/100
    4/4 [==============================] - 3s 687ms/step - loss: -178.9820 - accuracy: 0.1020 - val_loss: -175.5910 - val_accuracy: 0.1094
    Epoch 91/100
    4/4 [==============================] - 2s 604ms/step - loss: -180.6477 - accuracy: 0.1020 - val_loss: -176.9295 - val_accuracy: 0.1094
    Epoch 92/100
    4/4 [==============================] - 3s 642ms/step - loss: -180.9695 - accuracy: 0.1020 - val_loss: -178.2613 - val_accuracy: 0.1094
    Epoch 93/100
    4/4 [==============================] - 2s 580ms/step - loss: -183.1976 - accuracy: 0.1020 - val_loss: -179.6180 - val_accuracy: 0.1094
    Epoch 94/100
    4/4 [==============================] - 2s 589ms/step - loss: -185.0932 - accuracy: 0.1020 - val_loss: -180.9596 - val_accuracy: 0.1094
    Epoch 95/100
    4/4 [==============================] - 2s 533ms/step - loss: -185.6860 - accuracy: 0.1020 - val_loss: -182.3025 - val_accuracy: 0.1094
    Epoch 96/100
    4/4 [==============================] - 2s 620ms/step - loss: -187.5057 - accuracy: 0.1020 - val_loss: -183.6519 - val_accuracy: 0.1094
    Epoch 97/100
    4/4 [==============================] - 3s 659ms/step - loss: -186.8765 - accuracy: 0.1020 - val_loss: -184.9966 - val_accuracy: 0.1094
    Epoch 98/100
    4/4 [==============================] - 3s 709ms/step - loss: -189.9252 - accuracy: 0.1020 - val_loss: -186.3587 - val_accuracy: 0.1094
    Epoch 99/100
    4/4 [==============================] - 3s 687ms/step - loss: -192.4431 - accuracy: 0.1020 - val_loss: -187.7266 - val_accuracy: 0.1094
    Epoch 100/100
    4/4 [==============================] - 3s 657ms/step - loss: -191.8241 - accuracy: 0.1020 - val_loss: -189.1482 - val_accuracy: 0.1094





    <tensorflow.python.keras.callbacks.History at 0x282f316fe10>




```python
y_pred_lstm = model.predict_classes(X_test)
```


```python
from sklearn.metrics import accuracy_score
accuracy_score(Y_test,y_pred_lstm)
```




    0.109375




```python

```

### TF-IDF:


```python
final_tf = df_x
tf_idf = TfidfVectorizer(ngram_range=(1,2),max_features=5000)
tf_data = tf_idf.fit_transform(final_tf)
print(tf_data[1])
```

      (0, 3477)	0.03451900530181986
      (0, 4644)	0.04553529853589914
      (0, 2397)	0.08365062460886882
      (0, 3075)	0.04720353200871461
      (0, 4332)	0.04412482723197255
      (0, 1253)	0.04720353200871461
      (0, 1207)	0.030508993769107727
      (0, 3178)	0.04412482723197255
      (0, 2677)	0.04412482723197255
      (0, 1820)	0.0882496544639451
      (0, 4786)	0.09440706401742922
      (0, 2804)	0.09440706401742922
      (0, 1490)	0.0882496544639451
      (0, 4989)	0.04412482723197255
      (0, 2897)	0.03998918509561814
      (0, 2237)	0.04086126865526724
      (0, 3421)	0.03919303518245176
      (0, 3977)	0.03324019025330949
      (0, 258)	0.037782563878525174
      (0, 2997)	0.02305628441298117
      (0, 741)	0.04086126865526724
      (0, 4023)	0.10680458462718473
      (0, 4063)	0.04553529853589914
      (0, 2990)	0.06814515529412091
      (0, 2547)	0.06648038050661897
      :	:
      (0, 1000)	0.07997837019123628
      (0, 2553)	0.035483048950987035
      (0, 1482)	0.08648703679197389
      (0, 775)	0.013359417434252
      (0, 2676)	0.10327832007982579
      (0, 2946)	0.2509518738266065
      (0, 2223)	0.2462264238368193
      (0, 1118)	0.031773062719522314
      (0, 3759)	0.0381770720252551
      (0, 846)	0.1039319431338827
      (0, 1408)	0.038460648258251226
      (0, 936)	0.34471699337154704
      (0, 1096)	0.12258380596580172
      (0, 749)	0.01266194070840155
      (0, 1688)	0.0882496544639451
      (0, 780)	0.019664872681763663
      (0, 1044)	0.22767649267949572
      (0, 2294)	0.04052411402420102
      (0, 4125)	0.05987387102141208
      (0, 4680)	0.02560183148382079
      (0, 2402)	0.03600604568761869
      (0, 332)	0.023187940668397387
      (0, 1753)	0.028176741948372476
      (0, 4660)	0.02987500573725461
      (0, 1409)	0.010177758526163367



```python
pickle.dump(tf_idf, open("tfidf_vectorizer.pkl", "wb"))
```


```python
v = pickle.load(open('tfidf_vectorizer.pkl','rb'))
```


```python
tf_data.get_shape()
```




    (319, 5000)




```python
features = tf_idf.get_feature_names()
features[:10]
```




    ['aa',
     'ab',
     'abap',
     'abap kolkata',
     'abc',
     'abil',
     'abil quick',
     'abil work',
     'abl',
     'abroad']




```python
from sklearn.preprocessing import LabelEncoder
encode = LabelEncoder()
df_y1 = encode.fit_transform(df_y)
type(df_y1)
```




    numpy.ndarray




```python
# pd.DataFrame(['df_y1'])
```


```python
SEED =42
from sklearn.model_selection import train_test_split
x_train,x_val,y_train,y_val = train_test_split(tf_data,df_y,test_size = 0.2,random_state=SEED)
```


```python
from sklearn.naive_bayes import MultinomialNB
nb_model = MultinomialNB().fit(x_train, y_train)

y_pred_nb=nb_model.predict(x_val)
```


```python
from sklearn.metrics import accuracy_score
accuracy_score(y_val, y_pred_nb)
```




    0.09375



### Rf:


```python
from sklearn.ensemble import RandomForestClassifier
rf = RandomForestClassifier()
rf_model = rf.fit(x_train,y_train)
y_pred_rf = rf_model.predict(x_val)
accuracy_score(y_val,y_pred_rf)
```




    0.125




```python
import pickle
pickle.dump(rf_model,open('rf_score_model.pkl','wb'))
```


```python
#TESTING WITH SOME DATE
test = 'last updat st june debarghya das debarghyada com fb co dd deedi fb com dd cornel edu educ cornel univers meng comput scienc dec ithaca ny cornel univers bs comput scienc may ithaca ny colleg engin magna cum laud cum gpa major gpa la martinier boy grad may kolkata india link facebook dd github deedyda linkedin debarghyada youtub deedydash twitter debarghya das quora debarghya das coursework graduat advanc machin learn open sourc softwar engin advanc interact graphic compil practicum cloud comput evolutionari comput defend comput network machin learn undergradu inform retriev oper system artifici intellig practicum function program comput graphic practicum research asst teach asst x unix tool script skill program line java shell python javascript ocaml matlab rail latex line c c css php assembl familiar io android mysql experi facebook softwar engin jan present new york ny coursera kpcb fellow softwar engin intern june sep mountain view ca applic chosen kpcb fellow led ship yoda admin interfac new phoenix platform full stack develop wrote review code js use backbon jade stylus requir scala use play googl softwar engin intern may aug mountain view ca work youtub caption team javascript python plan design develop full stack add edit automat speech recognit caption product creat backbon js like framework caption editor phabric open sourc contributor team leader jan may palo alto ca ithaca ny phabric use daili facebook dropbox quora asana creat meme generat php shell led team mit cornel ic london uhelsinki project research cornel robot learn lab research jan jan ithaca ny work ashesh jain prof ashutosh saxena creat planit tool learn larg scale user prefer feedback plan robot trajectori human environ cornel phonet lab head undergradu research mar may ithaca ny led develop quicktongu first ever breakthrough tongu control game prof sam tilsen aid linguist research award top kpcb engin fellow st microsoft code competit cornel nation jump trade challeng finalist th cs cach race bot tournament nd cs biannual intra class bot tournament nation indian nation mathemat olympiad inmo finalist public jain das saxena planit crowdsourc approach learn plan path larg scale prefer feedback tech report icra press tilsen das b mckee real time articulatori biofeedback electromagnet articulographi linguist vanguard press'
snow = nltk.stem.SnowballStemmer('english')

corpus_test = []
# for i in range(0, len(df)):
review = re.sub('[^a-zA-Z]', ' ', test)
review = review.lower()
review = review.split()
    
review = [snow.stem(word) for word in review if not word in stopwords.words('english')]
review = ' '.join(review)
corpus_test.append(review)
```


```python
final_tf_test = corpus_test
# tf_idf_test = TfidfVectorizer(ngram_range=(1,2),max_features=5000)
tf_data_test = v.transform(final_tf_test)
tf_data_test.get_shape()
```




    (1, 5000)




```python
loaded_model = pickle.load(open('rf_score_model.pkl', 'rb'))
result = loaded_model.predict(tf_data_test)
print(result)
```

    [6]


### rf with rscv:


```python
# Number of trees in random forest
n_estimators = [int(x) for x in np.linspace(start = 200, stop = 2000, num = 10)]
# Number of features to consider at every split
max_features = ['auto', 'sqrt']
# Maximum number of levels in tree
max_depth = [int(x) for x in np.linspace(10, 110, num = 11)]
max_depth.append(None)
# Minimum number of samples required to split a node
min_samples_split = [2, 5, 10]
# Minimum number of samples required at each leaf node
min_samples_leaf = [1, 2, 4]
# Method of selecting samples for training each tree
bootstrap = [True, False]



random_grid = {'n_estimators': n_estimators,
               'max_features': max_features,
               'max_depth': max_depth,
               'min_samples_split': min_samples_split,
               'min_samples_leaf': min_samples_leaf,
               'bootstrap': bootstrap}

cls_rf_rscv = RandomForestClassifier()
from sklearn.model_selection import RandomizedSearchCV

random_search_rf = RandomizedSearchCV(cls_rf_rscv, random_grid,n_iter=5, n_jobs=1, cv=5,verbose=2)
random_search_rf.fit(x_train,y_train)
```

    Fitting 5 folds for each of 5 candidates, totalling 25 fits
    [CV] n_estimators=2000, min_samples_split=10, min_samples_leaf=4, max_features=auto, max_depth=110, bootstrap=False 


    [Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.


    [CV]  n_estimators=2000, min_samples_split=10, min_samples_leaf=4, max_features=auto, max_depth=110, bootstrap=False, total=   6.0s
    [CV] n_estimators=2000, min_samples_split=10, min_samples_leaf=4, max_features=auto, max_depth=110, bootstrap=False 


    [Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    5.9s remaining:    0.0s


    [CV]  n_estimators=2000, min_samples_split=10, min_samples_leaf=4, max_features=auto, max_depth=110, bootstrap=False, total=   6.1s
    [CV] n_estimators=2000, min_samples_split=10, min_samples_leaf=4, max_features=auto, max_depth=110, bootstrap=False 
    [CV]  n_estimators=2000, min_samples_split=10, min_samples_leaf=4, max_features=auto, max_depth=110, bootstrap=False, total=   7.5s
    [CV] n_estimators=2000, min_samples_split=10, min_samples_leaf=4, max_features=auto, max_depth=110, bootstrap=False 
    [CV]  n_estimators=2000, min_samples_split=10, min_samples_leaf=4, max_features=auto, max_depth=110, bootstrap=False, total=   6.5s
    [CV] n_estimators=2000, min_samples_split=10, min_samples_leaf=4, max_features=auto, max_depth=110, bootstrap=False 
    [CV]  n_estimators=2000, min_samples_split=10, min_samples_leaf=4, max_features=auto, max_depth=110, bootstrap=False, total=   6.4s
    [CV] n_estimators=1800, min_samples_split=2, min_samples_leaf=2, max_features=auto, max_depth=80, bootstrap=True 
    [CV]  n_estimators=1800, min_samples_split=2, min_samples_leaf=2, max_features=auto, max_depth=80, bootstrap=True, total=   7.1s
    [CV] n_estimators=1800, min_samples_split=2, min_samples_leaf=2, max_features=auto, max_depth=80, bootstrap=True 
    [CV]  n_estimators=1800, min_samples_split=2, min_samples_leaf=2, max_features=auto, max_depth=80, bootstrap=True, total=   9.2s
    [CV] n_estimators=1800, min_samples_split=2, min_samples_leaf=2, max_features=auto, max_depth=80, bootstrap=True 
    [CV]  n_estimators=1800, min_samples_split=2, min_samples_leaf=2, max_features=auto, max_depth=80, bootstrap=True, total=   6.8s
    [CV] n_estimators=1800, min_samples_split=2, min_samples_leaf=2, max_features=auto, max_depth=80, bootstrap=True 
    [CV]  n_estimators=1800, min_samples_split=2, min_samples_leaf=2, max_features=auto, max_depth=80, bootstrap=True, total=   7.1s
    [CV] n_estimators=1800, min_samples_split=2, min_samples_leaf=2, max_features=auto, max_depth=80, bootstrap=True 
    [CV]  n_estimators=1800, min_samples_split=2, min_samples_leaf=2, max_features=auto, max_depth=80, bootstrap=True, total=   7.1s
    [CV] n_estimators=1600, min_samples_split=10, min_samples_leaf=4, max_features=sqrt, max_depth=20, bootstrap=False 
    [CV]  n_estimators=1600, min_samples_split=10, min_samples_leaf=4, max_features=sqrt, max_depth=20, bootstrap=False, total=   6.0s
    [CV] n_estimators=1600, min_samples_split=10, min_samples_leaf=4, max_features=sqrt, max_depth=20, bootstrap=False 
    [CV]  n_estimators=1600, min_samples_split=10, min_samples_leaf=4, max_features=sqrt, max_depth=20, bootstrap=False, total=   4.8s
    [CV] n_estimators=1600, min_samples_split=10, min_samples_leaf=4, max_features=sqrt, max_depth=20, bootstrap=False 
    [CV]  n_estimators=1600, min_samples_split=10, min_samples_leaf=4, max_features=sqrt, max_depth=20, bootstrap=False, total=   5.8s
    [CV] n_estimators=1600, min_samples_split=10, min_samples_leaf=4, max_features=sqrt, max_depth=20, bootstrap=False 
    [CV]  n_estimators=1600, min_samples_split=10, min_samples_leaf=4, max_features=sqrt, max_depth=20, bootstrap=False, total=   9.4s
    [CV] n_estimators=1600, min_samples_split=10, min_samples_leaf=4, max_features=sqrt, max_depth=20, bootstrap=False 
    [CV]  n_estimators=1600, min_samples_split=10, min_samples_leaf=4, max_features=sqrt, max_depth=20, bootstrap=False, total=   5.2s
    [CV] n_estimators=400, min_samples_split=10, min_samples_leaf=1, max_features=sqrt, max_depth=10, bootstrap=True 
    [CV]  n_estimators=400, min_samples_split=10, min_samples_leaf=1, max_features=sqrt, max_depth=10, bootstrap=True, total=   1.2s
    [CV] n_estimators=400, min_samples_split=10, min_samples_leaf=1, max_features=sqrt, max_depth=10, bootstrap=True 
    [CV]  n_estimators=400, min_samples_split=10, min_samples_leaf=1, max_features=sqrt, max_depth=10, bootstrap=True, total=   1.2s
    [CV] n_estimators=400, min_samples_split=10, min_samples_leaf=1, max_features=sqrt, max_depth=10, bootstrap=True 
    [CV]  n_estimators=400, min_samples_split=10, min_samples_leaf=1, max_features=sqrt, max_depth=10, bootstrap=True, total=   1.1s
    [CV] n_estimators=400, min_samples_split=10, min_samples_leaf=1, max_features=sqrt, max_depth=10, bootstrap=True 
    [CV]  n_estimators=400, min_samples_split=10, min_samples_leaf=1, max_features=sqrt, max_depth=10, bootstrap=True, total=   1.2s
    [CV] n_estimators=400, min_samples_split=10, min_samples_leaf=1, max_features=sqrt, max_depth=10, bootstrap=True 
    [CV]  n_estimators=400, min_samples_split=10, min_samples_leaf=1, max_features=sqrt, max_depth=10, bootstrap=True, total=   1.5s
    [CV] n_estimators=1400, min_samples_split=10, min_samples_leaf=4, max_features=sqrt, max_depth=10, bootstrap=True 
    [CV]  n_estimators=1400, min_samples_split=10, min_samples_leaf=4, max_features=sqrt, max_depth=10, bootstrap=True, total=   4.1s
    [CV] n_estimators=1400, min_samples_split=10, min_samples_leaf=4, max_features=sqrt, max_depth=10, bootstrap=True 
    [CV]  n_estimators=1400, min_samples_split=10, min_samples_leaf=4, max_features=sqrt, max_depth=10, bootstrap=True, total=   4.1s
    [CV] n_estimators=1400, min_samples_split=10, min_samples_leaf=4, max_features=sqrt, max_depth=10, bootstrap=True 
    [CV]  n_estimators=1400, min_samples_split=10, min_samples_leaf=4, max_features=sqrt, max_depth=10, bootstrap=True, total=   4.3s
    [CV] n_estimators=1400, min_samples_split=10, min_samples_leaf=4, max_features=sqrt, max_depth=10, bootstrap=True 
    [CV]  n_estimators=1400, min_samples_split=10, min_samples_leaf=4, max_features=sqrt, max_depth=10, bootstrap=True, total=   4.0s
    [CV] n_estimators=1400, min_samples_split=10, min_samples_leaf=4, max_features=sqrt, max_depth=10, bootstrap=True 
    [CV]  n_estimators=1400, min_samples_split=10, min_samples_leaf=4, max_features=sqrt, max_depth=10, bootstrap=True, total=   4.0s


    [Parallel(n_jobs=1)]: Done  25 out of  25 | elapsed:  2.1min finished





    RandomizedSearchCV(cv=5, estimator=RandomForestClassifier(), n_iter=5, n_jobs=1,
                       param_distributions={'bootstrap': [True, False],
                                            'max_depth': [10, 20, 30, 40, 50, 60,
                                                          70, 80, 90, 100, 110,
                                                          None],
                                            'max_features': ['auto', 'sqrt'],
                                            'min_samples_leaf': [1, 2, 4],
                                            'min_samples_split': [2, 5, 10],
                                            'n_estimators': [200, 400, 600, 800,
                                                             1000, 1200, 1400, 1600,
                                                             1800, 2000]},
                       verbose=2)




```python
random_search_rf.best_params_
```




    {'n_estimators': 400,
     'min_samples_split': 10,
     'min_samples_leaf': 1,
     'max_features': 'sqrt',
     'max_depth': 10,
     'bootstrap': True}




```python
base_model = RandomForestClassifier(n_estimators= 2000,
                                     min_samples_split= 10,
                                     min_samples_leaf= 1,
                                     max_features= 'sqrt',
                                     max_depth= 90,
                                     bootstrap= True,
                                    random_state = SEED)
base_model.fit(x_train, y_train)
```




    RandomForestClassifier(max_depth=90, max_features='sqrt', min_samples_split=10,
                           n_estimators=2000, random_state=42)




```python
y_pred_rf_rscv = base_model.predict(x_val)
```


```python
# y_pred_rf_rscv = rf_rscv_model.predict(x_val)
accuracy_score(y_val,y_pred_rf_rscv)
```




    0.078125



#### fdjbklbb bnmn,


```python

```


```python

```


```python

```


```python

```


```python

```


```python

```


```python

```


```python

```


```python

```


```python

```


```python

```


```python

```


```python

```


```python

```


```python

```


```python

```


```python

```


```python

```


```python

```


```python

```


```python

```


```python

```


```python

```


```python

```


```python

```


```python

```


```python

```


```python

```


```python

```


```python

```


```python

```


```python

```


```python

```


```python

```


```python

```


```python

```


```python

```


```python

```


```python

```


```python

```
