<p>Happy news for you, there is.</p>

<p>A package called <strong>"SHAP"</strong> (<em>SHapley Additive exPlanation</em>) was recently released just for that purpose.
<a href="https://github.com/slundberg/shap" rel="nofollow noreferrer">Here's a link</a> to the github.</p>

<p>It supports visualization of complicated models (which are hard to intuitively explain) like boosted trees (and XGBOOST in particular!)</p>

<p>It can show you "real" feature importance which is better than the <code>"gain"</code>, <code>"weight"</code>, and <code>"cover"</code> <em>xgboost</em> supplies as they are not consistent.</p>

<p>You can read all about why SHAP is better for feature evaluation <a href="https://arxiv.org/pdf/1802.03888.pdf" rel="nofollow noreferrer">here</a>.</p>

<p>It will be hard to give you code that will work for you, but there is a good documentation and you should write one that suits you.</p>

<p>Here's the guide lines of building your first graph:</p>

<pre><code>import shap
import xgboost as xgb

# Assume X_train and y_train are both features and labels of data samples

dtrain = xgb.DMatrix(X_train, label=y_train, feature_names=feature_names, weight=weights_trn)

# Train your xgboost model
bst = xgb.train(params0, dtrain, num_boost_round=2500, evals=watchlist, early_stopping_rounds=200)

# "explainer" object of shap
explainer = shap.TreeExplainer(bst)

# "Values you explain, I took them from my training set but you can "explain" here what ever you want
shap_values = explainer.shap_values(X_test)

shap.summary_plot(shap_values, X_test)
shap.summary_plot(shap_values, X_test, plot_type="bar")
</code></pre>

<p>To plot the "<em>Why a certain sample got its score</em>" you can either use built in SHAP function for it (only works on a Jupyter Notebook). <a href="https://github.com/slundberg/shap#tree-ensemble-example-with-treeexplainer-xgboostlightgbmcatboostscikit-learn-models" rel="nofollow noreferrer">Perfect example here</a></p>

<p>I personally wrote a function that will plot it using <code>matplotlib</code>, which will take some effort.</p>

<p>Here is an example of a plot I've made using the shap values (features are confidential so all erased)
<a href="https://i.stack.imgur.com/b8E7z.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/b8E7z.png" alt="enter image description here"></a></p>

<p>You can see a 97% prediction to be <code>label=1</code> and each feature and how much it added or negate from the log-loss, for that specific sample.</p>
Visualize strengths and weaknesses of a sample from pre-trained model
<p>Let's say I'm trying to predict an apartment price. So, I have a lot of labeled data, where on each apartment I have features that could affect the price like:</p>

<ul>
<li>city</li>
<li>street</li>
<li>floor</li>
<li>year built</li>
<li>socioeconomic status </li>
<li>square feet </li>
<li>etc.</li>
</ul>

<p>And I train a model, let's say XGBOOST. Now, I want to predict the price of a new apartment. Is there a good way to show what is "good" in this apartment, and what is bad, and by how much (scaled 0-1)?</p>

<p>For example: The floor number is a "strong" feature (i.e. - in this area this floor number is desired, thus affects positively on the price of the apartment), but the socioeconomic status is a weak feature (i.e. the socioeconomic status is low and thus affects negatively on the price of the apartment).</p>

<p>What I want is to illustrate more or less why my model decided on this price, and I want the user to get a feel of the apartment value by those indicators.</p>

<p>I thought of exhaustive search on each feature - but I'm afraid that will take too much time.</p>

<p>Is there a more brilliant way of doing this?</p>

<p>Any help would be much appreciated...</p>