<p>There are four possible solutions:</p>

<ul>
<li><p>Learn 4 models and use SavedModel to save them.  Then, create a 5th model that restores the 4 saved models. This model has no trainable weights. Instead, it simply computes the context and applies the appropriate weight to each of the 4 saved models and returns the value. It is this 5th model that you will deploy.</p></li>
<li><p>Learn a single model. Make the context a categorical input to your model, i.e. follow the approach in <a href="https://arxiv.org/abs/1606.07792" rel="nofollow noreferrer">https://arxiv.org/abs/1606.07792</a></p></li>
<li><p>Use a separate AppEngine service that computes the context and invokes the underlying 4 services, weighs them and returns the result.</p></li>
<li><p>Use an AppEngine service written in Python that loads up all four saved models and invokes the 4 models and weights them and returns the result.</p></li>
</ul>

<p>option 1 involves more coding, and is quite tricky to get right.</p>

<p>option 2 would be my choice, although it changes the model formulation from what you desire. If you go this route, here's a sample code on MovieLens that you can adapt: <a href="https://github.com/GoogleCloudPlatform/cloudml-samples/tree/master/movielens" rel="nofollow noreferrer">https://github.com/GoogleCloudPlatform/cloudml-samples/tree/master/movielens</a></p>

<p>option 3 introduces more latency because of the additional network overhead</p>

<p>option 4 reduces network latency from #3, but you lose the parallelism. You will have to experiment between options 3 and 4 on which provides better performance overall</p>
How to use Cloud ML Engine for Context Aware Recommender System
<p>I am trying to build Context Aware Recommender System with Cloud ML Engine, which uses <code>context prefiltering</code> method (<a href="https://www.slideshare.net/irecsys/tutorial-context-in-recommender-systems" rel="nofollow noreferrer">as described in slide 55, solution a</a>) and I am using this <a href="https://cloud.google.com/solutions/machine-learning/recommendation-system-tensorflow-overview" rel="nofollow noreferrer">Google Cloud tutorial (part 2)</a> to build a demo. I have split the dataset to Weekday and Weekend contexts and Noon and Afternoon contexts by timestamp for purposes of this demo.</p>

<p>In practice I will learn four models, so that I can context filter by Weekday-unknown, Weekend-unknown, unknown-Noon, unknown-Afternoon, Weekday-Afternoon, Weekday-Noon... and so on. The idea is to use prediction from all the relevant models by user and then weight the resulting recommendation based on what is known about the context (unknown meaning, that all context models are used and weighted result is given).</p>

<p>I would need something, that responds fast and it seems like I will unfortunately need some kind of middle-ware if I don't want do the weighting in the front-end.</p>

<p>I know, that AppEngine has prediction mode, where it keeps the models in RAM, which guarantees fast responses, as you don't have to bootstrap the prediction models; then resolving the context would be fast.</p>

<p>However, would there be more simple solution, which would also guarantee similar performance in Google Cloud?</p>

<p>The reason I am using Cloud ML Engine is that when I do context aware recommender system this way, the amount of hyperparameter tuning grows hugely; I don't want to do it manually, but instead use the Cloud ML Engine Bayesian Hypertuner to do the job, so that I only need to tune the range of parameters one to three times per each context model (with an automated script); this saves much of Data Scientist development time, whenever the dataset is reiterated.</p>