<p>TFF does support different clients having different model architectures.</p>
<p>However, the <a href="https://www.tensorflow.org/federated/tutorials/federated_learning_for_image_classification" rel="nofollow noreferrer">Federated Learning for Image Classification tutorial</a> uses <a href="https://www.tensorflow.org/federated/api_docs/python/tff/learning/build_federated_averaging_process" rel="nofollow noreferrer"><code>tff.learning.build_federated_averaging_process</code></a> which implements the Federated Averaging (<a href="https://arxiv.org/abs/1602.05629" rel="nofollow noreferrer">McMahan et. al 2017</a>) algorithm, defined as each client receiving the same architecture. This is accomplished in TFF by &quot;mapping&quot; (in the functional programming sense) the model to each client dataset to produce a new model, and then aggregating the result.</p>
<p>To achieve different clients having different architectures, a different federated learning algorithm would need to be implemented. There are couple (non-exhaustive) ways this could be expressed:</p>
<ol>
<li><p>Implement an alternative to <a href="https://github.com/tensorflow/federated/blob/610843c724740e1b041837cc93501b609fb05d8f/tensorflow_federated/python/learning/federated_averaging.py#L43" rel="nofollow noreferrer"><code>ClientFedAvg</code></a>. This method applies a fixed model to the clients dataset. An alternate implementation could potentially create a different architecture per client.</p>
</li>
<li><p>Create a replacement for <code>tff.learning.build_federated_averaging_process</code>
that uses a different function signature, splitting out groups of clients
that would receive different architectures. For example, currently FedAvg
looks like:</p>
<pre><code>(&lt;state@SERVER, data@CLIENTS&gt; → &lt;state@SERVER, metrics@SERVER&gt;
</code></pre>
<p>this could be replaced with a method with signature:</p>
<pre><code>(&lt;state@SERVER, data1@CLIENTS, data2@CLIENTS, ...&gt; → &lt;state@SERVER, metrics@SERVER&gt;
</code></pre>
<p>This would allow the function to internally <code>tff.federated_map()</code> different model architectures to different client datasets. This would likely only be useful in FL simulations or experimentation and research.</p>
</li>
</ol>
<p><strong>However</strong>, in federated learning there will be difficult questions around how to aggregate the models back on the server into a single global model. This probably needs to be designed out first.</p>
In FL, can clients train different model architectures?
<p>I practice on this <a href="https://www.tensorflow.org/federated/tutorials/federated_learning_for_image_classification" rel="nofollow noreferrer">tutorial</a>, I would like that each client train a different architecture and different model, Is this possible?</p>