<p>I have a multi-part answer for you.</p>

<h2>How to freeze a Module</h2>

<p>It all comes down to how your optimizer is set up. The usual approach for TF1 is to initialize it with all Variables found in the TRAINABLE_VARIABLES <a href="https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/GraphKeys" rel="nofollow noreferrer">collection</a>. The <a href="https://www.tensorflow.org/hub/api_docs/python/hub/Module" rel="nofollow noreferrer">doc for hub.Module</a> says about <code>trainable</code>: "If False, no variables are added to TRAINABLE_VARIABLES collection, ...". So, <strong>yes</strong>, setting <code>trainable=False</code> (explicitly or by default) freezes the module in the standard usage of TF1.</p>

<h2>Why not to freeze BERT</h2>

<p>That said, BERT is meant to be fine-tuned. The <a href="https://arxiv.org/abs/1810.04805" rel="nofollow noreferrer">paper</a> talks about the <em>feature-based</em> (i.e., frozen) vs <em>fine-tuning</em> approaches in more general terms, but the <a href="https://tfhub.dev/google/bert_cased_L-12_H-768_A-12/1" rel="nofollow noreferrer">module doc</a> spells it out clearly: <strong>"fine-tuning all parameters is the recommended practice."</strong> This gives the final parts of computing the pooled output a better shot at adapting to the features that matter most for the task at hand.</p>

<p>If you intend to follow this advice, please also mind <a href="https://www.tensorflow.org/hub/tf1_hub_module#fine-tuning" rel="nofollow noreferrer">tensorflow.org/hub/tf1_hub_module#fine-tuning</a> and pick the correct graph version: BERT uses <a href="https://arxiv.org/abs/1207.0580" rel="nofollow noreferrer">dropout</a> regularization during training, and you need to set <code>hub.Module(..., tags={"train"})</code> to get that. But for inference (in evaluation and prediction), where dropout does nothing, you omit the <code>tags=</code> argument (or set it to the empty <code>set()</code> or to <code>None</code>).</p>

<h2>Outlook: TF2</h2>

<p>You asked about <code>hub.Module()</code>, which is an API for TF1, so I answered in that context. The same considerations apply for <a href="https://tfhub.dev/google/collections/bert/1" rel="nofollow noreferrer">BERT</a> in TF2 SavedModel format. There, it's all about setting <code>hub.KerasLayer(..., trainable=True)</code> or not, but the need to select a graph version has gone away (the layer picks up Keras' <code>training</code> state and applies it under the hood).</p>

<p>Happy training!</p>
Freezing of BERT layers while using tfhub module
<p>In this link <a href="https://medium.com/@prasad.pai/how-to-use-tensorflow-hub-with-code-examples-9100edec29af" rel="nofollow noreferrer">click here</a> the author says that:</p>

<pre><code>import tensorflow_hub as hub
module = hub.Module(&lt;&lt;Module URL as string&gt;&gt;, trainable=True)
</code></pre>

<p>If user wishes to fine-tune/modify the weights of the model, this parameter has to be set as True.
So my doubt is if I set this to false does it mean that I am freezing all the layers of the BERT which is my intension too. I want to know if my approach is right.</p>