<p>The documentation of <code>pos_weight</code> is indeed a bit unclear. For <code>BCEWithLogitsLoss</code> <code>pos_weight</code> should be a <code>torch.tensor</code> of size=1:</p>
<pre class="lang-py prettyprint-override"><code>BCE_With_LogitsLoss=nn.BCEWithLogitsLoss(pos_weight=torch.tensor([class_wts[0]/class_wts[1]]))
</code></pre>
<p>However, in your case, where pos class occurs only 2% of the times, I think setting <code>pos_weight</code> will not be enough.<br />
Please consider using <a href="https://stackoverflow.com/a/52161194/1714410">Focal loss</a>:<br />
<em>Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, Piotr Dollár</em> <a href="https://arxiv.org/abs/1708.02002" rel="nofollow noreferrer"><strong>Focal Loss for Dense Object Detection</strong></a> (ICCV 2017).<br />
Apart from describing Focal loss, this paper provides a very good explanation as to why CE loss performs so poorly in the case of imbalance. I strongly recommend reading this paper.</p>
<p>Other alternatives are listed <a href="https://stackoverflow.com/a/58213245/1714410">here</a>.</p>
pytorch BCEWithLogitsLoss calculating pos_weight
<p>I have a neural network as below for binary prediction. My classes are heavily imbalanced and class 1 occurs only 2% of times. Showing last few layers only</p>
<pre><code>self.batch_norm2 = nn.BatchNorm1d(num_filters)

self.fc2 = nn.Linear(np.sum(num_filters), fc2_neurons)

self.batch_norm3 = nn.BatchNorm1d(fc2_neurons)

self.fc3 = nn.Linear(fc2_neurons, 1)
</code></pre>
<p>My loss is as below. Is this a correct way to calculate <code>pos_weight</code> parameter? I looked into official documentation at this <a href="https://pytorch.org/docs/stable/generated/torch.nn.BCEWithLogitsLoss.html#torch.nn.BCEWithLogitsLoss" rel="nofollow noreferrer">link</a> and it shows that <code>pos_weight</code> needs to have one value for each class for multiclass classification. Not sure if for the binary class it is a difference scenario. I tried to input 2 values and I was getting an error</p>
<p><em><strong>My question: for binary problem, would <code>pos_weight</code> be a single value unlike multiclass classification where it needs to a list/array with length equal to number of classes?</strong></em></p>
<pre><code>BCE_With_LogitsLoss=nn.BCEWithLogitsLoss(pos_weight=class_wts[0]/class_wts[1])
</code></pre>
<p>My y variable is a single variable that has 0 or 1 to represent the actual class and the neural network outputs a single value</p>
<p><strong>--------------------------------------------------Update 1</strong></p>
<p>based upon the answer by Shai I have below questions:</p>
<ol>
<li><code>BCEWithLogitsLoss</code> - if it is a multiclass problem then how to use <code>pos_weigh</code> parameter?</li>
<li>Is there any example of using focal loss in pytorch? I found some links but most of them were old - dating 2 or 3 or more years</li>
<li>For training I am oversampling my class 1. Is focal loss still appropiate?</li>
</ol>