<p>I hate to break it to you, but this is an image (full of sound and fury) signifying nothing. When you flatten the output of your <code>Conv2d</code> layers and pass this output through 2 <code>Linear</code> layers, you lose any spatial meaning to the neurons. A &quot;linear&quot; or &quot;dense&quot; layer connects every node in the previous layer to every node in the next layer, effectively discarding the relation between any neuron/node and a locale in the original input image.</p>
<p>If you want to look at the parts of the image your network is attending to in order to make its decisions, you are going to want to look inside the convolutional layers. This is a nontrivial problem with many valid approaches to it. One popular method for doing this is <a href="https://arxiv.org/abs/1610.02391" rel="nofollow noreferrer">Grad-CAM</a>. If you want something simpler, you could try plotting each channel separately of the output of one of one of your convolutional layers; but even this will be difficult to interpret.</p>
What does the Linear regression layer, converted into an image tell me?
<p>I am following <a href="https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html" rel="nofollow noreferrer">this tutorial</a>.</p>
<p>However, I decided to take the linear layers, And then make them convertible into an image of 1 * 19 * 19. In doing so, I get a bunch of pixels at random places.
<a href="https://i.stack.imgur.com/8OtTa.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/8OtTa.png" alt="enter image description here" /></a></p>
<p>Here's what my modified code looks like. To describe what I did, I essentially cut from 0-10 for the labels, Then cut from 10-length of the array for my images. Just so I am separating the labels and jumbled images.</p>
<pre><code>import torch
import torchvision
import torchvision.transforms as transforms
import torch.optim as optim
import torch.nn as nn
import torch.nn.functional as F
import matplotlib.pyplot as plt


device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')


transform = transforms.Compose(
    [transforms.ToTensor(),
     transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])

batch_size = 4

trainset = torchvision.datasets.CIFAR10(root='./data', train=True,
                                        download=True, transform=transform)
trainloader = torch.utils.data.DataLoader(trainset, batch_size=batch_size,
                                          shuffle=True)

testset = torchvision.datasets.CIFAR10(root='./data', train=False,
                                       download=True, transform=transform)
testloader = torch.utils.data.DataLoader(testset, batch_size=batch_size,
                                         shuffle=False)

classes = ('plane', 'car', 'bird', 'cat',
           'deer', 'dog', 'frog', 'horse', 'ship', 'truck')

class Net(nn.Module):
    def __init__(self):
        super().__init__()
        self.conv1 = nn.Conv2d(3, 6, 5)
        self.pool = nn.MaxPool2d(2, 2)
        self.conv2 = nn.Conv2d(6, 16, 5)
        self.fc1 = nn.Linear(16 * 5 * 5,8 * 7 * 7) # 8 * 7 * 7
        self.fc2 = nn.Linear(8 * 7 * 7, 6 * 8 * 8) # 6 * 8 * 8
        self.fc3 = nn.Linear(6 * 8 * 8,19 * 19 + 10) # 19 * 19 + 10

    def forward(self, x):
        x = self.pool(F.relu(self.conv1(x)))
        #### IMPORTANT: this here is where I extract the picture of this layer, comparing against the
        # regression layers and this here!
        self.picx = self.pool(F.relu(self.conv2(x)))
        ####
        x2 = torch.flatten(self.picx, 1) # flatten all dimensions except batch
        x2 = F.relu(self.fc1(x2))
        x2 = F.relu(self.fc2(x2))
        x2 = self.fc3(x2)
        return x2



net = Net()

criterion = nn.CrossEntropyLoss()
optimizer = optim.SGD(net.parameters(), lr=0.001, momentum=0.9)



for epoch in range(5):  # loop over the dataset multiple times
    for i, data in enumerate(trainloader, 0):
        # get the inputs; data is a list of [inputs, labels]
        inputs, labels = data

        # zero the parameter gradients
        optimizer.zero_grad()

        # forward + backward + optimize
        outputs = net(inputs).to(device)
        loss = criterion(outputs, labels).to(device)
        loss.backward()
        optimizer.step()

        running_loss = loss.item()

with torch.no_grad():

    dataiter = iter(testloader)
    images, labels = dataiter.next()

    outputs = net.forward(images) ### pic = to the extracted convulotional layer, vs the regression layer.

    _, predicted = torch.max(outputs[...,0:10], 1)

    print(predicted, labels)

    preds = torch.reshape(outputs[...,10:1454], (4,19,19))

    plt.imshow(preds[0].detach().numpy())
    plt.show()



correct = 0
total = 0
# since we're not training, we don't need to calculate the gradients for our outputs
with torch.no_grad():
    for data in testloader:
        images, labels = data
        # calculate outputs by running images through the network
        outputs = net(images)
        # the class with the highest energy is what we choose as prediction
        _, predicted = torch.max(outputs[...,0:10], 1)
        total += labels.size(0)
        correct += (predicted == labels).sum().item()

print(f'Accuracy of the network on the 10000 test images: {100 * correct // total} %')
</code></pre>
<p>What is this image signifying? And is there any way to get a mask of where the pixels specifically detect the label? In other words, I want to have an image where it draws pixels of where it is seeing a subject, like a dog or a cat.</p>