Discussion on training CNN using pyTorch

ljg2np

Discussion on training CNN using pyTorch [Copy link]

 

CNN can be trained using pyTorch, either on the CPU or on the GPU.

1. Install pyTorch with GPU support;

2. Use the function in pytorch to determine whether GPU is supported:

g_support = torch.cunda.is_available()
if g_support:

device = torch.device('cuda:0')
else:
device = torch.device('cpu')

3. Transfer CPU to GPU:

net = Net()

net.to(device)

ljg2np

The statement to determine whether GPU is supported can also be coded as follows:

device = "cuda" if torch.cuda.is_available() else "cpu"

ljg2np

CNN is particularly suitable for processing image data and can capture spatial hierarchical structures in images, such as edges, textures, and more complex patterns; generally speaking, a CNN can be composed of convolutional layers, pooling layers, and fully connected layers.

ljg2np

As the building block of CNN, the convolutional layer generally consists of the following parts:

1. Convolution kernel;

2. Stride length (reduced);

3. Fill (enlarge);

4. Feature map.

ljg2np

Architecture code of CNN: Conv2D(), ReLU(), and MaxPool2D() layers perform convolution, activation, and pooling operations, and Linear() fully connected layers perform classification. The layers are combined using torch.nn.Sequential containers.

class CNN(torch.nn.Module):
...
self.model = torch.nn.Sequential(
torch.nn.Conv2d(in_channels = 3, out_channels = 32, kernel_size = 3, padding = 1),
torch.nn.ReLU(),
torch.nn.MaxPool2d(kernel_size=2),

...

torch.nn.Flatten(),
torch.nn.Linear(64*4*4, 512),
torch.nn.ReLU(),
torch.nn.Linear(512, 10)
)

ljg2np

Inherited by the Module class, use torch.nn.Conv2d to build a neural network with only one layer of two-dimensional convolution:

class MyNet(torch.nn.Module):

def __init__(self):

super(MyNet, self).__init__()

self.conv2d = torch.nn.Conv2d(in_channels=3,out_channels=64,kernel_size=3,stride=2,padding=1)

def forward(self, x):

x = self.conv2d(x)

return x

net = MyNet()

ljg2np

#定义CNN网络的例子（LeNet网络）：
class Net(torch.nn.Module):
def __init__(self):
super(Net, self).__init__()
self.conv1 = torch.nn.Conv2d(in_channels=3, out_channels=6, kernel_size=5)
self.pool1 = torch.nn.MaxPool2d(kernel_size=2, stride=2)
self.conv2 = torch.nn.Conv2d(in_channels=6, out_channels=16, kernel_size=5)
self.fc1 = torch.nn.Linear(in_features=16 * 5 * 5,out_features=120)
self.fc2 = torch.nn.Linear(in_features=120, out_features=84)
self.fc3 = torch.nn.Linear(in_features=84, out_features=10)

def forward(self, x):
x = self.pool1(torch.nn.functional.relu(self.conv1(x)))
x = self.pool1(torch.nn.functional.relu(self.conv2(x)))
x = x.view(-1, 16 * 5 * 5)
x = torch.nn.functional.relu(self.fc1(x))
x = torch.nn.functional.relu(self.fc2(x))
x = self.fc3(x)
return x

ljg2np

Use pyTorch to save and load the model:

torch.save(net, "abc.pth")

Define the model structure and load parameters

net = torch.nn.Sequential(...)

net = torch.load("abc.pth")

ljg2np

In CNN, the convolution kernel represents the feature. The feature map can be obtained through the convolution layer operation, which is a linear activation response. After that, the amount of data is reduced through the nonlinear activation response (Relu); and the amount of data is further reduced through the pooling layer.

ljg2np

As one of the earliest convolutional neural networks, LeNet's convolutional unit consists of a convolutional layer, a sigmoid (or ReLU) activation function, and an average (maximum) pooling layer.

ljg2np

The fully connected layer (FC) combines features, plays the role of a classifier in CNN, and maps them to the sample label space. The fully connected layer can be implemented by convolution operations.

ljg2np

卷积层通过局部映射来缩小实体对象在某一分类方向上的表示尺寸（投影）；池化层（也称汇聚层）通过指定操作进一步缩小信息块，以更显著的特征来生成特征图。

ljg2np

CNN is a feedforward neural network with a convolutional structure. The convolutional structure can reduce the amount of memory occupied by deep networks. Three key operations - local receptive field, weight sharing, and pooling layer - effectively reduce the number of network parameters and alleviate the overfitting problem of the model.

ljg2np

Artificial neural networks (ANN) process information by adjusting the weights between internal neurons. In the convolutional layer, each neuron in the output feature map of CNN is locally connected to its input, and the corresponding connection weights are weighted and summed with the local input plus the bias value to obtain the neuron input value. This process is equivalent to the convolution process, hence the name CNN.

ljg2np

PyTorch provides torch.library to extend, test, and create the pyTorch core operator library. There are several methods and their functions:
1. torch.library.custom_op is used to create new custom operators. This decorator wraps the function as a custom operator, enabling it to interact with various subsystems of PyTorch.

2. torch.library.opcheck is used to test whether custom operators are registered correctly and check whether operators behave consistently on different devices.

3. torch.library.register_kernel registers the implementation of a specific device type (such as CPU or CUDA) for the custom operator.

4. torch.library.register_autograd registers the backward pass formula of the custom operator so that it can correctly calculate the gradient during automatic differentiation.

5. torch.library.register_fake registers FakeTensor implementation for custom operators to support PyTorch compilation API.

通途科技

Study hard, make progress every day, come on, classmates, come on, yourself!!!