Upstage AI Lab 2기
2024년 3월 6일 (수) Day_058
수강생 TODO
목표 :
- Pytorch를 이용하여 다양한 딥러닝 모델 구현에 필요한 tensor 연산을 자유자재로 할 수 있다.
- 기본적인 딥러닝 모델들을 직접 구현하여 학습 및 추론할 수 있다.
- 다양한 라이브러리에 구현되어 있는 딥러닝 모델을 손쉽게 사용할 수 있고, 구현 과정에서 생기는 다양한 에러들을 해결할 수 있다.
수강생 미션 : 직접 데이터를 선정해서 학습이 가능한 형태로 변경함으로써 Pytorch의 data loading에 대해 이해한다.
수강생 세부 todo
1. UCI repository에서 학습하고 싶은 tabular 데이터를 선택. (Adult, Heart disease 등)
2. 해당 tabular data를 학습이 가능한 형태로 변경
2-1 preprocessing.ipynb 파일 생성2-2 UCI repository에서 다운로드 받은 csv파일을 pandas를 이용해 로딩2-3 categorical data가 있는 경우 label encoder 및 one-hot encoding으로 변경2-4 학습 데이터 (X)와 정답 (Y)를 분리2-5 train-test 비율을 6:4로 분리2-6 test set에서 5:5로 validation-test로 분리2-7 training, validation, test set의 X,y 데이터를 npy 또는 npz 파일로 저장
3. Dataset
- 3-1 앞서 진행했던 MNIST 코드를 참고하여 training 파일 새로 생성
- 3-2 저장된 npy 또는 npz 파일을 로딩 후, Pytorch를 이용해 CustomDataset 클래스를 생성
- 3-3 이후 MNIST코드와 같이 DataLoader를 통해 학습 진행
4. Training
- 4-1 기존 코드와 같이 training을 진행하되, 데이터가 변경됨에 따라 생긴 error들을 해결
-참고자료
UCI repository: https://archive.ics.uci.edu/
one-hot encoder (preprocessing): https://pandas.pydata.org/docs/reference/api/pandas.get_dummies.html
label-encoder: https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.LabelEncoder.html
train-test split: https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.train_test_split.html
Custom Dataset: https://pytorch.org/tutorials/beginner/basics/data_tutorial.html
선택한 학습 데이터
The CIFAR-10 dataset
https://www.cs.toronto.edu/~kriz/cifar.html
self.layer = nn.Sequential(
# input shape (batch, 3, 32, 32)
nn.Conv2d(in_channels=3, out_channels = 16, kernel_size = 5),
# output shape (batch, 16, 28, 28)
nn.ReLU(),
nn.Conv2d(in_channels = 16, out_channels = 32, kernel_size = 4, stride = 2),
# output shape (batch, 32, 13, 13)
nn.ReLU(),
nn.Flatten(),
nn.Linear(32*13*13, self.num_classes),
nn.LogSoftmax(dim=1)
)
Valid max accuracy : 0.6172
self.layer = nn.Sequential(
# input shape (batch, 3, 32, 32)
nn.Conv2d(in_channels=3, out_channels = 16, kernel_size = 5),
# output shape (batch, 16, 28, 28)
nn.LeakyReLU(),
nn.Conv2d(in_channels = 16, out_channels = 32, kernel_size = 5),
# output shape (batch, 32, 24, 24)
nn.LeakyReLU(),
nn.Conv2d(in_channels = 32, out_channels = 64, kernel_size = 5),
# output shape (batch, 64, 20, 20)
nn.LeakyReLU(),
nn.MaxPool2d(kernel_size = 2),
# output shape (batch, 64, 10, 10)
nn.Flatten(),
nn.Linear(64*10*10, 4*10*10),
nn.Linear(4*10*10, self.num_classes),
nn.LogSoftmax(dim=1)
)
Valid max accuracy : 0.6531
self.layer = nn.Sequential(
# input shape (batch, 3, 32, 32)
nn.Conv2d(in_channels=3, out_channels = 16, kernel_size = 5),
# output shape (batch, 16, 28, 28)
nn.LeakyReLU(),
nn.Conv2d(in_channels = 16, out_channels = 32, kernel_size = 5),
# output shape (batch, 32, 24, 24)
nn.LeakyReLU(),
nn.MaxPool2d(kernel_size = 2),
# output shape (batch, 32, 12, 12)
nn.Conv2d(in_channels = 32, out_channels = 64, kernel_size = 4, stride= 2),
# output shape (batch, 64, 5, 5)
nn.LeakyReLU(),
nn.Flatten(),
nn.Linear(64*5*5, self.num_classes),
nn.LogSoftmax(dim=1)
)
Valid max accuracy : 0.6793
self.layer = nn.Sequential(
# input shape (batch, 3, 32, 32)
nn.Conv2d(in_channels=3, out_channels = 16, kernel_size = 5),
# output shape (batch, 16, 28, 28)
nn.LeakyReLU(),
nn.Conv2d(in_channels = 16, out_channels = 32, kernel_size = 5),
# output shape (batch, 32, 24, 24)
nn.LeakyReLU(),
nn.AvgPool2d(kernel_size = 2),
# output shape (batch, 32, 12, 12)
nn.Conv2d(in_channels = 32, out_channels = 64, kernel_size = 4, stride= 2),
# output shape (batch, 64, 5, 5)
nn.LeakyReLU(),
nn.Flatten(),
nn.Linear(64*5*5, self.num_classes),
nn.LogSoftmax(dim=1)
)
Valid max accuracy : 0.6653
batch_size = 128 -> batch_size = 256
Valid max accuracy : 0.6566
batch_size = 256 -> batch_size = 512
Valid max accuracy : 0.6463
self.layer = nn.Sequential(
# input shape (batch, 3, 32, 32)
nn.Conv2d(in_channels=3, out_channels = 16, kernel_size = 3),
# output shape (batch, 16, 30, 30)
nn.LeakyReLU(),
nn.Conv2d(in_channels = 16, out_channels = 32, kernel_size = 5),
# output shape (batch, 32, 26, 26)
nn.LeakyReLU(),
nn.AvgPool2d(kernel_size = 2),
# output shape (batch, 32, 13, 13)
nn.Conv2d(in_channels = 32, out_channels = 64, kernel_size = 3, stride= 2),
# output shape (batch, 64, 6, 6)
nn.LeakyReLU(),
nn.Flatten(),
nn.Linear(64*6*6, self.num_classes),
nn.LogSoftmax(dim=1)
)
Valid max accuracy : 0.6415
self.layer = nn.Sequential(
# input shape (batch, 3, 32, 32)
nn.Conv2d(in_channels=3, out_channels = 8, kernel_size = 3, padding=1),
# output shape (batch, 8, 32, 32)
nn.LeakyReLU(),
nn.Conv2d(in_channels = 8, out_channels = 16, kernel_size = 3),
# output shape (batch, 16, 30, 30)
nn.LeakyReLU(),
nn.Conv2d(in_channels = 16, out_channels = 32, kernel_size = 3),
# output shape (batch, 32, 28, 28)
nn.LeakyReLU(),
nn.Conv2d(in_channels = 32, out_channels = 64, kernel_size = 3),
# output shape (batch, 64, 26, 26)
nn.LeakyReLU(),
nn.Conv2d(in_channels = 64, out_channels = 128, kernel_size = 3),
# output shape (batch, 128, 24, 24)
nn.LeakyReLU(),
nn.AvgPool2d(kernel_size = 2),
# output shape (batch, 128, 12, 12)
nn.Conv2d(in_channels = 128, out_channels = 256, kernel_size = 3),
# output shape (batch, 256, 10, 10)
nn.LeakyReLU(),
nn.Flatten(),
nn.Linear(256*10*10, self.num_classes),
nn.LogSoftmax(dim=1)
)
Valid max accuracy : 0.6503
self.layer = nn.Sequential(
# input shape (batch, 3, 32, 32)
nn.Conv2d(in_channels=3, out_channels = 8, kernel_size = 3, padding=1),
# output shape (batch, 8, 32, 32)
nn.LeakyReLU(),
nn.Conv2d(in_channels = 8, out_channels = 16, kernel_size = 3),
# output shape (batch, 16, 30, 30)
nn.LeakyReLU(),
nn.Conv2d(in_channels = 16, out_channels = 32, kernel_size = 3),
# output shape (batch, 32, 28, 28)
nn.LeakyReLU(),
nn.Conv2d(in_channels = 32, out_channels = 64, kernel_size = 3),
# output shape (batch, 64, 26, 26)
nn.LeakyReLU(),
nn.Conv2d(in_channels = 64, out_channels = 128, kernel_size = 3),
# output shape (batch, 128, 24, 24)
nn.LeakyReLU(),
nn.Conv2d(in_channels = 128, out_channels = 256, kernel_size = 3),
# output shape (batch, 256, 22, 22)
nn.LeakyReLU(),
nn.Conv2d(in_channels = 256, out_channels = 512, kernel_size = 3),
# output shape (batch, 512, 20, 20)
nn.LeakyReLU(),
nn.AvgPool2d(kernel_size = 2),
# output shape (batch, 512, 10, 10)
nn.Flatten(),
nn.Linear(512*10*10, self.num_classes),
nn.LogSoftmax(dim=1)
)
Valid max accuracy : 0.672
dropout 없이 했더니 완전히 오버피팅 됨
dropout 추가
dropout_ratio = 0.5
self.layer = nn.Sequential(
# input shape (batch, 3, 32, 32)
nn.Conv2d(in_channels=3, out_channels = 8, kernel_size = 3, padding=1),
# output shape (batch, 8, 32, 32)
nn.LeakyReLU(),
nn.Conv2d(in_channels = 8, out_channels = 16, kernel_size = 3),
# output shape (batch, 16, 30, 30)
nn.LeakyReLU(),
nn.Conv2d(in_channels = 16, out_channels = 32, kernel_size = 3),
# output shape (batch, 32, 28, 28)
nn.LeakyReLU(),
nn.Dropout(dropout_ratio),
nn.Conv2d(in_channels = 32, out_channels = 64, kernel_size = 3),
# output shape (batch, 64, 26, 26)
nn.LeakyReLU(),
nn.Conv2d(in_channels = 64, out_channels = 128, kernel_size = 3),
# output shape (batch, 128, 24, 24)
nn.LeakyReLU(),
nn.Dropout(dropout_ratio),
nn.Conv2d(in_channels = 128, out_channels = 256, kernel_size = 3),
# output shape (batch, 256, 22, 22)
nn.LeakyReLU(),
nn.Conv2d(in_channels = 256, out_channels = 512, kernel_size = 3),
# output shape (batch, 512, 20, 20)
nn.LeakyReLU(),
nn.AvgPool2d(kernel_size = 2),
# output shape (batch, 512, 10, 10)
nn.Flatten(),
nn.Linear(512*10*10, self.num_classes),
nn.LogSoftmax(dim=1)
)
Valid max accuracy : 0.6835
dropout_ratio = 0.2
Valid max accuracy : 0.6664
dropout_ratio = 0.3
Valid max accuracy : 0.6865
'Upstage AI Lab 2기' 카테고리의 다른 글
Upstage AI Lab 2기 [Day060] 수강생 TODO #02 - Pytorch Data Loading - 수정작업 #2 (4) | 2024.03.08 |
---|---|
Upstage AI Lab 2기 [Day059] 수강생 TODO #02 - Pytorch Data Loading - 수정작업 (0) | 2024.03.07 |
Upstage AI Lab 2기 [Day058] 수강생 TODO #01 - MNIST CNN (0) | 2024.03.06 |
Upstage AI Lab 2기 [Day058] CNN 구현 (0) | 2024.03.06 |
Upstage AI Lab 2기 [Day057] PyTorch 실습 - Tensor Manipulation (1) | 2024.03.05 |