| \n", " | inference (production) | \n", "training (development) | \n", "
|---|---|---|
| скорость работы (в FLOPs, FPS или seconds/sample) | \n",
" на целевом девайсе, с использованием inference-specific оптимизаций | \n",
" на девайсе для обучения, время необходимое на проведение экспериментов | \n",
"
| количество параметров (в штуках или байтах) | \n",
" размер приложения на диске, скорость загрузки на девайс | \n",
" репрезентативная способность модели, склонность к переобучению | \n",
"
| пиковое потребление памяти (в байтах или % от максимума) | \n",
" возможность \"уместиться\" на целевом девайсе, возможность параллельной обработки | \n",
" максимальный размер батча, требования к девайсу для обучения | \n",
"
![](\"https://courses.cv-gml.ru/storage/seminars/nn-training-basics/albumentations_logo.png\"){kind=logo}
"
]
},
{
"cell_type": "markdown",
"id": "three-devices",
"metadata": {},
"source": [
"Albumentations - это Python библиотека для быстрого и гибкого аугментирования изображений. Albumentations эффективно реализует широкий спектр операций преобразования изображений и обеспечивает при этом лаконичный, но мощный интерфейс для различных задач компьютерного зрения, включая классификацию, сегментацию и детекцию."
]
},
{
"cell_type": "markdown",
"id": "printable-geneva",
"metadata": {},
"source": [
"### Пример аугментаций из Albumentations:"
]
},
{
"cell_type": "markdown",
"id": "TODAY-cRN570",
"metadata": {},
"source": [
""
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "scheduled-pavilion",
"metadata": {},
"outputs": [],
"source": [
"import albumentations as A"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "universal-episode",
"metadata": {},
"outputs": [],
"source": [
"MyTransform = A.Compose(\n",
" [\n",
" A.RandomResizedCrop(size=(128, 128), p=0.3),\n",
" A.Rotate(limit=30),\n",
" A.HorizontalFlip(p=0.3),\n",
" A.RandomBrightnessContrast(brightness_limit=0.3, contrast_limit=0.3, p=0.5),\n",
" ]\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "beautiful-beginning",
"metadata": {},
"outputs": [],
"source": [
"show_images(weather_images, titles)\n",
"for _ in range(4):\n",
" transformed = [\n",
" MyTransform(image=np.array(image))[\"image\"]\n",
" for image in weather_images\n",
" #\n",
" ]\n",
" show_images(transformed)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5e31bd3c",
"metadata": {},
"outputs": [],
"source": [
"MyTransform = A.Compose(\n",
" [\n",
" A.SmallestMaxSize(max_size=160),\n",
" A.Affine(translate_percent=0.05, scale=(1.0, 1.05), rotate=15, p=0.5),\n",
" A.RandomCrop(height=128, width=128),\n",
" A.RGBShift(r_shift_limit=15, g_shift_limit=15, b_shift_limit=15, p=0.5),\n",
" A.RandomBrightnessContrast(brightness_limit=0.3, contrast_limit=0.3, p=0.5),\n",
" A.HorizontalFlip(p=0.3),\n",
" ]\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "244885e8",
"metadata": {},
"outputs": [],
"source": [
"show_images(weather_images, titles)\n",
"for _ in range(4):\n",
" transformed = [\n",
" MyTransform(image=np.array(image))[\"image\"]\n",
" for image in weather_images\n",
" #\n",
" ]\n",
" show_images(transformed)"
]
},
{
"cell_type": "markdown",
"id": "logical-output",
"metadata": {},
"source": [
"#### Ковариантное преобразование изображений и меток"
]
},
{
"cell_type": "markdown",
"id": "M5xzCPbnx_1h",
"metadata": {},
"source": [
""
]
},
{
"cell_type": "markdown",
"id": "confirmed-guard",
"metadata": {},
"source": [
"------"
]
},
{
"cell_type": "markdown",
"id": "EQrOZ7EKyXWo",
"metadata": {},
"source": [
"Добавим аугментации в обучение"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e455d2ab",
"metadata": {},
"outputs": [],
"source": [
"import albumentations.pytorch.transforms\n",
"\n",
"augmentations = [\n",
" A.SmallestMaxSize(max_size=160),\n",
" A.Affine(translate_percent=0.05, scale=(1.0, 1.05), rotate=15, p=0.5),\n",
" A.RandomCrop(height=128, width=128),\n",
" A.RGBShift(r_shift_limit=15, g_shift_limit=15, b_shift_limit=15, p=0.5),\n",
" A.RandomBrightnessContrast(brightness_limit=0.3, contrast_limit=0.3, p=0.5),\n",
" A.HorizontalFlip(p=0.3),\n",
"]\n",
"# Don't forget to prepare the image data for consumption by PyTorch\n",
"common_transforms = [\n",
" A.Resize(*NETWORK_SIZE),\n",
" A.ToFloat(max_value=255),\n",
" A.Normalize(max_pixel_value=1.0, mean=IMAGENET_MEAN, std=IMAGENET_STD),\n",
" A.pytorch.transforms.ToTensorV2(),\n",
"]\n",
"\n",
"MyTrainTransform = A.Compose(augmentations + common_transforms)\n",
"MyValidTransform = A.Compose(common_transforms)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "according-pharmacy",
"metadata": {},
"outputs": [],
"source": [
"def my_train_transform(image):\n",
" return MyTrainTransform(image=image)[\"image\"]\n",
"\n",
"\n",
"def my_valid_transform(image):\n",
" return MyValidTransform(image=image)[\"image\"]\n",
"\n",
"\n",
"ds_train_alb = MyCustomDataset(\n",
" mode=\"train\",\n",
" transform=my_train_transform,\n",
")\n",
"ds_valid_alb = MyCustomDataset(\n",
" mode=\"valid\",\n",
" transform=my_valid_transform,\n",
")\n",
"\n",
"dl_train_alb = data.DataLoader(\n",
" ds_train_alb,\n",
" batch_size=BATCH_SIZE,\n",
" shuffle=True,\n",
" drop_last=True,\n",
" num_workers=os.cpu_count(),\n",
")\n",
"dl_valid_alb = data.DataLoader(\n",
" ds_valid_alb,\n",
" batch_size=BATCH_SIZE,\n",
" shuffle=False,\n",
" drop_last=False,\n",
" num_workers=os.cpu_count(),\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "regulation-monitoring",
"metadata": {},
"outputs": [],
"source": [
"model = train(num_epochs=3, dl_train=dl_train_alb, dl_valid=dl_valid_alb)"
]
},
{
"cell_type": "markdown",
"id": "dffad066",
"metadata": {},
"source": [
"## Регуляризующие аугментации меток"
]
},
{
"cell_type": "markdown",
"id": "b1c7eef7",
"metadata": {},
"source": [
"### MixUp\n",
"\n",
"Попробуем использовать более сложную аугментацию из статьи [MixUp: Beyond Empirical Risk Minimization](https://arxiv.org/abs/1710.09412).\n",
"\n",
"MixUp - взвешенное усреднение двух изображений с соответствующим изменением one-hot таргета. Брать пары изображений можно внутри одного батча."
]
},
{
"cell_type": "markdown",
"id": "c7f5a3da",
"metadata": {},
"source": [
""
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f927d548",
"metadata": {},
"outputs": [],
"source": [
"def mixup_data(x, y, alpha=1.0):\n",
" lam = np.random.beta(alpha, alpha)\n",
"\n",
" batch_size = x.size()[0]\n",
" index = torch.randperm(batch_size, device=x.device)\n",
"\n",
" mixed_x = lam * x + (1 - lam) * x[index, ...]\n",
" y_a, y_b = y, y[index]\n",
"\n",
" return mixed_x, y_a, y_b, lam"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7a777d8d",
"metadata": {},
"outputs": [],
"source": [
"show_images(weather_images, titles)\n",
"for _ in range(2):\n",
" transformed = [\n",
" MyTransform(image=np.array(image))[\"image\"]\n",
" for image in weather_images\n",
" #\n",
" ]\n",
" transformed = torch.from_numpy(np.array(transformed))\n",
" transformed, _, _, _ = mixup_data(transformed, torch.zeros(5))\n",
" transformed = transformed.numpy().round().clip(0, 255).astype(np.uint8)\n",
" show_images(transformed)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3c8d05ca",
"metadata": {},
"outputs": [],
"source": [
"def mixup_loss_fn(loss_fn, p, y_a, y_b, lam):\n",
" loss_a = loss_fn(p, y_a)\n",
" loss_b = loss_fn(p, y_b)\n",
"\n",
" loss = lam * loss_a + (1 - lam) * loss_b\n",
"\n",
" # Or alternatively for smooth data,\n",
" # you can keep the loss function and\n",
" # mix the labels in mixup_data instead\n",
" #\n",
" # mixed_y = lam * onehot(y_a) + (1-lam) * onehot(y_b)\n",
" # loss = loss_fn(p, mixed_y)\n",
" return loss\n",
"\n",
"\n",
"# Mix-up is performed in TRAIN ONLY\n",
"def mixup_train_step(model, loss_fn, x, y):\n",
" x_mixed, y_a, y_b, lam = mixup_data(x, y)\n",
"\n",
" p = model(x_mixed)\n",
"\n",
" return mixup_loss_fn(loss_fn, p, y_a, y_b, lam)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c0060177",
"metadata": {},
"outputs": [],
"source": [
"model = train(\n",
" num_epochs=3,\n",
" dl_train=dl_train_alb,\n",
" dl_valid=dl_valid_alb,\n",
" train_step=mixup_train_step,\n",
")"
]
},
{
"cell_type": "markdown",
"id": "d8ab1498",
"metadata": {},
"source": [
"Другие похожие способы аугментаций:\n",
"* [CutMix](https://arxiv.org/abs/1905.04899)\n",
"* [FMix](https://arxiv.org/abs/2002.12047)"
]
},
{
"cell_type": "markdown",
"id": "668a6291",
"metadata": {},
"source": [
"### Label smoothing"
]
},
{
"cell_type": "markdown",
"id": "0324f30d",
"metadata": {},
"source": [
"Модель машинного обучения называется \"откалиброванной\" (calibrated), если выдаваемая ею вероятность отражает ее качество. Пусть у нас есть 100 примеров, на каждый модель выдает вероятность 0.9. Тогда, если модель \"откалибрована\", то ровно 90 примеров должны быть классифицированы корректно.\n",
"\n",
"Предсказания \"откалиброванной\" модели легче интерпретируются, к такой модели проще подбирать пороги и легче добавлять ее в ансамбль.\n",
"\n",
"Чересчур \"уверенная\" модель (overconfident) склонна выдавать очень высокие скоры. Побороть этот эффект помогает label smoothing. Он может быть использован в связке с softmax+cross-enropy loss. Метод заключается в том, чтобы преобразовать таргет по следующей формуле:\n",
"\n",
"$$y_{ls} = (1 - α) \\cdot y_{hot}\\ +\\ α \\cdot\\frac{1}{K}$$\n",
"\n",
"$y_{hot}$ - one-hot представление таргета, $K$ - число классов, $α$ - гиперпараметр, задающий силу сглаживания (обычному представлению таргета соответствует $α$ = 0)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e739f32e",
"metadata": {},
"outputs": [],
"source": [
"class LabelSmoothingCrossEntropyLoss(nn.Module):\n",
" def __init__(self, num_classes=len(CLASSES), smoothing=0.01):\n",
" assert 0 <= smoothing < 1\n",
"\n",
" super().__init__()\n",
" self.smoothing = smoothing\n",
" self.num_classes = num_classes\n",
"\n",
" def forward(self, p, y):\n",
"\n",
" with torch.no_grad():\n",
" K = self.num_classes\n",
" alpha = self.smoothing\n",
" y_one_hot = F.one_hot(y, num_classes=K)\n",
"\n",
" smooth_y = (1 - alpha) * y_one_hot + (alpha / K)\n",
"\n",
" loss = F.cross_entropy(p, smooth_y)\n",
" return loss"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ef771e44",
"metadata": {},
"outputs": [],
"source": [
"model = train(\n",
" num_epochs=3,\n",
" dl_train=dl_train_alb,\n",
" dl_valid=dl_valid_alb,\n",
" loss_cls=LabelSmoothingCrossEntropyLoss,\n",
")"
]
},
{
"cell_type": "markdown",
"id": "burning-donna",
"metadata": {},
"source": [
"---"
]
}
],
"metadata": {},
"nbformat": 4,
"nbformat_minor": 5
}