I and my colleague had Sanchit Singh had my honor to present at WeAreDevelopers Congress Vienna 2019. We wanted to support the whole talk with (ideally) real world examples. We decided to show most of the ideas on exaggerated use cases in Google Collab with training right on the stage. The code can be found here.

The video of the talk is not yet published, so for now following is a small summary of what we presented

Transfer learning (TR) is a way to

  • reuse a big model (trained once on millions of samples)
  • use Deep Learning even on normally too small datasets
  • increase accuracy by leveraging the extra information from different data

In this article we will work with ResNet501 architecture pretrained on the ImageNet2 dataset.

Example images from ImageNet.
Source: ImageNet

Basic TR

def get_resnet(data) -> Model:
    model = models.resnet50
    model = cnn_learner(data, model, metrics=[accuracy])
    model.model_dir = Path("/home/weights/")

    return model

def train(model: Model, cyc_len=1) -> Model:
    callbacks = [EarlyStoppingCallback(model, monitor='valid_loss', min_delta=0.01, patience=2)]
    model.fit_one_cycle(cyc_len=cyc_len, max_lr=1e-3, callbacks=callbacks)

    return model

References

  1. Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun. 2015. Deep Residual Learning for Image Recognition. arXiv:1512.03385 

  2. Deng, J. and Dong, W. and Socher, R. and Li, L.-J. and Li, K. and Fei-Fei, L. 2009. ImageNet: A Large-Scale Hierarchical Image Database. image-net.org