What is Transfer Learning
Is a method where a model developed for some tasks is reused and acts as the basis for a model on a second task which is similar in nature. In this way knowledge acquired in solving one problem is applied to a different but related problem.
It is quite popular in deep learning as the cost of resources required to train models is high and also due to the limitations in getting quality and quantity of relevant training datasets. In simple terms the objective is to transfer knowledge and learning gained in in one task and using it for another task.
Examples
In image recognition, an already trained model to recognise dog images can be re used to recognise cats or other animals.
In language processing weights generated in a model to understand general English language can be further reused for a domain specific model without having to train the latter to understand general English language.
Benefits
Reduction in effort and cost in creating new models.
Optimisation that allows rapid progress and improved performance when modelling the second task.
When to use
Transfer Learning can be best leveraged in the following situations.
Availability of data
In cases where there is limited availability of training data for training ML models , pre-trained models can be used
Rapid Development
When the turn around time required to build models is less , transfer learning can be utilised and pre-trained models can be used for further training and tuning for new requirements
Contextual AI
General pre trained models can be used to further train context specific models
Using a pre trained model
Prediction
Get a pre trained model and use it as it is to for new predictions.
Feature Extraction,
The pre-trained model’s final part is build to fit the new datasets. The output of the layer before the final layer is fed as input to a new model. The objective is to use the pre-trained model, to pre-process and get essential features. Pass these features to a new classifier.
Fine-tuning
Use weights from the pre-trained model to initialise weights in a new model.
References
http://research.aimultiple.com/transfer-learning
