What is Transfer learning

With the advent of emerging technologies and significant research in the field of computer science, there are no chapters left unturned for development. One of such miraculous discoveries is being experienced in the world of the Machine Learning and Deep Learning. If we want to relate in the simplest verse, what would be our requirement to lead a carefree and comfortable life? If someone could do all our works with pleasing accuracy and effectiveness, wouldn’t our purpose be fulfilled? And this is what exactly Machine Learning offers to us. When we dig deep into it, surely we will never get tired of complementing the eminent features it portrays but for now let’s stay connected to one of its very marvellous characteristics which is none other than Transfer Learning.

Using transfer learning on a pre trained model

A brief intro

The basement of this unique feature of Transfer Learning is very relative to the way we learn things in our life. People usually learn from some task A and apply the same in another task B which is similar to task A.For eg. while learning to drive a car if someone has  a prior experience of using gears that leads to learning to drive car much faster .

Transfer Learning works in a similar fashion, it is about applying the knowledge gained in training one model for some specific task to a different task. Let’s understand this in a clearer picture with an example. Suppose, we require a machine which can do image recognition of different objects. To attain this we will need to train this model with a plenty of images so that it can offer intelligent output. Now, after sometime we require another model who can be able to do radiology diagnosis. Here the connection lies as our previous model also works on images and the radiology diagnosis takes input as images too. This solves our problem as now we don’t need to train another machine learning model from scratch rather we can simply reap the benefits of our already deployed model. The only matter of concern is we need to improvise the knowledge of our older model and train it only in specific terms so that it sets its gears up to do radiology diagnosis now. Similarly, a model trained for speech recognition can be further used for making a wake-up call like “Hi Alexa”, “Hey Siri” or “OK Google”, this is the essence of Transfer Learning.

Pre-requisites for Transfer learning

There are some basic prerequisites which a model must fulfil to provide the best of results.

First of all, if a task B is going to be derived from a task A, then both the tasks must take the same inputs like in case of image recognition device and then radiology diagnosis device, their inputs were same i.e. images .Also the initial goal was image recognition which was further extended to radiology diagnosis.

Secondly, we must have a lot more data for task A if we wish to implement a task B based on it, only then it will serve our purpose intelligently keeping our expectations bounded. Also, it is extremely useful in a situation where we have less training data for a certain task and have a similar model trained on that.

Lastly, the low level features of task A must be helpful in learning for task B. With these parameters being on the same page, Transfer Learning can help accepting new challenges with minimal efforts and constant victory.


Why transfer learning is important to us?

At xlabs.in we provide the best deep learning solution to our clients.Our implementation of the model is mostly limited by the following factors to train a big network like the one shown in the image below:





  • Inadequate availability of training data for deep neural networks.
  • Limited cpu/gpu resources for training a model from the beginning.
  • Limited time period to complete a project

Apart from all the above limitations we sometime have a pertained model whose weights are used as a initialisation seed for the new model which helps in increasing accuracy and improving the model .It also helps in rapid prototyping of a model which helps to ensure us analyse a model quickly.

All the above factors lead us to used transfer learning heavily in our deep learning project.



Stay tuned for part 2 where we will explore transfer learning in action by implementing a python program for deep learning classifications using transfer learning.


To know more about transfer learning and deep learning please refer to the following links: