Keywords
Machine Learning, Deep Learning, Computer Vision
Select the category the research project fits.
Mathematical/Computational Sciences
Is this submission part of ICaP/PW (Introductory Composition at Purdue/Professional Writing)?
No
Abstract
Transfer learning is a common technique used in a wide variety of deep learning applications. Transfer learning methods are typically used to make use of a source domain, where there is an abundance of labeled data, to make inferences in a target domain, where labeled data is scarce. In the digital age, improving a model’s ability to generalize knowledge gained from the massive amount of data available online to new contexts is crucial. Most new contexts of interest, like radiological scans, have very few labels, an obstacle that can be overcome with improved transfer learning methods. A basic transfer learning technique involves resetting the weights and biases associated with the last few layers of a deep learning model that has been trained on the source domain, and then re-training the model on the target domain. This a very widely used technique, but can often times result in a phenomenon known as negative transfer. Negative transfer occurs when the knowledge gained in the source domain proves to be harmful when transferring to the target domain. In order to prevent this phenomenon, our team is focusing on making a systematic method for determining which weights and biases should be reset when transferring knowledge. The basic idea is that if the source and target domains are similar, then most of the models knowledge gained in the source domain will be transferred to the target domain. However, if the source and target domains are different, the model will forget that knowledge which would be harmful in its learning the target domain.
Recommended Citation
Ulmer, Andrew; Kapach, Zohar; Merrick, Daniel; Maiya, Karthik; Sasidharan, Abhay; Alikhan, Arshad; and Dang, David, "Actively Preventing Negative Transfer" (2019). Purdue Undergraduate Research Conference. 16.
https://docs.lib.purdue.edu/purc/2019/Posters/16
Actively Preventing Negative Transfer
Transfer learning is a common technique used in a wide variety of deep learning applications. Transfer learning methods are typically used to make use of a source domain, where there is an abundance of labeled data, to make inferences in a target domain, where labeled data is scarce. In the digital age, improving a model’s ability to generalize knowledge gained from the massive amount of data available online to new contexts is crucial. Most new contexts of interest, like radiological scans, have very few labels, an obstacle that can be overcome with improved transfer learning methods. A basic transfer learning technique involves resetting the weights and biases associated with the last few layers of a deep learning model that has been trained on the source domain, and then re-training the model on the target domain. This a very widely used technique, but can often times result in a phenomenon known as negative transfer. Negative transfer occurs when the knowledge gained in the source domain proves to be harmful when transferring to the target domain. In order to prevent this phenomenon, our team is focusing on making a systematic method for determining which weights and biases should be reset when transferring knowledge. The basic idea is that if the source and target domains are similar, then most of the models knowledge gained in the source domain will be transferred to the target domain. However, if the source and target domains are different, the model will forget that knowledge which would be harmful in its learning the target domain.