202405032341
Status: #idea
Tags: Deep Learning
Self-Supervised Learning
It is a technique that is used a lot in Natural Language Processing (NLP) for the training of fundamental models that have an understanding of language. It is essentially when the label is encoded in the independent variable instead of being labeled by an annotator.
For example, transformers predict words. The words are both the dependent and independent variables