DINO was created independent of JEPA but uses a similar principle of self supervised learning through minimizing the prediction error of a latent.
The difficulty in predicting a latent is so called "collapse"; the embedding neutral network can always output the zero vector and this would predict the output correctly.
There are different ways to solve this, DINO uses two different models - a teacher and a student and LeCunn uses an explicit term against collapsing to a single output.
The difficulty in predicting a latent is so called "collapse"; the embedding neutral network can always output the zero vector and this would predict the output correctly.
There are different ways to solve this, DINO uses two different models - a teacher and a student and LeCunn uses an explicit term against collapsing to a single output.
Yann mentions DINO in his talks