Current neuroscience suggests that brain operates in broadly two regimes - learning during wakefulness and model selection during sleep. Both these activities require "forgetting" of specific kinds. Specifically model selection involves trimming synapses. The goal is to learn a model of the world at the right level of detail, what AI folks call avoiding over fitting.