Neural representations are messy, this is both a strength and a weakness. It is a strength because it allows you to easily interpolate in the latent space of the representations in ways that might not be reflected by the training data or any rule-set that a human could come up with. This underlies the power of neural networks to generalize.
Symbolic representations are clean, this is both a strength and a weakness. You might have perfectly separated categories but the real world frequently presents inputs that break taxonomies.
We invented symbols like letters and numbers to reduce the complexity of the real world. Language and mathematics are lossy representations but also incredibly useful models.
Given the value that symbols and symbolic methods have for us I have little doubt that they will be an integral part of efficient AI systems in the future. You could train a neural world model on the ballistic properties of a rocket, but if it's orders of magnitude more efficient why not learn to calculate instead?
Symbolic representations are clean, this is both a strength and a weakness. You might have perfectly separated categories but the real world frequently presents inputs that break taxonomies.
We invented symbols like letters and numbers to reduce the complexity of the real world. Language and mathematics are lossy representations but also incredibly useful models.
Given the value that symbols and symbolic methods have for us I have little doubt that they will be an integral part of efficient AI systems in the future. You could train a neural world model on the ballistic properties of a rocket, but if it's orders of magnitude more efficient why not learn to calculate instead?