Thank you, I knew that derivatives were used in gradient descent, but automatic differentiation is new to me. The Wikipedia article is fairly opaque compared to what I learned in school, but this tidbit stood out:
Source code transformation (SCT): the compiler processes source code so that the derivatives are calculated alongside each instruction.
Operator overloading (OO): operators are overridden so that derivatives are calculated for numbers and vectors.
Based on the state of software these days, I'm guessing that OO (the "bare hands" method) is what's mainstream. It would be far better IMHO to use SCT, since it's a universal solution that doesn't require manually refactoring programs.
But stuff like SCT might be considered metaprogramming, which seems to have fallen out of fashion. I grew up with C++ macros and templates, so I feel that this is somewhat tragic, although readability and collaboration are much better today. A modern example might be something like aspect-oriented programming (AOP):
I once used the AOP library AspectJ to trace a Java app's execution, since Java made the (unfortunate) choice to focus on objects rather than functions, which makes it generally a poor fit for data processing, due to its high use of mutable state within objects (mutable state is what limits most object-oriented projects to around 1 million lines). Meaning that I couldn't remember the program's context as I was stepping through it, and had to analyze traces instead. AspectJ allows one to hook into the code without modifying it, sort of like a debugger, so that stuff like function calls and variable mutations can be watched:
If we had that, maybe we could automatically generate derivatives for the set operations. Then either access them as variables in stored procedures, or possibly as something like views or via metadata stored somewhere like MySQL's INFORMATION_SCHEMA.
I don't really know, but maybe these breadcrumbs could be helpful.
https://en.wikipedia.org/wiki/Automatic_differentiation#Impl...
Source code transformation (SCT): the compiler processes source code so that the derivatives are calculated alongside each instruction.
Operator overloading (OO): operators are overridden so that derivatives are calculated for numbers and vectors.
Based on the state of software these days, I'm guessing that OO (the "bare hands" method) is what's mainstream. It would be far better IMHO to use SCT, since it's a universal solution that doesn't require manually refactoring programs.
But stuff like SCT might be considered metaprogramming, which seems to have fallen out of fashion. I grew up with C++ macros and templates, so I feel that this is somewhat tragic, although readability and collaboration are much better today. A modern example might be something like aspect-oriented programming (AOP):
https://en.wikipedia.org/wiki/Aspect-oriented_programming
I once used the AOP library AspectJ to trace a Java app's execution, since Java made the (unfortunate) choice to focus on objects rather than functions, which makes it generally a poor fit for data processing, due to its high use of mutable state within objects (mutable state is what limits most object-oriented projects to around 1 million lines). Meaning that I couldn't remember the program's context as I was stepping through it, and had to analyze traces instead. AspectJ allows one to hook into the code without modifying it, sort of like a debugger, so that stuff like function calls and variable mutations can be watched:
https://en.wikipedia.org/wiki/AspectJ
https://www.eclipse.org/aspectj/doc/released/progguide/index...
Looks like this might still be an open problem in Python:
https://stackoverflow.com/questions/12356713/aspect-oriented...
https://docs.spring.io/spring-python/1.2.x/sphinx/html/aop.h...
But it seems like SQL would be a good candidate for AOP:
https://stackoverflow.com/questions/12271588/aspect-oriented...
https://technology.amis.nl/it/aspect-oriented-programming-ao...
If we had that, maybe we could automatically generate derivatives for the set operations. Then either access them as variables in stored procedures, or possibly as something like views or via metadata stored somewhere like MySQL's INFORMATION_SCHEMA.
I don't really know, but maybe these breadcrumbs could be helpful.