"Video card" was the more general word. "VGA" is one of the IBM video cards for PCs that later became a de facto standard, as its behavior was cloned by other companies. It's sometimes used descriptively to talk about the 640x480 resolution, or the DE-15 connector that remained a standard connection for analog video output on personal computers for a long time.
With some others like the Hercules which was MDA upward-compatible and did graphics as well as text.
They didn't really do any graphics "processing"; just displaying memory-mapped pixels in various formats.
They were memory-mapped, and the MDA used a different memory block than the CGA/EGA/VGA, so you could have two separate monitors simultaneously, doing things lke running something like Turbo Debugger on the MDA text display.