Correct. I run mpd on my desktop machine, then connect to it from a client either running on my desktop (https://github.com/floren/Ampd) or on my phone or my laptop. However, regardless of where my client is executing, the music comes out of the speakers attached to my desktop. If there's a way to stream music from the server to the client, I am not aware of it.
I've definitely done it before. It used to be my primary way to stream music at work from my home server. This was likely a decade ago so I couldn't say exactly how it was set up (I've since switched to the, now seemingly abandoned, Subsonic) but knowing me SSH probably played a key part.
