According to the almighty wikipedia, The connection is correct but it turned out to be an accident. David Hilbert who coined spectral theory was surprised when it was found to be applicable to solving quantum mechanical spectra.
If a spectrum is just a range of numbers, then the idea of spectrum should apply to many phenomenon. Many things can be described by a number. So, when matrix spectrum and atomic spectrum were formulated, perhaps spectrum was a word that was in vogue to describe a quantity. I think this is the explanation of the reason the same term is used for both. Because many connected phenomenon can be quantified. I know only a little about matrix spectrum and atomic spectrum, so take my thought with a grain of salt.
