Are you talking about Common Lisp, or are you talking about Scheme? Common Lisp does provide primitives for just about everything, in ridiculously generic forms. Scheme forces you to code it yourself.
I think the criticism leveled would go to both. If what you say is true, moreso to Scheme. But the criticism is more that it allows this. To many, it even encourages it.
For myself, I still like it, a lot. To the point that the more I learn of it, the more I'm confused on how it hasn't taken off better.
I would see more 'primitives' in a language than 'special forms'.
For example Common Lisp provides support for complex numbers and operations for them. These operations are 'primitives' in the language. This is unrelated to 'special forms'.
