I don't know if it's all that rare. My wife and I attended one in Asakusa on our last day in Japan and saw a couple of yakuza guys. (And she saw one of the woman yakuza on the women's side.) We didn't go looking for them, it was just the closest neighborhood bath house to the shrine.

The other bath house I went to, the one at the bottom of Mt. Takao, did not allow any tattoos. But it was much larger and more commercial.