> They lost the case, because the Supreme Court (of Canada) ruled you can’t double‐dip on IP protection by turning a functional (i.e., patentable) design into a trademark.
Nintendo did attempt that trick, although the case you’re thinking of is Sega v. Accolade, which as you said, ruled that forcing your competitor to display your trademark unwillingly doesn’t mean you can smack them down for trademark infringement.
The case was SEGA v. Accolade, but the lockout system on the Genesis worked on the same legal theory (e.g. induced trademark infringement equals Doctorowian[0] interpretations of "intellectual property"[1]). For the Game Boy, the problem was that Taiwan[2] didn't have a copyright treaty with Japan, but they did have a trademark treaty, so if you induced pirates to commit a trademark violation you could get them with something. Same idea, but the jurisdiction is different, so all the existing case law about not turning trademark into mutant perpetual copyright wouldn't apply here.
[0] "Intellectual property is any law that allows you to dictate the conduct of your competitors." - Cory Doctorow, paraphrased
[1] Yes, it's a shitty made-up term to make you confuse four different kinds of law, no I don't care. There's enough negative sentiment around "intellectual property" these days that the Stallmanian position sounds like a defense of it now.
There's case history on that in the US as well.