Even if you implemented the whole checkout process yourself, you'd still be paying at least ~3% to your credit card processor -- more if you're handling small payments, as most processors charge a fixed fee as well.
10% to handle the entire subscription/checkout process isn't bad at all, especially if it means that many of your customers can check out without entering any billing details.
Retail stores have a finite and very small amount of shelf space. Publishers pay a lot for a significant percentage of that shelf space devoted to their product.
The App Store has effectively infinite space. There are so many products and search is so broken that it's extremely difficult for users to discover your product.
Rather than comparing the experience to brick and mortar stores a more apt comparison would be to throw your products in a landfill, junkyard, or extremely large flea market and expect your users to find them. Would you pay high commissions to a junkyard for tossing your product in a pile "somewhere in the back"?