Before building HappyRhino, I tried a lot to improve Backbone because we were using it for building FriendCode (friendco.de) but the code base of Backbone is really not modular and hard to extend (the all backbone code is in one unique file, model/view/collection don't inherit from a "class", etc...)
Yes, I ran into this problem as well when building the first version (0.0.1) of ePantry (epantry.com).
I had a 3,000 line CoffeeScript file with 5-6 Backbone Models, matching Collections, things called "MicroViews" and "AdHocViews". It got heavy really fast.
Thankfully I am a vim'er. But I'm not sure if my use of vim engendered this or was the effect of all of Backbone's lack of modularity. Then again, there's the ultimate question of: What is a Module? (Is it material JS files or is it what is loved into the memory states of the Browser Application?)