Expensive to initialize doesn’t imply large. And wanting to avoid expensive initialization (runtime) is orthogonal to wanting to avoid a larger memory footprint. So I don’t quite buy your argument.
Your question was why the advice was given to avoid ThreadLocals. This is the primary reason. It's not necessarily related to avoiding a larger memory footprint.