Even i5 will only make it around twice as slow[1]. So from 5min -> 10min.
And that's a relatively pessimistic scenario - huge number of dependencies, slow processor (i5), fresh build. In incremental build, it goes from minutes to seconds.
The reason rustc and other compilers like it (e.g. GHC) are "slow" isn't because they are unoptimized; it's because they do A LOT of work!
rustc is enforcing a strong and expressive type system, checking for ownership, generating code from macros, etc. It will always be an order of magnitude slower than C/C++ compiler, kind-of by design.
Which is exactly why we traditionally don't build the world from scratch in C++, rather rely on binary libraries for 3rd party dependencies, or component frameworks.
Additionally rely on dynamic libraries during debug builds.