Aside from theoretic advice, one of the key things I've found euros best is _not_ to set out to build a big complex system.
Solve small problems that deliver value in distributed way and then iterate and expand. Doing this focuses engineering on the real problems that need to be solved at the scale you are working at. This delivers real understanding of what is going on, both theoretically and specially within your domain.
The retort to this is often you'll never get to scale, or that your architecture will be incoherent. Basically you need to see the whole picture and build towards it.
The caveat is that solving small problems doesn't mean 'do something quickly or hacky'. If you do, sure you'll end up with an incoherent mess.
If you maintain good engineering practices, and focus them on real problems, my experience is you will scale your platform. These practices will keep focused on solving real problems properly, and doing so will give you an understanding of what the next problem is going to be.
Solve small problems that deliver value in distributed way and then iterate and expand. Doing this focuses engineering on the real problems that need to be solved at the scale you are working at. This delivers real understanding of what is going on, both theoretically and specially within your domain.
The retort to this is often you'll never get to scale, or that your architecture will be incoherent. Basically you need to see the whole picture and build towards it.
The caveat is that solving small problems doesn't mean 'do something quickly or hacky'. If you do, sure you'll end up with an incoherent mess.
If you maintain good engineering practices, and focus them on real problems, my experience is you will scale your platform. These practices will keep focused on solving real problems properly, and doing so will give you an understanding of what the next problem is going to be.