> I just wanted to point out that there is a fundamental reason that for loops are inferior to map/reduce when working with data.
> For loops is "how" where map/reduce is "what" and that puts the burden on you when you need to parallelize your job.
That's not inferior. That's one advantage that map/reduce has in the set of trade-offs between them. I'm aware of that argument. It's part of what I was trying to get at with the "less powerful" comment (the other side of the coin of being less powerful/more constrained is that it's easier to parallelize).
But ~95% of code never makes it to that step. And I've found that defaulting to it leads to worse code bases, instead of defaulting to the simple intuitive thing and accepting the burden for the other 5% of the code.
Because the burden of that 5% of the code is never the biggest part of scaling. It's usually something like getting the business to understand the implications of CAP theorem and figuring out what trade-offs are best for them. The code is pretty easy, by comparison. Even if I have to chuck out 5% of the files and re-write them over a period of time.
I guess my recommendation is basically to do things that don't scale in your code, because 95% of code won't need to, and the other 5% has a good chance of being re-written anyway during the process of scaling.
> For loops is "how" where map/reduce is "what" and that puts the burden on you when you need to parallelize your job.
That's not inferior. That's one advantage that map/reduce has in the set of trade-offs between them. I'm aware of that argument. It's part of what I was trying to get at with the "less powerful" comment (the other side of the coin of being less powerful/more constrained is that it's easier to parallelize).
But ~95% of code never makes it to that step. And I've found that defaulting to it leads to worse code bases, instead of defaulting to the simple intuitive thing and accepting the burden for the other 5% of the code.
Because the burden of that 5% of the code is never the biggest part of scaling. It's usually something like getting the business to understand the implications of CAP theorem and figuring out what trade-offs are best for them. The code is pretty easy, by comparison. Even if I have to chuck out 5% of the files and re-write them over a period of time.
I guess my recommendation is basically to do things that don't scale in your code, because 95% of code won't need to, and the other 5% has a good chance of being re-written anyway during the process of scaling.