I'll expand a little, because I've been feeling the same thing recently.
Python has syntactic support for list[0] comprehensions, which can be used a little like maps:
def addOne(n):
return n+1
l = [1, 2, 3]
[ addOne(n) for n in l ] # [2, 3, 4]
a little like filters:
def isOdd(n):
return n % 2 == 1
l = [1, 2, 3]
[ n for n in l if isOdd(n) ] # [1, 3]
and a little like folds/reductions:
def accum(s):
acc = s
def a(n):
acc += a
return acc
return a
l = [1, 2, 3]
reduce = accum(0)
[ reduce (n) for n in l ][-1] # 6
You can also do Cartesian joins, though I rarely see these.
There are a couple of problems I've run into. The first is that Python's libraries are just not engineered with the idea of using list comprehensions in this way - folding is as awkward as it looks above, exceptions thrown in the list comprehension functions will terminate the comprehension, many python functions alter state and return None rather than a useful output, and so on. The second is that they're amazingly uncomposable, syntactically:
l.map(addOne).filter(isOdd).reduce(accum(0))
is what I'd write in Scala, which is extremely tractable. In comparison, here's the equivalent in python:
[ reduce(nr) for nr in [ nf for nf in [ addOne(nm) for nm in l ] if isOdd(nf) ] ][-1]
You note that I've had to rename the elements, because they "leak" to their surrounding comprehension - this can be quite confusing the first time you see it. Also these are fairly trivial comprehensions, which call functions rather than evaluate expressions in-place - this is well-supported and very idiomatic, but makes comprehension composition much harder.
I find Python's comprehension style very convenient, and I'm sure you could produce an excellent theoretical abstraction over it, but if you're coming at it from the point of view of wanting them to be map/reduce or something equally reasonable-about, you're going to be disappointed. Python isn't an object-oriented language, and isn't a functional language - the more I use it the more I think it's something akin to a collection-oriented language. Maybe that's just the way I use it. :)
[0] and also set comprehensions, dict comprehensions, and generator (lazy list) comprehensions, which are wonderful but exacerbate both the problems I talk about.
Of course you can. But the poor support for lambdas and higher-order-functions makes comprehensions a worse-is-better solution, because you can e.g. pickle comprehension expressions (which you can't do for lambdas), and you don't need to import a module for reduce (in 3.x). I gave up on using them when I realized they were just too frictive (or that they were "un-Pythonic", if you prefer).
I'm sorry, but can you clarify what you mean by poor support for higher-order-functions? And how can one pickle comprehension expressions?
I'm not trying to be argumentative, I just don't have much experience with that. I write functions that return functions/closures regularly, but they're always simple cases.
By poor support for higher-order functions, I mean that e.g. you have to do "from functools import reduce, partial" for fold or partial function application. It's a trivial complaint, I'll give you, but it's one that's bitten me on more than one occasion (you think I'd learn!). There's also no foldr unless you implement it yourself.
I badly misspoke when I said that you could pickle comprehensions, because what I meant was that the language gives you no hint that you might be able to. Pickling
sum([ os.stat(f).st_size for f in os.listdir(".") ])
is obviously (I hope) not going to work. On the other hand pickling
[ lambda n: n % 2 == 0 ]
intuitively ought to, since pickling [ isEven ] would work fine. I've had to rewrite a couple of modules because of this - again, maybe I should have learnt from my mistakes - but it gives me the general impression of "avoid lambdas and functions that regularly use lambdas, because they're occasionally a lot of unexpected work".
import marshal
marshal.dump((lambda n: n % 2 == 0).func_code, file)
#Loading is the same
isEven = types.FunctionType(marshal.load(file), globals())
isEven(4)
When you pickle a function, you get a reference to the function, not the actual function code.
Right - that's usually what I want. (I've used pickle to store tests against game assets, e.g. that a model has all of its textures checked into perforce, or that a texture for a model does not exceed 128x128, unless otherwise specified). Marshalling functions is usually a non-starter for this, since a) it's complicated to analyse the call graph before execution, and b) native functions can't be marshalled - an awful lot of code executed against this asset pipeline is thin bindings over native/Java/C# code. Maybe I have found the 1% of the Python use-cases where lambdas suck a bit and everywhere else it's fine--it would be great if my experience was exceptional and no-one else had ever had a similar problem doing something else.
Python has syntactic support for list[0] comprehensions, which can be used a little like maps:
a little like filters: and a little like folds/reductions: You can also do Cartesian joins, though I rarely see these.There are a couple of problems I've run into. The first is that Python's libraries are just not engineered with the idea of using list comprehensions in this way - folding is as awkward as it looks above, exceptions thrown in the list comprehension functions will terminate the comprehension, many python functions alter state and return None rather than a useful output, and so on. The second is that they're amazingly uncomposable, syntactically:
is what I'd write in Scala, which is extremely tractable. In comparison, here's the equivalent in python: You note that I've had to rename the elements, because they "leak" to their surrounding comprehension - this can be quite confusing the first time you see it. Also these are fairly trivial comprehensions, which call functions rather than evaluate expressions in-place - this is well-supported and very idiomatic, but makes comprehension composition much harder.I find Python's comprehension style very convenient, and I'm sure you could produce an excellent theoretical abstraction over it, but if you're coming at it from the point of view of wanting them to be map/reduce or something equally reasonable-about, you're going to be disappointed. Python isn't an object-oriented language, and isn't a functional language - the more I use it the more I think it's something akin to a collection-oriented language. Maybe that's just the way I use it. :)
[0] and also set comprehensions, dict comprehensions, and generator (lazy list) comprehensions, which are wonderful but exacerbate both the problems I talk about.