I'm sorry, but can you clarify what you mean by poor support for higher-order-functions? And how can one pickle comprehension expressions?
I'm not trying to be argumentative, I just don't have much experience with that. I write functions that return functions/closures regularly, but they're always simple cases.
By poor support for higher-order functions, I mean that e.g. you have to do "from functools import reduce, partial" for fold or partial function application. It's a trivial complaint, I'll give you, but it's one that's bitten me on more than one occasion (you think I'd learn!). There's also no foldr unless you implement it yourself.
I badly misspoke when I said that you could pickle comprehensions, because what I meant was that the language gives you no hint that you might be able to. Pickling
sum([ os.stat(f).st_size for f in os.listdir(".") ])
is obviously (I hope) not going to work. On the other hand pickling
[ lambda n: n % 2 == 0 ]
intuitively ought to, since pickling [ isEven ] would work fine. I've had to rewrite a couple of modules because of this - again, maybe I should have learnt from my mistakes - but it gives me the general impression of "avoid lambdas and functions that regularly use lambdas, because they're occasionally a lot of unexpected work".
import marshal
marshal.dump((lambda n: n % 2 == 0).func_code, file)
#Loading is the same
isEven = types.FunctionType(marshal.load(file), globals())
isEven(4)
When you pickle a function, you get a reference to the function, not the actual function code.
Right - that's usually what I want. (I've used pickle to store tests against game assets, e.g. that a model has all of its textures checked into perforce, or that a texture for a model does not exceed 128x128, unless otherwise specified). Marshalling functions is usually a non-starter for this, since a) it's complicated to analyse the call graph before execution, and b) native functions can't be marshalled - an awful lot of code executed against this asset pipeline is thin bindings over native/Java/C# code. Maybe I have found the 1% of the Python use-cases where lambdas suck a bit and everywhere else it's fine--it would be great if my experience was exceptional and no-one else had ever had a similar problem doing something else.
I'm not trying to be argumentative, I just don't have much experience with that. I write functions that return functions/closures regularly, but they're always simple cases.