One thing I've done is to deliver a scaled back prototype to let the client play with. It lets me explore the technically difficult areas, show the customer something interesting and get a feel for what I can do (and at what speed).

This lets me give a more realistic estimate for completion and cost. If the customer wants too much more or doesn't like my cost estimate, we're both better off knowing earlier than later.