Not a pilot myself but it seems that a large part of the danger with flying is that when something goes wrong you are much more likely to have a high speed crash. Cars don't even usually travel at speeds that planes crash at.