I programmed java web apps for 10 years before I switched to python, 4+ years ago. I feel that I'm much more productive using python and can get much more done in a shorter period of time, and to be honest, I'm much happier when I develop in python. Here are some of the reasons why I think python is better then Java based on my personal experience, your milage may very.
Web Frameworks:
When I first start programming web apps in Java, Struts just came out, and it wasn't great, but it was the best thing available. I created a bunch of struts apps, and a few in other frameworks along the way. Whenever a new framework came out (Tapestry, Wicket, GWT, stripe, grails, AppFuse, Play, RichFaces, Spring, etc), I would try it out and see if it was any better, and most times it was only a little better, and sometimes not better at all. I do have to say the play framework is a step in the right direction.
Batteries not included:
One of the most annoying parts of Java was the fact that most of the libraries that you use were not included in java itself, you had to include a ton of 3rd party libs from places like apache commons. If you use something like hibernate with any other large library, you end up in Jar dependency hell, where hibernate needs one version of a jar, and something else needs another version. If you load the jar files in the wrong order, you are out of luck. You need to depend on tools like maven, and ivy to manage your dependencies, and this just brings in more dependencies into your project which results in projects being huge. I had some war files 100MB+ war files for the simplest web apps.
Too many options:
For some reason there seems to be way too many different ways to do the same thing in Java. There are over 38 different web frameworks for java according to wikipedia (http://en.wikipedia.org/wiki/Comparison_of_web_application_f...) and 23 different ORM's (http://en.wikipedia.org/wiki/List_of_object-relational_mappi...) just to name a couple of examples. If you look at other languages they have a more reasonable number. Some people think that having lots of options is a good thing, but it isn't it leads to a lot of wasted effort in the developer community, everyone is reinventing the same wheel, and if you are a new person to the language you have too many option to pick from.
App servers:
Java web applications are really heavy, and require a lot of resources to run. They are especially memory hungry. Like any piece of software they can be tuned to reduce their resource footprint, but compared to other languages their out of the box setup is horrible. In my past I have used weblogic, websphere, Jboss, tomcat, and jetty. I only used the first three when I was forced to use EJB's, but even if you aren't using EJB's they were large app servers and sometimes hard to configure and get running correctly. Tomcat and Jetty are much better and easier to setup, but are still resource hogs.
App Hosting:
If you aren't running your own server it is real hard to find shared hosting for your java apps at a reasonable price. The main reason is because java apps require much more memory compared to other languages, so it doesn't make sense for a shared hosting provider to spend their valuable RAM running a java site, when they could run 5 php sites in the same place. That means there are less providers offering java hosting, which in turn means higher costs to run your website.
Development Time:
When I developing in java, I found myself much slower then what I can do in python. I would need to make a change, compile, redeploy and then test, and this slows down the iterative process. I know there are ways to make this faster, but even at it's best, I felt much slower then what I can do in python.
There is also a lot less boilerplate code to do the same thing in python, so I spend less time developing the code as well.
Java just feels over engineered in a lot of parts, A lot of the API's and interfaces are just way to complicated for what you want to do. And everyone and their brother thinks they are a java architect and this results in big complicated systems that are hard to use and develop with.
IDE:
When I was developing in Java, I felt stuck to the IDE, I was lost without it. IntelliJ is the best IDE's on the market, and it was hard switching to python because there wasn't anything like it for python. So instead of an IDE, I just used textmate, which is just a normal text editor. It was hard at first, but because it was just a text editor, it was a really fast and responsive application. I could open my whole project in a few seconds, whereas when I want to open a project in an IDE it could take a minute or more, with a machine with a ton of RAM. The makers of IntelliJ came out with a python editor called pycharm, I bought it when it first came out, and it is great. But what I realized is that I don't need an IDE for python, I'm fine with a text editor. When I go back to working on Java web apps which I have to do from time to time, I try to use the text editor, but I haven't quite mastered that yet. I personally need the IDE for Java more because If I mess up something it takes longer to recompile and redeploy, which slows me down.
ORM:
When I first started using Hibernate as an IDE, I thought it was great, it had it's problems, and it wasn't perfect, but it was better then what I was doing before. I was happy with it, until I did an application with Django's ORM on a python project, and that opened up my eyes, that is how an ORM is suppose to work. After that project I went back to hibernate, and I just felt disappointed, and longed for going back to Django's ORM. Another great python ORM is sqlalchemy, which is similar to Django's ORM, but a little different. I have limited experience with ROR's ORM, but from what I remember, it was pretty good as well.
Templates:
The web templating systems in Java aren't that good, and I think I have tried them all (tiles, freemarker, velocity, etc). Most of them offer only basic functionality and are a pain to work with. On the Python side, my two favorites are Django templates and Jinja2, they have everything that I could need in a templating engine, and are really easy to use.
I think all of this was true 4 years ago. But Java EE 6 has changed a lot of this. My main experience has been on the Microsoft stack, but I have learned and worked with a lot of tech, including Rails and Django. Last year I worked on a project for which we chose Java EE 6.
There's a lot of good stuff there; for instance simple, bloat-free architecture, JPA (the new persistence architecture), CDI (context and dependency injection), JAX-RS (REST), JSF + PrimeFaces (very productive front-end development) and lightweight, fast web application servers (Tomcat 7, Glassfish).
And you have access, when and if you need it, to a very mature, stable and capable set of libraries, open source projects and tools. Plus the ability to deploy to almost anywhere including PaaS platforms.
For what it is worth, I have used Java 7, JPA (with hibernate), tomcat 7 and glassfish, and I didn't notice much difference. Jersey (JAX-RS) is pretty nice, one of the better libraries to come out of Java in a while.
I do agree that Java is moving in the right direction, but it is a very slow progression. I think the major cause of the slowness is because of the fact that SUN was bought by Oracle, and it took a little while for that to settle down, and now things are slowly moving forward again.
One of the things that Java did right was their WAR files. It is the envy of most other programming languages. You compile your WAR file, and deploy anywhere assuming it is a compliant servlet container. I wish Python had something like this.
You are right the popularity of PaaS's like dotCloud make it much easier to deploy and run your application. On dotCloud you just need to push up your WAR file, and you are running, by far the easiest of all the languages to deploy.
I really like Java as a language, I just think it has a long way to go on the web application front, before it can catch up to the features and speed of development of the other languages.
This is a very thorough, albeit not necessary accurate, story of one's experience using Java.
I agree that there are TONS of duplicated effort out there:
Tapestry, Struts 1/2, JSF, Wicket, etc are all "MVC _and_ Component Oriented".
Hibernate vs JPA (yes, I know Hibernate is JPA 2 compliant, and yes, I know JPA 2 has less features).
XML processors. Java common utilities, the list goes on.
But these days there seems to be a quite select few of winners in my book: Google Guava, JUnit (TestNG is neat, but JUnit is good enough for 80% cases), GWT, Spring Framework or Java EE6 (your choice, or mixed).
Java based ORMs can't beat dynamic language ORMs in terms of "it's fun to use" (or less setup). Period. But having said that, Java based ORMs took different approach compare to ActiveRecord or Django Model. Java based ORMs are more similar to Ruby DataMapper; they required a little bit of setups (have to define your own model, etc) but in the long run, I think these second concept (or known as the Data Mapper pattern: http://martinfowler.com/eaaCatalog/dataMapper.html) may have been a better choice. Of course that is for the long run... not weekend project, get-prototype, get-funded, and get rich-quick scheme.
Java-based App Servers are heavy but with a caveat, depending on your loads, Java-based App Servers may use less resources (or degrade gracefully) because they are utilizing thread vs bringing up Rails + Ruby process per request.
When it comes to dev-time that depends on your experience. For example: I use JUnit, Eclipse, and JUnit plugin as faux-REPL. No re-compilation needed. Setting up the tools I mentioned is done via Maven + vanilla Eclipse. In another word: don't get too gung-ho with the number of tools I mentioned. It's fairly simple to get it to work.
I do have to say this though: as much as people hate Maven, there's no better and more complete build/dependency management/project management tools in other ecosystems (Ruby, Python, Lisp, Haskell) compare to Maven. None.
When it comes to IDE, I love the shortcut navigation keys of my IDE, Eclipse. I'm contemplating to purchase RubyMine in the future as well because quite frankly I can't live with VIM/Emacs. NetBeans has a decent Ruby IDE for a while until Oracle decided not to invest in Ruby anymore. Not sure what the stats these days. I need my "press X to go to Class/Method definition" to read the actual source code.
Yes, Java has been slow moving forward under SUN. Things have changed under Oracle. They're pushing it aggressively and toward the right direction.
People keep saying that the "WAR" deployment is one of the best thing that happened in Java. I heard Oracle is preparing Java EE7 that supports multi-tenant architecture.
Web Frameworks:
When I first start programming web apps in Java, Struts just came out, and it wasn't great, but it was the best thing available. I created a bunch of struts apps, and a few in other frameworks along the way. Whenever a new framework came out (Tapestry, Wicket, GWT, stripe, grails, AppFuse, Play, RichFaces, Spring, etc), I would try it out and see if it was any better, and most times it was only a little better, and sometimes not better at all. I do have to say the play framework is a step in the right direction.
Batteries not included:
One of the most annoying parts of Java was the fact that most of the libraries that you use were not included in java itself, you had to include a ton of 3rd party libs from places like apache commons. If you use something like hibernate with any other large library, you end up in Jar dependency hell, where hibernate needs one version of a jar, and something else needs another version. If you load the jar files in the wrong order, you are out of luck. You need to depend on tools like maven, and ivy to manage your dependencies, and this just brings in more dependencies into your project which results in projects being huge. I had some war files 100MB+ war files for the simplest web apps.
Too many options:
For some reason there seems to be way too many different ways to do the same thing in Java. There are over 38 different web frameworks for java according to wikipedia (http://en.wikipedia.org/wiki/Comparison_of_web_application_f...) and 23 different ORM's (http://en.wikipedia.org/wiki/List_of_object-relational_mappi...) just to name a couple of examples. If you look at other languages they have a more reasonable number. Some people think that having lots of options is a good thing, but it isn't it leads to a lot of wasted effort in the developer community, everyone is reinventing the same wheel, and if you are a new person to the language you have too many option to pick from.
App servers:
Java web applications are really heavy, and require a lot of resources to run. They are especially memory hungry. Like any piece of software they can be tuned to reduce their resource footprint, but compared to other languages their out of the box setup is horrible. In my past I have used weblogic, websphere, Jboss, tomcat, and jetty. I only used the first three when I was forced to use EJB's, but even if you aren't using EJB's they were large app servers and sometimes hard to configure and get running correctly. Tomcat and Jetty are much better and easier to setup, but are still resource hogs.
App Hosting:
If you aren't running your own server it is real hard to find shared hosting for your java apps at a reasonable price. The main reason is because java apps require much more memory compared to other languages, so it doesn't make sense for a shared hosting provider to spend their valuable RAM running a java site, when they could run 5 php sites in the same place. That means there are less providers offering java hosting, which in turn means higher costs to run your website.
Development Time:
When I developing in java, I found myself much slower then what I can do in python. I would need to make a change, compile, redeploy and then test, and this slows down the iterative process. I know there are ways to make this faster, but even at it's best, I felt much slower then what I can do in python.
There is also a lot less boilerplate code to do the same thing in python, so I spend less time developing the code as well.
Java just feels over engineered in a lot of parts, A lot of the API's and interfaces are just way to complicated for what you want to do. And everyone and their brother thinks they are a java architect and this results in big complicated systems that are hard to use and develop with.
IDE:
When I was developing in Java, I felt stuck to the IDE, I was lost without it. IntelliJ is the best IDE's on the market, and it was hard switching to python because there wasn't anything like it for python. So instead of an IDE, I just used textmate, which is just a normal text editor. It was hard at first, but because it was just a text editor, it was a really fast and responsive application. I could open my whole project in a few seconds, whereas when I want to open a project in an IDE it could take a minute or more, with a machine with a ton of RAM. The makers of IntelliJ came out with a python editor called pycharm, I bought it when it first came out, and it is great. But what I realized is that I don't need an IDE for python, I'm fine with a text editor. When I go back to working on Java web apps which I have to do from time to time, I try to use the text editor, but I haven't quite mastered that yet. I personally need the IDE for Java more because If I mess up something it takes longer to recompile and redeploy, which slows me down.
ORM:
When I first started using Hibernate as an IDE, I thought it was great, it had it's problems, and it wasn't perfect, but it was better then what I was doing before. I was happy with it, until I did an application with Django's ORM on a python project, and that opened up my eyes, that is how an ORM is suppose to work. After that project I went back to hibernate, and I just felt disappointed, and longed for going back to Django's ORM. Another great python ORM is sqlalchemy, which is similar to Django's ORM, but a little different. I have limited experience with ROR's ORM, but from what I remember, it was pretty good as well.
Templates:
The web templating systems in Java aren't that good, and I think I have tried them all (tiles, freemarker, velocity, etc). Most of them offer only basic functionality and are a pain to work with. On the Python side, my two favorites are Django templates and Jinja2, they have everything that I could need in a templating engine, and are really easy to use.