I really like databricks at the beginning, Spark clusters magically appear whenever I needed them. No more thoughts were given on how to store the data, etc.. After a few years, I've grown, so did my skill set and my employer. Databricks simply couldn't grow with us at the rate that we needed. No, not because of scaling, but the toolings such as AI/ML, what the market is demanding us to do. Faster iterations and experiments, less cost.

Databricks now reminds me of Oracle. Still a great product, but it's a melting delicious ice cream.