I feel like the writer of the above article answers part of your question when they write:
"My experience is that when you tackle big problems, that go beyond simple execution but require actual strong engineers, hiring will be a problem, there's just no way around it. Choosing people that fit your development culture and see themselves fit to tackle big problems is a long process, integrating them is also time consuming. In that picture, the chosen language isn't a huge deciding factor."
As to the issue of change, you go with something else when the current stack is bad and/or unable to do something you need. Colin Steele wrote about this when he decided to switch Hotelicopter to Clojure. He first wrote about how the existing PHP stack needed to be re-written:
"For example, at that point, the site ran out of one ginormous subdirectory with hundreds of PHP files scattered like chunks of gorgonzola on your salad, sticking to one another with tenacious glee. There was a “lib” directory, which you think would hold much of the supporting library code, but a good fraction of that actually lived in “site”, and some in “server”. The previous programming staff had felt it good and worthwhile to roll their own half-assed MVC framework, including a barely-baked library for page caching (which broke and took the site down at regular intervals), and components for database abstraction that only worked with - wait for it - MySQL. Every single goddamn file was littered with SQL, like bacon bits on this demonic salad. There was a “log” directory, but the search logs weren’t kept there, they were in “server”. Etc., etc. It made you want to eat a gun."
There are certainly situations where the codebase calls for a complete re-write, and, as the writer says in the quote above, finding good engineers is hard, and the choice of language is a side issue compared to the difficulty of finding good engineers.
Colin could've re-written it using Java. He didn't do that because he doesn't like Java ;)
Language may have not be a huge deciding factor until it does.
My question is focused more toward the existing team that probably has some pretty good skill in, let's just say, Java and OOP and now they're turning 180 degree to pure functional like Clojure as opposed to Scala (not that I'm a big fan of Scala or anything like that, but just as an example).
If it were that easy to learn new languages, we wouldn't have issue with hiring as most of the job opening requires the candidate to know specific language however we would like to believe it was not the case.
What do you think people will think when they read this?
"He didn't do that because he doesn't like Java "
Your words imply that somehow the preferences of the top engineers or CTO should not matter. But why should those preferences not matter? If we are talking about people who are talented, then we can start with the assumption that those preferences are probably the distilled wisdom of many years of experience with particular styles of development. We don't need to re-enact the millions of debates that have happened regarding whether Java is good or bad (One side shouting "It is verbose!" the other side shouting: "Type enforcement is good", etc, etc, until the last syllable of recorded time). It is enough to know that good engineers have preferences and those preferences probably have some benefits.
You also wrote:
"My question is focused more toward the existing team..."
What if the current team is terrible? Again, referring to Colin Steele, he fired/allowedd-to-leave the entire team that existed when he first arrived.
What if we turn your question around and ask it from the other side: is change ever justified? I'm guessing you would say "Yes".
By your statement that the preferences of the top engineers should matters is the time where "language does matter". It matters because it's a personal choice regardless whether it is the right choice or not.
"What if the current team is terrible" is not the focused of my question hence providing an example from Colin is arguably not in the right context of this discussion and I'm going to leave it to that because there's no point to discuss as anyone could have switched the underlying technology from Java to Ruby and fire all Java developers and hire Ruby developers.
Colin is a very specific example that is written by the man himself. The justification is sound. Having said that, I could pick some obscure language tomorrow (not that Clojure is these days) and forces my own preferences to go forward as long as I can reach the goal and claim in an article how my personal preferences were the best thing as well even though it may not be the case.
But nobody knows the truth... right?
I'm not trying to be negative here but at the same time no human willing to admit his/her mistakes to be honest. Especially when there are plenty at stakes.
Learning a new language _is_ easy for a good developers. I know of a lot of companies that are doing just that, trying to find good developers, and then train them on the fly in new language. And it works well for them.
The reason why some companies have "hiring problem" is that they require X years of experience with framework Y in language Z, although that language Z takes 1 month to master, and framework Y another two weeks.
Why are they doing it? Because HR is doing hiring, and because "hiring is problem" they get bigger budget for hiring, instead development department getting bigger budget for training.
It's true that you can't learn programming in few months, or even a year, but if you are already solid programmer learning yet another language or framework is easy. Heck, some of us are doing it just for fun.
One of the advantages of JVM based languages is that you always have a safety net - worst case you are going to fall back and do parts in Java or some other JVM based language which, while perhaps painful, is not going to destroy your entire company the way locking yourself into a poor technology choice might in other circumstances. Of course, the JVM has its own limitations, but it's been around long enough that at least they are known quantities.
IMHO there is no better and faster way to learn a new programming language in-depth than to use it seriously in a project. Of course you have to be an experienced developer to do that, and you have to have the time and freedom to afford it :-)
This article motivated me to consider Clojure for a new business project. Clojure can use the JVM infrastructure, that's a big advantage. Java has become too boring for me (C# also).
You know that saying that "the moment you drive a new car off the lot, its value depreciates by half"? Well, the moment you ship a product, the cost of upgrading the underlying technology stack doubles.
In my case at the previous company it was rather simple. The language known to the team didn't fit the bill. So after a trial period we decided on one.
At several companies I've worked at (and my current company) there have been severe limitations on future growth with the existing technology choices. It's more important to be able to remain agile and have a high ceiling for growth than to use the most familiar thing always.
> Sometimes I do wonder why people decided to use a new language for a critical/important piece of their product while learning on the go as well.
Because flying by the seat of your pants, making shit up as you go and putting something into production without any real understanding of what the fuck is about to happen is one hell of an adrenaline rush.
Sometimes I do wonder why people decided to use a new language for a critical/important piece of their product while learning on the go as well.