And then, five years after you've left the company and some system inevitably collapses with nobody having a clue as to what went wrong you'd finally realize the wisdom of all that.
But it's no longer your problem.
So, please don't take it personal, but 'the well intentioned types who write these articles' tend to be the people that then get called in to clean up the mess.
And then - belatedly - the job gets done properly to keep the company in business, assuming there is still time enough to do so.
Just this week I had a nice inside look at the kind of mess gets left behind when the original duct-tape-and-spit guy leaves the company and lets his former co-workers clean up their mess. It isn't pretty, and a chapter 11 isn't an unlikely scenario, so forgive me if I take a harsher than usual look at the attitude that causes this sort of thing.
Note that the 'free market' doesn't have a horizon much longer than the next quarterly shareholder report, and that your typical software product lives a multiple of that interval. So software made with short term goals in mind will create long term headaches.
Your two week marketing campaign gets a pass. But your decade long backend project does not, nor does your real time medical device controller, ECU, database system or operating system.
> And then, five years after you've left the company and some system inevitably collapses with nobody having a clue as to what went wrong you'd finally realize the wisdom of all that.
But it's no longer your problem.
If that were a problem in reality, the markets would be punishing companies where that happens. It's not a real problem that that happens. Management pretends to be upset, but in reality it's not a huge deal. Entropy is normal in apps and everything else. To continue the analogy from my original comment, do companies really go into crisis mode when one of the many cars in their fleet inevitably "collapses"? No. They build or buy a new one and life goes on.
> nor does your real time medical device controller, ECU, database system or operating system.
Yep. Those are the rare cases I talked about. It's a tiny fraction of total programmers building databases and stuff like that.
> If that were a problem in reality, the markets would be punishing companies where that happens.
Oh they do, I can show you plenty of examples. But it is never the problem of the people that created the issue in the first place.
Think of these things as time-bombs of technical debt. They'll blow up sooner or later, usually later, and that makes it that much harder to deal with the fall-out.
Also: for all the lessons about economy made here: I would happily argue that doing things right is actually cheaper in the long run and possibly also cheaper in the short term, by applying the techniques described in proper measure you can save yourself a ton of headache.
But of course that would first require a basic understanding of what the article is trying to put across, which if your time horizon is short and your deadlines are looming likely isn't going to be on your agenda.
> It's a tiny fraction of total programmers building databases and stuff like that.
Software you build tends to live longer than you think and tends to be incorporated into places that you can not foresee when you make it.
The 'tiny fraction of total programmers building databases' should include the huge fraction of programmers building embedded systems, APIs, operating systems, libraries and so on. All of those will have life-spans in the decades if they're done halfway right.
You seem to have a poor understanding of both entropy and markets. Even the perfectly built program will soon become useless. Car companies are quite profitable building cars that "collapse" far before the actual potential which might be a car that lasts 50 years. But no longer can pass emissions tests... Zuck is about to surpass Buffet as the third richest person in the world. On an app built with PHP! I don't think much more needs to be said to support my original point.
> You seem to have a poor understanding of both entropy and markets.
And you're a bit assuming and rude. Your argument also isn't as bulletproof you want to make it sound. What is the argument here, anyway? There's no need to improve technique for an average programmer because an outlier system (Facebook) is written in a language commonly associated with poor programming practices, with some handwaving about markets and entropy sprinkled on top?
Sorry if I came off that way. I was in a rush on the way to an event and I thought I was just being honest about the weakness in his argument.
What's the argument here? That stakeholders have requirements that don't have to do with robustness like budget and deadlines and that your software has a shelf life and sometimes it's ok if it eventually breaks, just like cars and even the laptop I'm typing this on will. Is that an unreasonable perspective?
And Facebook is an outlier? Really? Even when we add Wordpress, Wikpedia, Flickr, MailChimp and a long list of the most successful websites in the world to that list?
> And Facebook is an outlier? Really? Even when we add Wordpress, Wikpedia, Flickr, MailChimp and a long list of the most successful websites in the world to that list?
Yes, FB is an outlier -- one out of million companies. Only 5-10 companies out of those millions made this current model work. So their existence and "success" proves absolutely nothing.
You have a strange understanding of the word "successful".
Facebook is certainly not "successful" because it neglects good tech. If anything, they rewrote PHP itself so as not to have to rewrite their customer-facing software. How is that for your "tech excellence is not important" argument? They rewrote the damned runtime and even added a compiler.
So please define what "successful" means to you. "A lot of people using FB" is a temporary metric, even if it lasts for decades. It's not sustainable per se. It relies on hype and network effect. These fade away.
@jacquesm's points are better argued than yours. Throwing words like "free market" and "entropy" does not immediately prove a point.
I will give you the historical fact that there are many throwaway projects but he's also right that the fallout from the tech debt they incurred is almost never faced by the original author. Throw in the mix the fact that many businessmen are oblivious on what do the techies do in their work hours exactly and one can be easily misled that technology perfection is not important. Seems that you did.
Final point: I am not arguing for 100% technical excellence. That would be foolish. We would still be refining HTTP and the WWW in general even today and internet at large would not exist. But the bean counters have been allowed to negotiate down tech efforts to the bare minimum for far too long, and it shows everywhere you look.
(My local favourite restaurant waiters' smartphone-like devices for accepting and writing orders are faulty to this day because some idiot bought a cheap consumer-grade router AND made the software non-fault-tolerant, being an everyday example.)
> Only 5-10 companies out of those millions made this current model work
Stats? Evidence? I mean hundreds of of thousands of companies use PHP and other forms of less than perfect tech.
Websites all over the world seems to get the job done even when JavaScript with all its warts is used. I like JS for the record, but it does have warts.
> even if it lasts for decades.
You're saying the same thing I said. That stuff breaks. That companies come and go in and out of fashion. I also think it's interesting that you're calling FB an example of tech excellence but saying it's going to fade away. Choose one?
> How is that for your "tech excellence is not important" argument?
I never made any such argument. Not even close. I only said quality is not the only requirement and might sometimes not be a requirement at all.
Most of the code I write is high quality. I put a lot of effort into code reviews too. I mentor more junior devs around quality. My original post is actually much more nuanced than you are claiming.
> Final point: I am not arguing for 100% technical excellence. That would be foolish. We would still be refining HTTP and the WWW in general even today and internet at large would not exist.
Exactly. That's in the spirit of my original post. Maybe re-read it to see that we mostly agree instead of making my position into something it really isn't?
> Stats? Evidence? I mean hundreds of of thousands of companies use PHP and other forms of less than perfect tech.
Oh, I meant companies at the scale of Facebook. There aren't too many of them, would you not agree?
> I also think it's interesting that you're calling FB an example of tech excellence but saying it's going to fade away. Choose one?
FB does a lot of open-source projects. Their devs are excellent. That doesn't mean that their main value proposition is not comprised of code of the kind you speak about. No need to choose one, both can coexist in such a huge company like FB.
> I never made any such argument. Not even close. I only said quality is not the only requirement and might sometimes not be a requirement at all.
Well alright then. I am not here to pick a fight, you should be aware that you came off a bit more extremist to me and a bunch of others than you claim. But these things happen, I can't claim your intent because of a few comments, that's true.
Me and several others' point is that quality plays a part bigger than what you seem to claim. I also knew many devs that decided they won't ask for permission to take the [slightly / much] longer road and this decision paid off many times over in the following months and years.
Sometimes businessmen simply must not be listened to. I can ship it next week alright. But I can skip a few vital details, namely that I did not take into consideration some stupid micromanagement attempts to teach me how to do my job ("nobody cares about this arcane thing you call 'error logger' or 'fault tolerance', just get on with it already!"). Such toxic work places should be left to rot, that is a separate problem however.
> You seem to have a poor understanding of both entropy and markets.
You're hilarious. On an annual basis I end up being the deciding factor in the allocation of a fairly large sum of VC money and tech quality is a big deciding factor in that.
Fortunately there are plenty of successful companies that do a much better job than what you are describing you are doing.
What I described is taking stakeholder requirements into consideration like budget, deadlines, and the expected useful life of the software. That's not best practice? If it's not could you describe what I should be doing differently? I thought it was a step up when I finally realized software quality is not the only requirement competing for attention, but since I've spent my entire career paying close attention to what's best practice I'm also willing to learn from you and improve.
Every piece of software should be high quality even if it's a throw-away website used for 2 weeks? You'd expect the programmer to give a mathematical proof for the website code?
I also described that stuff does eventually break like the laptop I'm writing this on and it's not end of the world. We expect stuff to break.
Can you explain why my wife and almost every accountant in the world amortize intangible assets like in-house developed software and give it a useful life span?
> Can you explain why my wife and almost every accountant in the world amortize intangible assets like in-house developed software and give it a useful life span?
That's learned behaviour post factum. Had we (the software industry at large) done better then they wouldn't have the countless examples to learn from and turn them into a habit. Don't conflate things.
> I also described that stuff does eventually break like the laptop I'm writing this on and it's not end of the world. We expect stuff to break.
You are arguing extremes. The fact that a physical object will eventually suffer wear and tear no matter what has zero correlation to the fact that most software can be much more robust and long-lived but extreme time and money constraints prevent it from being so.
Our points of view can meet but not until you admit that a learned behaviour is something that can be changed if enough people with money stop turning cutting corners into an olympic sport.
> Had we (the software industry at large) done better then they wouldn't have the countless examples to learn from and turn them into a habit.
Nonsense. Stuff breaks. Everything. Even stuff made to a very high standard.
> most software can be much more robust and long-lived
It can't. Not because it can't be much more robust. But because most software is simply obsolete after a few years or maybe ten years if you're lucky. Software doesn't live in a static world where nothing changes. Laws changes, accounting practices are modernized, entire industries come and go, and everything is in a constant state of change. Maybe you haven't been around long enough to see it. I have. I've seen perfectly built in-house custom inventory software replaced a few years later by something like SAP because upper management decides the pros of having an integrated logistics system
far outweighs the features of one in-house app. Sorry, but I've been in the business world far too long to fall for the idea that software is ever long-lived. There are some rare cases that it can be. The 40 year old COBOL programs some banks run to process massive amounts of transactions over night. And guess what? As long lived as they are, they are being re-written little by little because it's damn hard to find anyone under 65 who is actually interested in maintaining them. Software does not live in a vacuum.
Oh well, guess I better go into research and find academic investments because all this "make this web app yesterday" crap is getting very old and annoying...
tl;dr I have thousands of lots of horse carriage wheels built to the highest standard that will last another 500 years. Any interest? Quite a few AM radios too...
They didn't know that. Things that last also retain their usefulness. My entire point really. As an aside, let's hope you didn't use up extra resources that are not renewable in making something so durable but eventually useless. Would I favor regulations that prevent the opposite? Stuff that breaks before the end of its useful life just to create another purchase? I would consider it. Both ends of the spectrum create problems.
You give too much credit to the businessmen. They don't care about environment, they care about more sales and there do not exist any lengths they would not go to in order to achieve those.
I'd say producing one durable piece of tech vs. 5-10 non-durable pieces of tech is still more sustainable for the environment, would you not agree?
I love strong regulations that try to reduce or eliminate negative externalities (like pollution and toxic waste), especially when the cost of the those externalities are collected at the point of purchase. They aren't very popular though. When we try things like a sugar tax to reduce the $$$ billions a year that diabetes costs us, people starting screaming about nanny state and their freedom.
I'm not sure how we would require products be durable though? Who would be Czar of how durable things must be? Seems like that person would have a huge amount of power over which industries are profitable and by how much. I'm open to ideas though.
As an Eastern European I would definitely entertain the idea that totalitarianism is not entirely wrong. IMO the world needs politicians with stronger will who are less obsessed by next elections and are religiously persecuting criminals and opportunists... however the politicians are parts of these criminals so yeah, welcome to 21st century.
In any case, regulations have proven time and again to mean absolutely nothing unless enforced very strictly and with a very heavy hand (flat percents of the offender's gross income, and I mean from 20% and up, not some petty 1-2%). But that won't happen -- lobbies, rings of companies, "anonymous" donations, things like that... the status quo is too deeply entrenched. But we can dream, right?
My programming career has gone from "I don't know the best way to write something", to "I know the best way to write something and I'll do it this way every time", to "often it doesn't fucking matter, just get it done and move along to the next problem".
It depends on the part of the industry one works in. I'm basically a backend developer of web and mobile applications. I don't only code them, I also design or help designing most of the ones I work on.
Most of what we do in this market is reading some form data, JSON, XML, parse it, read/write to a database or some other API, gather results, send them back to the browser as either HTML or JSON.
I checked right now when it was the last time I had to devise some clever algorithm. I was at the end of 2015 in a Rails project. I remembered all the stuff about invariants from my CTO back in 1995 and it did really help. However in this market that's a once in 10 years occurrence, unless one keeps grinding through coding interviews on artificial problems (because those companies are affected by the streetlight effect.)
So, from my point of view "for most applications it does not matter, but for a few of them it does." It probably matters when writing some of the algorithm in the web browser I'm using right now.
It's clear a lot of people are taking my initial posts and comments wrong. I work really hard to not create technical debt. All I said was that there are competing requirements you should be weighing and not all of those requirements are technical. And that even the highest quality stuff eventually does break. Of course you should always strive to build high quality and even beautiful code. I take pride in my work. But part of that pride comes from being able to juggle multiple competing requirements and make the best decision for the company.
Sometimes creating technical debt is the right decision. Sometimes it's "get over this budget hump using two devs instead of five or we go out of business". And then you do get over it and all of a sudden the company is hugely successful and it's a good problem but you're working really long hours just trying to keep up with helping all that cash flow in... the real world rarely makes conditions so perfect you can write perfect code.
I strongly dislike it when I make the decision to create tech debt, but I will at least leave comments or documentation for the next guy on the parts I think could use more love.
And it's rare I do create it. I actually spend a large part of my time refactoring code and making it better and reducing technical debt. It's one of my favorite things to do. And that points right back to my original post. You know what's interesting about the code I refactor? It's working code. It solves the problem. I wouldn't want to build something bigger on it without refactoring. But I also wouldn't curse out the guy who wrote it. He solved the problem at the time within the budget and time constraints and other competing requirements he was juggling. Good for him.
> Sometimes it's "get over this budget hump using two devs instead of five or we go out of business". And then you do get over it and all of a sudden the company is hugely successful...
Seems you are judging from the SV / startup / USA bubble. Most of the world works in VERY different conditions than that.
I mean yeah, bosses whipping their devs for maximum throughput happens everywhere but the combination of factors you describe seems to be specific for USA.
I've worked in Europe, Asia, the USA, for quite a few startups, huge corps, and have started a couple of successful companies here and there myself. I have a long and varied career that I am very grateful for.
A lot of startups are actually obsessed with quality to the point of failing. I've seen that. I've also seen the opposite. There is a lot of variation in the myths founders create that they follow like a religion because they believe it's the one trick they need for success ;-)
You do eventually run out of money. It's so much more important to ship something to customers to get some feedback than it is to get it perfect. It's a very tricky balance to get right though. It has to look and work good enough to not scare potential customers away.
> It's so much more important to ship something to customers to get some feedback than it is to get it perfect.
I don't disagree, that's unequivocally true.
As a programmer myself however I know that "I'll get back to it later and fix it" is usually a lie...
Haven't founded a business yet and I think I'll do that eventually but it also seems that "do your market research first and foremost" is an universal rule.
EDIT: As an European I should add that most of us never start a business unless they already have several customers willing to pay lined up in an orderly queue. I feel that too many Americans (probably not only them) start a business based on pure enthusiasm and hand-waving that motivated them during a few business events where people vaguely expressed an interest in their idea.
Sometimes you just don't get a choice to do it the right way, or the most elegant way.
You've got a project manager breathing down your neck, pressure to deliver functionality, you've explained the technical debt issues etc - and you're still told to do it the wrong way.
Then years later, the only thing on the code is your name and some very dodgy looking decisions. That project manager is still probably kicking around blaming people and earning a fortune hehe.
> You've got a project manager breathing down your neck, pressure to deliver functionality, you've explained the technical debt issues etc - and you're still told to do it the wrong way.
Do the right thing and quite that toxic work place. You will help natural selection: extreme corner-cutters should go out of business. I knew a guy (programmer) who destroyed a company by simply leaving.
You can only tell a shoe craftsman to create shoes out of cow dung and grass for so long.
"If that were a problem in reality, the markets would be punishing companies where that happens."
Quite the opposite: markets have been rewarding it for some time. The richest companies mostly had buggy software. What got them revenue was everything but flawless quality. Then, once their customers were locked in via other tactics, the customers kept paying them so long as the software continued to work with a switch costing too much. They also often patented anything that could block competitors.
Even quality-focused customers often want specific features even if it leads to occasional downtime. Also, releases improving on features fast. I think Design-by-Contract with automated testing can help plenty there with the pace necessary for competitiveness in a lot of product areas. The markets don't care about perfection, though. The company's priorities better reflect that.
Why should it be a priority? Who should pay for it if customers are ok with the status quo? Where is the competition offering to fill the market gap with products that are security minded? I'm not in love with free markets as the end all to solve all problems worth solving, but I think these questions are worth answering. It's either customers willing to pay for something or taxes. Security will probably end up being much like national defense. No one willing to voluntarily pay for it, but it being in the best interest of all to be "forced" to pay for it.
Because otherwise one day you might find yourself facing bankruptcy.
I'm a strong advocate for liability for software producers because it seems we as an industry are categorically incapable of doing the right thing. Until it directly affects the bottom line this likely won't change.
Customers are not 'ok with the status quo', they're clueless, and the only thing that changes is corporate profits.
In the end the difference between doing it right and doing it wrong is more related to long term vs short term thinking than that it would affect the bottom line in a more dramatic fashion (such as would be the case with liability).
> Because otherwise one day you might find yourself facing bankruptcy.
> I'm a strong advocate for liability for software producers because it seems we as an industry are categorically incapable of doing the right thing. Until it directly affects the bottom line
These two statements seem to contradict each other. If it's not directly affecting the bottom line today, how would one go bankrupt?
I do agree with you there should be some force pushing to eliminate this negative externality. We could compare poor security practice with toxic waste. In general the force I'm talking about is government that creates smart regulations. You'd like to do it by allowing consumers to sue after the damage has already been done. I'm not going to get into that debate, but both of us have proposed solutions and I agree either would be an improvement over what we have today.
That's exactly my point. The markets pay for what they care about and ignore/punish what they don't. They rarely pay for security. They rarely punish insecurity. Even in security, it's usually just enough to not look incompetent when a breach or lawsuit happens. Both consumers and businesses care very little about software quality or security if assessing by what they buy, use, and stick with. You can easily prove this by giving them choices between feature- and security-focused products. Even when the latter are free/cheap and highly usable, the market still decides against them massively. The voters also don't push for regulation or liability of this stuff. Many straight-up vote against it.
So, the management at these companies operates in a market that barely cares about security or mostly cares about appearances/perception. The incentive structure rewards working against quality or security. The costs are externalized with little happening to counter that. So, the rational actors ignore quality/security as much as they can. Programmers should act no different in a system if maximizing selfish gain or minimizing work.
Personally, I'm a utilitarian that considers security a public need. I strongly favor regulations and liabilities to increase the baseline of our security. Just cover the basics like memory safety, input validation, secure admin interfaces, error handling, backups, and recovery if nothing else. The stuff we can already do today with free tools that the suppliers just don't care about. That's not what the market is, though. So, I can't blame people in it for giving it what it wants if they risk losing money or perishing focusing on idealistic goals. I do encourage those doing business with utilitarian style, though. It ranges from easy to hard work they don't even have to do. Also especially glad when I'm one of their customers. :)
Mothers used to die from doctors not washing their hands. The lack of a price signal didn't mean it wasn't a problem, it meant none of the doctors understood how to solve the problem (and neither did the patients).
Just to add, when Lister introduced antiseptic methods he was met with strong resistance from those same doctors who were equal parts annoyed with the messenger, and the message. It’s a hard thing to realize that you’d be killing thousands of people in your ignorance after all. It took quite a long time for his methods to be widely accepted and put into practice. Even when understanding emerges, you have to watch out for the entrenched interests defending themselves against change.
The truth is somewhere in the middle. What I’ve noticed over the years is that if you allow yourself to get in the habit of writing quick and dirty code you learn the wrong habits and gradually lose the ability to write correct code for complex problems. So I do favor the correctness approach.
But ... code has to be maintainable, meaning it should be simple to the person that maintains it next. Typically that means no cleverness and no obscure languages or frameworks. Choosing eiffel only makes sense if you know the next maintainer will be proficient at eiffel.
There is no need to resort to tools such as Eiffel to take some very good lessons about what the article is trying to say. Time has moved on since then, Eiffel has had its day, but just like Smalltalk and other obscure languages there is some underlying truth that is well worth studying.
> Choosing eiffel only makes sense if you know the next maintainer will be proficient at eiffel.
Choosing Eiffel makes the next maintainer proficient in Eiffel by definition, because that will be the job requirement for the maintainer position. Unless the people responsible for hiring cheap out, that is.
What you're advocating here is optimizing solutions strongly towards being maintainable by cheap, interchangeable workforce. It's a valid goal - presumably one the management would like - but sometimes (often) it's not worth the extra cost in complexity, both early and later on.
(Tangentially: programming is a profession. It should be entirely expected of people to be able to learn new things on the job.)
There's another issue here: many companies, when finding themselves in need of a proficient Eiffel programmer will just take one of their good programmers in other languages and ask: Would you like to learn Eiffel? And programmers usually enjoy learning new things, so someone will say yes.
I personally did that few times in my career: accepting job with technology I barely knew at the time, because I thought it would be fun to learn it.
You can imagine how that usually works out: software written by someone who was learning the programming language on the job.
> What I’ve noticed over the years is that if you allow yourself to get in the habit of writing quick and dirty code you learn the wrong habits and gradually lose the ability to write correct code for complex problems
As the original commenter who started this thread, I'd like to make it clear that I agree with you and I don't write quick and dirty code. Or at least very rarely. Even for stuff that has a very short shelf life, I write code that usually has very few bugs and that I'm usually proud of because of exactly what you said: I've done it so often it's a habit now.
I've always strived to write the best quality code possible within the constraints. Sometimes those constraints were even my own lack of knowledge. But after three decades of doing this I'm starting to think I'm actually getting to be pretty good at it. ;-)
So I wasn't suggesting to just write bad code in my original comment. Just to have a broader view of where quality goals sit within the many competing stakeholder requirements. A programmer who doesn't let perfect be the enemy of good is a better programmer.
The idea that it doesn't matter seems like survivorship bias. People see all the companies getting their databases dumped without a hit to their stock price/valuation and declare caution a waste of time, but what of the companies you never hear about that died because someone messed up in the rush to ship?
Five years as an example is probably the outer limit and so it supports your case well. But I've seen badly designed software cost twice as much to put right (compared to putting the effort in at the start) within a few months. It's a false economy over and over but that gets buried in all the drama that follows.
>>For a website that will support a two week marketing campaign, you don't need anything talked about in this article
>[...]
>And then, five years after you've left the company and some system inevitably collapses with nobody having a clue as to what went wrong you'd finally realize the wisdom of all that.
So guy puts up a web site for $500 in consulting fees that is a 2-week project. It makes the company $7 million over the next 72 months because it becomes literally the biggest inbound channel.
Are you saying he shouldn't have built it for $500? What should he have done?
But it's no longer your problem.
So, please don't take it personal, but 'the well intentioned types who write these articles' tend to be the people that then get called in to clean up the mess.
And then - belatedly - the job gets done properly to keep the company in business, assuming there is still time enough to do so.
Just this week I had a nice inside look at the kind of mess gets left behind when the original duct-tape-and-spit guy leaves the company and lets his former co-workers clean up their mess. It isn't pretty, and a chapter 11 isn't an unlikely scenario, so forgive me if I take a harsher than usual look at the attitude that causes this sort of thing.
Note that the 'free market' doesn't have a horizon much longer than the next quarterly shareholder report, and that your typical software product lives a multiple of that interval. So software made with short term goals in mind will create long term headaches.
Your two week marketing campaign gets a pass. But your decade long backend project does not, nor does your real time medical device controller, ECU, database system or operating system.