http://marktarver.com/thecathedralandthebizarre.html
The above is a very good article describing why most of the promises and predictions made in ESR's The Cathedral and the Bazaar (1998,) about the coming triumph of open source over closed source, failed to come true. As someone who read TCatB and fell for its promises -- in 1998, no less -- I cannot help but nod along to the arguments presented. It's helpful to go back to the early promises because while in 2019 open source may feel dominant, it has nevertheless clearly failed to live up to the original hype.
If anything, the author isn't critical enough when it comes to the claim that the world would converge on optimal solutions and avoid duplicate effort. I've got about ten thousand Javascript frameworks waiting outside to have a word with that one.
But I want to come at it from a different angle, and make what may be a novel criticism of open source.
Back in the day, if you wanted software to run your business on, you had to pay for it. Even if you only need the tools, to allow you to build your own software with which to run your business, you still had to pay for it. Operating systems, compilers, databases: money, money, money. The obvious consequence was that running a business on software was rather expensive. (This world sort of still exists, in Microsoft shops.)
But there was a secondary consequence that made things even more expensive than that: you needed to hire people that knew how to use and take care of all this software. This feels a bit like adding insult to injury, but it was actually something that businesses didn't mind at all, because it fit their existing mental model of how industrial systems worked. You see, if I had a factory, it was obviously necessary to hire people to work in it. If I bought a Bridgeport mill, it was only sensible that I needed to hire a machinist to run it. A crane needed an operator, a forklift needed a driver, and so on. Capital was never sufficient on its own, it was always necessary to add labor to get output.
So even as late as 1999, if you forked out a few million dollars a year for Oracle Database, it only made sense to spend a few hundred thousand extra employing a couple of professional Oracle DBAs. Likewise if you had a fleet of Windows NT servers in the racks, you would have a team of administrators trained (and likely Certified) on Microsoft software to look after them. And so it went for all the large proprietary business software vendors.
Then along comes open source software and... it costs nothing. Oh sure, the message is "free as in speech" not "free as in beer", but in practice it's all priced at $0 and if we're honest that's a big part of the attraction. An interesting thing happens psychologically. Paying $100k/yr for a professional DBA to support a $1M/yr Oracle installation feels very reasonable. Paying $100k/yr for a professional DBA to support a $0/yr MySQL installation... somehow does not.
There's another phenomenon developing right around the same time that reinforces this: the amateurization of business software development. Used to be one needed all this expensive software (and hardware!) to get a tech business off the ground. Then suddenly all you need is a cheap x86 server plus the zero-dollar LAMP stack and you're off to the races. For a while it was easy to dismiss this approach as the domain of hobbyists, but then the hobbyists starting launching successful businesses with it, forcing the entire industry to take it seriously. I say "amateurization" because the key driver here was the availability of free (as in beer) software that ran on cheap hardware, which allowed motivated hackers to get experience doing stuff without training, certifications, mentorship, or even (in many cases) college.
This deeply affected the culture of tech companies. In the proprietary high-dollar era, a developer was happy to enlist the help of a DBA, because the DBA was the expert on the database. The DBA was happy to enlist the help of the SysAdmin, becuase the SysAdmin was the expert on the OS and hardware. The SysAdmin was happy to enlist the help of the Network Admin... and so on. In the LAMP era, it's just four guys in a garage, and they all have to do everything just good enough to ship. The hardware, OS, network, database, compiler suite, various server software, and everything else is easy enough to procure, install, and configure that any motivated hacker can do it. There's neither a need nor time for specialized professionals.
This in turn has deeply affected the career development of technologists. Oracle DBA and Microsoft Server Admin used to be stable, high-paying jobs with long-term career prospects. Satellite firms built businesses around selling tools to these folks. These career-slash-cultures had their own conferences, newsletters, even glossy monthly magazines. Almost all of that is absent from the open source world. Do you know anyone who got training on how to install Linux? Anyone who's made a career out of MySQL administration? Someone certified on nginx?
I think it's been about 20 years since this evolution got going in earnest, so it seems reasonable to take a look back, as the author of the opening link did, and ask where it's gotten us.
In the "pro" column, it's a hell of a lot easier to start a company than it ever has been. If you have an idea and the drive to pursue it, it's never been cheaper or easier to try giving it a go.
In the "con" column, we have a systematic loss of expertise and deep understanding. We assume now that any piece of software that's no further away than apt-get install should be something we can run professionally, in production, with real money on the line, with no training, no practice, hell maybe not even a skim of the documentation.
