The open source revolution is over…
Spoiler, Open Source won the cloud wars. Any discussion of the technology revolution represented by the rise of cloud computing, specifically things classified as “Cloud Native” innovations, would not be complete without paying tribute to the revolution that has been Open Source. When open source software was starting to gain traction there were those who hailed it as the downfall of the software behemoths like Microsoft, IBM, Oracle, and others.
Few of them truly foresaw the rise of Cloud Computing which was built on a foundation of using open source software to wrest profits from those industry stalwarts. Virtually every cloud provider, including the modern incarnation of those industry stalwarts, is built on top of Open Source solutions. They each have their own managed services that are based on some of the most successful open source projects of all time. Linux, arguably the god father of the open source movement, is at the core of cloud services. Open source databases like MySQL, Postgres, or Mongo paved the way for hosting providers to find profit in open source. But I would be doing a great disservice to the term Open Source if I was to only enumerate software products that are licensed under one of the common software licenses referred to as “Open Source”. In truth, the battle that was won was more a battle of hearts and minds.
The real revolution was not in the licensing, although that is critical, the real revolution is in the creation of community and the transparency of developing “in the open”. It has been a monumental shift in the way that an engineer views a source code repository. Most engineers take some level of pride in their work, but working with the attitude that your source code repository is a place where you invite peers to learn about what your doing was something new. To think of my work-in-progress as having value to someone who wanted to learn, and to invite their contribution in a meaningful way, took the quality of my work to a whole new level. That affect was widespread, and profound, reaching well beyond the strict confines of codified language into the broader engineering world. I would argue that this attitude of working “in the open” is the real revolution of Open Source. The community that attitude builds around a problem space is what makes it a superior way to design, build, and run software solutions, or research, or legislation, or…. In our industry, these communities have created real disruption in hosting with a family of products including Docker, K8S, Helm, ArgoCD, CircleCI, Prometheus, Grafana, and Terraform. Likewise, open source has defined the core of Big Data and the concept of a Data Lake with tools like Hadoop, Hive, Spark, and others. In yet another example, Kafka, and the Kafka Connector ecosystem are at the heart of many event driven microservice architectures. These “open source” success stories are truly driven by the communities that have been built up around the problem space, and the innovation that those communities can create to constantly improve the solutions they use. In some cases, the software itself is a byproduct of the work the larger community is doing and becomes far less important than the overall ecosystem.
An important post script, using these open source solutions at the core of your own architecture allows you to invest in a cloud provider and explore the benefits of a strong partnership, without worrying about vendor lock-in. If the need should arise you can move to any of a number of other cloud hosting providers that support the same open source tooling.