[This is Craig -- CEO of this new venture, co-founder of K8s (with Joe and Brendan), and person who started CNCF (with Jim Zemlin, and a bunch of community wonks from big tech)]
It is a funny you say this. I spent a lot of time looking around the community at what existed before starting CNCF, and agonized over this. We needed to take K8s to foundation so that it wouldn't be a 'Google project'. Google was actually the best steward of the tech you could imagine, because the plan was always to make k8s ubiquitous and just win on quality of infrastructure but the community had no way to know that.
I looked at OpenStack hard, and like the energy and enthusiasm but really worried about (1) balkanization that was emerging with no 'true north' -- it just didn't have technical taste, (2) the tragedy of the commons -- most vendors were focused on their own interests and neglected the end users, (3) lack of coherence.
When designing CNCF I tried hard to work through this by creating a better foundation structure.
(1) the business board has very limited authority over projects, hopefully making sure that we avoid it being a pay-for-play affair.
(2) we made provisions for little companies to get top level seats based on community contributions (ditto)
(3) we created an empowered end user group that would have equal authority to any other affair to make sure real users interest are promoted
(4) we added a TOC (technical oversight committee) that was the most empowered group to establish true north that is community elected -- the idea is they need to champion the projects and establish technical 'taste' (e.g. Brian Grant from Google -- the guy who drives consistency sat on this group, not me the guy who had access to the purse strings and who was focused on the business).
(side note: i picked this structure because i was geeking on government structures at the time, and figured that the separation of powers yields more sustainable administration)
So being a PTL in OpenStack I have some various comments and questions that would be nice to have your thoughts on.
In terms of looking at OpenStack hard; and reaching decisions based on various <things> did you do any reach out to the OpenStack community to actually communicate the things you found or heard or concluded so that the group there (including myself) can actually work on improving itself (or perhaps some of the reasons you stated aren't even correct and the community could have helped you clarify those)? If not then it concerns me that you may have reached conclusions without actually talking with that community (but I don't want to jump to any conclusions without getting your thoughts/input).
So far from looking outwards in on the CNCF and seeing how it compares to the OpenStack community (which I am more involved with, including other small side-communities that I also work in) I've yet to understand what exactly the CNCF is targeting. It seems to be a body that is just adopting various projects that align to some mission (?); I have personally a hard time understanding the reasoning some of the projects have been adopted, maybe you can shed some light on that (what is true north for the CNCF, where is it written down, what is the TOC actually making adoption yes/no decisions on? what criteria? what is the technical taste you talk about, where is it written down?)
The nice thing about OpenStack is that they are writing most/all of this down and agreeing on those kinds of questions in public:
A fair point. One thing worth remembering is that this was a point in time thing. I have seen a lot of movement and some very positive signals around convergence of OpenStack, and a real focus on the end user community. When I was doing the digging things felt different and there is a decent chance that were open stack where it is now I would have taken a different position.
The mission of CNCF is the promotion of 'cloud native technologies' -- specifically container packaged, dynamically scheduled, micro-services oriented workloads. It isn't about picking winners, it is about establishing a safe space for innovation and bringing to bear the collective communities. We have legitimately taken some time in getting the identity of the foundation established, but I feel like Dan Kohn (our new ED) is doing super work in creating a collaborative space for new projects.
Thanks for the part of the response though I'm still concerned at the 'felt' part though, especially if that felt part didn't involve talking with much/any(?) of that community in a public forum. Is there anything I can do to help you understand it better, I'd at least like to be able to echo whatever concerns you had to that community, because at that point it can be actual data that made the decision to go with CNCF creation and not just feelings or thoughts of movement or positive signals (all very fluffy things IMHO).
So in the CNCF, are competing implementations allowed / encouraged?
For me that would have repercussions for what other tech a CNCF project supports (e.g. is all stats monitoring based on Prometheus, or can 2 projects in the CNCF support different technologies)
to elaborate for @hueving @mugsie et al., consider that OpenStack is organised around Nova, the scheduler++ that is at the heart of any OpenStack deployment. If the CNCF was "like OpenStack" then it could mandate that all projects are organised around Kubernetes, playing a role analogous to Nova. But we didn't want to be solely a "Kubernetes foundation". The market is early stage, and there are other valid approaches to orchestration, including Docker Swarmkit, Hashicorp Nomad, Mesos & DCOS templates like Marathon, and others. So, we need a different approach.
Of course there are people who want a KubeStack that is like OpenStack, for better or worse. That's fine too! We just don't want that to be the ONLY choice for customers.
Do you allow competing APIs for the same service? If not, how is that any different from OpenStack? If so, how do you address the issue of fragmentation across deployments?
You said it yourself in another comment on here: "It's never blatant, it's always calls for seemingly good things like extra pluggable points to make sure we don't favor particular solutions. Then it's making sure that any decision is brought to a huge vote by a giant committee that spends weeks arguing about if it's something they should even decide on, etc."
This kind of premature generalisation by committee is what has pulled OpenStack down; a situation from which it is now apparently recovering. CNCF seeks to avoid this, by encouraging projects towards interop but not in mandated ways.
Projects can do what they like. We believe that users, communities, market pressures, and so on, will drive good outcomes here. For example to date, all projects have worked to interoperate of their own volition. No committees were formed to achieve this.
So, also being a PTL in OpenStack I would say that the foundation design is very similar to the OpenStack one.
1 - the openstack board has 0 technical input to projects - the way to get things done in OpenStack is still to throw developers at it.
This does somewhat push the balance of power to larger companies - they have the money to employ developers.
2 - we have "community directors" who are elected by the people who actually commit code
3 - definite improvement over the initial setup of openstack, but that is currently changing with the User Committee
One question I would have about this - how are the end user groups requirements put forward? what mechanisms is there to ensure developers work on the defined priorities?
4 - Yup - we have the equivalent with the Technical Committee (the TC in OpenStack slang)
Separation of powers is ++ - but how does that play out when the TOC decides that they want to do something that does not mesh with the boards plans?
"how are the end user groups requirements put forward? what mechanisms is there to ensure developers work on the defined priorities?" --> projects are run by their leads, they are not told what to work on. In this sense, CNCF operates more like IETF/ASF but with (arguably) less intrusive governance.
The underlying idea here is that a well-run open source project gets plenty of strong direction from actual users, who must be interacted with directly.
There is a still-forming End User Board designed to create a strong forum for some types of User-Project discussion. But overall CNCF will lean towards "voluntary" and not "mandatory" requirements.
It is a funny you say this. I spent a lot of time looking around the community at what existed before starting CNCF, and agonized over this. We needed to take K8s to foundation so that it wouldn't be a 'Google project'. Google was actually the best steward of the tech you could imagine, because the plan was always to make k8s ubiquitous and just win on quality of infrastructure but the community had no way to know that.
I looked at OpenStack hard, and like the energy and enthusiasm but really worried about (1) balkanization that was emerging with no 'true north' -- it just didn't have technical taste, (2) the tragedy of the commons -- most vendors were focused on their own interests and neglected the end users, (3) lack of coherence.
When designing CNCF I tried hard to work through this by creating a better foundation structure. (1) the business board has very limited authority over projects, hopefully making sure that we avoid it being a pay-for-play affair. (2) we made provisions for little companies to get top level seats based on community contributions (ditto) (3) we created an empowered end user group that would have equal authority to any other affair to make sure real users interest are promoted (4) we added a TOC (technical oversight committee) that was the most empowered group to establish true north that is community elected -- the idea is they need to champion the projects and establish technical 'taste' (e.g. Brian Grant from Google -- the guy who drives consistency sat on this group, not me the guy who had access to the purse strings and who was focused on the business).
(side note: i picked this structure because i was geeking on government structures at the time, and figured that the separation of powers yields more sustainable administration)