Skip to content

Clarify namespace sameness control via optional policy #6748

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
76 changes: 54 additions & 22 deletions sig-multicluster/namespace-sameness-position-statement.md
Original file line number Diff line number Diff line change
@@ -1,38 +1,42 @@
# Namespace Sameness - SIG Multicluster Position Statement

Author: Jeremy Olmsted-Thompson (**[@jeremyot](https://github.com/jeremyot)**), Google
Last Edit: 2020/04/20
Last Edit: 2023/04/22
Status: RELEASED

## Goal

To establish a normative statement for multi-cluster namespace semantics and
governance as a building block for further development which will require
specifying behaviors across clusters.

## Context

Users are reaching for multi-cluster deployments for a
[variety of reasons](http://bit.ly/k8s-multicluster-conversation-starter-doc).
However, Kubernetes treats the cluster boundary as the edge of the universe.
There are currently no standard practices for how to extend the Kubernetes
resource model across multiple clusters. Without common patterns we can’t build
portable tooling to facilitate multi-cluster capabilities and know that behavior
will be consistent for each user.
However, Kubernetes has historically treated the cluster boundary as the edge
of the universe. Without common patterns we can’t build portable tooling to
facilitate multi-cluster capabilities and know that behavior will be consistent
for each user.

## Scope

A single organization may need multiple, disjoint sets of clusters. They may,
for example, represent different phases of the development lifecycle (dev,
staging, prod) or support unrelated projects. Each organization governs their
own clusters in isolation, so the scope of a namespace can only reasonably be
declared within the organizational boundary. The scope of namespace identity is
defined as the union of clusters, governed by a single authority, that are
defined as the set of all clusters, governed by a single authority, that are
expected to work together. An authority is a company, organization, team,
individual, or other entity which is entrusted to manage these clusters and, in
particular, to create namespaces in them.

## Position
**For a set of related clusters governed by a single authority, all namespaces of
a given name are considered to be the same namespace. A single namespace should
have a consistent owner across the set of clusters.**

**Within a set of related clusters which are governed by a single authority (a
"clusterset"), samely named namespaces in all clusters are considered to be
related, to have the same owner(s), and to represent the same intention,
unless otherwise dictated by some overriding policy, including, but not limited
to, implementation-specific configuration.**

## Appendix

Expand All @@ -46,12 +50,12 @@ Consider an organization that runs two clusters, "A" and "B". They have a
cluster-ops team which is responsible for keeping those clusters alive and
healthy. They also have app-teams, "foo" and "bar" which use those clusters.

Cluster-ops owns the creation of namespaces in both clusters. They can choose
Cluster-ops owns the creation of namespaces in all clusters. They can choose
to delegate the ability to create namespaces to foo-team and bar-team, but any
namespace name that is allocated in "reserved" in all clusters. How the
namespace name that is allocated is "reserved" in all clusters. How the
delegation works is an area for innovation, but some possible examples
include:
* A self-service portal which allocate a name in a global database and
* A self-service portal which allocates a name in a global database and
creates the namespace on the user's behalf
* Tooling that allocates a name in a global database and issues a credential
to the user which is checked in an admission controller
Expand Down Expand Up @@ -79,7 +83,7 @@ If there are special RBAC rules that need to be applied in some clusters (e.g.
clusters in EU have more limited access), the LDAP-to-RBAC sync implementation
can apply specializations based on whatever criteria it understands.

### Example 3: Multi-cluster services
### Example 3: Multi-cluster Services

Consider the same organization from previous examples. They have enabled an
implementation of multi-cluster Services, which lets them access backends which
Expand All @@ -88,14 +92,42 @@ are running on other clusters in the group.
As in example 2, cluster-ops run a per-cluster metrics service, It does not
make sense for clients in cluster A to access the metrics server of cluster B.
Even though this runs in the same "metrics" namespace in both clusters (and
thus namespce sameness applies), the implementation of multi-cluster services
thus namespace sameness applies), the implementation of multi-cluster services
probably should not be on-by-default. The details of how multi-cluster
services will work are an area for innovation, but ideas include:
* Opt-in: services must be "exported" to be merged across clusters
* Opt-out: services or namespace can be opted out of service merging
services work are out of scope for this document, but valid approaches could
include:
* Opt-in: services must be explicitly "exported" to be merged across clusters
* Opt-out: services or namespaces can be opted out of service merging
* Different discovery: merged services and "raw" services use different names
or other discovery mechanisms
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this section could benefit from a larger update to reflect concrete implementations such as ServiceExport, ServiceImport and .clusterset.local DNS.

Do any implementations actually support a service opt-out pattern, where a default or configured behavior is for all services in a given namespace to global/merged/same, but some could be reserved/un-exported, or perhaps more confusingly, exported but "not-same"?

"Export all services in namespace" feels like a reasonable option, and if more granularity is needed, following a service opt-in pattern with manual ServiceExports definitions instead or moving the cluster-local services to a local namespace feels preferable and less confusing.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but some could be reserved/un-exported, or perhaps more confusingly, exported but "not-same"?

I understand namespace sameness to allow for only the first, not the second (I would prefer the line here should say what is being opted out of is, as originally, "service merging", or even better "exporting", but definitely not as proposed here "sameness"). Most (all?) MCS implementations today are opt-in so implementation wise opt-out is only its corollary: by not having ServiceExports made in 1-N clusters with the same named namespace/service in the first place or removing them. Istio et al probably have more to say on opt out patterns of behavior since those implementations are more on-by-default.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll update. At the time this was written, MCS semantics had not really been pinned down. That said, I like it as an example, but I don't want to inline that spec here.

Will try to balance.


However the implementation works, the metrics service is not automatically
merged across clusters, though the LDAP-to-RBAC sync from example 2 can still
apply consistent policies.
However the implementation works, the metrics service is not _necessarily_
merged across clusters, even if the LDAP-to-RBAC sync from example 2 still
applies consistent RBAC policies. Those are independent capabilities.

### Example 4: Authority sameness policy
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A comment thread got lost on the push:

#6748 (comment)


Consider the same organization from previous examples, but with more clusters:
"A", "B", "C", and "D". Business rules dictate that the foo-team and the
bar-team have a common security posture and may share clusters, but the
billing-team has stricter requirements and must not share clusters with other
teams. Cluster-ops wants to assign foo-team's and bar-team's namespaces to
clusters A and B, and billing-team's namespaces to clusters C and D. They also
want to ensure that the opposite is never true - foo-team's and bar-team's
namespaces cannot be used from clusters C or D, and billing-team's namespaces
cannot be used from clusters A or B.

What exactly it means to "be used" depends on each specific multi-cluster
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A comment thread got lost on the push:

#6748 (comment)

capability. For multi-cluster Services, the billing-team's endpoints from
clusters C and D would be eligible for merging, but billing endpoints from
clusters A and B would not. This can be used by implementations to add another
layer of safety - even if cluster A were compromised and the billing-team's
namespaces was created, the MCS implementation can use the policy described
above to know that it should NOT merge billing endpoints from cluster A.

As an implementation choice, the authority may offer controls which govern
which cluster-namespaces are considered for sameness and which are not.
The details of how these policies might be expressed could include:
* In-cluster namespace labels or annotations
* In-cluster custom-resources
* An external control-plane API