squat
topology setup question
I'm trying to create a k8s cluster with k0s edge nodes behind NAT (basically in peoples homes), with a couple k8s clusters at AWS that should act as the "router" for all the k8s nodes.
The setup looks like this:
- home1:
kilo.squat.ai/location=home1 - home2:
kilo.squat.ai/location=home2 - home3:
kilo.squat.ai/location=home3 - aws nodes:
kilo.squat.ai/location=aws
kg config: --mesh-granularity=location
my problem is that kilo thinks that there should be a wireguard connection be created between home1, home2 and home3, but this is not possible because of the NAT case. So I'm trying to find a solution where the home nodes only create a wireguard connection to the aws leader
Added a picture for clarity, how the topology looks today and I would like to get rid of connections crossed as red.
I'm trying to remove all peers that are not in the K8s master location so the nodes cannot communicate with pods from other worker nodes. Is this possible @squat ?
@Schnitzel What I am going to do is only run kilo on the desired machines (in your case the aws nodes) and then create the peers manually using the Peer CR/CRD
@Jonathan-Eid yea manually works, but I'm looking at an automated way, as in my situation I'm planning to run 1000+ k8s clusters, so doing it manually it's a bit tedious :D
@Jonathan-Eid negative, I ended up looking into ways to create the wireguard networking before k8s with netmaker, which definitely works and it is more cleverer in realizing which node can talk to which.
the project though has been a little bit on ice, to the need is also gone and I haven't really looked into it yet
@Schnitzel I just took the 0.6.0 release and hardcoded these lines into the func Conf() in topology.go, works like a charm
for _, s := range t.segments {
if s.location == t.location || (s.location != "location:gcp" && t.location != "location:gcp") {
continue
}