Networking in Kubernetes clusters is an abstracted implementation that can be configured per cluster basis. Pharos supports few different options how to configure the networking provider.
Supported Network Providers
Weave is the default networking provider in Pharos clusters. Weave Net creates a virtual network that connects containers across multiple hosts and enables their automatic discovery. Weave also supports network policies.
If the cluster is deployed on multiple regions/data centers*, Weave networking is configured so that each node within a region connects to other nodes in the same region through private interfaces/addresses. When nodes peer with nodes outside their own region the peering uses public addresses of the nodes. This configuration is fully dynamic and handled by an additional side-car component on the networking deployment. The users needs to just ensure the nodes have proper region labels in place.
*) Nodes region is determined by
failure-domain.beta.kubernetes.io/region annotations value.
network: provider: weave service_cidr: 172.31.0.0/16 pod_network_cidr: 188.8.131.52/16 weave: trusted_subnets: - 10.10.0.0/16 no_masq_local: true
An array of trusted subnets where overlay network can be used without IPSEC. By default Weave creates secure tunnels between nodes with IPSEC. In environments where the node-to-node networking is secure and trusted you can disable the IPSEC tunneling for better performance.
no_masq_local (optional, default:
Whether to preserve the client source IP address when accessing Service annotated with
service.spec.externalTrafficPolicy=Local. For more information look here: https://kubernetes.io/docs/tutorials/services/source-ip/#source-ip-for-services-with-type-nodeport
Calico creates and manages a flat layer 3 network, assigning each workload a fully routable IP address. Workloads can communicate without IP encapsulation or network address translation for bare metal performance, easier troubleshooting, and better interoperability. In environments that require an overlay, Calico uses IP-in-IP tunneling
network: provider: calico pod_network_cidr: 172.31.0.0/16 service_cidr: 184.108.40.206/16 calico: ipip_mode: CrossSubnet
Always(default) - Calico will route using IP-in-IP for all traffic originating from a Calico enabled host to all Calico networked containers and VMs within the IP Pool.
Never- Never use IP-in-IP encapsulation.
CrossSubnet- IP-in-IP encapsulation can also be performed selectively, only for traffic crossing subnet boundaries. This provides better performance in AWS multi-AZ deployments, and in general when deploying on networks where pools of nodes with L2 connectivity are connected via a router.
For more details on IP-in-IP configration and usability see https://docs.projectcalico.org/v3.3/usage/configuration/ip-in-ip.
Whether or not calico should apply NAT on the kubernetes nodes to outgoing packets from pods. Supported options:
Firewalld provides a dynamically managed firewall with support for network/firewall zones that define the trust level of network connections or interfaces.
By default Kontena Pharos does not enable any firewalld rules. Firewalls rules are only applied to cluster hosts if
network.firewalld is enabled. When enabled following rules are applied by default:
22/tcp- ssh is opened to all hosts
80/tcp- http is opened to all hosts
443/tcp- https is opened to all hosts
6443/tcp- kubernetes api is opened to master hosts
30000-32767tcp+udp- nodeports are opened to all hosts
Traffic between the cluster hosts is whitelisted automatically.
network: firewalld: enabled: true open_ports: # these are the defaults if firewalld is enabled - port: "22" protocol: tcp roles: - "*" - port: "80" protocol: tcp roles: - worker - port: "443" protocol: tcp roles: - worker - port: "6443" protocol: tcp roles: - master - port: "30000-32767" protocol: tcp roles: - "*" - port: "30000-32767" protocol: udp roles: - "*" trusted_subnets: - "192.168.10.0/24"
Specify if firewalld rules are applied. Supported options:
Specify ports that are opened to the outside world.
An array of trusted subnets which can access all ports.