K3S installation

We use Kubernetes, an open-source container orchestration system to manage our applications.

At the moment the only Kubernetes distribution supported is K3s (click here for the official documentation) by Suse Linux for both on-premise and cloud deployments.

Why K3s?

Kubernetes has been widely adopted in modern software development as it offers a powerful, portable and open-source platform that automates the management of containerized applications.

When setting up a Kubernetes environment, it comes in two flavors: vanilla Kubernetes and managed Kubernetes. With vanilla Kubernetes, a software development team has to pull the Kubernetes source code binaries, follow the code path, and build the environment on the machine. On the other hand, managed Kubernetes comes pre-compiled and pre-configured with tools that improve features to enhance a certain focus area, such as storage, security, deployment, monitoring, etc. Managed Kubernetes versions are also known as Kubernetes distributions. Some popular Kubernetes distributions are Rancher, Red Hat OpenShift, Mirantis, VMware Tanzu, EKS, GKE and AKS.

Kubernetes distributions can have different components that may cause applications that work in one distribution to not necessarily work or even crash into another. Some of the most important components that differ between distributions are:
  • Container Runtime: The container runtime is the software that is responsible for running containers. Each Kubernetes Distribution may offer support for different Container Runtimes. Some popular container runtimes include Docker, CRI-O, Apache Mesos, CoreOS, rkt, Canonical LXC and frakti among others.
  • Storage: Storage is important for Kubernetes applications as it offers a way to persist this data. Kubernetes Container Storage Interface (CSI) allows third-party vendors to easily create storage solutions for containerized applications. Some Kubernetes Distributions build their own storage solutions while others integrate with existing third-party solutions. Popular storage solutions for Kubernetes include Amazon ElasticBlock Storage (EBS), GlusterFS, Portworx, Rook, OpenEBS among others.
  • Networking: Kubernetes applications are typically broken down into container-based microservices which are hosted in different PODs, running in different machines. Networking implementations allow for the seamless communication and interaction between different containerized components. Networking in Kubernetes is a herculean task, and each distribution may rely on a networking solution to facilitate communication between pods, services and the internet. Popular networking implementations include Flannel, Weave Net, Calico and Canal among others.
In order to offer our customers a better and more seamless experience while configuring, running, upgrading and troubleshooting our products while also avoiding compatibility issues between different distributions we decided to officially support ONLY ONE Kubernetes distribution: K3s. The main reasons for choosing K3s are:
  1. Costs — K3s is 100% open source and there is no need to pay for any expensive licenses.
  2. Less setup overhead — a lot of time is saved when setting up a new environment because you do not need to go through a lengthy process of acquiring extra licenses based on how many CPU cores you have. Also, K3s can be installed using only one command.
  3. It supports many Linux distros — K3s supports popular Linux distributions including open-source ones, it can also run both on- premises and in the cloud (AWS, Azure, GCP).
  4. It is fast and lightweight — K3s is packaged as a single <100MB binary and its lightweight architecture makes it faster than stock Kubernetes for the workloads that it runs.
  5. Easy to update — Thanks to its reduced dependencies.
  6. Batteries included — CRI, CNI, service load balancer, and ingress controller are included.
  7. Smaller attack surface — Thanks to its small size and reduced amount of dependencies.
  8. Certified — K3s is an official CNCF project that delivers a powerful certified Kubernetes distribution.
  9. Flexible — you can run K3s using single-node or multi-node cluster setup.

K3s and Antivirus

Antivirus software adds an unknown variable to the existing complexity of Kubernetes. Most of them have not yet kept up with newer technologies such as Kubernetes and have not reached a CNCF certified status. In environments where antivirus software had been enabled, there has seen issues stemming from interfering actions from such software. As an example, there have been incidents where the antivirus software had pruned files in the ContainerD filesystem incorrectly, causing the ContainerD mounts to go corrupt and causing data loss.

Issues resulting from third-party tools, such as antivirus and intrusion detection software, interfering with ContainerD or other necessary system calls are deemed resolved should disabling such tools restore functionality.

Also, having antivirus software running in the nodes can result in poor performance. Some antivirus software may continue to impact even if all paths our products rely on are excluded. Whilst performing scanning and inspection of files, disk I/O and CPU usage can increase, thus slowing our product performance. In some cases, this reduction in performance can be dramatic and can even render our products impossible to use.

Lastly, where a customer environment has deviated from our supported configurations, we always reserve the right to recommend the customer to revert to a supported configuration to resolve the reported issue.

Secure mode K3s

If the cluster has been deployed before January 2023, most probably it will come across this issue at some point.

If cluster runs a pod that uses a public image that has not yet been copied to our public repo, error ErrImagePullwith the message Failed to pull image will arise.

To address this issue this configuration file needs to be added,
mirrors:
  "*":
    endpoint:
      - "https://images.master.k3s.getvisibility.com"
    rewrite:
      "(.*)": "gv-public/$1"
  "images.master.k3s.getvisibility.com":
    endpoint:
      - "https://images.master.k3s.getvisibility.com"

This file instructs K3s to rewrite all images (except images.master.k3s.getvisibility.com) and redirect them to Getvisibility’s public repository.

Requirements

The minimum requirement for the Kubernetes cluster is a single node (1 virtual machine) with the following specs:

Table 1.
  FDC Forcepoint DSPM Ultimate
CPU cores 8 16 20
Memory 32GB 64GB 80GB
Storage 500GB

min 32M inodes

600GB

min 39M inodes

700GB

min 32M inodes

Also refer to Estimate hardware capacity needs section.

Installation

Ensure the following items are in place and configured:

  • Domain Name Service (DNS) with public name resolution enabled.
  • Network Time Protocol (NTP)
  • Software Update Service- access to a network-based repository for software update packages.
  • Fixed private IPv4 address
  • Unique static hostname

For details on how to configure Rancher behind a proxy, refer to the official Rancher documentation and ensure the network settings mentioned above are applied.

If using proxy, please run this before using starting k3s.sh installation:
export http_proxy="$PROXY_IP"
export https_proxy="$PROXY_IP"
no_proxy="$NODE_IP,localhost,127.0.0.0/8,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,.svc,.cluster.local"

When running k3s.sh script you need to provide product name in a form of PRODUCT_NAME argument. This will instruct the installer to test your current environment against product requirements, which are different.

Allowed product names are:

  • synergy
  • dspm
  • enterprise
  • ultimate

Capitalization of the name is important. If you provide name that cannot be recognized or if you do not provide product name at all, the script will default to PRODUCT_NAME="dspm".

Regular installation of k3s
curl -sfL https://assets.master.k3s.getvisibility.com/k3s/k3s.sh | \
INSTALL_K3S_VERSION="v1.26.10+k3s1" K3S_KUBECONFIG_MODE="644" PRODUCT_NAME=enterprise sh -s - server --node-name=local-01

This is just a regular k3s installation command when you want to install Forcepoint Enterprise product.

Run precheck only
curl -sfL https://assets.master.k3s.getvisibility.com/k3s/k3s.sh | \
INSTALL_K3S_VERSION="v1.26.10+k3s1" K3S_KUBECONFIG_MODE="644" PRODUCT_NAME=ultimate ONLY_PRECHECK=true sh -s - server --node-name=local-01

If you want just to see if your environment meets all the requirements use the command above.

Custom command
curl -sfL https://assets.master.k3s.getvisibility.com/k3s/k3s.sh | \
INSTALL_K3S_VERSION="v1.26.10+k3s1" K3S_KUBECONFIG_MODE="644" SKIP_SYSTEM_CHECKS=true sh -s - server --node-name=local-01

Above command will allow you to skip checking how installed memory size, number of required CPU cores and storage requirements fare against your environment.

curl -sfL https://assets.master.k3s.getvisibility.com/k3s/k3s.sh | \
INSTALL_K3S_VERSION="v1.26.10+k3s1" K3S_KUBECONFIG_MODE="644" SKIP_PRECHECK=true sh -s - server --node-name=local-01

If you want to skip both hardware and connectivity checks use this argument.

Note: Be cautious when skipping built-in checks- we built them for the product to achieve optimal performance with minimal required maintenance.
Table 2.
Argument Description
SKIP_PRECHECK=true to skip all built in checks
SKIP_SYSTEM_CHECKS=true to skip hardware checks
SKIP_NETWORK_CHECKS=true to skip connectivity checks
ONLY_PRECHECK=true will run precheck only and stop after that

The installer, depending on the combination of arguments provided, will perform a set of actions before returning back to the command line.

Provided not all the checks have been skipped, it is possible for the installer to abort the installation process. If that happens, review the output paying special attention to any warn messages. Should you have any concerns or questions, contact the Support with the result screen attached.

Note: Run the kubectl registration command. The command below is just an example, it will not work during deployment. For direct customers, Customer Support Team will provide the registration command, otherwise you should have been provided registration command in the Welcome Email.
kubectl apply -f https://....k3s.getvisibility.com/v3/import/dxslsxcf84....yaml
Note: For security reasons the registration command can be used only a single time, the command becomes invalid after the first use. In case you need to run it again you must contact the support team for a new registration command.

Monitor the progress of the installation:

watch -c "kubectl get deployments -A"

  • The K3s deployment is complete when elements of all the deployments (coredns, local-path-provisioner, metrics-server, traefik and cattle-cluster- agent) show at least "1" as "AVAILABLE"
  • In case of errors, you can inspect the logs of a pod using kubectl logs , for example:
    kubectl logs cattle-cluster-agent-d96d648d8-wjvl9 -n cattle-system