User Guide¶
Welcome to the Flux Operator user guide! If you come here, we are assuming you have a cluster with the Flux Operator installed and are interested to submit your own custom resource to create a MiniCluster, or that someone has already done it for you. If you are a developer wanting to work on new functionality or features for the Flux Operator, see our Developer Guides instead.
Local Install¶
If you are trying this out on your own, here is a quick start to getting the operator installed on MiniKube (or similar), and installing directly from the repository (with the development use case in mind). This assumes some experience with Kubernetes (or applying yaml configs) and using MiniKube or similar. More advanced users can try out the Production install detailed below.
This setup is intended if you want to clone the codebase and use the same tools that we use to develop! You’ll first want to clone the codebase.
$ git clone https://github.com/flux-framework/flux-operator
$ cd flux-operator
And then start a cluster with minikube:
$ minikube start
And make a flux operator namespace
$ kubectl create namespace flux-operator
namespace/flux-operator created
You can set this namespace to be the default (if you don’t want to enter -n flux-operator
for future commands:
$ kubectl config set-context --current --namespace=flux-operator
If you haven’t ever installed minkube, you can see install instructions here. And then officially build the operator,
$ make
(optionally) to make your manifests:
$ make manifests
And install. Note that this places an executable bin/kustomize
that you’ll need to delete first if you make install again.
$ make install
At this point, you can kubectl apply your custom resource definition to define your MiniCluster to your cluster to either run a job or start a Flux Mini Cluster.
$ kubectl apply -f config/samples/flux-framework.org_v1alpha1_minicluster.yaml
Note that we have other examples (using the web interface in examples/flux-restful
and headless examples for testing in examples/tests).
When you are all done, cleanup with kubectl delete
commands and/or!
$ make clean
And to stop MiniKube.
$ minikube stop
Production Install¶
To deploy the Flux Operator, you have a few options! Choose “quick,” “repository,” or “tool” deploy below, and then continue to read how to check that everything worked.
Quick Deploy¶
This works best for production Kubernetes clusters, and comes down to downloading the latest yaml config, and applying it.
wget https://raw.githubusercontent.com/flux-framework/flux-operator/main/examples/dist/flux-operator.yaml
kubectl apply -f flux-operator.yaml
Note that from the repository, this config is generated with:
$ make build-config
and then saved to the main branch where you retrieve it from.
Repository Deploy¶
If you want to possibly develop and then deploy what you are working on, you can start with a clone:
$ git clone https://github.com/flux-framework/flux-operator
$ cd flux-operator
A deploy will use the latest docker image from the repository:
$ make deploy
...
namespace/operator-system created
customresourcedefinition.apiextensions.k8s.io/miniclusters.flux-framework.org unchanged
serviceaccount/operator-controller-manager created
role.rbac.authorization.k8s.io/operator-leader-election-role created
clusterrole.rbac.authorization.k8s.io/operator-manager-role configured
clusterrole.rbac.authorization.k8s.io/operator-metrics-reader unchanged
clusterrole.rbac.authorization.k8s.io/operator-proxy-role unchanged
rolebinding.rbac.authorization.k8s.io/operator-leader-election-rolebinding created
clusterrolebinding.rbac.authorization.k8s.io/operator-manager-rolebinding unchanged
clusterrolebinding.rbac.authorization.k8s.io/operator-proxy-rolebinding unchanged
configmap/operator-manager-config created
service/operator-controller-manager-metrics-service created
deployment.apps/operator-controller-manager created
Tool Deploy¶
We maintain a tool Flux Cloud that is able to bring up clusters, install the operator, and optionally run experiments and bring them down. We currently support a handful of clouds (AWS and Google) and if you find yourself wanting a way to easily generate and save results for experiments, this might be the way to go. If you have a cloud or environment you want to deploy to that isn’t supported, please let us know.
Next Steps¶
For next steps, you might do the following:
1. Verify Install¶
Regardless of what you chose above, from this point on (after the operator is installed)
there are some checks that you can do to see that everything worked.
First, ensure the operator-system
namespace was created:
$ kubectl get namespace
NAME STATUS AGE
default Active 12m
kube-node-lease Active 12m
kube-public Active 12m
kube-system Active 12m
operator-system Active 11s
$ kubectl describe namespace operator-system
Name: operator-system
Labels: control-plane=controller-manager
kubernetes.io/metadata.name=operator-system
Annotations: <none>
Status: Active
No resource quota.
No LimitRange resource.
You can find the name of the operator pod as follows:
$ kubectl get pod --all-namespaces -o wide
<none>
operator-system operator-controller-manager-6c699b7b94-bbp5q 2/2 Running 0 80s 192.168.30.43 ip-192-168-28-166.ec2.internal <none> <none>
2. Create Namespace¶
You’ll likely Make your namespace for the flux-operator:
$ kubectl create namespace flux-operator
3. Apply your custom resource definition¶
Then apply your custom resource definition or CRD - importantly, the localDeploy needs to be false if you are using a production cluster. Basically, setting to true uses a local mount, which obviously won’t work for different instances in the cloud!
spec:
# Set to true to use volume mounts instead of volume claims
localDeploy: false
Also ensure that your custom resource definition matches the namespace you just created. Note that we don’t have yet a production solution for a shared filesystem, but @vsoch is going to test this soon. Then apply your CRD. You can use the default testing one from the repository or any in our examples folder. Here is using the default we provide:
$ wget https://raw.githubusercontent.com/flux-framework/flux-operator/main/config/samples/flux-framework.org_v1alpha1_minicluster.yaml
$ kubectl apply -f flux-framework.org_v1alpha1_minicluster.yaml
Please let us know if you would like an example type added - we have plans for many more but are prioritizing them as we see them needed. And now you can get logs for the manager:
$ kubectl logs -n operator-system operator-controller-manager-6c699b7b94-bbp5q
And then watch your jobs!
$ kubectl get -n flux-operator pods
And don’t forget to clean up! Leaving on resources by accident is expensive! This command will vary depending on the cloud you are using. Either way, it’s good to check the web console too to ensure you didn’t miss anything. Next, you might be interested in ways to submit jobs or how to build images in our Developer Guides.