Argo CD vs Flux

Two GitOps approaches.

Argo CD and Flux both do one stubborn thing: they watch a Git repository, treat whatever is committed there as the truth, and keep bending your cluster back to match it, over and over, forever. That is GitOps (running infrastructure by storing the desired state in Git and letting a controller apply it on a loop, the mechanism from ag-reconcile). Both projects are graduated members of the CNCF (the Cloud Native Computing Foundation, the open-source home of Kubernetes), so neither is a hobby project. The loop underneath them is the same. What differs is the machine wrapped around that loop, and for a defender the machine is the thing you have to guard.

Here is the shape of the difference. Argo CD is a staffed control room: one big board on the wall shows every application in green or red, and a single door (a web login) lets operators walk in and push buttons. Flux is a rack of small, single-purpose relays bolted inside the cluster: no board, no door, each relay quietly doing one job, and you read them one at a time with a handheld meter (kubectl, the Kubernetes command-line tool). Same electricity running through both. Very different room to lock.

The Same Loop, Two Machines

In Argo CD you write one Application, a CRD (Custom Resource Definition, the way you teach Kubernetes a brand-new kind of object). That single object names a Git repo, a path inside it, and a destination cluster and namespace (a named partition inside a cluster). Behind it sits a small crew of pods (the running containers that make up the app). argocd-repo-server renders the raw manifests (the YAML files that describe each Kubernetes object). argocd-application-controller compares the rendered result against what is live and syncs the difference. argocd-server serves the API (Application Programming Interface, the control surface every tool and human login talks to) and the web UI (user interface, the dashboard). argocd-dex-server brokers SSO (Single Sign-On) logins, and redis plus the applicationset and notifications controllers round it out. One object per app, one console over all of them.

kubectl get pods -n argocd

NAME                                                READY   STATUS    RESTARTS   AGE
argocd-application-controller-0                     1/1     Running   0          6d
argocd-applicationset-controller-6b8c9f5d4-x2k9t    1/1     Running   0          6d
argocd-dex-server-5c8f7b6d9-qz4mn                   1/1     Running   0          6d
argocd-notifications-controller-7d9f6c8b5-lm2wp     1/1     Running   0          6d
argocd-redis-6f9d8c7b4-nk8vx                        1/1     Running   0          6d
argocd-repo-server-7b6c9d8f5-r4t2h                  1/1     Running   0          6d
argocd-server-6d8f7c9b6-p9x3k                       1/1     Running   0          6d

Flux has no single app object and no server at all. It ships as the GitOps Toolkit (GOTK for short), a set of controllers that are ordinary Kubernetes deployments (long-running services the cluster keeps alive). You declare a GitRepository, which is the source (where the manifests live and how often to pull them), and then one or more reconciler objects that point at a path inside that source and say apply this. A Kustomization applies a folder of plain manifests. A HelmRelease installs a Helm chart (Helm being Kubernetes' package manager, the apt or Homebrew of the cluster). The split is on purpose: source-controller fetches the repo once, and many reconcilers can chew on the same fetched copy. A default bootstrap installs four controllers.

kubectl get deploy -n flux-system

NAME                      READY   UP-TO-DATE   AVAILABLE   AGE
helm-controller           1/1     1            1           5d
kustomize-controller      1/1     1            1           5d
notification-controller   1/1     1            1           5d
source-controller         1/1     1            1           5d

So the same repo takes two shapes. Argo CD folds fetch, render, and apply behind one Application and paints the result on a board everyone can watch. Flux separates where the manifests live from what to do with them, and hands you controllers instead of a board. Neither is more correct. They draw the boundary in a different spot, and that boundary is the line your threat model has to trace.

# Argo CD: one Application = fetch + render + sync, shown on the board
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: shop
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://github.com/acme/shop
    targetRevision: main
    path: deploy/prod
  destination:
    server: https://kubernetes.default.svc
    namespace: shop
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
---
# Flux: a source, then a reconciler that consumes it
apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
metadata:
  name: shop
  namespace: flux-system
spec:
  interval: 1m
  url: https://github.com/acme/shop
  ref:
    branch: main
---
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: shop
  namespace: flux-system
spec:
  interval: 10m
  sourceRef:
    kind: GitRepository
    name: shop
  path: ./deploy/prod
  prune: true

Bootstrapping: Who Installs What

Getting each running says a lot about how you will secure it. With Argo CD you apply a bundle of manifests into the argocd namespace, then log in. The first admin password is generated for you and dropped into a Secret (a Kubernetes object that holds sensitive values like passwords and keys) named argocd-initial-admin-secret. You read it, log in, and then you delete that Secret so it can never be read again.

kubectl create namespace argocd
kubectl apply -n argocd \
  -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml
kubectl -n argocd get secret argocd-initial-admin-secret \
  -o jsonpath='{.data.password}' | base64 -d; echo

namespace/argocd created
customresourcedefinition.apiextensions.k8s.io/applications.argoproj.io created
customresourcedefinition.apiextensions.k8s.io/appprojects.argoproj.io created
serviceaccount/argocd-application-controller created
serviceaccount/argocd-server created
...
deployment.apps/argocd-repo-server created
deployment.apps/argocd-server created
statefulset.apps/argocd-application-controller created
8kJ2mNpQ4rL9vX3z

With Flux you run flux bootstrap, which commits Flux's own controller manifests into your Git repo and then reconciles itself from there. Flux installs Flux. That is a security property worth naming out loud: the platform's own configuration lives under the same branch protection, review, and audit trail as your applications, and bootstrap also creates a deploy key (a read credential scoped to that one repository) so the controllers can read it. With Argo CD the install is a pile of YAML (the plain-text config format Kubernetes reads) you applied out of band and now have to keep patched yourself, which is why many teams end up putting Argo CD under Argo CD, or under Helm, so its own version is managed too.

flux bootstrap github \
  --owner=acme --repository=fleet \
  --branch=main --path=./clusters/prod

► connecting to github.com
► cloning branch "main" from Git repository "https://github.com/acme/fleet.git"
✔ cloned repository
► generating component manifests
✔ generated component manifests
✔ committed component manifests to "main" ("a1b2c3d")
► pushing component manifests to "https://github.com/acme/fleet.git"
► installing components in "flux-system" namespace
✔ installed components
✔ reconciled components
► determining if source secret "flux-system/flux-system" exists
✔ configured deploy key "flux-system-main-flux-system" for "https://github.com/acme/fleet"
► generating sync manifests
✔ reconciled sync configuration
◎ waiting for Kustomization "flux-system/flux-system" to be reconciled
✔ Kustomization reconciled successfully
► confirming components are healthy
✔ source-controller: deployment ready
✔ kustomize-controller: deployment ready
✔ helm-controller: deployment ready
✔ notification-controller: deployment ready
✔ all components are healthy

Watching For Drift, Two Ways

The reward for running the loop is drift detection (spotting when the live cluster has quietly wandered away from the committed state). If an attacker or a rushed operator hand-edits a live Deployment, both engines notice within a cycle, and both can shove it back to what Git says (self-heal). The difference is how you see it happen. Argo CD gives you a status you can read from the command line, the exact picture the UI shows.

argocd app get shop

Name:               argocd/shop
Project:            default
Server:             https://kubernetes.default.svc
Namespace:          shop
URL:                https://argocd.example.com/applications/shop
Source:
- Repo:             https://github.com/acme/shop
  Target:           main
  Path:             deploy/prod
SyncWindow:         Sync Allowed
Sync Policy:        Automated (Prune)
Sync Status:        OutOfSync from main (a1b2c3d)
Health Status:      Progressing
 
GROUP  KIND        NAMESPACE  NAME   STATUS     HEALTH       MESSAGE
       Service     shop       shop   Synced     Healthy      service/shop unchanged
apps   Deployment  shop       shop   OutOfSync  Progressing  deployment.apps/shop configured

OutOfSync from main means the live cluster no longer matches the repo. With self-heal turned on, the application-controller reverts the change within a couple of minutes, so an intruder's manual kubectl edit is undone and leaves an event you can alert on. Flux reports the same facts, without the wall board.

flux get kustomizations

NAME         REVISION            SUSPENDED  READY  MESSAGE
flux-system  main@sha1:a1b2c3d4  False      True   Applied revision: main@sha1:a1b2c3d4
shop         main@sha1:9f8e7d6c  False      True   Applied revision: main@sha1:9f8e7d6c

Flux ships no dashboard, so if you want to know that a tenant fell OutOfSync at three in the morning, you wire the notification-controller to Slack, a webhook, or your alerting stack, or you are refreshing kubectl by hand. Argo CD gives you a screen many teams keep open at once; Flux gives you events and expects you to plumb them somewhere useful. Neither one sees more than the other. One shows it, the other emits it.

The Blast Radius Is Not The Same

This is where the choice stops being a matter of taste. Argo CD's control room has a door, and behind that door sit the credentials for every cluster it manages. The argocd-server pod holds the API and the UI, and the cluster connection secrets live right beside it in the argocd namespace. Phish an admin, steal an initial-admin secret nobody deleted, or pop that server pod, and you can push a sync to anything Argo reaches. One box, the whole fleet. That concentration is handy for operators and just as handy for an attacker, so you lock the door hard: SSO backed by a real identity provider, RBAC (Role-Based Access Control, Kubernetes' permission system) scoped tight in the argocd-rbac-cm config map (a Kubernetes object holding plain configuration), anonymous access off, and the server kept off the public internet.

Flux has no door because it has no server. There is nothing to log into and nothing to phish. The way in is the Git repository, since anyone who can push can change the cluster, plus the controllers' own RBAC in the flux-system namespace. That swaps one juicy central target for a quieter, more spread-out one: the kustomize-controller and helm-controller carry broad permissions and no human is watching a screen. For both tools the uncomfortable fact is identical. Git is the real control plane. Branch protection, required reviews, and signed commits on that repo matter more than any toggle inside either engine (ag-hardening).

Tenancy And Least Privilege

Argo CD draws a tenant boundary with an AppProject, an object that openly gates which repos an app may pull from, which clusters and namespaces it may write to, and which resource kinds it may create. It reads like a whitelist because it is one, and it is enforced centrally by the controller for every app in the project.

apiVersion: argoproj.io/v1alpha1
kind: AppProject
metadata:
  name: team-shop
  namespace: argocd
spec:
  sourceRepos:
    - https://github.com/acme/shop        # only this repo may be a source
  destinations:
    - server: https://kubernetes.default.svc
      namespace: shop                     # only this namespace may be written
  clusterResourceWhitelist: []            # no cluster-scoped resources at all
  namespaceResourceWhitelist:
    - group: apps
      kind: Deployment
    - group: ''
      kind: Service
  signatureKeys:
    - keyID: 4AEE18F83AFDEB23             # only commits signed by this key sync

Flux draws the same boundary in a different place, and this is the trap that catches teams. A Flux Kustomization can name a serviceAccountName. When it does, the kustomize-controller applies that tenant's manifests as that service account (SA for short, the identity a workload uses against the Kubernetes API), so ordinary RBAC on the SA caps what the tenant can create. When the field is missing, the controller applies with its own identity, which in a default install is bound to cluster-admin. The isolation is invisible until you switch it on, and you have to switch it on for every reconciler.

apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: shop
  namespace: team-shop
spec:
  interval: 10m
  sourceRef:
    kind: GitRepository
    name: shop
  path: ./deploy/prod
  prune: true
  targetNamespace: shop
  serviceAccountName: shop-deployer   # apply AS this SA, not the controller's

Make it fail closed. Start the kustomize-controller and helm-controller with the flag --default-service-account set to a low-privilege account, so a Kustomization that forgets serviceAccountName falls back to that account instead of cluster-admin. Add --no-cross-namespace-refs=true so a tenant cannot point its reconciler at another tenant's source. In a bootstrapped repo you set both with a patch on the toolkit components, and it applies to every reconciler from then on.

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - gotk-components.yaml
  - gotk-sync.yaml
patches:
  - target:
      kind: Deployment
      name: "(kustomize-controller|helm-controller)"
    patch: |
      - op: add
        path: /spec/template/spec/containers/0/args/-
        value: --default-service-account=flux-restricted
      - op: add
        path: /spec/template/spec/containers/0/args/-
        value: --no-cross-namespace-refs=true

Verifying The Commit Signature

Both engines can refuse to apply anything that is not signed by a key you trust, which upgrades whoever can push into whoever can push and sign with an approved key. In Argo CD you list allowed GPG (GnuPG, the GNU Privacy Guard signing tool) key IDs in the AppProject signatureKeys field you saw above, and load the matching public keys into a config map; a sync from an unsigned commit, or one signed by the wrong key, is rejected. In Flux you set spec.verify on the GitRepository, so source-controller checks the commit signature before any downstream controller is allowed to touch the fetched copy.

apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
metadata:
  name: shop
  namespace: team-shop
spec:
  interval: 1m
  url: https://github.com/acme/shop
  ref:
    branch: main
  verify:
    provider: openpgp          # Git commits are checked with GnuPG
    mode: HEAD                 # verify the signature on the tip commit
    secretRef:
      name: shop-signing-keys  # Secret holding trusted GnuPG public keys

Whichever engine you run, prove the lockdown actually took hold. Push an unsigned commit to a throwaway branch and confirm the sync is refused. Hand-edit a managed Deployment with kubectl and confirm self-heal reverts it inside the interval and fires an event you can catch. Point a tenant reconciler at a repo it should not reach and confirm it is denied. When all three behave the way you expect, the engine is holding its end. Then put the rest of your attention on the Git repository, because that is the lock both tools hang everything else from.

Try this

Run kubectl get pods -n argocd on a scratch host or disposable cluster and read the output against what this lesson described. Then change one input so it fails, and re-run: the error you get is the one you will meet in production.

Takeaway

The trap worth remembering here: flux tenancy is impersonation you opt into. Check that on your own systems before you need to, because it is cheaper to find on a quiet afternoon than during an incident.

Related