Building dynamic stages from pull requests

[FrederikNJS]

I'm working on improving the CI/CD pipeline we have in our company. I have managed to use Kargo to model different environments and promotions and validations between them. But there one thing I can't quite seem to figure out whether is supported, or even possible.

We have 4 Kubernetes clusters, a dev, a stage and a prod cluster, these are our regular workload clusters. We also have an ops cluster, where stuff like ArgoCD and Kargo lives.

The regular flow is that something is merged to main, which builds an image, and that starts the regular pipeline of promoting the artifact through dev -> stage -> production stages.

The developers however also want to try out their new code is a somewhat realistic environment before merging their code to master. Therefore we want to create parallel dev environment per Pull Request in github. These are just regular deployments that are deployed into the dev cluster, but all the resources have a different prefix so they don't name-clash.

Is it somehow possible to model these dynamic dev environment in Kargo? I know that I can generate Applications through an ApplicationSet in ArgoCD based on the pull requests, but can these be visualised, triggered and validated in Kargo? To some extent I guess you could consider this a pre-step to the regular dev stage, but are there any way to have dynamically generated stages based on Pull Requests?

[BenHesketh21]

I have been trying to do a similar thing and what I have done is add another application set for the PR stages themselves. So not only does ArgoCD sync the branch containing your application manifests, it syncs the stage and warehouse manifests.

The issue I am currently facing is the ability to have dynamic promotion policies to allow auto promotions to the PR stages.

[krancour]

It's not part of the current feature set. We have been focused only on picking up where CI leaves off. i.e. Progressing artifacts produced by CI through known environments.

There has been some prior discussion of doing exactly what you're asking about. We know this is something people want, but the initial consensus between @jessesuen and myself was that this seems like a distinctly different thing from what Kargo currently does and that rather than trying to make it fit, this could (eventually) be the foundation of a whole secondary feature set like "pre-pipelines." That's basically where we'd left this.

In the meantime, an ApplicationSet with a PR generator, as you mentioned, is exactly how I'd personally approach this.