Increase appstudio pods quota #993
Conversation
Even though this quota is named for-<some-tenant>-deployments implying that it only affects deployments, this is not the case and it does block the build. This quota cap the number of pods in a namespace, including the completed ones. Not only the build tasks can no longer start, it looks like this does block some kind of pod cleanup because as soon as I increased the quota, not only builds started but overall number of pods started to go down after slightly increasing over 300. PipelineRun usually have about 10 tasks or more so 300 pods, not counting deployments, allows maximum 30 PipelineRun which is too low. The pod counts includes the completed ones so since PipelineRun are not deleted right after completion, we need to increase that quota to not end up in this situation all the time. The final number will need fine tuning but for now increase it to 600 as I experimented on tenant's namespace where the issue was seen and 600 allowed to not get the issue again until now. KONFLUX-2454 Signed-off-by: Hugo Arès <[email protected]>
hugares
requested review from
MatousJobanek,
xcoulon,
alexeykazakov,
rajivnathan,
ranakan19,
sbryzak and
mfrancisc
as code owners
|
Hi @hugares. Thanks for your PR. I'm waiting for a codeready-toolchain member to verify that this patch is reasonable to test. If it is, they should reply with Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
|
Paired with: codeready-toolchain/host-operator#993 Signed-off-by: Hugo Arès <[email protected]>
Paired with: codeready-toolchain/host-operator#993 KONFLUX-2454 Signed-off-by: Hugo Arès <[email protected]>
|
/ok-to-test |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: alexeykazakov, hugares, mfrancisc The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Codecov Report
Additional details and impacted files@@ Coverage Diff @@
## master #993 +/- ##
=======================================
Coverage 84.73% 84.73%
=======================================
Files 55 55
Lines 5039 5039
=======================================
Hits 4270 4270
Misses 585 585
Partials 184 184 |
alexeykazakov
merged commit 188cb8a
into
codeready-toolchain:master
Paired with: codeready-toolchain/host-operator#993 KONFLUX-2454 Signed-off-by: Hugo Arès <[email protected]>
Even though this quota is named
for-<some-tenant>-deploymentsimplying that it only affects deployments, this is not the case and it does block the build. This quota cap the number of pods in a namespace, including the completed ones.Not only the build tasks can no longer start, it looks like this does block some kind of pod cleanup because as soon as I increased the quota, not only builds started but overall number of pods started to go down after slightly increasing over 300.
PipelineRun usually have about 10 tasks or more so 300 pods, not counting deployments, allows maximum 30 PipelineRun which is too low. The pod counts includes the completed ones so since PipelineRun are not deleted right after completion, we need to increase that quota to not end up in this situation all the time.
The final number will need fine tuning but for now increase it to 600 as I experimented on tenant's namespace where the issue was seen and 600 allowed to not get the issue again until now.
KONFLUX-2454