Syncotta logoGet notified at launch

Troubleshooting

Symptom, likely cause, fix.

Alpha. Tested end to end on Argo CD 3 with a Jira Cloud development site. Details may change before 1.0.

Format: symptom, likely cause, fix. For setup see Getting started; for chart values see Helm values.

Useful logs:

  • Argo CD notifications controller: kubectl logs deploy/argocd-notifications-controller -n <argocd namespace>. It logs the webhook URL and, on delivery errors, the full payload including the commit message. Restrict access to it.
  • Gate Job: kubectl logs job/<release>-syncotta-gate -n <application namespace>. The gate client never prints the URL or the key.

Deployments do not appear in Jira

Cause How to tell Fix
The Application has no syncotta.com/payload label Nothing in the notifications controller log about Syncotta Add syncotta.com/payload=default (or =single-source), see Getting started, step 2.
The Application matches both labels Reported twice or not as expected Keep only one of the two labels.
single-source label on a multi-source app Nothing is sent; the commit message lookup fails for multi-source apps Use syncotta.com/payload=default and report issue keys from CI (Getting started, step 3).
Wrong or old notifications token HTTP 401 in the notifications controller log Check that argocd-notifications-secret holds the current token under notifications.tokenKey (default syncotta-token). After New token the old one works for 24 hours only.
Wrong instance id HTTP 401 instanceId must equal the id in the admin page.
The Syncotta licence or trial is not active The admin page shows “Syncotta licence not active”; the controller log shows HTTP 202, but nothing reaches Jira Start a trial or renew the subscription in Jira’s Manage apps. Deployments sent while the licence was inactive are not stored and do not appear later.
The chart is installed but Argo CD does not use it argocdValues mode: the generated values were never applied Run the helm upgrade of the argo-cd release with -f syncotta-argocd-values.yaml. If you had your own notifications.subscriptions, copy them into the file (Helm replaces lists).
configMap mode overwrote your other notification config Other services or triggers gone Move them to notifications.extraConfig.
The sync has not finished or the app is not Healthy Deployment stays in progress, or never becomes successful The successful state is sent when the sync succeeded and the app is Healthy.
No issue keys found Deployment is not on any work item Single-source: the commit message must contain a key such as PAY-123. Multi-source or no key in the commit: use the GitHub Action. Keys need at least two letters before the dash.
The image is only known by a mutable tag CI report accepted but never links Tags such as latest, main, master, dev, develop, stable, edge, nightly are ignored. Push a digest or a version tag.
The GitHub Action found no keys status: skipped in the step output Use fetch-depth: 0, put the key in the commit, PR title or branch name, or pass issue-keys.
Looking at a Jira Service Management space Issue view has no Deployments section Service Management spaces do not show deployments in the issue view. Check a software space.
Just synced Not there yet Deployments can take a few seconds to show up. Reload the page.

If Syncotta cannot be reached, Argo CD retries a failed webhook delivery on the next reconcile; the chart does not use oncePer, so lost deliveries are sent again.

The gate Job answers

The Job logs one line per attempt. What each HTTP answer means:

Answer Meaning What the Job does What to do
200 Allowed (change approved, or gate mode off, or errors with the allow policy). Exits 0, the sync continues. Nothing.
202 Waiting: the change is not approved yet. Waits (15 to 30 s, growing) and asks again. Approve the change, see below for Standard mode.
409 Prevented: the change was rejected. Exits 1, the sync fails. A new approval is needed, then re-sync by hand (see “Prevented changes”).
422 Invalid. The request is wrong, the Standard mode settings do not work (unknown space or work type, missing permission, deleted gate work item), or the Syncotta licence is not active with the licence policy on block. Exits 1, the sync fails. See “422 in Standard mode” and “Fail policy”.
401 Authentication failed. Exits 1. Check the instance id, the HMAC key, and the node clock (requests older than 5 minutes are refused). A new HMAC key works only after you update the Secret.
400 The request was refused as malformed. Exits 1. Check the Helm parameters, especially that gate.revision and gate.appName are not empty or unsubstituted.
503 or other error Temporary problem on the Syncotta side, or with the network. Waits and asks again. If it lasts, see “Deadline reached”.
Deadline reached No final answer within gate.deadlineSeconds (default 1800). Exits 1, the sync fails. Fix the cause shown by the last answers. Argo CD’s sync retry then runs a new Job.

Premium mode comes after the alpha and the admin page does not offer it. An instance that was set to Premium before gets 503 on every call (block policy) or 200 (allow policy); switch it to Standard.

422 in Standard mode

Symptom: the Job log says HTTP 422 (invalid) or the sync waits although the work item is already done.

Causes and fixes:

  • Status names. The gate compares the work item’s status name with the names in “Statuses that allow” and “Statuses that reject”, ignoring case. Syncotta reads the names as the app sees them, which can differ from what you see:
    • In classic Service Management spaces the built-in statuses are untranslated for the app (Open, Work in progress, Pending, Reopened, Done) even if you see them translated.
    • A status may really have another name than you expect, for example Gotowe in a team-managed space created in Polish, so Done never matches and the Job waits until its deadline.
    • The admin page checks the names when you click Save. If one is wrong it refuses and lists the valid names, for example “Statuses of this work type in Jira: Open, Work in progress, Pending, Reopened, Done.” Copy the names from that message.
  • A status renamed after saving. The gate keeps waiting. Save the settings again.
  • Unknown space key or work type, or a work type the space does not have. Fix Space key and Work type. The error is shown on save.
  • The gate work item was deleted. The gate answers 422 for that revision. Sync again after changing the settings, or contact support if the work item should exist.
  • The licence is not active and the licence policy is block. See “Fail policy”.

Syncotta does not retry a 422: waiting would not fix a wrong setting, and failing open on a typo would quietly ungate production.

The Job cannot read the Secret

Symptom: gate: missing /etc/syncotta/gate-url or permission denied in the Job log.

Cause: files from a Secret volume with mode 0400 are owned by root, and the gate client runs as a non-root user.

Fix: the chart sets fsGroup: 65534 and file mode 0440. If you changed the Job (a patch, a policy engine or a mutating webhook that drops fsGroup), restore both. The Secret needs the keys gate-url and hmac-key.

Other start-up errors: gate-url must be an https URL; APP_NAME is empty or REVISION is empty (the Helm parameters $ARGOCD_APP_NAME and $ARGOCD_APP_REVISION were not passed or not substituted; the chart then fails rendering when the value is missing, but a literal unsubstituted $ARGOCD_... value is treated as empty by the client).

Prevented changes do not re-sync by themselves

After a 409 the Application stays OutOfSync. Argo CD does not sync that state again on its own, so approving the change later does not resume the deployment. Re-sync by hand (Argo CD UI or argocd app sync <app name>) after the new approval.

An automated sync retries a failed PreSync hook up to 5 times by default, so a rejected change is asked about up to 6 times in a few minutes. This is normal.

If you want a “not approved yet” state that waits instead of failing, leave the work item in a status that is neither allowing nor rejecting. Only use a rejecting status for an explicit “no”.

Self-heal syncs skip the gate

Symptom: someone changes a resource by hand, self-heal reverts it, and the gate is not asked.

Cause: self-heal runs a selective sync of the drifted resources, and Argo CD does not run hooks in it. This is intended: self-heal only re-applies the revision that already passed the gate.

Related: ApplyOutOfSyncOnly=true does not skip the gate. A normal automated sync with that option still runs the PreSync Job.

The gate is per revision

  • The gate approves one revision of one application for one environment at a time. A new commit is a new request and needs a new approval (in Standard mode: a new work item).
  • Once a revision is allowed, later syncs of the same revision (retries, a manual re-sync) get 200 at once and no second work item is created.
  • A sync that only changes Helm parameters or values, not the source revision, keeps the same revision and reuses the earlier answer. With a chart from a Helm repository the revision is the chart version.
  • gate.revision is the revision of the source that holds the chart: the Git commit for a chart in Git, the chart version for a chart in a Helm repository. In multi-source apps it is the revision of that one source.
  • An empty revision fails the sync. Pass $ARGOCD_APP_REVISION as a Helm parameter.
  • Do not pass $ARGOCD_APP_NAMESPACE as gate.appNamespace. It is the destination namespace, and the gate then uses a different application id than the deployments reported from notifications. The default argocd is right unless your Applications live elsewhere.

Fail policy: errors and licence

In the admin page, Settings of the instance:

Situation Policy Result
Syncotta has an error reading Jira or its storage (including a gate backend that is not built) If Syncotta cannot reach Jira: Block the sync (default) The Job gets 503 and keeps polling; it fails the sync at its deadline.
Same Let the sync continue The Job gets 200 and the sync goes on; Syncotta logs it as fail-open.
The Syncotta licence (or trial) is not active If the licence is not active: Block the sync (default) The Job gets 422 and the sync fails at once.
Same Let the sync continue The Job gets 200.
The change was rejected, or a Standard mode setting is wrong Any Always stops the sync (409 or 422). The policy never turns a rejection into a pass.

If production syncs wait or fail and you did not expect it: look at the Job log first. Repeated 503 means Syncotta had errors, 202 means a work item is waiting for approval, 422 means settings or licence. To let syncs through while you fix it, switch the gate mode to Off or the error policy to Let the sync continue.

Sync times out or the Job disappears

Argo CD has no per-hook timeout and does not set controller.sync.timeout.seconds by default. If you set it, the controller ends the sync when the time is up (checked periodically, so up to about a minute late), deletes the waiting gate Job and does not retry. Keep gate.deadlineSeconds below that value.