Keep an Integration Studio workflow running

At 09:00 somebody asks whether last night's import happened. This guide takes you from that question to an answer and a fix, for one example: the shipped Customers CSV import from SFTP template, or a workflow built the same way. A schedule starts it at night, it lists the inbox folder on your supplier's SFTP server, loops over the files in it, imports each one into Revenexx, and moves each file to an archive folder on success or to a failed folder on error. As step 6 wires it, the error branch then ends the run, so one bad file stops that night's import and the files after it wait in the inbox. File transfer shows that pattern. You find out whether it ran, which step stopped it and what that step had already written, and you know which button finishes the job. Then you make the workflow safe to run twice, so the next bad night costs less.

This is for integrators who run Integration Studio workflows unattended. You need to be able to open Integration Studio in Cockpit, the revenexx admin interface at app.revenexx.com: a tenant with the studio enabled and permission to open it, as Getting started describes. You also need a workflow that starts from a schedule or a webhook and writes somewhere. For the example that is a nightly SFTP import with two credentials: an SFTP credential for the supplier's server, and a Revenexx API credential, which holds an API key created in Cockpit. That key needs the io scope (which scope covers which app), which lets the import and export nodes use the platform's Import & Export service; see Reaching Revenue Cloud. Step 6 adds a third if you send the failure email, an SMTP credential. If you have no workflow yet, Templates lists the shipped SFTP imports. New to the platform? Prerequisites covers Cockpit, your tenant slug, installing an app and a test tenant.

Three words carry the page. A firing is a trigger going off. A run is one execution of the workflow, and a firing may or may not become one. A build is the preparation a workflow goes through after a save that changes its graph. Its status is pending, building, ready or failed, and Cockpit shows it on the workflow; the workflows list does not refresh by itself, so reload it. While the status is not ready, runs are refused (a Run you press in the editor is held until the build is ready, see step 3). A failed build stays failed until you fix the workflow and save again, which queues a new build.

Everything here is a screen in Cockpit. Nothing needs code. When Cockpit refuses an action, it shows an error message with the platform's reason. The status codes this page quotes, such as 409, are what the same refusal returns over the API.

What a bad night leaves behind

Find the row that matches before you press anything.

1. Read a failed run

Open Integration Studio → Runs and filter the status to failed. You can also filter by trigger type and search. There is no date filter, so find last night's run by its Started time. The dashboard's Needs attention list names up to three workflows whose latest run failed and links to the workflow, not the run. A workflow whose failed night was followed by a successful run is no longer listed there, so the Runs list is the reliable place to look.

Open the run. Details shows how it ended, the platform's reason for a failure and the result. Switch to Timeline for the Steps list, with one entry per step in the order they ran, above a time chart of what the engine did. Open a step to read its Input, its Output and, for a failed step, its error and how often it was tried. Input is what the node received after every expression was resolved, so a mapping that produced nothing shows as an empty field. Replay steps through the graph as the run walked it.

The Steps list answers the question from 09:00, what was already written:

• Steps before the failed one succeeded. A step that failed and then succeeded on a later attempt shows as succeeded, with no trace of the failed attempt.
• A loop's steps are recorded once per item, one after another. For a file import, the Steps list shows an entry per file and step, so the last file with a succeeded entry tells you how far the loop got, and the failed step's input names the file that stopped it. A failure that is not routed to an error port ends the run, so the files before it were processed and the ones after it never started. If the step is routed to an error port, the loop moved on to the next file.
• The failed step may have written part of its work. A timeout or a dropped connection says nothing about the far side. Ask the target system what it holds from that step before you run anything again. For the import, open the failed file, take the customers in it by the key you import on, and look them up in the target itself, here Revenexx's customers, to see which exist.

A step you routed to its error port does not fail the run, so a run whose error branch ends without a Stop and Error node looks green in the list. Step 6 wires that.

2. Choose retry, resume or replay

Five buttons look like "run again". Four of them start a new run, each in a different way. Replay on a run row starts nothing.

Pick by what already happened:

• Resume does not apply a fix you saved to the workflow: it continues the run on the revision that was saved when that run started. Use it when earlier steps did work you must not repeat, such as a file already staged or a batch already accepted, and the cause of the failure was outside the workflow and is gone, for example the far side is back. Inside a loop it continues at the failed item: the items before it keep their results and are not run again, the item list is the one the original run read (a file that arrived since is not in it), and the items after the failed one still run.
• Retry when the whole job should happen again from the top and step 5's habits make that safe. It runs the workflow you have saved now, so it is also how a fix gets applied. To tell whether the write is keyed, open the import step in the editor and read its key columns: on Import without profile (Ad-hoc) they are settings of the node, on Import from profile they are the profile's mapping.keys. No key columns means you are in the next case. The shipped template's own key columns are not documented, so check your copy rather than assume.
• After Stop and Error, Retry. A run that ended at Stop and Error failed on purpose, and Resume does not finish it: Resume restarts at the first step that failed in the run, which is either the Stop and Error step itself, so it fails again at once, or an earlier step that the error branch had already caught, so it runs that step and everything after it again, including the steps that already parked the file. Retry lists the inbox again, finds the files that never started, and does not meet the bad one, which is in the failed folder.
• Neither yet when the failed step may have written part of its work. Look up what the failed step already wrote in the target system first, as step 1 describes. Then either remove those partial writes by hand, or make the write keyed (step 5) so a repeat overwrites them. If a bug in the workflow caused the failure, fix it, save, wait until the build is ready, and then Retry: Retry runs the workflow as saved now, and Resume would not.

On a run row, Retry and Resume each start a new run and leave the original as it was, so a resumed night still shows its original as failed. A Missed runs row offers only Replay and Discard, and Replay starts a new run. A Retry or a Replay can be refused with 409 while the workflow is building or its build failed, and a Replay also if the missed run was already handled. Both are refused with 422 if the trigger they came from no longer exists, for example after a restore. Resume is refused with 409 when the run did not fail or while a resume of it is still running, and with 422 when the run's history has no point to resume from. A run that was terminated (stopped by hand with Terminate) also reads as failed, and offers only Retry.

3. Catch up a firing that never became a run

A run is refused while the workflow's build status is anything other than ready. A save that changes the graph starts a build, so a nightly import saved at 01:59 is still building at 02:00. A save that only changes the name or description, or a trigger, does not. What becomes of a firing in that window depends on the trigger, and it is the same for pending, building and failed:

Missed runs is the Missed runs tab of Integration Studio → Runs, next to the Runs tab; the count also shows on the tab and on the dashboard. Each row has Replay and Discard, one row at a time, with no select-all, and only rows still waiting can be acted on. An event row carries the event. A schedule row carries no payload, only the fact that the firing was missed.

For the nightly file drop, catch up once. Every run lists the inbox when it starts, so a single run after a week of missed nights imports every file that waited, and seven replays would run the same import seven times. That holds because the example lists its inbox: if it has Missed runs rows, replay one and discard the rest. A workflow that reads one named file per firing needs every replay.

Three more cases fire nothing and keep nothing:

• An inactive workflow or a paused trigger fires nothing. Integration Studio → Schedules names the obstacle for each schedule: paused, workflow inactive, or workflow not built. Activate a workflow from the toolbar in its editor. A paused trigger is separate: its card on the canvas is badged Inactive, and you resume it by selecting the trigger, switching Active on in its settings and saving.
• A platform that was down at a schedule's time does not fire it late, and nothing in Cockpit lists the missed time.
• A restore can remove a trigger. Step 4 explains.

If a night has neither a run nor a missed run, start the workflow by hand with Run in the editor header, beside Save, or Run workflow in the canvas's right-click menu. For the example, one such run imports every file that waited. A manual run does not need the workflow to be active, only a ready build and a configuration with nothing missing, so it works while you find out why the schedule did not fire. A run pressed while the workflow is still building is held and starts on its own once the build is ready. See Runs.

4. Roll back a bad save

Open the workflow's History tab. Revisions are listed newest first with the current one marked. Select an older revision to preview its graph, read-only, next to the list. To restore one, press Restore on its row and confirm. There is no button called rollback and no separate publish step: a save that changes the graph is what builds the workflow, and Activate or Pause in the toolbar is separate.

A restore is a save, so mind what it brings with it:

• It starts a build. Runs are refused until it finishes, which opens the step 3 window again. Restore at a quiet hour.
• It brings back the old triggers and removes newer ones. A trigger added since that revision is gone, including one another system depends on. Read the workflow's triggers first.
• It switches the workflow off if the old graph no longer validates, for example when it uses a node version that has since been retired. Check that it is active afterwards.
• It keeps the current name, description and execution mode, the setting for whether a caller waits for a run's result, and rewrites no run.
• Unsaved editor changes are discarded. Save or abandon them first.

Only a change to the graph writes a revision. If someone edited a schedule and nothing else, there is no earlier revision to go back to. Open Integration Studio → Audit Log, which records who changed what and when, and set the trigger back by hand. See Revisions and audit.

Authorization inside Integration Studio is not role-based yet (see that page), so these buttons are not limited by role. Treat Restore, Discard and Replay as available to everyone who can open the studio.

5. Make it safe to run twice

Every recovery above ends with the workflow running again, so decide what a second run does. The trigger will not decide it for you:

So the safety has to be in what the workflow writes. For the file drop, three habits cover it:

• Write by key. Import with a key the data carries, such as the customer number, so a second run updates the rows the first one made instead of adding new ones. On Import without profile (Ad-hoc) you set the key columns and the apply mode on the node. Import from profile runs an Import & Export profile, a saved mapping created through /v1/io/profiles or by the Ensure Profile exists node, and takes both from it: mapping.keys names the columns a row is matched on, and apply_mode is upsert (the default), append or full-sync. Keep upsert here: full-sync deletes every row of the entity that is not in the file. See Import and export for profiles and Import a BMECAT catalog for one in use.
• Move what you processed. List the inbox (SFTP: List Directory), loop over the files (Iterator), import each one, and move it with SFTP: Move File to an archive folder on success and to a failed folder on the error branch. A second run then finds an empty inbox. The same holds for a recovery: Retry and Replay on a Missed runs row list the inbox again, so move a file back from the failed folder to the inbox, with your own SFTP client, before you use either. See File transfer.
• Keep the schedule interval longer than a run takes. With the first two habits an overlap is harmless, but it still does the work twice.

On a node that writes without a key, open the node's Settings tab, set Max attempts under Retry on failure to 1, and set On error to Route to error output, so a failure lands on the error port (step 6) instead of repeating a write. See Failure handling.

6. Know it failed without watching

Integration Studio sends no push notification when a run fails, and has no alert setting. What Cockpit does show by itself is the Missed runs count and, on the dashboard, Needs attention with the latest failure of up to three workflows. So build the watching in:

• In the workflow: route the error ports of the nodes that write to a Send Email node, which sends over SMTP and needs an SMTP credential, then to Stop and Error with a message that names what failed, for example Customer import failed, see the run's steps. Stop and Error always fails the run, whatever On error says, so in a loop it ends the whole run at the first bad file: the files before it were processed, and the files after it never start and stay in the inbox. Put the move to the failed folder before it, so the bad file is parked and the next Retry skips it. A node that declares an error port and is set to Route to error output, with nothing wired to the port, fails the run and names the step and the port. A branch of an If condition node with nothing wired to it ends quietly. This covers failures the workflow can catch, not a firing that never started.
• In your routine: each morning, open Runs filtered to failed and look at the Missed runs count. A night you already resumed still shows its original run as failed, and Cockpit has no way to mark a failure as handled or to filter by date, so order the list by Started and read only the runs since your last check. Then open Integration Studio → Schedules: it lists every schedule with its status and its last run, so a night that produced nothing shows as a last run more than a day ago, and a paused schedule or an inactive or unbuilt workflow is named there. A refused webhook call shows nowhere in Cockpit; only the system that called saw the 409. For an integration that matters, make it the first thing the person on duty does.

Two limits shape how long evidence lasts. Run history is not an archive: no retention period is promised, so write what you need to keep somewhere you own. The audit log keeps entries for a year by default, and that default is not a commitment.

One gap to plan around. Cockpit cannot show which workflows use a credential, and deleting a credential is not blocked by use. Before you change or delete the SFTP credential, open the workflows that could be using it and look. The next run fails at the step that needs it, not at the delete. See Credentials.

Troubleshooting

Where to go next

• Hand an order to the ERP — a workflow whose retries and replays have to be safe, built end to end.
• Debug a failed request — when the failure is a call to the platform API rather than a step in a workflow.
• Runs and Missed runs — each screen in this guide, in full.
• Failure handling — retries, onError routing and timeouts per node.