Talend

Talend Job Scheduling and Orchestration: Complete TAC Guide

Petru Constantin
••13 min read
#talend#tac#scheduling#orchestration#etl

In Talend Administration Center (TAC) 8.0 you schedule a Job by creating a task for it in Job Conductor and adding a Simple, CRON or File trigger to that task. To chain tasks, you build an Execution Plan whose child nodes run OnOk, OnError or After their parent, and external systems start a task through the MetaServlet API with runTask. This guide covers those pieces for on-premises TAC; Qlik Talend Cloud uses Talend Management Console instead.

Petru Constantin has built Talend ETL pipelines and APIs between enterprise systems since 2022. Code examples are written against the Talend Administration Center 8.0 R2026-10 and Talend Studio 8.0.1 R2026-10 documentation.

TAC Architecture Overview

TALEND ADMINISTRATION CENTER (TAC) 8.0
│
├── Web application
│     ├── Job Conductor     tasks + triggers (Simple, CRON, File)
│     ├── Execution Plan    task trees: OnOk / OnError / After, parallel root tasks
│     ├── Servers           registered Talend JobServers
│     └── MetaServlet API   runTask, getTaskExecutionStatus, ...
│
├── TAC database            tasks, triggers, plans, execution history, users
│
└── Execution servers (Talend JobServer)
      ├── JobServer 1       Standard Jobs
      ├── JobServer 2       Standard Jobs (batch window)
      └── JobServer 3       Spark Batch Jobs, submitted to the cluster

Job Deployment

Publishing Jobs to TAC

#!/usr/bin/env bash
# run_tac_task.sh - start an existing Job Conductor task through the TAC MetaServlet API
#
# Deployment happens before this script: publish the Job from Talend Studio
# (or a CI build) to the artifact repository, then create a task for it in Job Conductor.
set -euo pipefail
 
TAC_URL="http://tac.example.com:8080/org.talend.administrator/"
TAC_USER="admin@example.com"
TAC_PASS="${TAC_PASS:?export TAC_PASS first}"
TAC_CLASSES="${TAC_CLASSES:?path to the TAC WEB-INF/classes folder}"
TASK_ID="${1:?usage: run_tac_task.sh TASK_ID}"
 
# Values go inside single-quoted JSON strings: reject input that would break out of them
[[ "${TASK_ID}" =~ ^[0-9]+$ ]] || { echo "taskId must be numeric" >&2; exit 1; }
[[ "${TAC_PASS}" != *"'"* && "${TAC_PASS}" != *'\'* ]] || { echo "unsupported character in TAC_PASS" >&2; exit 1; }
 
# MetaServletCaller.sh ships in the TAC WEB-INF/classes folder
cd "${TAC_CLASSES}"
./MetaServletCaller.sh --tac-url="${TAC_URL}" \
  --json-params="{'actionName':'runTask','authUser':'${TAC_USER}','authPass':'${TAC_PASS}','mode':'synchronous','taskId':'${TASK_ID}','waitWhileUnknownStatus':true}" \
  --format-output
 
# Documented synchronous result on success:
#   errorStatus = NO_ERROR, execDetailedStatus = ENDED_OK, exitCode = 0

Job Server Configuration

# Talend JobServer, JMX monitoring settings
# File: <jobserver_path>/conf/TalendJobServer.properties
 
# JMX monitoring port (default 8888) and on/off switch (enabled by default)
org.talend.remote.jobserver.server.TalendJobServer.MONITORING_PORT=8888
org.talend.remote.jobserver.server.TalendJobServer.ENABLE_MONITORING_PORT=true
 
# true (default): listen on all network interfaces; false: localhost only
org.talend.jmxmp.wildcard=true
 
# Command port, file transfer port and SSL: follow the "Talend JobServer"
# section of the Talend installation guide for your platform before changing them.

Trigger Configuration

Time-Based Triggers (Cron)

TAC "Add CRON trigger" form (Job Conductor > Triggers)
Fields: Label, Description, Minutes, Hours, Days of month, Months, Days of week, Years
There is no seconds field. Use Ctrl + click to select several values in a list.
Fill in either Days of week or Days of month.
 
Daily load at 02:00
  Minutes: 0   Hours: 2   Days of month: every day   Months: every month
 
Weekdays at 08:30
  Minutes: 30  Hours: 8   Days of week: Monday to Friday (Ctrl + click)
 
Month-end close at 23:00
  Minutes: 0   Hours: 23  Days of month: L   (L = last day of the month)
 
Quarter start at 01:00
  Minutes: 0   Hours: 1   Days of month: 1   Months: January, April, July, October
 
Example from the TAC 8.0 guide:
  15 10 ? * 6L  ->  10:15 a.m. on the last Friday of every month
 
Misfire rule: if the task is still running when the trigger fires, Job Conductor
waits; if the task has not finished after 60 seconds, the execution becomes "misfired".

File-Based Triggers

/*
 * TAC file trigger (Job Conductor > Triggers > Add file trigger), TAC 8.0 fields:
 *   Label / Description
 *   Pollint                : polling interval in seconds
 *   Folder Path            : /data/incoming/sales   (on the JobServer file system)
 *   File mask              : sales_*.csv
 *   Exist / Created / Modified : file events that start the task (several allowed)
 *   Execution server       : the JobServer where the file arrives
 *   Pause trigger on error
 *   Trigger once for any number of events
 */
 
// Define these context variables in the Job (Contexts view in Talend Studio);
// Job Conductor passes their values at run time: folderPath, filePath, fileName
 
// tJava at the start of the Job
String inputFile = context.filePath;
String fileName = context.fileName;
System.out.println("Processing file: " + inputFile);
 
// The trigger form has no archive setting: move or delete the file
// in the Job itself after a successful load.

Event-Based Triggers

#!/usr/bin/env bash
# start_from_event.sh - TAC triggers are Simple, CRON and File; for event-driven
# starts, the calling system runs the task through the MetaServlet API.
set -euo pipefail
 
TAC_URL="http://tac.example.com:8080/org.talend.administrator/"
TAC_USER="admin@example.com"
TAC_PASS="${TAC_PASS:?export TAC_PASS first}"
TAC_CLASSES="${TAC_CLASSES:?path to the TAC WEB-INF/classes folder}"
TASK_ID="${1:?usage: start_from_event.sh TASK_ID}"
 
[[ "${TASK_ID}" =~ ^[0-9]+$ ]] || { echo "taskId must be numeric" >&2; exit 1; }
[[ "${TAC_PASS}" != *"'"* && "${TAC_PASS}" != *'\'* ]] || { echo "unsupported character in TAC_PASS" >&2; exit 1; }
 
cd "${TAC_CLASSES}"
 
# Asynchronous run: the response contains an execRequestId
./MetaServletCaller.sh --tac-url="${TAC_URL}" \
  --json-params="{'actionName':'runTask','authUser':'${TAC_USER}','authPass':'${TAC_PASS}','mode':'asynchronous','taskId':'${TASK_ID}'}" \
  --format-output
 
# Follow the run with getTaskExecutionStatus and that execRequestId
# (see "Job Execution Monitoring" below).

Execution Plans

Creating Execution Plans

Execution Plan: Daily_Sales_Pipeline          (Execution Plan page > Add)
  Timeout (s) : 14400        plan and its tasks are killed by the server after 4 hours
  Rollback    : Notify_Admin task run if the plan is killed by the timeout
  Trigger     : CRON trigger, Minutes 0, Hours 2
 
Root: Extract_Sales_Data
 └─ OnOk: Extract_Customer_Data
     ├─ OnOk: Transform_Sales
     │   └─ OnOk: Load_Data_Warehouse
     │       └─ OnOk: Generate_Reports
     │           └─ After: Send_Notifications     (runs whatever the report status)
     └─ OnError: Alert_Pipeline_Failure
 
Rules from the TAC 8.0 guide:
  - OnOk and OnError can coexist under one parent; After cannot sit next to either.
  - Tasks set for parallel execution at the root cannot have OnOk, OnError or After
    sub-nodes. To run both extracts in parallel and then transform, call them from
    one master Job (tParallelize, see below) and put that Job's task at the root.
  - Each node is an existing Job Conductor task with its own context.

Dependency Management

/*
 * Dependencies in TAC Execution Plans:
 *   OnOk    : child runs if the parent task ends without error
 *   OnError : child runs if the parent task ends with an error
 *   After   : child runs after the parent, whatever its status
 *
 * Inside one master Job, tRunJob gives the same control:
 *   tRunJob_1 (Extract) ─OnSubjobOk─► tRunJob_2 (Transform) ─OnSubjobOk─► tRunJob_3 (Load)
 *        └─OnSubjobError─► tRunJob_4 (Alert)
 *
 * tRunJob_1 Basic settings:
 *   Job                    : Extract_Sales_Data
 *   Context                : Production
 *   Transmit whole context : selected (child uses the parent's context values)
 *   Die on child error     : cleared, so the parent can read the return code below
 */
 
// tJava linked from tRunJob_1 with OnSubjobOk
Integer childExitCode = (Integer) globalMap.get("tRunJob_1_CHILD_RETURN_CODE");
String childStack = (String) globalMap.get("tRunJob_1_CHILD_EXCEPTION_STACKTRACE");
 
if (childExitCode == null || childExitCode != 0) {
    System.err.println("Child job failed with code: " + childExitCode);
    if (childStack != null) {
        System.err.println(childStack);
    }
    // route to the alert subjob with a Run if trigger, or stop with tDie
}

Parallel Execution

Parallel execution
 
1. Plan level (Execution Plan page)
   Root node: select "Use parallel execution", click Add Task once per task,
   and set "Simultaneous exec." (MAX = all selected tasks at once).
   The same task cannot be added twice to one parallel node, and parallel
   tasks cannot have OnOk / OnError / After sub-nodes.
 
2. Several instances of one Job
   Create one Job Conductor task per instance, each with its own context
   parameter values (region = US / EU / APAC), and run them as parallel root tasks.
 
3. Inside one Job: tParallelize
   tParallelize ──Parallelize──► Subjob A (file type A)
                ──Parallelize──► Subjob B (file type B)
                ──Parallelize──► Subjob C (file type C)
                ──Synchronize──► tRunJob (next step)
 
   Wait For       : end of all subJobs  (the Synchronize branch starts when A, B and C end)
   Sleep Duration : interval in ms between checks of the subjobs
   Die when one of parallelize subjobs fails : selected to stop on the first failure

Monitoring and Alerting

Job Execution Monitoring

# Status of an asynchronous run (execRequestId comes from the runTask response)
./MetaServletCaller.sh --tac-url="http://tac.example.com:8080/org.talend.administrator/" \
  --json-params="{'actionName':'getTaskExecutionStatus','authUser':'admin@example.com','authPass':'...','execRequestId':'1406817216723_97nsV'}" \
  --format-output
 
# Values documented in the TAC 8.0 guide:
#   still running     : execDetailedStatus = RUNNING
#   success (sync)    : errorStatus = NO_ERROR, execDetailedStatus = ENDED_OK, exitCode = 0
#   failure (async)   : execBasicStatus = ERROR, execDetailedStatus = JOB_ERROR, exitCode = 1

Alert Configuration

Failure alerts inside the Job (same behaviour under TAC, TMC or a shell scheduler)
 
  tLogCatcher ──► tMap (subject + body) ──► tSendMail
 
tLogCatcher Basic settings:
  Catch Java Exception : selected
  Catch tDie           : selected
  Catch tWarn          : selected (if warnings should alert too)
 
Put tDie or tWarn on the error paths you want to report (for example after a
reject flow or on OnSubjobError), with a message that names the source and batch.
In the tMap, build the mail subject from the Job name and the caught message.

SLA Monitoring

SLA guardrails available in TAC 8.0 without custom code
 
Execution Plan "Daily_ETL"
  Timeout (s) : 14400              the server kills the plan and its tasks after 4 hours
                                   (the plan status is checked once a minute)
  Rollback    : Alert_SLA_Breach   task run when the plan is killed by the timeout,
                                   for example a Job with tSendMail to the on-call list
  Pause triggers on error          the plan trigger pauses only when a failed task
                                   has an OnOk child branch
 
Start time: a CRON trigger at 02:00 gives the 4-hour plan its 06:00 deadline.

Advanced Patterns

Dynamic Job Generation

// Generate runs dynamically from a configuration table
 
/*
 * tDBInput (etl_job_config, WHERE enabled = 1 ORDER BY priority)
 *     │ row1
 *     ▼
 * tFlowToIterate   ("Use the default (key, value) in global variables": selected,
 *     │             each column is stored as "row1.<column>")
 *     │ iterate
 *     ▼
 * tRunJob
 */
 
// Job configuration table:
/*
 * CREATE TABLE etl_job_config (
 *     config_id INT PRIMARY KEY,
 *     job_name VARCHAR(100),
 *     source_system VARCHAR(50),
 *     source_table VARCHAR(100),
 *     target_table VARCHAR(100),
 *     enabled BIT,
 *     priority INT,
 *     parameters TEXT
 * );
 */
 
// tRunJob Basic settings:
/*
 * Use dynamic job : selected (only the latest version of each called Job is used)
 * Context job     : ((String) globalMap.get("row1.job_name"))
 * Context         : Production
 *
 * Context parameters passed to the child:
 * - source_system: ((String) globalMap.get("row1.source_system"))
 * - source_table : ((String) globalMap.get("row1.source_table"))
 * - target_table : ((String) globalMap.get("row1.target_table"))
 */

Cross-Environment Promotion

Promotion DEV → PROD without rebuilding
 
1. Build once: publish the Job from Talend Studio (or a Maven CI build) to the
   artifact repository with a version number, for example 1.4.0.
2. Keep environment values in context groups (Development, Production) or pass
   them as context parameters on the task; the artifact stays the same.
3. DEV TAC: the task points at artifact 1.4.0 with context Development; test it.
4. PROD TAC: update the PROD task to artifact 1.4.0 with context Production,
   after your approval step.
5. Run the PROD task once by hand (or with runTask) before re-enabling its triggers.
 
Exported Jobs outside TAC take the same context switches:
  ./Daily_ETL_Job_run.sh --context=Production --context_param batch_date=2026-11-15

Recovery and Restart

// Checkpoint-based restart with a small routine and three components
 
// 1) Routine: Repository > Code > Routines > Checkpoint
package routines;
 
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.nio.file.StandardCopyOption;
 
public class Checkpoint {
 
    public static String read(String dir, String key) throws java.io.IOException {
        Path p = Paths.get(dir, key + ".chk");
        if (!Files.exists(p)) {
            return null;
        }
        return new String(Files.readAllBytes(p), StandardCharsets.UTF_8).trim();
    }
 
    public static void save(String dir, String key, String value) throws java.io.IOException {
        Path tmp = Paths.get(dir, key + ".chk.tmp");
        Files.write(tmp, value.getBytes(StandardCharsets.UTF_8));
        Files.move(tmp, Paths.get(dir, key + ".chk"),
            StandardCopyOption.REPLACE_EXISTING, StandardCopyOption.ATOMIC_MOVE);
    }
 
    public static void delete(String dir, String key) throws java.io.IOException {
        Files.deleteIfExists(Paths.get(dir, key + ".chk"));
    }
}
 
// 2) tJava at the start of the Job
String checkpointKey = "checkpoint_" + jobName + "_" + context.batch_date;
String lastCheckpoint = Checkpoint.read(context.checkpoint_dir, checkpointKey);
globalMap.put("checkpoint_key", checkpointKey);
globalMap.put("resume_from", lastCheckpoint);       // use it in the input query, e.g. WHERE id > resume_from
globalMap.put("is_restart", lastCheckpoint != null);
System.out.println(lastCheckpoint != null
    ? "Resuming from checkpoint: " + lastCheckpoint
    : "Starting fresh run");
 
// 3) tJavaRow in the main flow: save a checkpoint every 10000 rows
Integer counter = (Integer) globalMap.get("row_counter");
int currentRow = (counter == null ? 0 : counter) + 1;
globalMap.put("row_counter", currentRow);
if (currentRow % 10000 == 0) {
    Checkpoint.save(context.checkpoint_dir,
        (String) globalMap.get("checkpoint_key"), String.valueOf(input_row.id));
}
output_row.id = input_row.id; // map the other columns too
 
// 4) tJava on OnSubjobOk of the last subjob: clear the checkpoint
Checkpoint.delete(context.checkpoint_dir, (String) globalMap.get("checkpoint_key"));
System.out.println("Job completed, checkpoint cleared");

Common mistakes

  • Typing a Quartz expression with a seconds field. The TAC CRON form has separate Minutes, Hours, Days of month, Months, Days of week and Years fields.
  • Giving child nodes to tasks set for parallel execution. A task defined for parallel execution cannot have OnOk, OnError or After sub-nodes.
  • Mixing After with OnOk or OnError under one parent. OnOk and OnError can coexist; After cannot sit next to either.
  • Firing a trigger while the previous run is still going. Job Conductor waits, and if the task has not finished after 60 seconds the execution is marked misfired.
  • Forgetting the file trigger context variables. Define folderPath, filePath and fileName in the Job's contexts so Job Conductor can pass them.

Best Practices

scheduling_best_practices:
  design:
    - "Use execution plans for multi-job pipelines"
    - "Implement proper dependencies between jobs"
    - "Design for restart/recovery capability"
    - "Use meaningful job and plan names"
 
  scheduling:
    - "Schedule resource-intensive jobs during off-peak hours"
    - "Stagger job start times to avoid contention"
    - "Leave buffer time between dependent jobs"
    - "Consider timezone differences for global operations"
 
  monitoring:
    - "Set up alerts for all critical jobs"
    - "Monitor SLAs proactively"
    - "Track job duration trends"
    - "Review failed job logs promptly"
 
  maintenance:
    - "Archive old execution logs regularly"
    - "Review and clean up unused jobs"
    - "Test recovery procedures periodically"
    - "Document scheduling decisions"
 
  security:
    - "Use role-based access control"
    - "Encrypt sensitive context parameters"
    - "Audit all job executions"
    - "Separate environments properly"

Conclusion

Effective job scheduling with TAC transforms individual jobs into reliable automated pipelines. Implement proper triggers, dependencies, and execution plans for complex workflows. Monitor SLAs, set up comprehensive alerting, and design for recovery. These patterns ensure consistent, reliable ETL operations at enterprise scale.


Official references

Related on this site: Talend Job Optimization: Complete Performance Tuning Guide, Talend REST API Integration: Complete Developer Guide, Talend Big Data: Complete Spark and Hadoop Integration Guide. More articles in the Talend category, and the engineering work behind them is on the services page.

Need an AI system built and running in production, with the scope agreed in writing and a delivery guarantee? See how a DeviDevs Sprint works →

Want AI agents and automations your business can actually run on?

We ship production AI in fixed two-week sprints, then secure them. Book a 30-minute call to scope yours.

Book a Call

Before you book, you can see what we build, how a sprint works, how our pricing works, what we have already built, or write to us directly.

Weekly AI Agents, Automation & Security Digest

How we build AI agents and automations with Claude Code, plus practical AI security, integrations, and platform notes, delivered weekly.

No spam. Unsubscribe anytime.