10.1 Multi-Stage Processing Pipelines

You've learned individual automation building blocks—columns, dependencies, triggers, and custom code. Now let's combine them into multi-stage processing pipelines: sophisticated automations where data flows through multiple phases, each building on the previous one.

What Is a Multi-Stage Pipeline?

Simple Analogy: Assembly Line

Think of a car manufacturing assembly line:

Stage 1: Frame assembly

Input: Raw materials
Output: Car frame

Stage 2: Engine installation

Input: Frame (from Stage 1)
Output: Frame + engine

Stage 3: Paint and finishing

Input: Frame + engine (from Stage 2)
Output: Painted car

Stage 4: Quality inspection

Input: Painted car (from Stage 3)
Output: Approved car OR back to Stage 3

Stage 5: Delivery

Input: Approved car (from Stage 4)
Output: Car shipped to dealer

Frame Assembly

Input: Raw materials → Output: Car frame

→

Engine Installation

Input: Frame → Output: Frame + engine

→

Paint & Finishing

Input: Frame + engine → Output: Painted car

→

Quality Inspection

Input: Painted car → Output: Approved OR back to Stage 3

→

Delivery

Input: Approved car → Output: Shipped to dealer

✓

Key characteristics:

Sequential stages (order matters)
Each stage adds value
Later stages depend on earlier stages
Quality gates (inspection, approval)
Different paths possible (pass/fail)

In Taibles

A multi-stage pipeline is a series of columns where:

Early columns collect and validate data
Middle columns enrich and analyze data
Later columns make decisions and take actions
Dependencies ensure correct order
Conditions create quality gates
Parallel branches handle different scenarios

Lead enters system

↓

Stage 1: Collection

Validate email and company

↓

Stage 2: Enrichment

Fetch company data from APIs

↓

Stage 3: Analysis

Calculate lead score

↓

Stage 4: Routing

Assign to sales rep

↓

Stage 5: Outreach

Send personalized email

Core Concepts

Concept 1: Sequential Stages

Definition: Processing happens in specific order

Why it matters: Can't analyze data before you enrich it, can't route before you analyze

Example:

❌Wrong Order

1. Assign to sales rep

↓

2. Calculate score

↓

3. Enrich data

Can't assign before scoring, can't score before enriching!

✅Correct Order

1. Enrich data

↓

2. Calculate score

↓

3. Assign to sales rep

Each stage uses data from the previous stage

In Taibles: Use dependencies to enforce order

Concept 2: Quality Gates

Definition: Checkpoints that validate data before next stage

Why it matters: Don't waste expensive API calls on bad data

Stage 1: Email validation

DoneEmail validated

⚡ Quality Gate

Is email valid?

YES

Continue to enrichment

DoneEnrichment completed

External data fetched successfully

Skip enrichment

SkippedEnrichment skipped

Condition not met: invalid email

In Taibles: Use conditions to create gates

When a column has a condition:

If condition passes → Column runs normally
If condition fails → Column shows Skipped status with gray badge

Concept 3: Parallel Branches

Definition: Multiple columns processing simultaneously (no dependencies between them)

Why it matters: Faster processing, different data sources

Email validated

Done

Enrich from Clearbit

Processing

Independent - runs immediately

Enrich from LinkedIn

Processing

Independent - runs immediately

Both complete

Now merge results

⚡ Performance Benefit

Both branches run at the same time = 2× faster than sequential processing

In Taibles: Columns with same dependencies run in parallel

Concept 4: Aggregation Points

Definition: Where parallel branches reconverge

Why it matters: Combine data from multiple sources

Clearbit data

Done

Job title, company size, industry

LinkedIn data

Done

Skills, experience, education

Combined enrichment

Processing

Waits for BOTH sources before starting

Lead score

Done

Calculated from all available data

In Taibles: Column depends on multiple columns (all must complete first)

Example Pipeline: Lead Processing

Let's build a complete lead processing pipeline with 5 stages.

Business Goal

Objective: Automate lead qualification from initial contact to sales outreach

Process:

Capture lead data (email, company)
Enrich with external data
Score and qualify lead
Route to appropriate sales rep
Generate and send personalized outreach

Expected outcome: Only qualified, enriched leads reach sales team

Lead Processing Pipeline

From initial contact to qualified outreach

Data Collection

Capture email, company, validate format

✓ Quality gate

Enrichment

Fetch external data (parallel processing)

⚡ Parallel

Analysis

Score and qualify lead

⚡ Aggregation

Routing

Assign to appropriate sales rep

✓ Quality gate

Outreach

Generate and send personalized email

👤 Manual approval

Expected Outcome

Only qualified, enriched leads with complete data reach the sales team

Stage 1: Data Collection

Purpose

Capture raw lead data and validate essential fields

Columns to Create

Column Name	Type	Purpose
email	Text	Contact email (manual entry)
company_name	Text	Company name (manual entry)
website	Text	Company website (optional)
email_valid	Custom Code	Validate email format

Implementation: Email Validation Column

email_valid

Email Validation Column

Custom Code

Run mode:

Always (recalculate when data changes)

Dependencies:

Logic:

Checks if email exists
Validates format (contains @ and .)
Blocks disposable email domains
Returns validation result with details

Example outputs:

{"valid": true, "email": "john@company.com", "domain": "company.com"}

{"valid": false, "error": "Disposable email not allowed"}

⚡ Quality Gate

Only emails with valid = true will proceed to enrichment

What it does:

Checks if email exists
Validates email format (contains @ and .)
Blocks disposable email addresses

Quality gate: Only valid emails proceed to enrichment

What you'll see in the UI:

Valid email → Column shows Done with green checkmark
Invalid email → Column shows Done with validation error details
Other columns with condition checking this → Skipped with gray badge if invalid

Stage 2: Enrichment

Purpose

Fetch external data to enrich lead information

Columns to Create

Column Name	Type	Dependencies	Condition
basic_info	Clearbit API	email_valid	email is valid
company_details	Company API	company_name OR website	at least one present
social_profiles	Social Scraper	basic_info	basic_info not null

Implementation: Basic Info Enrichment

Configure Column: basic_info

Clearbit Person Enrichment

Column Name

Display Label

Email Address

Click to select which column contains the email address

Final Configuration

Run ModeRun once (don't re-enrich)

Dependenciesemail_valid

ConditionReference email_valid → Select valid → Type == true

Configuration Steps:

Add the Column
- Click the + Add Column button in the toolbar
- Search for your enrichment service (e.g., "Clearbit")
- Select the appropriate node
Configure Basic Settings
- Column name: basic_info
- Display label: "Basic Info"
- Email field: Click to select your email column
Set Dependencies
- Click the Dependencies tab
- Select email_valid from the list
- Click Save
Add Condition (Quality Gate)
- Click the Conditions tab
- The editor opens with autocomplete support
- Type: Reference email_valid column → Select valid property → Type == true
- Click Save
Set Run Mode
- Click the Run Configuration tab
- Select Run once (don't re-enrich)
- Click Save

Result:

Only enriches leads with valid emails
Runs once per lead (doesn't re-enrich if data changes)
Shows Skipped badge if email validation failed

Parallel Processing

Notice: basic_info and company_details have different dependencies

email_valid completes

Done

↙↘

basic_info starts

Processing

Uses email

company_details starts

Processing

Uses company_name

↓

Both complete independently

⚡ Faster! Parallel processing saves time

They run in parallel for faster processing!

Stage 3: Analysis

Purpose

Analyze enriched data to score and qualify leads

Columns to Create

Column Name	Type	Dependencies	Purpose
lead_score	Custom Code	basic_info, company_details	Calculate score
match_score	LLM	company_details, social_profiles	AI relevance scoring
qualification_status	Custom Code	lead_score, match_score	Final decision

Implementation: Lead Score

Component visualization placeholder

What it calculates:

Company size score (more employees = higher score)
Industry score (target industries get bonus)
Role score (decision-makers get bonus)
Returns total score and letter grade (A, B, C)

Configuration:

Run mode: Always (recalculate if data changes)
Dependencies: basic_info, company_details

Implementation: Qualification Status

Component visualization placeholder

What it does:

Checks both lead score and match score
Qualified if both scores meet thresholds
Returns decision with reasoning

Configuration:

Run mode: Always (update if scores change)
Dependencies: lead_score, match_score

Aggregation Point

Notice: qualification_status depends on both scoring columns

Component visualization placeholder

Waits for BOTH to complete before executing

You'll see in the UI:

lead_score shows Done with green checkmark
match_score shows Done with green checkmark
Only then qualification_status starts: Processing with animated spinner

Stage 4: Routing

Purpose

Route qualified leads to appropriate sales reps

Columns to Create

Column Name	Type	Dependencies	Condition	Run Mode
email_template	LLM	All previous	lead is qualified	Run once
email_sent	SMTP	email_template, assigned_to	None	👤 Manual

Implementation: Assignment

Component visualization placeholder

What it does:

Routes by geography (US West, US East, Europe, etc.)
Assigns to specific sales rep based on territory
Records assignment date

Configuration:

Run mode: Run once (assign once, don't reassign)
Dependencies: qualification_status
Condition: Only run if lead is qualified

What you'll see:

Qualified lead → Column shows Done with assignment details
Unqualified lead → Column shows Skipped (condition not met)

Implementation: Priority

Component visualization placeholder

What it calculates:

Combines lead score and match score
High scores (>70) → HOT (4 hour SLA)
Medium scores (>50) → WARM (24 hour SLA)
Low scores → COLD (72 hour SLA)

Configuration:

Run mode: Always (update if scores change)
No condition (calculate for all leads)

Stage 5: Outreach

Purpose

Generate personalized outreach and send to lead

Columns to Create

Column Name	Type	Dependencies	Condition
assigned_to	Custom Code	qualification_status	lead is qualified
priority	Custom Code	lead_score, match_score	None

Implementation: Email Template Generation

Component visualization placeholder

What it does:

AI generates personalized email based on all lead data
References industry, role, company size
Proposes discovery call
Keeps it concise (under 150 words)

Configuration:

Run mode: Run once (generate once per lead)
Dependencies: All previous columns
Condition: Only for qualified leads

Implementation: Send Email

Component visualization placeholder

Why Manual?: Sales rep reviews before sending

Workflow:

Email template generates automatically
Shows Done with green checkmark
Email content appears in the cell
Sales rep reviews email
If approved: Right-click cell → Recalculate cells
Email column starts: Processing → Done
Email sent!

Complete Pipeline Architecture

Visual Flow

Component visualization placeholder

What You See in the UI

As a row processes through the pipeline:

Stage 1 (Collection):

Columns show Processing (animated spinner, green)
Email validation completes → Done (green checkmark)

Stage 2 (Enrichment):

Enrichment columns start in parallel: Both show Processing
Complete at different times → Each shows Done

Stage 3 (Analysis):

Scoring columns start: Processing
Complete → Done
Qualification waits for both → Then starts Processing → Done

Stage 4 (Routing):

If qualified: Assignment starts → Done with rep name
If unqualified: Assignment shows Skipped (gray badge)
Priority always runs → Done

Stage 5 (Outreach):

If qualified: Email template generates → Done
Email send waits: Shows Idle (manual trigger needed)
After review: Manual trigger → Processing → Done

Building Your Pipeline: Step-by-Step

Step 1: Design on Paper

Before creating columns:

Component visualization placeholder

Step 2: Create Columns in Order

Best practice: Create columns in dependency order

Component visualization placeholder

Step 3: Set Dependencies Correctly

For each column:

Component visualization placeholder

What you'll see:

Open column configuration dialog (gear icon)
Click Dependencies tab
Multi-select list appears
Check columns this column needs
Save

Testing dependencies:

Add test row
Watch execution order in the grid
Earlier columns complete first (green checkmarks)
Dependent columns wait (stay Idle)
When dependencies done → Start Processing

Step 4: Add Quality Gates (Conditions)

For columns that should skip on bad data:

Component visualization placeholder

What you'll see:

Open column configuration
Click Conditions tab
Expression editor opens
Type condition with autocomplete
Save

Testing conditions:

Add row with invalid data
Column shows Skipped (gray badge, condition icon)
Click cell → Details show why it was skipped
Add row with valid data
Column shows Processing → Done

Step 5: Set Appropriate Run Modes

For each column, choose:

Component visualization placeholder

Where to set it:

Open column configuration
Click Run Configuration tab
Select mode from dropdown
Save

Step 6: Test the Complete Pipeline

Component visualization placeholder

Benefits of Multi-Stage Pipelines

Benefit 1: Quality Control

Component visualization placeholder

Result: 80% cost reduction, 10× better lead quality

Benefit 2: Parallel Processing

Component visualization placeholder

Benefit 3: Clear Workflow

Component visualization placeholder

Benefit 4: Reusable Stages

Each stage can be:

Tested independently
Modified without affecting others
Reused in other pipelines

Example: Enrichment stage

Use in lead pipeline
Reuse in contact pipeline
Reuse in account pipeline

Common Pipeline Patterns

Pattern 1: Linear Pipeline

Structure: A → B → C → D

Use case: Simple sequential processing

Example: Data entry → Validation → Enrichment → Export

Component visualization placeholder

Pattern 2: Branching Pipeline

Structure: Different paths based on conditions

Use case: Route data differently based on criteria

Example:

Qualified → Route to sales
Unqualified → Route to nurture

Component visualization placeholder

Pattern 3: Fan-Out Pipeline

Structure: One input, multiple parallel processes

Use case: Parallel enrichment from multiple sources

Example:

Enrich from Clearbit
Enrich from LinkedIn
Enrich from company website

Component visualization placeholder

Pattern 4: Fan-In Pipeline

Structure: Multiple inputs, single output

Use case: Combine multiple sources

Example:

Clearbit data → Combined profile
LinkedIn data → Combined profile
Website data → Combined profile

Component visualization placeholder

Summary: Multi-Stage Pipelines

You now understand multi-stage pipelines:

Core concepts:

Sequential stages (order matters with dependencies)
Quality gates (conditions that skip processing)
Parallel branches (faster processing)
Aggregation points (combining data)

Complete example: 5-stage lead processing pipeline

Implementation steps:

Design on paper first
Create columns in order
Set dependencies correctly
Add quality gates (conditions)
Choose appropriate run modes
Test thoroughly

Benefits:

Quality control (80% cost reduction)
Parallel processing (37% faster)
Clear workflow (maintainable)
Reusable stages (modular)

Common patterns: Linear, branching, fan-out, fan-in

Next Steps

Now that you understand multi-stage pipelines, Section 10.2 will cover:

Event-driven architectures
Triggers that start pipelines
Event processing patterns
Real-time automation

Let's explore event-driven automation!

10.1 Multi-Stage Processing Pipelines ​

What Is a Multi-Stage Pipeline? ​

Simple Analogy: Assembly Line ​

In Taibles ​

Core Concepts ​

Concept 1: Sequential Stages ​

Concept 2: Quality Gates ​

Concept 3: Parallel Branches ​

Concept 4: Aggregation Points ​

Example Pipeline: Lead Processing ​

Business Goal ​

Lead Processing Pipeline

Stage 1: Data Collection ​

Purpose ​

Columns to Create ​

Implementation: Email Validation Column ​

email_valid

Stage 2: Enrichment ​

Purpose ​

Columns to Create ​

Implementation: Basic Info Enrichment ​

Configure Column: basic_info

Parallel Processing ​

Stage 3: Analysis ​

Purpose ​

Columns to Create ​

Implementation: Lead Score ​

Implementation: Qualification Status ​

Aggregation Point ​

Stage 4: Routing ​

Purpose ​

Columns to Create ​

Implementation: Assignment ​

Implementation: Priority ​

Stage 5: Outreach ​

Purpose ​

Columns to Create ​

Implementation: Email Template Generation ​

Implementation: Send Email ​

Complete Pipeline Architecture ​

Visual Flow ​

What You See in the UI ​

Building Your Pipeline: Step-by-Step ​

Step 1: Design on Paper ​

Step 2: Create Columns in Order ​

Step 3: Set Dependencies Correctly ​

Step 4: Add Quality Gates (Conditions) ​

Step 5: Set Appropriate Run Modes ​

Step 6: Test the Complete Pipeline ​

Benefits of Multi-Stage Pipelines ​

Benefit 1: Quality Control ​

Benefit 2: Parallel Processing ​

Benefit 3: Clear Workflow ​

Benefit 4: Reusable Stages ​

Common Pipeline Patterns ​

Pattern 1: Linear Pipeline ​

Pattern 2: Branching Pipeline ​

Pattern 3: Fan-Out Pipeline ​

Pattern 4: Fan-In Pipeline ​

Summary: Multi-Stage Pipelines ​

Next Steps ​

10.1 Multi-Stage Processing Pipelines

What Is a Multi-Stage Pipeline?

Simple Analogy: Assembly Line

In Taibles

Core Concepts

Concept 1: Sequential Stages

Concept 2: Quality Gates

Concept 3: Parallel Branches

Concept 4: Aggregation Points

Example Pipeline: Lead Processing

Business Goal

Stage 1: Data Collection

Purpose

Columns to Create

Implementation: Email Validation Column

Stage 2: Enrichment

Purpose

Columns to Create

Implementation: Basic Info Enrichment

Parallel Processing

Stage 3: Analysis

Purpose

Columns to Create

Implementation: Lead Score

Implementation: Qualification Status

Aggregation Point

Stage 4: Routing

Purpose

Columns to Create

Implementation: Assignment

Implementation: Priority

Stage 5: Outreach

Purpose

Columns to Create

Implementation: Email Template Generation

Implementation: Send Email

Complete Pipeline Architecture

Visual Flow

What You See in the UI

Building Your Pipeline: Step-by-Step

Step 1: Design on Paper

Step 2: Create Columns in Order

Step 3: Set Dependencies Correctly

Step 4: Add Quality Gates (Conditions)

Step 5: Set Appropriate Run Modes

Step 6: Test the Complete Pipeline

Benefits of Multi-Stage Pipelines

Benefit 1: Quality Control

Benefit 2: Parallel Processing

Benefit 3: Clear Workflow

Benefit 4: Reusable Stages

Common Pipeline Patterns

Pattern 1: Linear Pipeline

Pattern 2: Branching Pipeline

Pattern 3: Fan-Out Pipeline

Pattern 4: Fan-In Pipeline

Summary: Multi-Stage Pipelines

Next Steps