Typed Datasets

Datasets are pre-generated entity tables (products, locations, campaigns) that journeys reference during execution. They are generated once before any journey runs and are immutable during execution.

Defining a Dataset

Use defineDataset with field generators. Each field can be a static value or a function receiving the context and row metadata.

import { defineDataset, oneOf } from '@synode/core';

const productsDef = defineDataset({
  id: 'products',
  name: 'Product Catalog',
  count: 500,
  fields: {
    id: (ctx, row) => `prod-${row.index}`,
    name: (ctx) => ctx.faker.commerce.productName(),
    price: (ctx) => ctx.faker.number.float({ min: 9.99, max: 499.99, fractionDigits: 2 }),
    category: oneOf(['electronics', 'clothing', 'home', 'sports', 'books']),
    inStock: (ctx) => ctx.faker.datatype.boolean({ probability: 0.85 }),
  },
});

Field generators receive (context, { index, data }) where index is the row number and data is the partially-built row (for referencing earlier fields).

Type Inference with InferDatasetRow

Extract the row type from a dataset definition for type-safe access downstream.

import { type InferDatasetRow } from '@synode/core';

type Product = InferDatasetRow<typeof productsDef>;
// Result: { id: string; name: string; price: number; category: string; inStock: boolean }

This works by unwrapping generator functions and Promises -- if a field is (ctx) => string, the inferred type is string.

Accessing Datasets in Journeys

Untyped Access

Use ctx.dataset('id') for a DatasetHandle with unknown row types.

const product = ctx.dataset('products').randomRow();
const allProducts = ctx.dataset('products').getAllRows();
const count = ctx.dataset('products').size();

Typed Access

Use ctx.typedDataset<T>('id') with the inferred type for full type safety.

import { defineAction, type InferDatasetRow } from '@synode/core';

type Product = InferDatasetRow<typeof productsDef>;

const addToCart = defineAction({
  id: 'add-to-cart',
  name: 'add_to_cart',
  handler: (ctx) => {
    const product = ctx.typedDataset<Product>('products').randomRow();
    return [
      {
        id: ctx.generateId('event'),
        userId: ctx.userId,
        sessionId: ctx.sessionId,
        name: 'add_to_cart',
        timestamp: ctx.now(),
        payload: { productId: product.id, productName: product.name, price: product.price },
      },
    ];
  },
});

DatasetHandle Methods

Method	Return	Description
randomRow()	TRow	Random row from the dataset
getRowById(id)	TRow | undefined	Lookup by id field value
getRowByIndex(i)	TRow | undefined	Lookup by zero-based index
getAllRows()	TRow[]	All rows
size()	number	Row count

Registering Datasets with generate

Pass dataset definitions to generate. They are hydrated before any journey runs.

import { generate } from '@synode/core';

await generate(purchaseJourney, {
  users: 1000,
  datasets: [productsDef, locationsDef],
  lanes: 4,
});

Import and Export

Use importDataset and exportDataset for file-based dataset I/O. Supported formats: csv, json, jsonl.

import { exportDataset, importDataset } from '@synode/core';

// Export to file (formats: 'csv', 'json', 'jsonl')
await exportDataset(dataset, './data/products.csv', 'csv');

// Import from file
const products = await importDataset('products', 'Product Catalog', './data/products.csv', 'csv');

Pass imported datasets via preloadedDatasets -- these skip generation and are injected directly.

const products = await importDataset('products', 'Products', './products.json', 'json');
await generate(journey, { users: 500, preloadedDatasets: [products] });

For in-memory operations, use exportDatasetToString and importDatasetFromString.

Cross-Field References

Later fields can reference earlier fields in the same row via the data parameter.

const orders = defineDataset({
  id: 'orders',
  name: 'Orders',
  count: 200,
  fields: {
    quantity: (ctx) => ctx.faker.number.int({ min: 1, max: 10 }),
    unitPrice: (ctx) => ctx.faker.number.float({ min: 5, max: 100, fractionDigits: 2 }),
    total: (_ctx, row) => (row.data.quantity as number) * (row.data.unitPrice as number),
  },
});