configuring-imports

Configure Celigo imports -- the destination step that writes records to external systems. Use when creating or editing imports, choosing the adaptor type, setting up lookups, upsert or exactly-once writes, or file-based imports. For mapping rules, use writing-mappings; for AI agent steps, configuring-ai-agents.

2,923
installs: 0 up / 0 down, score 0.00
votes
Updated: less than a month ago
In the catalog since: 1 Oct 2026
From: celigo/ai

Get it through your agent

Skill instructions

Rendered from the published SKILL.md, 498 lines.

Configuring Imports

An import is the data destination in a Celigo integration. It takes records from an upstream step and writes them to an external system -- REST APIs, databases, ERPs, file servers, or AI models. Every import has one adaptor type and, apart from the connection-less kinds in Minimum Required Fields, is bound to exactly one connection.

Imports handle six concerns:

Field mapping -- transforming source fields into the destination system's expected format (including value resolution via static maps and lookup tables). Uses Mapper 2.0 (mappings[] array) by default, including NetSuiteHTTPImport. SalesforceImport and WrapperImport support only Mapper 1.0 (mapping.fields[] / mapping.lists[]), and NetSuiteDistributedImport uses its own Mapper 1.0 dialect at netsuite_da.mapping
Operation logic -- create, update, upsert, delete, attach/detach
Hooks -- JavaScript pre/post processing at various pipeline stages (preMap, postMap, postSubmit). File-based imports that generate files from records also support postAggregate; an HTTP records import refuses it (422 invalid_hook)
One-to-many -- fan out a child array already on the input record. Set oneToMany: true and pathToMany to that path (e.g., "lineItems") when one source record should create one import operation per child. After fan-out the child is record ({{record.sku}}) and the parent (without the array) is record._PARENT ({{record._PARENT.id}}). Leave pathToMany blank when the arriving record is itself an array (grouped rows from a groupByFields export or a file group) -- each element then becomes the record, with no _PARENT. A path on an array record processes zero children and still reports success; a blank path on an object record fails with oneToMany_error. oneToMany doesn't batch by itself: templates see batch_of_records only when http.batchSize is above 1 (the default varies by API) or an RDBMS query runs per_page
Response mapping -- extract fields from the import's API response back into the record for downstream steps. Configured on the flow's pageProcessors[] entry, but planned when building the import. Extracts read the response envelope: id (the destination-assigned record ID), statusCode, ignored, errors, _headers, dataURI (the record's link, when the import sets dataURITemplate), and _json (the response body after any response transform, whose top-level fields are also flattened onto the envelope). Use id for a created record's ID and _json.fieldName for other response fields (e.g., _json.output.1.content.0.text for OpenAI responses) -- _json.id works only when the body itself has a top-level id. Flow test runs never send imports, so id is empty there. Response mapping uses Transformation 1.0 syntax (extract/generate pairs), not the newer expression-based transforms
postResponseMap hook -- JavaScript processing after response mapping merges the response back into the record. Configured on the flow's pageProcessors[] entry, but planned when building the import. Use to transform or enrich the merged record before downstream steps

Imports are used across flows, APIs, and tools.

Import Execution Pipeline

When records arrive at an import step, this pipeline executes in strict order:

Input filter (optional) -- records that don't match skip this import (see Gotchas). Configured as filter on the import resource itself -- an inputFilter key on an import is silently discarded. See configuring-filters
preMap hook (optional) -- JavaScript processing before field mapping
Field mapping -- Mapper 2.0 or 1.0 maps source fields to destination fields, including lookups and hardcoded values
postMap hook (optional) -- JavaScript processing after field mapping, before submission
Submit to destination -- writes the mapped record to the external system
postSubmit hook (optional) -- JavaScript processing after the destination responds (access response data, log results, trigger side effects)
Response transform (optional, responseTransform) -- reshapes the response body (_json) after postSubmit. Its expression rules are Transformation 1.0 only (expression.version: "1"; "2" is refused with 422), and it keeps only the fields it maps, so a field it drops is gone for response mapping. Its execution-log stage is responseTransformation
Response mapping (optional) -- carries data from the destination response back into the record for downstream steps. Configured on the flow's pageProcessors[] entry, not on the import itself
postResponseMap hook (optional) -- JavaScript processing after response mapping merges response data back into the record

Key distinction: Response mapping lives on the flow's pageProcessors[] entry, not on the import resource. preMap, the mapping and postMap shape only the request this import sends: the next step receives the record as it arrived plus what response mapping and postResponseMap add, so a field added in steps 2-4 never reaches it. When building an import that needs to pass data downstream, plan the response mapping at flow design time.

What an HTTP import's templates see. http.relativeURI and header values render against the record before mapping, and an undefined path fails the request: a field the record lacks in the URI -- or a {{lookup.<name>}} whose lookup found nothing -- fails the record before anything is sent, with handlebars_template_parse_error (Failed to generate request url from template: … Details: "<field>" not defined in the model.). That is missing data, not a template bug, so a retry fails the same way; an update import that some records reach without the destination id needs a composite UPDATE/CREATE or ignoreMissing (Common Errors). A field holding "" is defined: it renders empty, and the request goes out without the id. A field holding null renders the text null instead (the request goes to …/null), and a field under an object that is null fails the record like a missing one ("<field>" not defined in null) -- see writing-handlebars, gotcha 16. http.body renders against the mapped record, where an undefined path renders empty and the record still succeeds (strictHandlebarEvaluation doesn't change that). Leave http.body unset to send the mapped record; when you template it, use the mapped (destination) field names, and double braces for string values in a JSON body (gotcha 11).

Categories of Import

Record-Based Imports

Submit structured records to APIs, databases, or ERPs. The vast majority of imports.

NetSuiteDistributedImport -- high-performance SuiteApp writes (add, update, addupdate, delete, attach, detach). The default for NetSuite when the Celigo SuiteApp or bundle is installed. netsuite_da.restletVersion is fixed when the step is created, so author new steps on suiteapp2.0; a hook's SuiteScript version must match the step's
NetSuiteHTTPImport -- NetSuite's REST APIs on a netsuite connection (nsDomainType: "suitetalk": POST /services/rest/record/v1/<type> to insert, PATCH .../<type>/{id} to update; "restlet" for a custom RESTlet). Use it when neither the SuiteApp nor the bundle is installed, or the user asks for REST
HTTPImport -- REST/GraphQL APIs (POST, PUT, PATCH, DELETE). Supports connector-assisted (formType: "assistant") and GraphQL (graph_ql, assistant_graphql) modes; see GraphQL imports
SalesforceImport -- Salesforce writes through salesforce.api: soap (the default), rest, or compositerecord (Composite Graph, a tree of related records in one request); metadata deploys configuration, not records. salesforce.api has no Bulk value
RDBMSImport -- SQL databases (Snowflake, PostgreSQL, MySQL, SQL Server, Oracle). rdbms.queryType is a one-element array whose value fixes the companion field: per_record, per_page (renders against batch_of_records) and first_page (runs once, on the first page) take rdbms.query, an array of SQL strings; bulk_insert takes rdbms.bulkInsert (pure INSERT, with no upsert or ignore-existing logic); bulk_load takes rdbms.bulkLoad (a staged load that becomes a MERGE with primaryKeys, or runs your own SQL with overrideMergeQuery). Prefer bulk_load where the database supports it, then bulk_insert for pure INSERTs, then per_page, and per_record only when the logic can't be batched. The legacy uppercase INSERT (and the retired MERGE, COPY and BULK INSERT) fail any save with 422 "Import query types … are deprecated"; UPDATE is legacy too, so don't use it. Other strings, such as lowercase insert, fail as an invalid enum value. A bulk_load import's GET also returns a file block describing its staging file; the platform derives it on every read, so a save that changes it is accepted and discarded. The SQL itself: writing-sql
MongodbImport, DynamodbImport, JDBCImport -- other databases

GraphQL imports

A GraphQL import is an HTTPImport with http.formType graph_ql (or assistant_graphql, the connector-assisted GraphQL form). Each http.body[] entry is a string holding the {"query", "operationName", "variables"} envelope, one per operation on an update/create pair -- never an object, which is stored as-is and doesn't render -- and the mutation's input comes from the record through Handlebars in variables.

Endpoint. The import posts to the connection's base URI -- its form has no relative-URI field and writes relativeURI: ["/"] on every save -- so the connection must supply the GraphQL endpoint: a graph_ql connection, or a connector connection bound to the app's GraphQL API. On a connection bound to the app's REST API every record fails (HTTP 406 on Shopify); add a second connection bound to the GraphQL API, re-point the GraphQL steps to it, and leave the shared REST connection unchanged. The form type is set on the step, so a plain http connection whose base URI is the endpoint works too.
Variables. Under graph_ql or assistant_graphql, a variables value stored as a JSON string is sent as an object; under http it goes out as a string. An object takes expressions inside string values only: "variables": {"input": {"email": "{{record.email}}"}}. A string can also carry bare values, but its string values sit two JSON levels deep, so give each one jsonEncode in double braces: "variables": "{\"input\": {\"note\": \"{{jsonEncode record.note}}\", \"quantity\": {{record.qty}} } }". An object or array goes in as {{jsonSerialize record.items}}, also in double braces. With {{record.note}}, or triple braces on either helper, a value holding a quote fails its record with 422 invalid_json before anything is sent.
Errors come back as HTTP 200. A GraphQL server usually answers a failed mutation with 200 and an errors array, and with no response settings the import counts the record as written. Set http.response.resourceIdPath to the id only a successful mutation returns (["data.customerCreate.customer.id"]): any other answer then fails the record with missing_id, top-level errors and the mutation's own userErrors alike. For the server's message on top-level errors, also set failPath: ["errors[0].message"] and errorPath: "errors[0].message"; failPath: ["errors"] catches a response with exactly one error and lets two or more through.

File-Based Imports

Write or upload files to remote storage. Require the file{} configuration block. Two modes:

Record-to-file -- aggregates incoming records into a file (CSV, JSON, XML, XLSX). The file{} block defines the output format, and file.aggregation.type how a run's records become files: all (one file per run), page (one per page) or key (EDI file definitions only, with B2B Manager). Send the legacy file.skipAggregation as its pair -- true with page, false with all or key -- because validation reads the value you send: an EDI file definition on page without true, or a file.batchSize without it, fails with 422 not_supported. The format is file.type (csv, json, xml, xlsx), with its settings under that name: a CSV's file.csv.includeHeader defaults to true, so set it false for a file with no header row. The columns are the import's mappings rows, in order, each row's generate naming its column; there is no file.columns.
Blob passthrough -- transfers a file as-is from an upstream blob export or blob lookup. Set blob: true, and blobKeyPath to the record field holding the blobKey that step put there (FTP and Google Drive imports fall back to the record's root blobKey when it's unset). Base64 content in a record is record data, not a blobKey. AI agent, guardrail and tool imports reject both fields with 422 not_supported.

Adaptor types:

HTTPImport with http.type: "file" -- the folder mode of the cloud file providers (Google Drive, Box, Dropbox, Azure Blob Storage, Google Cloud Storage, Azure Data Lake Storage Gen2, Celigo Storage). Address the folder by path in http.relativeURI (a folder, never an API endpoint) or by the provider's folder id in file.directory (pathMode: "directoryId"), and name the file in file.fileName; provider-specific location rules are in file.yml
FTPImport -- CSV, XML, JSON, XLSX, EDI files to FTP/SFTP. The folder goes in ftp.directoryPath and the file's name in ftp.fileName, which takes no directory separators (the UI stores the same name in file.fileName too); a dated name is a Handlebars template (writing-handlebars > File paths and names)
S3Import -- objects to Amazon S3
AS2Import -- AS2 EDI file transmission
FileSystemImport -- local/on-premise filesystem writes

AI Imports

Invoke AI models for classification, extraction, or safety checks. No _connectionId required unless using BYOK.

AiAgentImport -- LLM invocations (any provider in the aiAgent.provider enum) with structured output, tool use, and reasoning
GuardrailImport -- PII detection, content moderation, or custom AI-based validation

Stack and Tool Imports

WrapperImport -- custom connector code, typically a function on a stack (wrapper.function)
ToolImport -- runs the Celigo Tool named by tool._toolId. It needs no _connectionId; instead it must bind each connection the tool's steps use in tool.overrides.connections as {_abstractId, _id} -- set _id equal to _abstractId to keep the tool's own connection. A connection left unbound fails the run with connection_not_overridden

Topic Imports (Event Streams)

TopicImport -- publishes each record as a message to an Event Streams topic, which topic listeners (TopicExport) read. Set topic._topicId to the topic's _id, never its name (celigo topics list; an unknown id fails the save with 422 invalid_ref), and optionally topic.batchSize (default 1000). There's no connection and no create/update logic: each record becomes one message, reshaped by the mapping if there is one; with one-to-many, each element is its own message carrying the parent under _PARENT. Register the topic on the flow's integration so its flows can bind to it (celigo integrations register-topics <integrationId> <topicId>); the account needs the Event Streams entitlement. Test runs publish nothing; an invoke publishes for real

Import Matching (create / update / upsert)

The core decision on most record-based imports is the operation -- what happens to each record at the destination. Create requires no match key; update and delete require one; upsert checks first and does whichever applies. Users describe the intent in business terms ("look up the customer and update them", "match by email and upsert", "skip the ones that already exist") that resolve into five behaviors:

Always create -- every record is submitted as new. No matching, no checks.
Create only if missing -- match-check first; submit as create if not found, skip silently if found.
Update only if exists -- match-check first; submit as update if found, skip silently if not.
Update only if exists, fail if missing -- strict variant; no-match records error out instead of skipping.
Upsert -- submit a create when no match is found, an update when matched. The catch-all, and the right default when the user is vague.

When matching applies, decide four things: the matching behavior, the match key field(s) (email, external_id, customer.id -- required for anything other than always-create), the on-match action (update, skip, fail), and the on-no-match action (create, skip, fail). A lookup on a cross-reference field (NetSuite externalid, or a custom field that holds the source id) finds only records whose field holds that id, so the same import must write the source id into the field when it creates them; without that mapping no later run finds them, and each run adds the record again.

How a destination implements matching is adaptor-specific -- there is no single "matching mode" field. A destination might expose: a native upsert keyed off an external ID (Salesforce upsert, NetSuite upsert, RDBMS ON CONFLICT); a distinct addupdate operation that handles both paths in one call; an ignoreExisting flag paired with a lookup that probes before writing (lock it when repeats can arrive concurrently -- see Writing Exactly Once); two separate create and update endpoints with no upsert variant (see composite imports below); or a lookup endpoint plus separate create and update endpoints, where the lookup runs pre-write to drive the create-vs-update decision. Some destinations have no matching concept at all -- writing a CSV to FTP, sending an email, posting to a webhook -- so every record goes out as-is.

Prefer import-level matching over a separate lookup step. Imports natively support this pre-write check, so a single import that does the matching and the write together means fewer steps, fewer round-trips, and no glue logic to maintain. A standalone lookup earns its place only when the looked-up data has a consumer beyond the write -- a router branching on something other than "does this exist", an AI agent reasoning over the result, or multiple downstream steps reading different fields. If the only consumer is the destination call itself, the work belongs inside the import. It's also the only check that runs under the import's concurrency lock, which matters whenever the same record can arrive twice (Writing Exactly Once). An import's own lookups refuse an ambiguous match only when told to: an HTTP lookup that finds more than one match fails the record only with allowFailures: false, and otherwise sends the first match with no error (gotcha 14); a NetSuite DA lookup with allowFailures: true also resolves to one of the matching records unless useDefaultOnMultipleMatches sends it to the default (netsuitedistributed.yml). So search on a key that identifies one record. When an update's key must come from a separate lookup that can match several records, guard it -- fail or skip on multiple matches, or check for exactly one (configuring-lookup-caches > Safe Lookup-to-Update).

Composite (two-endpoint) imports

When an HTTP destination has no native upsert but exposes separate create and update endpoints, a composite import pins both endpoints on a single import node, role-tagged create and update. The runtime picks per record via a match-key check -- the same way a native upsert would -- so the flow stays one step. Prefer this over two separate imports driven by an upstream lookup or router; reach for separate imports only when the create and update paths must diverge beyond endpoint selection (different mappings, different downstream consumers, or different hook chains).

The operations sit in positional arrays, one entry per operation: http.requestType (["UPDATE", "CREATE"]), http.method and http.relativeURI. Exactly one decider picks per record: http.existingExtract, a field the record already carries (non-empty means UPDATE, whose URI reads it as {{{data.0.<field>}}}), or http.existingLookupName, a lookup in http.lookups that probes the destination (the UPDATE URI reads {{lookup.<name>}}). Never add ignoreExisting to a composite import, and give every http.response.* array one entry per operation.

existingExtract must hold the destination's id (put there by an upstream step's response mapping, or kept as a cross-reference), never the source record's own id: every record carries one, so every record would go to UPDATE. With existingLookupName, a lookup that comes back empty sends the record to CREATE, with no error -- and so does one whose extract resolves nothing, so a $.-prefixed extract sends every record to CREATE. A templated http.body has one entry per operation too; left unset, both requests send the mapped record. A search-driven upsert against an API that finds contacts by email:

"http": {
  "requestType": ["UPDATE", "CREATE"],
  "method": ["PATCH", "POST"],
  "relativeURI": ["/contacts/{{lookup.findContact}}", "/contacts"],
  "existingLookupName": "findContact",
  "lookups": [{
    "name": "findContact",
    "method": "POST",

Show the whole skill (378 more lines)

"relativeURI": "/contacts/search",
"body": "{\"email\": \"{{email}}\"}",
"extract": "results[0].id"

}] }


The lookup sits in `http.lookups` (a `lookups` array sent only at the top level is dropped on save), its request template is in `body`, and the template names the record's fields at the top level (`{{email}}`, never `{{record.email}}`, which renders empty, so every search misses and every record is created again; gotchas 10 and 11). Its `extract` is the id itself, because that is what the UPDATE URI receives. An HTTP lookup's `extract` is a path into the response written without `$.` -- `results[0].id`, or `ids` for an array of ids -- unlike a Mapper 2.0 row's `extract`, which needs `$.`. A `$.` prefix, `[*]`, or a dot path through an array (`results.id`) resolves nothing, so the lookup always comes back empty.

### Writing Exactly Once

Every source can hand an import the same record twice: a sender fires a webhook again, a failed or canceled run is re-run over the same delta window, an error is retried, a record that wrote this step but failed a later one comes back on the next run, or another flow writes the same destination. **A trace key never de-duplicates** -- it correlates a record's errors and is the default lock key below; a record that arrives twice is written twice. Make the write itself idempotent, in this order of preference:

| Rank | Pattern | Why it holds under concurrency |
|---|---|---|
| 1 | **A key the destination enforces** -- an upsert or insert on a primary key or unique constraint (`ON CONFLICT`, `MERGE` -- see [writing-sql](../writing-sql/SKILL.md)), a unique external ID, an API that rejects a duplicate reference | The destination refuses the second copy |
| 2 | **`ignoreExisting: true` + the existence check on this import + `idLockTemplate` on the same business key.** `ignoreExisting` (like `ignoreMissing`) is a top-level import field, not part of `http`. The check is `http.ignoreLookupName` (a lookup on the key) or `http.ignoreExtract` (the record already carries the destination ID) -- exactly one -- or the adaptor's equivalent (NetSuite `internalIdLookup`, Salesforce `idLookup.whereClause`) | The existence check runs inside the per-key lock |
| 3 | **A ledger step** in front of a destination that can't be queried -- [configuring-lookup-caches > Deduplication Ledger](../configuring-lookup-caches/SKILL.md#deduplication-ledger) | The ledger's claim is itself rank 2 |

A match that is really a search run before the write -- a lookup-driven composite create/update, an `addupdate` found by search -- is a check-then-write, not rank 1, unless the destination also rejects the duplicate key. A natural id is unique only within its source: when two sources can produce the same one (two stores, two accounts), namespace the key per source (`{{record.storeId}}-{{record.orderId}}`), or one source's record collides with or overwrites the other's. Rank 2 is only as good as its lookup: a lookup that fails counts as "not found", so the record is written (gotcha 12).

**The concurrency ID lock.** `idLockTemplate` renders a key per record; records that render the same key are never submitted at the same time, and different keys run in parallel.

- **Default:** the record's trace key (a source's `traceKeyTemplate` arrives on the record as `io_custom_trace_key`). Without a template, the trace key is the first top-level id field the record has -- `id`, `internalId`, `_id`, `internal_id` or `identifier`, in a few spellings each -- and a nested id is never used. A blank trace key means **no lock**, so set `idLockTemplate` when the identity is nested or composite. A template replaces the default rather than adding to it.
- **Render time:** from the record as it arrives, before this import's own lookups and mapping. A key built from `{{lookup.<name>}}` or a mapped field renders empty -- and an empty key is no lock at all. Key it on a field the record carries (`{{{record.orderId}}}`), or one an upstream lookup step added through response mapping.
- **Scope:** this import across every page, every concurrent run, and every flow that uses it (`celigo imports dependencies <id>`) -- not the connection: two imports rendering the same key still run in parallel. So two imports that both create the same entity (an order flow that creates missing customers, and a separate customer sync) can each miss the other's write and create it twice: create the entity through one import, or schedule the flows so they don't overlap. Records within one page are sent one at a time anyway, so a one-page test never shows the race.
- **Coverage:** the `ignoreExisting` existence check runs inside the lock. Nothing documents the same for `ignoreMissing`, the composite create/update decision, or one-to-many children -- where those must write once, connection `concurrencyLevel: 1` closes the check-then-write race (the import's pages run one at a time), at the cost of serializing everything on that connection.

## Quick Reference

### Adaptor Decision Matrix

| Your data goes to... | Use adaptorType | Category | Read schema |
|---|---|---|---|
| REST or GraphQL API | `HTTPImport` | Record-based | [http.yml](references/schemas/http.yml) |
| NetSuite, Celigo SuiteApp or bundle installed | `NetSuiteDistributedImport` | Record-based | [netsuitedistributed.yml](references/schemas/netsuitedistributed.yml) |
| NetSuite, neither installed (or REST requested) | `NetSuiteHTTPImport` + `nsDomainType` | Record-based | [http.yml](references/schemas/http.yml) |
| Salesforce objects | `SalesforceImport` | Record-based | [salesforce.yml](references/schemas/salesforce.yml) |
| SQL database (Snowflake, PostgreSQL, etc.) | `RDBMSImport` | Record-based | [rdbms.yml](references/schemas/rdbms.yml) |
| MongoDB | `MongodbImport` | Record-based | [mongodb.yml](references/schemas/mongodb.yml) |
| DynamoDB | `DynamodbImport` | Record-based | [dynamodb.yml](references/schemas/dynamodb.yml) |
| JDBC database (non-built-in) | `JDBCImport` | Record-based | [jdbc.yml](references/schemas/jdbc.yml) |
| Files to cloud storage (Google Drive, Box, Dropbox, Azure Blob, Google Cloud Storage, Celigo Storage) | `HTTPImport` with `http.type: "file"` | File-based | [http.yml](references/schemas/http.yml) |
| Files to FTP/SFTP | `FTPImport` | File-based | [ftp.yml](references/schemas/ftp.yml) |
| Files to S3 | `S3Import` | File-based | [s3.yml](references/schemas/s3.yml) |
| AS2 EDI transmission | `AS2Import` | File-based | [as2.yml](references/schemas/as2.yml) |
| Local filesystem | `FileSystemImport` | File-based | [filesystem.yml](references/schemas/filesystem.yml) |
| An LLM (any provider in the `aiAgent.provider` enum) | `AiAgentImport` | AI | [aiagent.yml](references/schemas/aiagent.yml) |
| PII detection / content moderation | `GuardrailImport` | AI | [guardrail.yml](references/schemas/guardrail.yml) |
| Celigo Tool | `ToolImport` | Tool | [tool.yml](references/schemas/tool.yml) |
| Custom connector code on a stack | `WrapperImport` | Stack | [wrapper.yml](references/schemas/wrapper.yml) |
| An Event Streams topic | `TopicImport` | Topic | [topic.yml](references/schemas/topic.yml) |

**Raw HTTP is the fallback, not the default.** Pick the most specific match, in order:

1. **Native adaptor** -- if the application has its own row (NetSuite, Salesforce, databases, FTP/S3), use it. Do not build an `HTTPImport` against that app's REST API.
2. **Pre-built HTTP connector** -- for any other REST/GraphQL app, check the 550+ connector catalog before writing HTTP config (see [Check for a pre-built connector](#3-check-for-a-pre-built-connector)). The step is still an `HTTPImport`, but it runs on a connector-backed connection and takes its endpoint config from the connector.
3. **Manual HTTP** -- hand-write the config from public API docs only when no connector exists or it doesn't cover the operation you need.

For NetSuite, run `celigo connections suiteapp-status <connectionId>`: `suiteapp.success` or `bundle.success` true means `NetSuiteDistributedImport`; both false, each with `statusCode` 404 and an install `URL`, means neither is installed, so use `NetSuiteHTTPImport`. A different `statusCode` means the check itself failed, not that the components are missing: fix the connection and re-run it before choosing. `NetSuiteImport` is legacy -- don't use it unless asked.

`adaptorType` is **case-sensitive**: `NetSuiteDistributedImport`, not `netsuitedistributedimport`.

### Minimum Required Fields

Every import needs at minimum: `name`, `adaptorType`, and the adaptor config block (`http{}`, `netsuite_da{}`, `salesforce{}`, `tool{}`, `topic{}`, etc.). `_connectionId` is required except on the connection-less imports: `ToolImport` (connections are bound per tool step in `tool.overrides`), `TopicImport` (must not carry one -- with a connection, the save is validated as that connection's adaptor and fails), and `AiAgentImport` / `GuardrailImport` unless they use a BYOK connection.

### Which Schemas to Read

1. Always: [request.yml](references/schemas/request.yml) (base fields)
2. Plus: the adaptor-specific file from the decision matrix
3. If file-based: also [file.yml](references/schemas/file.yml)
4. If cloning: [clone-request.yml](references/schemas/clone-request.yml), [clone-response.yml](references/schemas/clone-response.yml)

### Schema Index

All schemas are in [references/schemas/](references/schemas/):

- **Base fields (all imports):** [request.yml](references/schemas/request.yml)
- **Response shape:** [response.yml](references/schemas/response.yml)
- **Adaptor-specific config:**
  - [http.yml](references/schemas/http.yml) -- HTTP/REST/GraphQL (methods, URIs, headers, response parsing, lookups, composite upsert via `existingExtract` or `existingLookupName`)
  - [netsuitedistributed.yml](references/schemas/netsuitedistributed.yml) -- NetSuite SuiteApp (operation, recordType, internalIdLookup, mapping, lookups)
  - [netsuite.yml](references/schemas/netsuite.yml) -- NetSuite legacy
  - [salesforce.yml](references/schemas/salesforce.yml) -- Salesforce (sObjectType, operation, api, idLookup)
  - [rdbms.yml](references/schemas/rdbms.yml) -- SQL databases (queryType, query, bulkInsert, bulkLoad)
  - [ftp.yml](references/schemas/ftp.yml) -- FTP/SFTP
  - [s3.yml](references/schemas/s3.yml) -- Amazon S3
  - [mongodb.yml](references/schemas/mongodb.yml) -- MongoDB (method, collection, filter, upsert)
  - [dynamodb.yml](references/schemas/dynamodb.yml) -- DynamoDB
  - [jdbc.yml](references/schemas/jdbc.yml) -- JDBC databases
  - [as2.yml](references/schemas/as2.yml) -- AS2 EDI
  - [wrapper.yml](references/schemas/wrapper.yml) -- custom stack connectors
  - [tool.yml](references/schemas/tool.yml) -- Tool imports (`_toolId`, connection overrides)
  - [topic.yml](references/schemas/topic.yml) -- Event Streams publisher (`_topicId`, `batchSize`)
  - [filesystem.yml](references/schemas/filesystem.yml) -- local filesystem
- **AI config:**
  - [aiagent.yml](references/schemas/aiagent.yml) -- AI agent (provider, model, instructions, tools, structured output)
  - [guardrail.yml](references/schemas/guardrail.yml) -- guardrails (PII, moderation, AI-agent validation)
- **File output:** [file.yml](references/schemas/file.yml) (CSV, XML, JSON, XLSX config for file-based imports)
- **Clone:** [clone-request.yml](references/schemas/clone-request.yml), [clone-response.yml](references/schemas/clone-response.yml)

## Related Skills

- [configuring-connections > Quick Reference](../configuring-connections/SKILL.md#quick-reference) -- connection types and auth for import destinations
- [managing-file-storage > Use stored files in a flow](../managing-file-storage/SKILL.md#8-use-stored-files-in-a-flow) -- Celigo Storage folders as a file destination
- [writing-mappings > Mapper 2.0 Workflow](../writing-mappings/SKILL.md#mapper-20-workflow) -- field mappings on imports
- [writing-scripts > Data Pipeline Hooks](../writing-scripts/SKILL.md#data-pipeline-hooks) -- preMap, postMap, postSubmit, postAggregate hooks
- [writing-handlebars > Quick Reference](../writing-handlebars/SKILL.md#quick-reference) -- dynamic values in URIs, HTTP bodies, SQL queries
- [building-flows > How to Build a Flow](../building-flows/SKILL.md#how-to-build-a-flow) -- wiring imports into flow pipelines as page processors
- [configuring-topics > How to Build an Event Stream](../configuring-topics/SKILL.md#how-to-build-an-event-stream) -- the topic a publisher writes to: registration, retention, messages
- [troubleshooting-flows > Diagnostic Workflow](../troubleshooting-flows/SKILL.md#diagnostic-workflow) -- diagnosing import-related failures

<!-- TIER:2 -->

## How to Build an Import

### 1. Identify the target application

What system are you writing data to? This determines adaptor type, connection type, and configuration shape.

### 2. Check for existing patterns

Before building from scratch, look at what already exists:

```bash
# Search your account (fast, uses local index)
celigo account search "<keyword>"

# Show what an existing import uses (connection) and what uses it (flows)
celigo account dependencies import <id>

# Find orphaned imports not referenced by any flow
celigo account lint

# Search marketplace templates
celigo templates marketplace

# Extract just imports from a template
celigo templates preview <id> --model Import
celigo templates preview <id> --summary

The account index auto-refreshes when stale (older than 15 minutes by default). Force a fresh snapshot with celigo account snapshot.

3. Check for a pre-built connector

Always run this check before writing any HTTP config. Celigo maintains 550+ HTTP connectors with pre-configured auth, endpoints, and resources. Hand-write a manual HTTPImport from public API docs only when this search comes up empty or the connector doesn't cover the operation you need.

# Search HTTP connectors
celigo http-connectors list | grep -i "<application-name>"
celigo http-connectors get <id> --full    # see endpoints, resources, auth config

# Drill into the endpoints the connector defines for imports
celigo http-connectors catalog <id> --resource-type import --published-only
celigo http-connectors endpoint-detail <id> --resource-type import --resource-id <rid> --endpoint-id <epid>
celigo http-connectors endpoint-detail <id> --resource-type import --resource-id <rid> --endpoint-ids <createId>,<updateId>   # a create/update pair

# Search trading partner connectors (EDI, AS2)
celigo tp-connectors list

If a connector exists, create the connection from it (http._httpConnectorId -- see configuring-connections > Check for a pre-built connector and global iClient) and take the import's relativeURI, method, and body/response shapes from the connector's endpoint metadata rather than reconstructing them from public API docs. Write each operation's relativeURI and method exactly as the endpoint defines them: saving links the import to those endpoints by setting http._httpConnectorVersionId, http._httpConnectorResourceId and http._httpConnectorEndpointIds[] (one per operation; imports drop the singular _httpConnectorEndpointId), but pointers you send for the same connector are kept as sent. Read the import back: missing pointers mean a URI matched no endpoint. When retargeting an import, update or drop the old pointers in the same save -- a GET-modify-PUT that changes only the URI leaves the import naming the old endpoint.

endpoint-detail on the import plane returns the resource, whose resourceFields catalog the record's fields, and each endpoint's supportedBy: what the connector's own form applies. preConfiguredFields are values it sets, fieldsToUnset are fields it clears, and fieldsUserMustSet is what it asks for -- on Shopify's create-customer endpoint, ignoreExisting and the mapping targets mapping.fields.generate.customer.email, …first_name and …last_name. The paths are the form's, not the import's. On create and update endpoints, lookupToIdentifyExisting is the existence lookup to build (the endpoint to call and the extract that reads the existing id, such as customers[0].id), and pathParameterToIdentifyExisting names the update URI's id parameter.

4. Query metadata for the target system

For NetSuite, Salesforce, and RDBMS connections, discover available record types and fields:

celigo metadata types <connectionId>          # List record types / sObjects / tables
celigo metadata fields <connectionId> <type>  # List fields for an entity type
celigo metadata fields <connectionId> <type> --refresh   # Bypass the cache after a field is created in the app

Metadata is cached on the server, so a field or record type created since the last read stays missing until you re-read with --refresh.

NetSuite: metadata fields returns field IDs, types, and groups. On NetSuiteDistributedImport, use the IDs in netsuite_da.mapping.fields[].generate and netsuite_da.mapping.lists[].fields[].generate; sublist names (groups such as "item", "addressbook") go in netsuite_da.mapping.lists[].generate. NetSuiteHTTPImport maps with Mapper 2.0 mappings[]. Lookup field IDs here are the searchField/resultField values for netsuite_da.lookups[].
Salesforce: metadata fields returns field API names, types, and relationship info. Use field API names for salesforce.sObjectType lookups and for discovering which fields are createable/updateable.
RDBMS: metadata fields returns column names and types for a table — use these to write SQL queries (see writing-sql) and verify column names before building bulkInsert.tableName or bulkLoad.tableName.

5. Determine the category

Is this a record-based import (submit records to an API/database/ERP), a file-based import (write files to storage), or an AI import (invoke a model)?

6. Choose the right adaptor type

Refer to the Adaptor Decision Matrix in the Quick Reference above. For NetSuite, Diagnosing NetSuite Steps has the account-side checks (the SuiteApp and bundle, the account's concurrency limit) and the errors NetSuite steps return.

7. Build the import JSON

Reference the Schema Index for the exact fields needed. Use the Which Schemas to Read decision rule to determine which files to consult.

File Uploads over HTTP (multipart/form-data)

Some destination APIs accept files only as multipart/form-data POSTs (Jira attachments, QuickBooks attachables, OpenAI file uploads). This is an HTTPImport with blob: true and blobKeyPath, posting to the API's upload endpoint, and it works nothing like a JSON record write. Do not set http.type: "file": that is the folder mode of file-storage connectors (Google Drive, Box, Dropbox, Azure Blob), which treats relativeURI as a folder path -- combined with an API endpoint URL it saves cleanly and fails every record with "Directory is not found." Four pieces have to line up:

Where the bytes come from. The import never carries the file itself. A preceding blob export or blob lookup pulls the file into blob storage, and that step's response mapping puts the reference onto the record -- the idiom is {"extract": "data.0.blobKey", "generate": "blobKey"}. The record then carries a blobKey (a storage pointer, not content).
The media type. The connection's media type (or the import's request media type override) is multipart/form-data; success/error response media types usually override to JSON so replies parse normally.
The request body. Not raw MIME and not a normal handlebars payload -- a JSON array of parts, each {name, value, type} plus optional filename (include the extension) and mime-headers. The file part is "type": "attachment" with "value": "{{blob}}" -- {{blob}} is the only accepted value for an attachment (anything else 422s). Fields the API wants alongside the file ride as "type": "inline" parts; an inline part whose value is a JSON object must be serialized. One file reference per import.
blob: true + blobKeyPath (Advanced settings) -- blobKeyPath is the JSON path in the record where the blobKey lives (blobKey, or file.blobKey if nested). At send time the platform follows it into blob storage and streams the real bytes into the attachment part.

The platform assembles the final MIME body itself -- it generates the boundary (never hardcode one), writes each part's Content-Disposition, and substitutes the attachment part with the raw file bytes. The parts array is a build recipe, not the payload.

Not every multipart API is form-data. Some upload endpoints expect multipart/related instead (e.g. Google Drive's /upload/drive/v3/files?uploadType=multipart), and the parts-array machinery does NOT apply. There the request body is the literal MIME document: explicit boundary, a JSON metadata part, and a content part referencing {{blob}} (double braces). Check which flavor the destination API documents before building -- mixing them produces an import that saves cleanly and fails at runtime.

Async Destinations (submit, poll, confirm)

Some destination APIs only acknowledge a write (an HTTP 202, a job ticket) and finish it in the background -- bulk loads, file ingestion, document conversion. Attach an async helper to the import (http._asyncHelperId) so the step submits, polls a status export until the external work completes, and only then resolves. The mechanics and constraints (a status export with done/error value lists and poll intervals; no transform, output filter, or hook on the async-configured step; helpers cannot nest) are identical to the export side -- see configuring-exports > Async APIs (submit, poll, fetch). Only add one when the API genuinely cannot confirm the write synchronously.

Testing an Import Without Writing

Most ways of trying an import send the request to the destination for real. Know which is which before you point one at a production system:

What you run	Reaches the destination?
`celigo imports invoke <id>` with records on stdin	Yes -- a real write. One result per record, every adaptor; no job is created
`celigo imports invoke` with an import document on stdin (the preview route)	Yes -- a real write, despite the name. Only the first record of `data` runs, so try one record per call. NetSuite, Salesforce, RDBMS, and FTP imports are rejected with 422 `invalid_adaptor`; a bare document piped without `data` runs against one empty record
`celigo imports invoke [<id>] --connection <connectionId>` (a virtual import)	Yes -- a real write. Runs the saved import, or a piped `{"import": {...}, "data": [...]}` document, through that connection, NetSuite and RDBMS imports included; one result per record
`celigo flows test-run <flowId> --export <exportId>`	No. Imports never execute in a test run -- each returns its saved `mockResponse`. A NetSuite DA import is the exception: the page still goes to NetSuite, where the SuiteApp or bundle runs the lookups and SuiteScript hooks and builds each record without saving it. Its stages are `mapping` and `lookup` (plus one per SuiteScript hook), not `request` and `response`, and an error NetSuite raises while building a record is real and counts on the job
A real flow run, or `celigo flows retry-errors`	Yes

Invoking by id or as a virtual import runs the step with the import's own settings only: hooks get no integration or flow settings, and {{settings.integration.*}} and {{settings.flow.*}} render empty. To run it as a step of its flow, tool or API, add --integration <id> and --flow, --tool or --api <id> to the --connection form.

In the flow builder's import editor, Preview builds the request without sending it (HTTP imports only), while Send is a real write. No CLI command offers the compose-only preview. From the CLI, test-run the flow and read the import's stages with celigo flows test-run-step-results: the mapping stage's output is the record after mapping, while the request stage shows the body template unrendered, and its input is the record before mapping. Database imports have no compose-only surface at all; the SQL editor's evaluation is their only preview.

Save a realistic mockResponse. In test runs and flow-builder previews, an import with no saved mock answers with the placeholder {"id": "1234567890", "mockResponse": "replace me"} (numbered per record on a page: 1234567890-1, 1234567890-2), so response mappings extract nothing real and downstream steps see empty data. Save an array of records shaped like the destination's answer -- statusCode, the created record's id, and the response body under _json:

celigo imports set <importId> 'mockResponse=[{"statusCode": 200, "id": "SO-1001", "_json": {"id": "SO-1001", "status": "created"}}]'

A saved mockResponse substitutes only in test runs and flow-builder previews -- imports invoke, real runs, and retries always send. sampleResponseData only seeds the response-mapping editor; test runs and previews never use it.

CLI Commands

# CRUD
celigo imports list
celigo imports get <id>
celigo imports create < import.json
celigo imports update <id> < import.json
celigo imports set <id> key=value [key2=value2 ...]
celigo imports delete <id> [-y]
celigo imports audit <id>          # change history
celigo imports dependencies <id>   # the flows, APIs and tools that use it

# Invoke -- a real write: each record is sent to the destination (no job is created).
# HTTP 200 even when records fail; check each result's statusCode and errors
echo '[{"name":"test"}]' | celigo imports invoke <id>
# Virtual import: an unsaved or modified document through a connection (also a real write)
echo '{"import": {...}, "data": [{"name":"test"}]}' | celigo imports invoke --connection <connectionId>

# Mock response for test runs and flow-builder previews (never used by invoke or real runs)
celigo imports set <id> mockResponse=file://./mock-response.json

# Clone and connection management
echo '{"connectionMap":{"oldConnId":"newConnId"}}' | celigo imports clone <id>
celigo imports replace-connection <id> <newConnectionId>

# Discovery
celigo templates marketplace
celigo http-connectors list
celigo http-connectors catalog <id> --resource-type import --published-only
celigo http-connectors endpoint-detail <id> --resource-type import --resource-id <rid> --endpoint-id <epid>
celigo tp-connectors list
celigo metadata types <connectionId>
celigo metadata fields <connectionId> <entityType>

# Debug -- a window of at most 60 minutes (debugUntil, set by PATCH; a PUT doesn't set it).
# It records each request the import sends while open: not earlier ones, and nothing from test runs
celigo imports enable-debug <id> [--duration <minutes>]
celigo imports disable-debug <id>
celigo flows debug-requests <flowId> <importId>   # read them from the consumer (or apis / tools debug-requests)

Pre-Submit Checklist

Required (all imports)

name is set
adaptorType exact case matches connection type (request.yml > adaptorType)
_connectionId references a valid, online connection (skip for ToolImport, TopicImport, and AI imports without BYOK)
Adaptor config block name matches adaptorType (http{} for HTTPImport, netsuite_da{} for NetSuiteDistributedImport, etc.)

Adaptor-specific

HTTP: pre-built connector was checked (celigo http-connectors list) -- hand-written config only because no connector covers the app or operation
HTTP: http.method and http.relativeURI are set (http.yml)
HTTP: string values in a JSON http.body or lookup body are double-braced inside quotes; triple braces only for numbers, booleans and serialized JSON (gotcha 11)
HTTP lookups: every lookup is in http.lookups (the save keeps that array and mirrors it to the top-level lookups; one sent only at the top level is dropped), each request template is in body, not postBody, and names the record's fields without record. (gotcha 10), and each extract is a path without $. to the id itself or an array of ids, from a search on a key that matches one record (gotcha 14)
NetSuite DA: netsuite_da.operation and netsuite_da.recordType are set, and update, addupdate and delete carry netsuite_da.internalIdLookup -- the lookup, not a mapped internalid, picks the record (on update it overrides a mapped one). When the record already carries the internal id, use the expression form ["internalid","is","{{{id}}}"] rather than dropping the lookup (netsuitedistributed.yml)
NetSuite DA fields set at record initialization (NetSuite preloads them when it creates or transforms the record, before the other fields): one body row {"generate": "celigo_initializeValues", "hardCodedValue": "entity,subsidiary"} listing their ids, comma-separated and without .internalid, next to the rows that map those fields. It's what the mapping editor's "Use this field for NetSuite record initialization" saves; a useAsAnInitializeValue key on a row isn't stored
NetSuite DA serialized items: the SuiteApp and bundle take a line's serialnumbers as <serial>(<quantity>) text when the account's Advanced Bin / Numbered Inventory Management feature is off; with it on, give the serials through the line's inventory detail subrecord (celigo_inventorydetail, netsuitedistributed.yml)
NetSuite DA keyed on externalid (["externalid","is","{{{id}}}"]): a mapping writes that same source id to externalid ({"extract": "id", "generate": "externalid"}). The lookup finds only records whose externalid was set, so without the mapping an addupdate adds the record again on every run
RDBMS: rdbms.queryType is one current value (bulk_load, bulk_insert, per_page, per_record or first_page) with its companion field set -- never the legacy uppercase INSERT or UPDATE (rdbms.yml)
Salesforce: salesforce.sObjectType and salesforce.operation are set. upsert also needs salesforce.upsert.externalIdField (the Salesforce external id field) and salesforce.idLookup.extract (the record field matched against it); update, delete, addupdate and an insert with ignoreExisting need idLookup.whereClause instead (salesforce.yml). The mapping is Mapper 1.0: mapping.fields[] rows of {extract, generate} (or hardCodedValue), never mappings[]

Cross-resource consistency

Connection type matches the import's adaptorType
If response mapping needed: configured on the flow's pageProcessors[] entry, not on the import itself
If one-to-many: oneToMany: true, with pathToMany set to the child array path for an object record (child fields then use record.<field> and parent fields record._PARENT.<field>) and left blank for an array (grouped) record
If downstream steps read this import's response: a realistic mockResponse is saved, so test runs carry real-looking data past this step
If a repeated record would duplicate data: the write is keyed (a destination-enforced key, or ignoreExisting + an existence check) and idLockTemplate renders a non-empty key from a field the record carries on arrival -- never {{lookup.*}} (Writing Exactly Once), and its existence lookup has been checked against a record that exists (gotcha 12)
Mapper 1.0 only where required: mapping.fields[] / mapping.lists[] on Salesforce and wrapper imports, netsuite_da.mapping on NetSuite DA -- never mappings[] there. NetSuiteHTTPImport uses mappings[]

Gotchas

PUT erases omitted fields. Always GET first, modify, then PUT. The set command handles this. The import PATCH route accepts only debugUntil (any other path is refused with 422), so a change to a mapping, body or lookup is always this full PUT, never a reason to send the user to the UI. Leave the read-only fields a GET adds (_id, lastModified, createdAt) out of the body.
Including a rest: block creates a legacy RESTImport. Use only http: for new imports.
Input filter skips that import, not the record. Filtered records skip the current import step but continue to subsequent steps in the flow, without this import's response-mapped fields, so a later step that writes those fields back needs its own filter on them. Check numIgnore on the job if records seem to bypass a step.
Multipart file parts only accept {{blob}}. In a multipart/form-data parts array, the file part must be "type": "attachment" with "value": "{{blob}}" -- any other value 422s. Never hardcode the MIME boundary; the platform generates it. Fields the API wants alongside the file ride as "type": "inline" parts.
bodyKey/blobKey in logs is an artifact, not a payload field. Audit and debug logs never show the assembled multipart body -- an internal storage pointer appears where the body would be. But if the destination actually received the literal string bodyKey or blobKey, the file part is misconfigured (an inline part where an attachment belongs, or a blobKeyPath that doesn't resolve).
imports invoke is a real write. "No job created" doesn't mean nothing is sent: every form executes the request against the live destination, and a saved mockResponse doesn't stop them. It also runs the import's hooks, postSubmit included, and its response transform (the editor's Send stops before postSubmit). A 200 can still carry per-record failures in each result's errors, and on a file import (FTP, S3, file-mode HTTP) it means the file was composed, not delivered: an FTP import to an unreachable host still answers {name, fileLocation}. Point it at a sandbox, or test-run the flow when you only need to check mapping output.
A trace key is not duplicate detection. Repeated keys are normal (delta re-emits, retries) and never skip, merge, or reject a record. Only a keyed write stops the second copy -- see Writing Exactly Once.
A lock keyed on this import's own lookup is no lock. idLockTemplate renders before the import's lookups and mapping run, so {{lookup.<name>}} renders empty. Check the key against a real record (an error's retry data: celigo flows error <flowId> <importId> <errorId> --retry-data), not a sample that may lack the field.
Lookups can't chain. A lookup's own URI, body and headers render before any lookup has run, so {{lookup.<name>}} there is undefined: it either fails the render (handlebars_template_parse_error: Unable to read "<name>" from undefined) or renders empty, and either way that lookup fails or asks the wrong question for every record. Test runs never execute imports, so the first real run is where it shows. Resolve the value in one query, or add an upstream lookup step and reference the field it adds to the record.
An HTTP import's lookups send body, not postBody, and read the record's fields at the top level. Put a lookup's request template in http.lookups[].body. A template saved as postBody, the field http.yml documents, saves without complaint, but the lookup goes out with an empty body, so it asks the wrong question for every record and nothing errors. The CLI's schema check calls body undocumented; it's still the field that's sent. Inside that template the record's fields sit at the top level: {{email}} renders the record's value, and {{record.email}} renders empty.
In a JSON body, string values take double braces. {"name": "{{record.name}}"} is JSON-escaped at runtime, so a quote, backslash, newline or tab in the value arrives intact. Triple braces insert the raw text: such a value fails its record with 422 invalid_json before anything is sent, and in a lookup's body the lookup quietly finds nothing (gotcha 12). Keep triple braces for numbers, booleans and serialized JSON ({{{jsonSerialize record}}}).
A failing existence lookup counts as "not found". With ignoreExisting, an http.ignoreLookupName lookup that fails -- a wrong URI answering 404, a body broken by a raw quote -- extracts nothing, so the record is written as new and no error is raised. Before trusting the check, invoke the import in a sandbox with a record that already exists there and confirm the result comes back ignored.
A 200 isn't always success. When an API answers 200 for a failed write, set http.response.successPath + successValues or failPath + failValues (one entry per operation; values compare as strings). A record that trips them fails: an invoke reports statusCode 422, and a run records an error whose code is the destination's status (200), source application, with the response body as its message -- or response_failure ("response failed using path") when a successPath value doesn't match.
An ambiguous HTTP lookup fails only with allowFailures: false. When a lookup's extract resolves to two or more values (an array of ids or of result objects), the record fails with lookup_conflict ("Multiple matches found for lookup.") only if the lookup sets allowFailures: false. With it left out or true, the first value is sent and nothing errors -- from an array of objects, the literal text [object Object] -- and useDefaultOnMultipleMatches hasn't been seen to send it to the default instead. An indexed extract (results[0].id, ids[0]) takes the first match silently, whatever allowFailures says: right for a search that can match only one record, and never a fix for lookup_conflict, which it turns into writes against whichever record comes first. Fix ambiguity in the search: query on a key that identifies one record (an email, an external id), taken from a field the record carries -- a failed record's retry data shows them -- and keep allowFailures: false, so a search that still finds two fails rather than writing against the wrong record.

Common Errors

Error	Cause	Fix
422 "Invalid value '' for field 'adaptorType'"	Wrong case (`httpimport`) or an unknown adaptor	Use the exact case from the decision matrix: `HTTPImport`, `NetSuiteDistributedImport`, etc.
422 "Expected field: _connectionId to be present"	No connection on an adaptor that needs one	Set `_connectionId`; only the connection-less imports in Minimum Required Fields omit it
422 `invalid_ref` ("connection not found.; is not a valid _connectionId")	The connection id doesn't exist in this environment	Use an id from `celigo connections list`
Copying an Integration App's import fails: 403 `create_not_allowed` ("Cannot create an Integration App resource"), 422 `invalid_ref` ("The referenced Connection document ... belongs to different Integration App installation"), or 400 from `imports clone` ("Cloning can't be performed for ... that were installed from an Integration App.")	The body still carries the app's `_connectorId`, or the copy keeps the app's connection, which only the app's own resources can use; a clone of an app's import is refused outright	Build the copy as your own import, on a connection of your own, with `_connectorId`, `_id` and the timestamps left out
422 "http subschema not defined" (or another connection type's name)	The `adaptorType` doesn't match the connection: the save validates the import as the connection's adaptor	Pick the adaptor for the connection's type from the decision matrix, with its config block
422 "`<value>` is not a valid enum value for path `<field>`"	A value outside the field's enum, such as `http.method: ["PUSH"]`	Use a value the adaptor's schema lists
422 `Import query types … are deprecated`	`rdbms.queryType` holds the legacy uppercase `INSERT` (or a retired `MERGE`, `COPY` or `BULK INSERT`)	Use one current value -- `bulk_load`, `bulk_insert`, `per_page`, `per_record` or `first_page` -- and set its companion field
422 `… is not a valid enum value for path rdbms.queryType.0`	A `queryType` string outside the enum, such as lowercase `insert`	Use one of the current values above
422 `distributed required`	Missing NetSuite flag	Use `NetSuiteDistributedImport` with `distributed: true` on the import (there's no such connection flag); check the SuiteApp with `celigo connections suiteapp-status <connectionId>`
422 `mapping invalid`	Wrong mapper version	Salesforce uses Mapper 1.0 (`mapping.fields[]`) and NetSuite DA uses `netsuite_da.mapping`, not Mapper 2.0 (`mappings[]`); `NetSuiteHTTPImport` uses `mappings[]`
`Unable to find lookup in mapping configs - <name>` on every record (source `mapping`)	A mapping row's `lookupName` names a lookup the import doesn't define -- on an HTTP import, often one sent only in the top-level `lookups`, which the save drops; the save doesn't check it	Define the lookup under that exact name (in `http.lookups` on an HTTP import), or fix the row
Every record fails with `Cannot set properties of undefined (setting '_hasStarNotation')`	A NetSuite DA mapping condition (`conditional.lookupName`) names a lookup the import doesn't define; on a sublist row the job can stall instead	Fix the name or restore the lookup -- changing the SuiteScript version doesn't help
`INVALID_HANDLE_BAR_EXPRESSION` on a NetSuite DA sublist lookup	The lookup's `expression` reads a bare line field, which is undefined at the record root	Address the line as `<array>[].<field>` (`{{{line_items[].sku}}}`)
`VALUE_LOOKUP_FAILED` "Could not find a match for …"	The rendered criteria, quoted in the message, matched no NetSuite record; inactive records don't count unless the lookup sets `includeInactive`	Fix the criteria or the data, or set `allowFailures: true` with a `default` to fall back
`VALUE_LOOKUP_FAILED` "More than one match for …" (`MORE_THAN_ONE_MATCH_FOUND` on a `suiteapp2.0` step)	The criteria matched several records; an empty source value searches `["<field>","isempty"]`, which matches every blank one	Keep an empty key out of the search (branch in the `expression`), or set `useDefaultOnMultipleMatches` with `allowFailures: true` and a `default`
NetSuite `INVALID_KEY_OR_REF` ("Invalid reference key .") on some records	The field references another NetSuite record, and the value NetSuite received isn't a key it accepts. When the quoted value is a name or code and the row writing that field sets `internalId: true` (its value must be the referenced record's internal id), those records carry the name where the id belongs (Diagnosing NetSuite Steps has the other causes)	Resolve the name to the id on that row with a lookup: a static `map` from each name to its internal id (the ids are the user's to supply), or a NetSuite lookup that searches the name field and returns `resultField: "internalid"` (netsuitedistributed.yml). Then retry the failed records; a retry before the change fails again
NetSuite `USER_ERROR` (`source: application`) whose message states a business rule ("PO # is required for customers in the Distributor category")	A rule in the NetSuite account -- often its own user event script or workflow -- refused the record. Celigo's hooks report their own `source` (`pre_map_hook`, `post_submit_hook_ss` and the like), so `application` means NetSuite raised it	Read a failing record's retry data for the value the rule asks for and check whether a mapping row writes it: map it, or have a NetSuite administrator change the rule. Don't edit a Celigo hook for it (Diagnosing NetSuite Steps)
NetSuite `INVALID_FLD_VALUE` "You have entered an Invalid Field Value for the following field: " on some records, on a list or select field	The row sends text, and a NetSuite DA import sets a select field's text by the option's display name: the list has no option named exactly that value -- often a new value the source started sending. On a date field the same code means a value the import can't read as a date (Diagnosing NetSuite Steps)	Compare the quoted value with what the records that succeed send. Add the option to the NetSuite list with that exact name, or translate the values with a static lookup on the row (to the options' internal ids, with `internalId: true`). Then retry the failed records
NetSuite `USER_ERROR` "Please enter value(s) for: " on some records only	The field is mandatory for those records and the mapping leaves it empty for them: their source value is blank, or another field makes NetSuite require it -- a customer whose `isperson` is `F` (a company) needs Company Name, while an individual (`isperson` `T`) needs first and last name instead	Read a failing record's retry data next to one that succeeded. Map the field from a value those records have, give the row a default, or set the deciding field from the data (`isperson` `T` when the company is empty). Then retry the failed records
NetSuite `SSS_INVALID_SUBLIST_OPERATION` ("…trying to add or remove lines from a static sublist", often ending "Tip: Please make sure that all the line items that you're trying to use exist on the record.") on an item fulfillment	A mapped line matched none of the fulfillment's lines, so the import tried to add one, and a fulfillment can't add item lines: its lines are its order's. Lines match on the sublist's `isKey` columns, or on the item when none is marked (writing-mappings > Mapper 1.0)	Find the line whose key value none of the order's lines holds -- in its retry data, or from a change to the row or lookup that resolves the item -- and fix the mapping or the data. A retry before that fails the same way
NetSuite `MULTIPLE_LINES_WITH_SAME_KEYS` "Multiple lines found with keys item= …"	Two of the record's lines hold the mapped line's key values: on a fulfillment, receipt or invoice keyed on the item (no column is `isKey`), an order with that item on two lines	Key each line on what only it holds: map the order line's id (its `line` on the order) to `orderline` and mark that row `isKey` -- a mapped `orderline` without `isKey` isn't a key, so the item still is. Then retry the failed records; a retry before the change fails the same way
A NetSuite update repeats the record's lines on every run, with no error	The sublist has no `isKey` column, so no mapped line matches an existing line and each is added; only sublists with a default key fall back to it -- the item on fulfillment-style transactions, the applied transaction on a payment's or credit's	Mark the row that identifies a line `isKey` -- a line id the record's create wrote to a line field -- so each mapped line updates its own. To also drop lines the source no longer has, map `celigo_replaceAllLines_<sublist id>` to `true`. Records already updated keep their repeated lines, and since the repeats carry the same id, their next update with the key set fails with `MULTIPLE_LINES_WITH_SAME_KEYS`: clean those up first
Records the destination already has show as ignored, and their changes never arrive	`ignoreExisting` skips each record its existence check finds; it never updates one	To update them, make the import a composite UPDATE/CREATE that decides on the same lookup (`http.existingLookupName`; Composite imports) and drop `ignoreExisting`
`The mapping condition:"record_updated" not supported for the operation`	A `record_updated` condition on a NetSuite DA `add` import	Use `addupdate` or `update`, or drop the condition
A record fails with code `200`, source `application`, and the response body as its message (or `response_failure`)	The import's `failPath` / `successPath` rules judged a 200 a failure	Expected when the API reports failures in a 200 body (gotcha 13); fix the record, or the rule if it misfires
422 attachment `value` invalid	Multipart file part is not `{{blob}}`	Set the file part to `"type": "attachment"`, `"value": "{{blob}}"`; let the platform generate the `boundary`
422 `invalid_json` ("An invalid JSON is sent in the request.body") on some records only	A triple-braced string in a JSON `http.body` met a value holding a quote, backslash, newline or tab; in a GraphQL `variables` string, any string value without `jsonEncode`	Double-brace string values inside quotes (gotcha 11); inside a GraphQL `variables` string, `{{jsonEncode record.x}}` (GraphQL imports)
A GraphQL import reports every record written, and the destination has none	The server answered 200 with an `errors` array or the mutation's `userErrors`, and the import sets no response rules	Set `resourceIdPath` to the created object's id, and `failPath`/`errorPath` to `errors[0].message` (GraphQL imports)
`missing_id` ("could not extract id using path: …")	No id at `resourceIdPath`: the write failed (the response, quoted in the message, carries the server's errors) or the path is wrong	Read the quoted response; fix the data, or the path
`handlebars_template_parse_error`: "Failed to generate request url from template: … Details: "" not defined in the model." (source `application`, classification `parse`) on some records	Those records lack the field the `relativeURI` names, or its `{{lookup.<name>}}` found nothing -- missing data, not template syntax. Nothing was sent, and a retry fails the same way	When the records may not exist at the destination yet (new items, no destination id), make the import a composite UPDATE/CREATE that decides on the field (`http.existingExtract`; Composite imports), or skip them with `ignoreMissing: true` and `http.ignoreExtract` set to the field. Otherwise get the id onto the record upstream, then retry
422 `unsupported_type` ("NetSuiteHTTPImport does not support type: file")	`http.type: "file"` on a `NetSuiteHTTPImport`: the NetSuite REST import writes records, and the save refuses file mode	Remove `http.type`
Every record fails with "Directory is not found"	`http.type: "file"` (file-storage folder mode) combined with an API upload endpoint	Remove `http.type: "file"`; keep `blob: true` + `blobKeyPath` and post to the endpoint
422 `invalid_adaptor` ("Import preview is not supported for adaptor type")	`celigo imports invoke` with a document on stdin, for a NetSuite, Salesforce, RDBMS, or FTP import	Save the import and invoke it by ID, or run the document as a virtual import with `--connection` (both real writes); or test-run the flow to check mapping output without sending
422 `Connection is offline` from a mapping preview or a NetSuite field check	The destination connection is marked offline (its `offline` flag), so the platform doesn't call it; nothing is wrong with the import or its mapping	Fix the connection (test it, re-authorize), then retry
A Salesforce import fails with `response stream exceeded limit of 5242880 bytes.`	The platform reads at most 5 MB of Salesforce's answer to a request, and this batch's results were larger	Send fewer records per request
Salesforce `UNABLE_TO_LOCK_ROW` on some records -- on the `rest` and `compositerecord` APIs the platform replaces Salesforce's message with "Salesforce creates a lock on a record while updating it. If another request is trying to update the same record which is already locked, Salesforce would throw this error." and a help link	Another write held the Salesforce record this one needed: this import's own pages or runs submitting in parallel, or another import or flow writing the same records	Set `idLockTemplate` to the key of the record the writes contend for -- the record's own id, or the parent id they share when they update children of one record, taken from a field the incoming record carries (`{{{record.AccountId}}}`; gotcha 8) -- so pages carrying the same key never submit together through this import. The lock doesn't reach other imports: when another flow writes the same records, lower the connection's `concurrencyLevel` instead. Then retry the failed records
A Salesforce import reports a field error for a record that Salesforce saved	On the `soap` API (the default), when Salesforce refuses a value in a field the import mapped (the error names the field), the platform drops that field and sends the record again, up to `maxAttempts` more times (default and maximum 3; `maxAttempts: 1` turns it off). A resend that succeeds saves the record without the field, and its error still counts against the job	Look the record up in Salesforce before retrying: a retry of an `insert` creates it a second time. Fix the value at the source, update the saved record, and resolve the error
A `compositerecord` import fails a record with `Composite Operation failed due to the following error(s)`, then one line per failure (`<message> for <object>[<index>]`)	One object in the record's tree failed. The platform sends a record and its related records as one all-or-none Composite API request, so Salesforce saved none of them; the rollback notices it returns for the other objects (`PROCESSING_HALTED`, `ALL_OR_NONE_OPERATION_ROLLED_BACK`) are left out, and the error's code is the first real failure's	Fix the field each line names on that object, then retry the whole record
Salesforce `ENTITY_IS_DELETED` on an `update` or `delete`	The record was deleted in Salesforce	With `ignoreMissing: true` the import counts such records as ignored instead of failing them; without it, resolve the errors once the deletions are confirmed as intended
The destination refuses the text `Invalid date` in a date field (its error quotes that value)	A date mapping (`extractDateFormat`, `extractDateTimezone` or `generateDateFormat` set, Mapper 1.0 or 2.0) couldn't read the source value with its `extractDateFormat` -- `YYYY-MM-DD HH:mm:ss` when unset -- so it wrote the text `Invalid date`	Set `extractDateFormat` to the format the source actually sends (a failed record's retry data shows the value), then retry the failed records
Downstream steps see `1234567890` or `"replace me"` in a test run	No `mockResponse` saved on the upstream import -- test runs never call the destination	Save a realistic `mockResponse` on that import
The destination receives the same record twice	A bare create; `ignoreExisting` with no lock on records whose trace key is blank, or with an existence lookup that fails (gotcha 12); or a lock keyed on the import's own lookup	Key the write and lock it on the business key (Writing Exactly Once); find which runs wrote it first (building-flows > Duplicate Records)
A NetSuite `addupdate` adds the same record again on every run	`internalIdLookup` searches `externalid`, but no mapping writes `externalid`, so the lookup never finds what earlier runs added	Map the source id to `externalid` and keep the lookup; the duplicates already added stay until someone merges them
`lookup_error`: "Could not find value for static lookup: {Lookup Key: } …" on some records	The value isn't a key of that lookup's `map`, and the lookup has `allowFailures: false`; the other records succeed	Add each missing value as a key, exactly as the message quotes it (case, spacing and suffixes count), with the destination's value (ask the user -- a guessed code sends the record wrong), keeping every existing key in the save, then retry the failed records. A retry runs the step as it is now, so one before the map changes fails again. A `default` would instead send every unmapped value as the default, with no error
`lookup_conflict`: "Multiple matches found for lookup." (source `lookup`) on some records	An HTTP lookup with `allowFailures: false` got two or more values at its `extract`: the search key isn't unique (a name several records share)	Search on a key that identifies one record and keep `allowFailures: false`. Don't index the `extract` (`ids[0]`) or allow failures to get past it: either one writes against whichever match comes first (gotcha 14)
`batch_lookup_evaluation_failed`: ": The batch lookup could not be evaluated. Check the value in the key '', and try again." (source `internal`)	An HTTP lookup with `allowFailures: false` got nothing usable for that value: the search matched nothing, or the `extract` resolves nothing (a `$.` prefix, `[*]`, a dot path through an array) or points at a result object rather than an id	Check the `extract` against a real response -- a path without `$.` to the id (`results[0].id`) or an array of ids; if the value truly has no match, fix the data or the search, then retry
422 `cannot_set_traceKeyTemplate`	`traceKeyTemplate` set on a one-to-one import	Set it on the source export or listener -- on an import it is valid only with `oneToMany: true`. To change the lock key, set `idLockTemplate`
`oneToMany_error`	A blank `pathToMany` on an object record ("Record at index 0 was not an array, and no PathToMany was provided"). The editor's Preview can also show it falsely on a step fed an array, because the preview re-splits the record it displays	Object record: set `pathToMany` to the child array. Array (grouped) record: blank is right, and each row is the record, with no `_PARENT` -- confirm with a test run before changing the configuration

Get it through your agent

Call the Graph-of-Skills MCP server's retrieve_skill_bundle tool and it returns this skill's text inside your budget.

{
  "tool": "retrieve_skill_bundle",
  "arguments": {
    "query": "configuring-imports"
  }
}

New to this? Set up Graph-of-Skills or get a free key. The text on this page comes from its authors: how skills are sourced and what we check. If none fits your task, your agent can write a skill for your own library. To keep a skill from your own library on a machine, see how to install one of your skills.

Share on X Share on LinkedIn Share on Bluesky Share on Hacker News

Share on Reddit

Source

Listing on skills.sh

From celigo/ai via skills.sh

Related skills

configuring-exports
Configure Celigo export resources -- the data source step that fetches records from external systems. Use when creating or editing exports, choosing the right adaptor type for a target application, setting up delta (incremental) exports, webhooks, file transfers, or lookup steps. To replicate tables into a data warehouse, use building-syncs.
2,923 installs
configuring-filters
Configure Celigo filter rules on exports, imports, and flow branches that control which records continue through the pipeline. Use when adding filters, setting branch routing conditions, or choosing between expression and script filters.
2,880 installs
netlify-database
Zero-config Postgres for Netlify apps via @netlify/database — querying data from Functions/Edge Functions, writing schema migrations, setting up Drizzle ORM, local dev with netlify dev, database branches for deploy previews, and migrating an existing Postgres project onto Netlify. Use when adding a database, building a contact form or CRUD API, writing SQL migrations, wiring up Drizzle, running netlify database commands, testing with a local Postgres, or switching from Neon/Supabase/RDS to Netlify Database.
2,894 installs
drizzle
Use for Drizzle schemas and queries: tables, indexes, relations, joins and inferred types. Rollout belongs to db-migrations.
5,419 installs
terraform-search-import
Discover existing cloud resources using Terraform Search queries and bulk import them into Terraform management. Use when bringing unmanaged infrastructure under Terraform control, auditing cloud resources, or migrating to IaC.
4,851 installs

More from celigo/ai

writing-scripts
Write Celigo JavaScript hook scripts -- preSavePage, preMap, postMap, postSubmit, postResponseMap, filter, transform, branching, handleRequest. Use when creating or editing scripts, choosing the right hook point, understanding input/output data shapes, or debugging script behavior.
2,950 installs
writing-handlebars
Write Handlebars template expressions for Celigo integrations -- dynamic values in mappings, HTTP bodies, SQL queries, URIs, and filters. Use when building any resource configuration that needs computed, conditional, or formatted field values.
2,926 installs
configuring-exports
Configure Celigo export resources -- the data source step that fetches records from external systems. Use when creating or editing exports, choosing the right adaptor type for a target application, setting up delta (incremental) exports, webhooks, file transfers, or lookup steps. To replicate tables into a data warehouse, use building-syncs.
2,923 installs
configuring-ai-agents
Configure Celigo AI agent imports -- LLM-powered steps that classify, extract, or generate data within flows. Use when creating agent imports on any supported model provider, or configuring prompts, structured output, tools, or BYOK connections. For guardrails (PII, moderation, policy verdicts), use configuring-guardrails.
2,921 installs
writing-sql
Write SQL queries for Celigo RDBMS exports and imports -- SELECT, INSERT, UPDATE, UPSERT, MERGE, delta, once, and bulk operations across Snowflake, Postgres, MySQL, SQL Server, Oracle, BigQuery, and Redshift. Use when editing rdbms.query or troubleshooting SQL errors.
2,919 installs

Content is provided as published by its creators; you are responsible for how you use it.

To request removal, use the contact form.