Before creating a new cache, search for existing ones in the account:

# List all lookup caches
celigo lookup-caches list

# Search across the account
celigo account search "lookup cache"
celigo account search "<keyword related to your use case>"

3. Create the cache

Create the cache metadata first. Data is loaded separately.

# Create a new empty cache
echo '{"name":"Product SKU to NetSuite ID","description":"Maps Shopify SKUs to NetSuite internal IDs for order import"}' | celigo lookup-caches create

# With template inclusion, then registered on the integration whose flows use it
echo '{"name":"State Shipping Overrides","includeDataInTemplatesAndCloning":true}' | celigo lookup-caches create
celigo integrations register-lookup-caches <integrationId> <cacheId>

Through MCP, upsert_lookup_cache creates the cache; register it through the CLI or the REST route.

4. Load data into the cache

Use the put-data command to upsert key-value entries. The CLI auto-batches by count (1000 entries) and size (5 MB) per request.

# Upsert entries
echo '{"data":[{"key":"SKU-001","value":{"nsId":"12345","name":"Widget A"}},{"key":"SKU-002","value":{"nsId":"12346","name":"Widget B"}}]}' | celigo lookup-caches put-data <cacheId>

# Simple string values for deduplication
echo '{"data":[{"key":"order-10001","value":"processed"},{"key":"order-10002","value":"processed"}]}' | celigo lookup-caches put-data <cacheId>

For large datasets, pipe from a script or file that generates the { "data": [...] } JSON. The CLI handles batching automatically, and so does MCP upsert_lookup_cache_data.

5. Verify the data

# Get first page of all data (max 1000 keys)
celigo lookup-caches get-data <cacheId>

# Get specific keys
echo '{"keys":["SKU-001","SKU-002"]}' | celigo lookup-caches get-data <cacheId>

# Get keys by prefix
echo '{"startsWith":"SKU-"}' | celigo lookup-caches get-data <cacheId>

# Check cache size (bytes, not entries)
celigo lookup-caches get <cacheId>

6. Reference the cache in a lookup

On the import or export resource, add a lookup entry referencing the cache ID (on an HTTP import, in http.lookups), name it from the mapping row that needs the value, and set allowFailures for what a missing key should do (Referencing Caches in Lookups). See writing-mappings > Add lookups for value translation for the full configuration pattern.

7. Plan the data refresh strategy

Lookup caches are not automatically refreshed. Choose a strategy:

• Manual -- update data via CLI or API as needed. Good for static reference tables.
• Flow-driven -- a scheduled flow that reads the system of record and writes the cache through an import on Celigo's own API (POST /v1/lookupcaches/<id>/data; references/flow-steps.md has the step bodies). Good for cross-reference caches that need periodic sync. Write from a hook (lookupcaches.putData) only when the write needs logic a step can't express: the cache must be registered to the flow's integration, and the hook's writes are real in test runs too.
• Purge and reload -- purge all data and reload from scratch on a schedule. Good when the full dataset is small enough to reload quickly.

Purge vs Delete -- Data vs Resource

Two different operations both sound like "clearing" or "removing" a cache. Choose by blast radius:

• Purge empties the data (celigo lookup-caches purge-data <id>; MCP delete_lookup_cache_data with all: true) while the cache resource and every lookup that references its _lookupCacheId stay valid. After a purge, lookups just miss until data is reloaded, so their default / allowFailures behavior takes over. "Clear the cache" and "start over with fresh data" almost always mean purge.
• Delete removes the resource itself (celigo lookup-caches delete <id>; MCP delete_resource). Reserve it for "we don't use this cache anymore." The platform refuses it while an export's or import's lookup uses the cache (422 export <id> must be deleted before you can delete lookupcache <id>, naming each step and its flow) or while an alias points at it (422 ... is referenced by an alias on integration <id>. Alias must be deleted before resource can be deleted.). Remove the lookup from those steps -- editing the step is enough -- and the entry from the integration's aliases[]; removing the alias breaks every {{getLookupCacheData}} template that reads the cache through it. A delete also takes the cache off every integration it was registered to.

Before a delete, list what uses the cache: celigo lookup-caches dependencies <id> names the exports and imports whose lookups read it, their flows, and the integrations that register it or hold its alias. A template that reads the cache through {{getLookupCacheData}} names the alias, not the id, so search templates for the alias too. The CLI answers 404 for an id that doesn't exist; the REST route (GET /v1/lookupcaches/{id}/dependencies) answers {} for one, the same as for a cache nothing uses, so confirm the id first.

Purge can't be undone, on either surface. The entries come back only if you reload them from their source, so confirm with the user before purging, and never purge a cache that is the only copy of its data -- a deduplication ledger or a state cache -- unless every record should look new again. Deleting a cache is a soft delete: for 30 days you can restore it from the recycle bin under the same _id (celigo recycle-bin restore <cacheId>). It comes back unregistered, so register it again. Whether a restored cache's entries come back isn't documented, so read them after restoring. celigo recycle-bin purge <cacheId> removes a deleted cache for good; one that has held data can refuse that with 422 this document is referenced by LookupCacheUsage : <id>, and then leaves the bin when its 30 days end.

Loads are upserts keyed by key: re-loading a refreshed dataset overwrites matching keys and adds new ones, but it does not remove keys that are absent from the new load. A true "replace the whole table" is therefore purge + reload, not a plain reload.

Deduplication Ledger

A ledger remembers which keys were processed so a repeat -- a re-fired webhook, a re-run, another flow -- is skipped. Use one only when the destination can't answer "does this exist?"; otherwise match on the destination import (configuring-imports > Writing Exactly Once). The wiring decides whether it actually de-duplicates:

1. Claim with an import step, under a lock. A cache write is an upsert -- no insert-if-absent, no compare-and-set -- so the claim has to be one import step: an HTTP import to the cache's data endpoint (POST /v1/lookupcaches/<id>/data, through a connection to Celigo's own API -- configuring-connections > Minimum Required Fields says how to build one) with ignoreExisting: true, an ignore lookup that calls getData for the record's key, and idLockTemplate on the same key. The lock isn't optional: without it, two pages or runs that both read a miss both write. A hook that calls getData and then writes has the same race and no lock to close it. Point the ignore lookup's extract at the entry's value (data[0].value): a keyed getData returns an entry for a miss too, with the key echoed and an error instead of value, so an extract of data, data[0] or data[0].key is never empty and the import ignores every record. Give every ledger entry a non-empty value, because an empty one reads as not found. Write the lookup's request in body with the record's fields named at the top level ({{orderId}}, not {{record.orderId}}); a postBody isn't sent (configuring-imports > Gotchas).
2. Let ignored records through, then filter. A record the ledger ignores still continues to the next step. Response-map the ledger step's ignored flag onto the record ({"extract": "ignored", "generate": "alreadyDone"}) and give the destination import a filter that drops records where alreadyDone is true (configuring-filters).
3. Mark processed; never delete after processing. A cache emptied as records are handled is a queue, not a ledger: a record that wrote one step and failed the next never reaches the delete, so the next run writes it again, and a later repeat of the same event looks new. Keep the entry with a status and timestamp in its value, have any flow that reads the cache as its source skip processed entries, and delete old entries by key on a schedule (delete-data; caches have no TTL).
4. The claim lands before the destination write. If the destination step then fails, retry that step's error (celigo flows retry-errors <flowId> <importId> <retryDataKey>) -- a fresh run would find the claim and skip the record for good.

CLI Commands

# CRUD
celigo lookup-caches list
celigo lookup-caches get <id>
celigo lookup-caches create < cache.json
celigo lookup-caches update <id> < cache.json
celigo lookup-caches set <id> key=value [key2=value2 ...]
celigo lookup-caches dependencies <id>                                  # what uses it, before a delete
celigo lookup-caches delete <id> [-y]
celigo recycle-bin restore <id>                                         # within 30 days; comes back unregistered

# Register on an integration (the only tie a cache has to one)
celigo integrations register-lookup-caches <integrationId> <id>
celigo integrations deregister-lookup-caches <integrationId> <id> [-y]

# Data operations
echo '{"data":[{"key":"k","value":"v"}]}' | celigo lookup-caches put-data <id>
celigo lookup-caches get-data <id>                                      # one page (~1000), all keys
echo '{"keys":["k1","k2"]}' | celigo lookup-caches get-data <id>       # specific keys
echo '{"startsWith":"prefix"}' | celigo lookup-caches get-data <id>     # prefix search
echo '{"keys":["k1","k2"]}' | celigo lookup-caches delete-data <id> [-y]
celigo lookup-caches purge-data <id> [-y]                               # every entry, irreversible

Gotchas

1. PUT erases omitted fields. Always GET first, modify, then PUT. The set command handles this.
2. Data upsert uses POST, not PUT. POST /v1/lookupcaches/{id}/data upserts entries. This is different from the resource-level PUT that updates cache metadata.
3. Values keep their JSON type -- except top-level arrays. Null, number, boolean, string, and object values round-trip as written. A top-level array, or an object with its own top-level key property, fails the whole batch with 400 invalid_value and nothing is written; wrap arrays ({"items": [...]}). Store objects as objects, not JSON strings, so the lookup's extract path can read their fields.
4. 50 MB per cache, 1 GB per environment. Caches near the 50 MB limit, such as a large NetSuite item cache, risk hitting the ceiling on the next upsert. Monitor size and sizeInMB on the response -- they measure bytes, not entries.
5. get-data returns one page of at most 1000 entries. An unfiltered or startsWith read returns up to 1000 entries in no particular order and, while more remain, a nextPageURL (/v1/lookupcaches/<id>/getData?startAfterKey=...&shardId=...): POST the same body to it for the next page, until a response comes back without one. celigo lookup-caches get-data and MCP list_lookup_cache_data send one request and don't follow it (celigo pull does, to save a cache's whole data set), so on those surfaces read a larger cache in startsWith slices or by keys.
6. No automatic TTL or expiry. Lookup cache data persists until explicitly deleted or purged. Stale data is a common source of bugs -- plan a refresh strategy.
7. includeDataInTemplatesAndCloning defaults to false. Set it to true only when the entries are configuration -- static reference tables, shipping overrides -- or that data is lost when cloning or installing from a template. Keep it false for runtime state a flow builds in its own environment (cross-references, ledger entries, state markers): a clone or template that carries a ledger makes records in the target environment look already processed.
8. A cache belongs to one environment. Each environment has its own caches and its own 1 GB aggregate. sandbox is a legacy read-only flag the API no longer returns, so don't set it or filter on it.
9. Deleting a cache is a soft delete. The cache stays in the recycle bin for 30 days, where it can be restored under the same _id -- unregistered, so register it again. After that, or a purge from the bin, it's gone for good (Purge vs Delete).
10. A cache write is an upsert -- there is no insert-if-absent. Writes never fail because a key exists, so a "first writer wins" claim built from a read then a write (two steps, or a hook) lets every concurrent writer win. The claim holds only as one import step with ignoreExisting + an existence lookup + idLockTemplate on the key (Deduplication Ledger).

Safe Lookup-to-Update -- Never Write Off an Unguarded First Match

Cross-reference resolution -- the flagship cache use case -- resolves a key (an internal ID, an account ID) that then feeds a downstream update or write. A cache read returns exactly one value per key, so resolving on a genuinely unique key (an external ID, a primary key) is safe by construction. The danger lives in any lookup that can return more than one match -- typically a live/dynamic lookup on a fuzzy or human key (name, email, phone number), where duplicates are normal.

Feeding the first result (data[0]) of a multi-match-capable lookup into an update writes an arbitrary wrong record on every duplicate -- a silent data-corruption bug, not a skipped record, and one wrong-record update is worse than a skipped record. Never feed, skip on, or branch on an unguarded first match. Guard it one of two ways:

• Configure the resolving lookup to fail or skip the record on multiple matches, so a duplicate never reaches the write, OR
• Check that the result count equals one before the write -- an input filter on the writing step, or a postResponseMap hook that drops or errors records whose lookup returned more than one result.

Make the guard explicit when you build a lookup-then-update: a bare data[0] mapping paired with an assumption that the lookup "should" return a single match is exactly the shape that corrupts data in production.

Common Errors