Skip to content

Conversation

@tak-amboss
Copy link
Contributor

@tak-amboss tak-amboss commented Oct 1, 2025

What?

This PR adds atomic update operations ($push and $remove) to Local API update functions for relationship fields configured with hasMany: true. The operations work for both single-document updates (updateById) and multi-document updates. It is related to #13891, which introduced atomic operations at the database adapter layer.

New API:

// Single document: apply atomic operations by ID
await payload.update({
  collection: 'posts',
  id: '507f1f77bcf86cd799439011',
  data: {},
  operations: {
    $push: { categories: ['featured'] },
  },
})

// Multiple documents: apply atomic operations to all matches
await payload.update({
  collection: 'posts',
  where: { status: { equals: 'published' } },
  data: {},
  operations: {
    $remove: { categories: ['outdated'] },
  },
})

Why?

Right now, updating relationship arrays generally means replacing the whole array. That forces clients to first fetch the existing document, merge changes locally, and then send everything back. Atomic operations let clients express intent directly (e.g., “add this category” or “remove that relation”) without shuttling full arrays back and forth.

How?

LocalAPI update got a new operations property where atomic operations for hasMany: true relationship fields can be submitted. Atomic operations are resolved in updateById / update internally before collection- and field-level hooks are executed so existing hooks and validations are not affected by this change.

Implementation Notes:

  • Local API only. REST and GraphQL support will be added separately.
  • $push and $remove for relationship fields only. Similar operations such as $inc might be added later.
  • Handles localized and nested fields
  • Validates field conflicts between data and operations

Considerations

This change proposes introducing a dedicated operations property to carry atomic updates such as $remove, $push, and $inc. Keeping these mutation operators separate from data preserves the existing semantics of data as pure payload and avoids breaking changes - especially hook implementations - that currently assume data contains only field data. From a typing perspective, operations can be described with a focused, well-scoped type that models update operators without altering the shape of data.

An obvious alternative would be to allow these operators directly within data. While superficially simpler, that approach risks breaking existing hooks that are not prepared to receive operator syntax mixed with collection data. It would also force a significant expansion in type complexity: we’d need recursive types that permit operators at every nesting level, which would likely cascade into type errors for downstream implementations and reduce overall readability. For these reasons, the proposal favors a new operations field, keeping backward compatibility intact and making both validation and type definitions more tractable.

@tak-amboss tak-amboss changed the title feat(payload): add atomic operations support to bulk update (Local API) Oct 1, 2025
@tak-amboss tak-amboss changed the title feat: add atomic operations support to bulk update (Local API) Oct 1, 2025
@tak-amboss tak-amboss changed the title feat: add $push and $remove atomic operations to localAPI update Oct 1, 2025
@tak-amboss tak-amboss force-pushed the feat/local-api-bulk-atomic-operations-clean branch 3 times, most recently from 4b53f94 to 77d79c4 Compare October 1, 2025 07:53
@tak-amboss tak-amboss force-pushed the feat/local-api-bulk-atomic-operations-clean branch from d2c960c to 920fd79 Compare October 10, 2025 07:24
…tionships

Introduces atomic operations for relationship fields with hasMany: true,
enabling efficient array modifications without replacing entire arrays.

Key Features:
- $push: Add items to relationship arrays (prevents duplicates)
- $remove: Remove specific items from relationship arrays
- Works with updateByID and bulk update operations
- Supports polymorphic relationships, localized fields, and nested fields
- Validates conflicts between data and operations parameters

Implementation:
- New atomic utilities (apply.ts, validate.ts) for operation processing
- Updated updateByID to support operations parameter
- Extended bulk update to apply operations per-document
- Each document fetched fully for accurate operations
- Only selected fields returned in response

Testing:
- Comprehensive test suite covering all operation types
- Tests for select interaction and bulk updates
- Error handling and validation tests

Documentation:
- Complete guide for high-level and low-level API usage
- Examples for simple, polymorphic, and nested relationships
- Bulk update documentation with query examples

Related to payloadcms#13891 (database adapter-level atomic operations)

Local API only. REST and GraphQL support will be added separately.
@tak-amboss tak-amboss force-pushed the feat/local-api-bulk-atomic-operations-clean branch from 920fd79 to 031ead2 Compare November 12, 2025 11:16
)
}

const resolvedData = deepCopyObjectSimple(data)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we avoid deep copying when it's not needed? Same in other files.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment on lines +53 to +90
const applyOperationRecursively = (
currentDoc: Record<string, unknown>,
data: Record<string, unknown>,
operationData: Record<string, unknown>,
operationFn: (currentValue: unknown[], items: unknown[]) => unknown[],
path = '',
): void => {
for (const [fieldName, value] of Object.entries(operationData)) {
const currentPath = path ? `${path}.${fieldName}` : fieldName

if (Array.isArray(value)) {
// This is a leaf field - apply the operation
const currentValue = getObjectDotNotation(currentDoc, currentPath, [])

if (!Array.isArray(currentValue)) {
throw new Error(`Cannot execute atomic operation on non-array field "${currentPath}"`)
}

// Apply the operation using the callback function
const newValue = operationFn(currentValue, value)

// Apply the change directly to the data object
ensureNestedPath(data, fieldName)
data[fieldName] = newValue
} else if (value && typeof value === 'object') {
// This is a container field - recurse deeper
ensureNestedPath(data, fieldName)
applyOperationRecursively(
currentDoc,
data[fieldName] as Record<string, unknown>,
value as Record<string, unknown>,
operationFn,
currentPath,
)
}
}
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I might be wrong, but it doesn't seem like this function actually uses DB atomic operations which are processed here

const $inc: Record<string, number> = {}
const $push: Record<string, { $each: any[] } | any> = {}
const $addToSet: Record<string, { $each: any[] } | any> = {}
const $pull: Record<string, { $in: any[] } | any> = {}
transform({
$addToSet,
$inc,
$pull,
$push,
adapter: this,
data,
fields,
operation: 'write',
})
and it just merges the new item into the existing data, right? If yes, the performance of using payload.update({ data: { hasManyRelationship: [...prev, new] }}) and payload.update({ data: {}, $push: { hasManyRelationship: new }) should be the same here. To actually benefit from this functionallity I think we need to rely on DB atomic operations

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right that it doesn't use the atomic operations on a database level. I don't think it is feasible to use the database operations here directly because this way we would skip all hooks and other field-level operations and introduce breaking changes once someone wants to start using atomic operations. That's why I avoided that.

Nevertheless it has significant benefits:

  1. On LocalAPI level it allows you to use atomic operations without considering any side effects (such as skipped hooks, dependencies to other fields, etc.).
  2. To achieve the same via the local API in the current setup, you would need to fetch the data, change the data, and save it. With the new atomic operations we can shortcut this.
  3. As a next step, we can bring this to REST and GraphQL APIs. This will eventually allow us to utilize them in the admin UI, e.g. on the bulk edit UI to allow adding / removing items from hasMany relationship fields which is currently impossible to achieve.

I agree on the performance aspect, but that's not the primary motivation here. If there is later on a way to also improve the performance, that'd certainly great, but the current way enables already a whole bunch of things which are IMO totally worth it.

Does that make sense to you?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

2 participants