feat: depth generic, fully type safe result types for relationships and joins

[r1tsuu]

This feature allows you to have fully type safe results for relationship / joins fields depending on depth. Currently this is opt-in with typescript.typeSafeDepth: true property, as it may break existing types. Meaning without enabling it - there's no any effect on your existing project. Discussion - #8229

For example, with the given config:

export default buildConfig({
  collections: [
    {
      slug: 'books',
      fields: [
         {
          type: 'relationship',
          name: 'relatedMovie',
          relationTo: 'movies',
        },
        {
          type: 'join',
          on: 'books',
          name: 'author',
          collection: 'authors',
        },
      ],
    },
    {
      slug: 'authors',
      fields: [
        {
          type: 'relationship',
          hasMany: true,
          relationTo: 'books',
          name: 'books',
        },
      ],
    },
    {
      slug: 'movies',
      fields: [
        {
          type: 'relationship',
          relationTo: 'authors',
          name: 'author',
        },
      ],
    },
  ],
  // default in Payload
  defaultDepth: 2,
  typescript: {
    typeSafeDepth: true,
  },
})

Let's try to fetch some movies using the Local API:

Without specifying depth, the result type for each movie is ApplyDepth<Movie, 2>: (From defaultDepth)

movie.author is ApplyDepth<Author, 1>, without this feature it'd have been annoying string | Author :

movie.author.books is ApplyDepth<Book, 0>[] because of the hasMany relationship:

But then, if we try to access movie.author.books[0].relatedMovie we get string which is exactly what we expect from depth: 2:

However, with depth: 3 we get relatedMovie: ApplyDepth<Movie, 0>:

Now, let's try to specify depth: 0:
The result type is ApplyDepth<Movie, 0> and movie.author is string:

This also works for join fields as well.

Challenges:

• This works across for any nesting of your relationship fields (including polymorphic ones) to arrays / groups / blocks. To determine whether the current type is a relationship or not, we need some indicator and there's no currently one. This PR adds an internal __collection property (if typescript.typeSafeDepth for collection generated types which the generic uses.

  payload/test/relationships/payload-types.ts

  Line 127 in a245bf7

  __collection?: 'posts';

• There's no really an easy way to subtract number types in Typescript like this:

type X = 2 - 1

There are hacks https://softwaremill.com/implementing-advanced-type-level-arithmetic-in-typescript-part-1/ that potentially may screw Typescript Server performance, so in this PR I went with even better approach to generate types for depth specifically:

payload/test/relationships/payload-types.ts

Lines 73 to 80 in a245bf7

	depth: {
	allowed: 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10;
	/**
	* @minItems 11
	* @maxItems 11
	*/
	decremented: [null, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9];
	default: 2;

depth.default we use when depth isn't passed, allowed - an enum of allowed depth values. For example we can't pass 11:

And depth.decremented is used for internal typescript purposes which allows us to easily decrement depth for typescript:

payload/packages/payload/src/index.ts

Lines 227 to 233 in a245bf7

	export type AllowedDepth = TypedDepthConfig['allowed']
	export type DefaultDepth = TypedDepthConfig['default']
	export type DecrementedDepth = TypedDepthConfig['decremented']
	export type DecrementDepth<Depth extends AllowedDepth> = DecrementedDepth[Depth]

For example DecrementDepth<2> - result 1.

[denolfe]

I'd be curious if we would benefit from introducing a type assertions tool like tsd or tstyche.

• https://github.com/tsdjs/tsd
• https://github.com/tstyche/tstyche

[r1tsuu]

I'd be curious if we would benefit from introducing a type assertions tool like tsd or tstyche.

• https://github.com/tsdjs/tsd
• https://github.com/tstyche/tstyche

We have now type assertions here - https://github.com/payloadcms/payload/pull/9782/files#diff-8dec54cdc3beba0979d4e6eceb4130340bd2ee3bc91f1e779145baaeae8ea440 works perfectly!
Here are also assertions that it doesn't break current types https://github.com/payloadcms/payload/pull/9782/files#diff-5e3d49183adb6900b95513d3511004e26e5426bfe5bd555a1609d43cf1321dae (if the feature isn't enabled)

[DanRibbens]

I'll need more time to test this on a canary version with an actual project.

[DanRibbens]

We should enable this as the default option for all templates/examples.

[V1RE]

@DanRibbens and @r1tsuu Would it be possible to mark this feature as experimental, and include it in a release? That way people can start using it if they opt into experimental usage, and collect feedback from real-world usage.

[AndKenneth]

I'd be happy to try this out on our project if it ended up behind an experimental flag.

[r1tsuu]

I think it'd be reasonable, but we shouldn't enable it for templates/examples then at the start, but rather after some feedback on this feature.

[V1RE]

@DanRibbens What are your thoughts on this?

[V1RE]

@r1tsuu @DanRibbens Sorry for bothering you guys once again, but this feature is absolutely mindblowing. Is there anything I can do to help this feature get shipped in the next update?

[DanRibbens]

@r1tsuu @DanRibbens Sorry for bothering you guys once again, but this feature is absolutely mindblowing. Is there anything I can do to help this feature get shipped in the next update?

You're not bothering us at all. Thank you for the nudge. I will try and make more time for this soon.

[robertmalicke]

Fantastic feature, excited to see it, great job Payload team

[V1RE]

@r1tsuu Is it possible to somehow start using this inside a project? If not, pkg.pr.new might be interesting so the community can start testing these types of changes

[V1RE]

@DanRibbens By any chance had time to look into this?

[crcorbett]

This feature would be massively helpful! 🙏

[renekahr]

Any updates on this?

[ericwaetke]

I’m super stoked for this as well and can’t wait for this feature to be ready. This would dramatically improve my code quality!

[renekahr]

@ericwaetke Agree!
By the way, how do you currently solve this?
I can't find a proper way to type my queries that use depth without either writing the types myself or checking for the existence of each field in the code.

[ericwaetke]

@ericwaetke Agree! By the way, how do you currently solve this? I can't find a proper way to type my queries that use depth without either writing the types myself or checking for the existence of each field in the code.

I’ve got a function like this, pretty sure it’ll work most of the time

async function fetchOrReturnRealValue<T extends keyof Config['collections']>(
	item: number | Config['collections'][T],
	collection: T
): Promise<Config['collections'][T]> {
	if (typeof item === "number") {
		const payload = await getPayload()
		return await payload.findByID({
			collection,
			id: item
		}) as Config['collections'][T]
	} else {
		return item as Config['collections'][T]
	}
}

[DanRibbens]

@r1tsuu we should revisit this one.

[Patrickroelofs]

Wondering if this could get a follow-up, this PR increases DX by a lot!

[kolyasya]

I wish this one to be implemented. This is really annoying in terms of DX to have a string | [Collection] type

[HananoshikaYomaru]

@r1tsuu does this PR make the select and join option in the query statement type safe as well?