parallel cognifying process in cognee

[LStromann]

=lets say i have an endponit and each time I trigger it a cognifying process starts each with differnt datasets so is the execution parallel or sequential

This discussion was automatically pulled from Discord.

[LStromann]

Execution is sequential for datasets, but it is parallel for documents in a dataset in the SDK cognify function. Up to 10 documents are executed in parallel, there is an env variable you can set to change this number to make it higher or lower

[LStromann]

but if you call the endpoint two times each with different datasets then it's in parallel