Description
This issue comes from #13881 (review):
I see now what I did wrong. The stats, warnings etc are joined in Downstream here but my accumulator is returning only one result.
I don't know how adventurous your are but the current state has a little code smell. I think the join of the stats should be moved out of Downstream. Furthermore, if you follow the code for the streams you'll see that we run
iter.NewSortEntryIterator
on the results. However, NewStreamAccumulator(params) returns only one result. So the sort iterator is not really required.I suggest to change the
Accumulator.Result
return type tologqlmodel.Result
instead of a slice. And then move the logic of the merge for theBufferedAccumulator
. I'd do this as a follow up. The big work is that this would require moving the logic ofNewMergeLastOverTimeStepEvaluator
,NewMergeFirstOverTimeStepEvaluator
andNewConcatStepEvaluator
into new Accumulators.Stepping back a little, my refactoring from #11863 was intended to go down that path.
Let me explain why this makes sense:
It will enable us to move the joining of stats into a central logic so it's not lost on further refactoring.
- Accumulate and drop samples as they arrive. E.g. last_over_time does not have to buffer all the results anymore. Same with sum by in the future. This should reduce the memory footprint a little.
- This will allow us to define special topk or sum by accumulators in the future..
- The previous point will then enable us to print and investigate a query plan without running the step evaluators.
TLDR:
- We should land this and verify that it works.
- Replace BufferedAccumulator with ConcatAccumulator, LastOverTimeAccumulator and FirstOverTimeAccumulator.