-
Notifications
You must be signed in to change notification settings - Fork 3.7k
feat(logical planning): Planner playground #16403
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
21 commits
Select commit
Hold shift + click to select a range
033b06c
playing with a planner description
owen-d dc76102
autogen comments
owen-d 2550bb5
more progress, inc scan & project
owen-d 60a3b49
aggregate plans & dataframe init
owen-d e0c05a6
format pkg
owen-d 7b03abb
formatting
owen-d e5b5a06
typo
owen-d 979c22a
lint
owen-d 18a77e2
remove datasource, s/scan/maketable/
owen-d 9620b6e
[wip] SSA progress
owen-d 3e1e1df
plan interfaces vs impls; reimplements tree formatter
owen-d c761b93
[wip] more ssa hacking
owen-d 1196ed4
rename
owen-d 3e91374
doc work
owen-d 80959b2
lint
owen-d 16cb270
nit: s/type/category/
owen-d 8318bef
Revert " nit: s/type/category/"
owen-d ad48cf9
abandons interface approach for variant signal approach (similar to d…
owen-d c22d7b9
doc
owen-d 0739f41
adds limit plan
owen-d ac15f25
sort impl
owen-d File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,109 @@ | ||
package logical | ||
|
||
import ( | ||
"github.com/grafana/loki/v3/pkg/dataobj/internal/metadata/datasetmd" | ||
"github.com/grafana/loki/v3/pkg/dataobj/planner/schema" | ||
) | ||
|
||
// AggregateOp represents the type of aggregation operation to perform. | ||
// It is a string-based enum that identifies different aggregation functions | ||
// that can be applied to expressions. | ||
type AggregateOp string | ||
|
||
const ( | ||
// AggregateOpSum represents a sum aggregation | ||
AggregateOpSum AggregateOp = "sum" | ||
// AggregateOpAvg represents an average aggregation | ||
AggregateOpAvg AggregateOp = "avg" | ||
// AggregateOpMin represents a minimum value aggregation | ||
AggregateOpMin AggregateOp = "min" | ||
// AggregateOpMax represents a maximum value aggregation | ||
AggregateOpMax AggregateOp = "max" | ||
// AggregateOpCount represents a count aggregation | ||
AggregateOpCount AggregateOp = "count" | ||
) | ||
|
||
// Convenience constructors for each aggregate operation | ||
var ( | ||
// Sum creates a sum aggregation expression | ||
Sum = newAggregateExprConstructor(AggregateOpSum) | ||
// Avg creates an average aggregation expression | ||
Avg = newAggregateExprConstructor(AggregateOpAvg) | ||
// Min creates a minimum value aggregation expression | ||
Min = newAggregateExprConstructor(AggregateOpMin) | ||
// Max creates a maximum value aggregation expression | ||
Max = newAggregateExprConstructor(AggregateOpMax) | ||
// Count creates a count aggregation expression | ||
Count = newAggregateExprConstructor(AggregateOpCount) | ||
) | ||
|
||
// AggregateExpr represents an aggregation operation on an expression. | ||
// It encapsulates the operation to perform (sum, avg, etc.), the expression | ||
// to aggregate, and a name for the result. | ||
type AggregateExpr struct { | ||
// name is the identifier for this aggregation | ||
name string | ||
// op specifies which aggregation operation to perform | ||
op AggregateOp | ||
// expr is the expression to aggregate | ||
expr Expr | ||
} | ||
|
||
// newAggregateExprConstructor creates a constructor function for a specific aggregate operation. | ||
// This is a higher-order function that returns a function for creating aggregate expressions | ||
// with a specific operation type. | ||
func newAggregateExprConstructor(op AggregateOp) func(name string, expr Expr) AggregateExpr { | ||
return func(name string, expr Expr) AggregateExpr { | ||
return AggregateExpr{ | ||
name: name, | ||
op: op, | ||
expr: expr, | ||
} | ||
} | ||
} | ||
|
||
// Type returns the type of the expression. | ||
// For aggregate expressions, this is always ExprTypeAggregate. | ||
func (a AggregateExpr) Type() ExprType { | ||
return ExprTypeAggregate | ||
} | ||
|
||
// Name returns the name of the aggregation. | ||
// This is used as the column name in the output schema. | ||
func (a AggregateExpr) Name() string { | ||
return a.name | ||
} | ||
|
||
// Op returns the aggregation operation. | ||
// This identifies which aggregation function to apply. | ||
func (a AggregateExpr) Op() AggregateOp { | ||
return a.op | ||
} | ||
|
||
// SubExpr returns the expression being aggregated. | ||
// This is the input to the aggregation function. | ||
func (a AggregateExpr) SubExpr() Expr { | ||
return a.expr | ||
} | ||
|
||
// ToField converts the aggregation expression to a column schema. | ||
// It determines the output type based on the input expression and | ||
// the aggregation operation. | ||
func (a AggregateExpr) ToField(p Plan) schema.ColumnSchema { | ||
// Get the input field schema | ||
|
||
return schema.ColumnSchema{ | ||
Name: a.name, | ||
// Aggregations typically result in numeric types | ||
Type: determineAggregationTypeFromFieldType(a.expr.ToField(p).Type, a.op), | ||
} | ||
} | ||
|
||
// determineAggregationTypeFromFieldType calculates the output type of an aggregation | ||
// based on the input field type and the aggregation operation. | ||
// Currently, this is a placeholder that always returns int64, but it should be | ||
// implemented to handle different input types and operations correctly. | ||
func determineAggregationTypeFromFieldType(_ datasetmd.ValueType, _ AggregateOp) datasetmd.ValueType { | ||
// TODO: implement | ||
return datasetmd.VALUE_TYPE_INT64 | ||
} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,74 @@ | ||
package logical | ||
|
||
import ( | ||
"github.com/grafana/loki/v3/pkg/dataobj/planner/schema" | ||
) | ||
|
||
// Aggregate represents a plan node that performs aggregation operations. | ||
// The output schema is organized with grouping columns followed by aggregate expressions. | ||
// It corresponds to the GROUP BY clause in SQL and is used to compute aggregate | ||
// functions like SUM, AVG, MIN, MAX, and COUNT over groups of rows. | ||
type Aggregate struct { | ||
// input is the child plan node providing data to aggregate | ||
input Plan | ||
// groupExprs are the expressions to group by | ||
groupExprs []Expr | ||
// aggExprs are the aggregate expressions to compute | ||
aggExprs []AggregateExpr | ||
} | ||
|
||
// NewAggregate creates a new Aggregate plan node. | ||
// The Aggregate logical plan calculates aggregates of underlying data such as | ||
// calculating minimum, maximum, averages, and sums of data. Aggregates are often | ||
// grouped by other columns (or expressions). | ||
// A simple example would be SELECT region, SUM(sales) FROM orders GROUP BY region. | ||
func newAggregate(input Plan, groupExprs []Expr, aggExprs []AggregateExpr) *Aggregate { | ||
return &Aggregate{ | ||
input: input, | ||
groupExprs: groupExprs, | ||
aggExprs: aggExprs, | ||
} | ||
} | ||
|
||
// Schema returns the schema of the data produced by this aggregate. | ||
// The schema consists of group-by expressions followed by aggregate expressions. | ||
// This ordering is important for downstream operations that expect group columns | ||
// to come before aggregate columns. | ||
func (a *Aggregate) Schema() schema.Schema { | ||
var columns []schema.ColumnSchema | ||
|
||
// Group expressions come first | ||
for _, expr := range a.groupExprs { | ||
columns = append(columns, expr.ToField(a.input)) | ||
} | ||
|
||
// Followed by aggregate expressions | ||
for _, expr := range a.aggExprs { | ||
columns = append(columns, expr.ToField(a.input)) | ||
} | ||
|
||
return schema.FromColumns(columns) | ||
} | ||
|
||
// Type implements the ast interface | ||
func (a *Aggregate) Type() PlanType { | ||
return PlanTypeAggregate | ||
} | ||
|
||
// GroupExprs returns the list of expressions to group by. | ||
// These expressions define the grouping keys for the aggregation. | ||
func (a *Aggregate) GroupExprs() []Expr { | ||
return a.groupExprs | ||
} | ||
|
||
// AggregateExprs returns the list of aggregate expressions to compute. | ||
// These expressions define the aggregate functions to apply to each group. | ||
func (a *Aggregate) AggregateExprs() []AggregateExpr { | ||
return a.aggExprs | ||
} | ||
|
||
// Child returns the input plan. | ||
// This is a convenience method for accessing the child plan. | ||
func (a *Aggregate) Child() Plan { | ||
return a.input | ||
} |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we should use the same enum pattern (int type + iota + String() function) as in the rest of the PR.