Adding functions
Functions live in two places: the language-independent IR registry, which the engine implements, and the PPL catalogs, which map PPL names onto it.
Scalar functions
- Register the function in
pkg/ir/funcs_catalog.gowithscalar(category, name, minArgs, params, result, doc). Parameters carry a type (p), constant-only flag (pc) or lambda arity (pl); the result rule (Returns,SameAs,UnifyArgs,NumericResult, ...) computes the static result type. Wrap the call to set the remaining properties:awarefor functions that handle NULL arguments themselves (the default is strict: NULL in, NULL out),variadic,volatile(never folded, not row-local),queryConst(the same value for every row of one execution, such asnow), anderrsto document how invalid arguments are reported. TheDocsentence appears in the generated function table and inGET /api/v1/catalog. - Implement it in
pkg/vm/ir_funcs*.go.TestIRRegistryHasImplementationsfails until every registered scalar function has an implementation. Invalid values should produce NULL rather than an error. For the columnar executor, implement it inpkg/colstore/exec/irfuncwith the same semantics; the executor runs a function missing there through the VM over the fields it reads, and leaves a volatile one to the row operators. - Map the PPL name in
pkg/ppl/functions.go:direct(name, irName)for a one-to-one mapping or an alias,lowered(name, construct, f)for a function compiled to IR operators, orunsupportedFunc(reason, names...)for a PPL function LynxDB does not execute. Functions with named arguments are registered like the relevance functions inpkg/ppl/relevance.go. - Test evaluation in
pkg/vm/ir_compiler_test.goand, when the PPL mapping is not direct, add a compile golden underpkg/ppl/testdata/compile. - Regenerate the reference tables with
go run ./internal/docgen.
Foldable functions (scalar, not volatile, not query-constant) are evaluated at planning time when all arguments are literals, so they must be pure.
Aggregation functions
- Register the aggregate in
pkg/ir/funcs_catalog.gowithaggregate(name, minArgs, params, result, merge, doc). The merge kind (exact,approximate, none) states whether partial results combine; mark order-dependent aggregates withordered. - Map it to a pipeline aggregator in
pkg/engine/physical/operators.go(aggregateNames) and implement the aggregator inpkg/engine/pipeline. For the columnar executor, map it inaggFuncsofpkg/colstore/exec/irplanonto an aggregate ofpkg/colstore/exec; an aggregate it does not map is run by the row operators. - Add the PPL name to
aggregateCataloginpkg/ppl/aggs.go, with a note for any deviation from OpenSearch and for any difference between the two executors. Special argument handling (such aspercentile's constant percent) goes inectx.aggCall. - Test it in
pkg/database/execute_test.goand regenerate the tables.
Window-only functions (row_number, rank, ...) are listed in
windowOnlyFuncs and fail with UNSUPPORTED_FUNCTION.