pipelineBuilder.go

pipelineBuilder.go - Overview

1. Overview This file contains functions for preparing log parsing pipelines for use in a collector configuration. It handles the transformation of pipeline configurations into a format suitable for execution, including adding filters, routers, and handling specific operator types like regex, grok, JSON, add, move, copy, remove, trace_parser, time_parser and severity_parser. It also generates nil checks for referenced fields to prevent errors during pipeline execution.

2. Detailed Documentation

CollectorConfProcessorName

• Purpose: Generates a normalized processor name for a given pipeline, suitable for use in collector configurations. It replaces invalid characters with hyphens and prefixes the name with logs_ppl_.
• Parameters:
  • p (pipelinetypes.GettablePipeline): The pipeline for which to generate the processor name.
• Returns:
  • string: The normalized processor name.

PreparePipelineProcessor

• Purpose: Prepares a list of pipelines by configuring their operators, adding a router, and creating a processor map for use in collector configurations. It also handles pipeline filtering and ensures unique processor names.
• Parameters:
  • gettablePipelines ([]pipelinetypes.GettablePipeline): A slice of pipelines to prepare.
• Returns:
  • map[string]interface{}: A map of processor names to processor configurations.
  • []string: A slice of processor names.
  • error: An error if any step fails.

getOperators

• Purpose: Filters and configures operators within a pipeline, adding conditional checks (If clauses) based on the operator type and its parameters.
• Parameters:
  • ops ([]pipelinetypes.PipelineOperator): A slice of pipeline operators to process.
• Returns:
  • []pipelinetypes.PipelineOperator: A slice of filtered and configured pipeline operators.
  • error: An error if any step fails.

cleanTraceParser

• Purpose: Cleans up a trace parser operator by setting TraceId, SpanId, and TraceFlags to nil if their ParseFrom fields are empty.
• Parameters:
  • operator (*pipelinetypes.PipelineOperator): A pointer to the trace parser operator to clean.
• Returns: None

fieldNotNilCheck

• Purpose: Generates an expression string that checks if a given field path has a non-nil value in a log record. It handles nested field paths and array indexing.
• Parameters:
  • fieldPath (string): The field path to check.
• Returns:
  • string: The generated expression string.
  • error: An error if the field path is invalid.

rSplitAfterN

• Purpose: Splits a string after a separator from the right, creating up to n parts.
• Parameters:
  • str (string): The string to split.
  • sep (string): The separator to split after.
  • n (int): The maximum number of parts to create.
• Returns:
  • []string: A slice of strings representing the split parts.

reverseString

• Purpose: Reverses a string.
• Parameters:
  • s (string): The string to reverse.
• Returns:
  • string: The reversed string.

fieldsReferencedInExprNotNilCheck

• Purpose: Generates an expression that checks if all fields referenced in a given expression have non-nil values in a log record.
• Parameters:
  • expr (string): The expression to analyze.
• Returns:
  • string: The generated expression string.
  • error: An error if the expression is invalid or if a nil check cannot be created.

logFieldsReferencedInExpr

• Purpose: Extracts all log fields referenced in a given expression.
• Parameters:
  • expr (string): The expression to analyze.
• Returns:
  • []string: A slice of strings representing the referenced log fields.
  • error: An error if the expression cannot be parsed.

logFieldsInExprExtractor

• Purpose: Expr AST visitor for extracting referenced log fields.
• Fields:
  • referencedFields ([]string): Stores the list of referenced fields.
• Methods:
  • Visit(node *ast.Node): Implements the ast.Visitor interface. Adds MemberNode strings that begin with "attributes" or "resource" to referencedFields.

3. Code Examples

No examples are necessary.

Include in Getting Started: NO