sql.go

sql.go - Overview

1. Overview This file contains functions to generate SQL queries for retrieving Kafka messaging statistics from a ClickHouse database (signoz_traces.distributed_signoz_index_v3). These queries are used for different views and aggregations of Kafka producer and consumer data.

2. Detailed Documentation

generateConsumerSQL

• Purpose: Generates a SQL query to retrieve consumer-related metrics such as p99 latency, total requests, error count, average message size, error rate, and throughput, grouped by service name.
• Parameters:
  • start (int64): Start timestamp in nanoseconds.
  • end (int64): End timestamp in nanoseconds.
  • topic (string): Kafka topic name.
  • partition (string): Kafka partition ID.
  • consumerGroup (string): Kafka consumer group name.
  • queueType (string): Messaging queue type (e.g., "kafka").
• Returns:
  • string: The generated SQL query.

generatePartitionLatencySQL

• Purpose: Generates a SQL query to retrieve partition-level latency and throughput metrics for a given queue type, grouped by topic and partition.
• Parameters:
  • start (int64): Start timestamp in nanoseconds.
  • end (int64): End timestamp in nanoseconds.
  • queueType (string): Messaging queue type (e.g., "kafka").
• Returns:
  • string: The generated SQL query.

generateConsumerPartitionLatencySQL

• Purpose: Generates a SQL query to retrieve consumer partition latency metrics, including p99 latency, error rate, and throughput, grouped by consumer group and service name.
• Parameters:
  • start (int64): Start timestamp in nanoseconds.
  • end (int64): End timestamp in nanoseconds.
  • topic (string): Kafka topic name.
  • partition (string): Kafka partition ID.
  • queueType (string): Messaging queue type (e.g., "kafka").
• Returns:
  • string: The generated SQL query.

generateProducerPartitionThroughputSQL

• Purpose: Generates a SQL query to retrieve producer partition throughput metrics, including p99 latency, error rate, and throughput, grouped by topic and service name.
• Parameters:
  • start (int64): Start timestamp in nanoseconds.
  • end (int64): End timestamp in nanoseconds.
  • queueType (string): Messaging queue type (e.g., "kafka").
• Returns:
  • string: The generated SQL query.

generateProducerTopicLatencySQL

• Purpose: Generates a SQL query to retrieve producer topic latency metrics, including p99 latency, error rate, and throughput, grouped by partition.
• Parameters:
  • start (int64): Start timestamp in nanoseconds.
  • end (int64): End timestamp in nanoseconds.
  • topic (string): Kafka topic name.
  • service (string): Service name.
  • queueType (string): Messaging queue type (e.g., "kafka").
• Returns:
  • string: The generated SQL query.

generateConsumerLatencySQL

• Purpose: Generates a SQL query to retrieve consumer latency metrics, including p99 latency, error rate, ingestion rate, and byte rate, grouped by topic and service name.
• Parameters:
  • start (int64): Start timestamp in nanoseconds.
  • end (int64): End timestamp in nanoseconds.
  • queueType (string): Messaging queue type (e.g., "kafka").
• Returns:
  • string: The generated SQL query.

generateConsumerServiceLatencySQL

• Purpose: Generates a SQL query to retrieve consumer service latency metrics, including p99 latency, error rate, and throughput, grouped by partition.
• Parameters:
  • start (int64): Start timestamp in nanoseconds.
  • end (int64): End timestamp in nanoseconds.
  • topic (string): Kafka topic name.
  • service (string): Service name.
  • queueType (string): Messaging queue type (e.g., "kafka").
• Returns:
  • string: The generated SQL query.

generateProducerConsumerEvalSQL

• Purpose: Generates a SQL query to evaluate the time difference between producer and consumer spans. It calculates the number and percentage of spans that breach a defined evalTime, and lists the top 10 trace IDs with the highest time differences.
• Parameters:
  • start (int64): Start timestamp in nanoseconds.
  • end (int64): End timestamp in nanoseconds.
  • queueType (string): Messaging queue type (e.g., "kafka").
  • evalTime (int64): Evaluation time in nanoseconds.
• Returns:
  • string: The generated SQL query.

generateProducerSQL

• Purpose: Generates a SQL query to retrieve producer metrics, including p99 latency, error percentage, and throughput, grouped by service name.
• Parameters:
  • start (int64): Start timestamp in nanoseconds.
  • end (int64): End timestamp in nanoseconds.
  • topic (string): Kafka topic name.
  • partition (string): Kafka partition ID.
  • queueType (string): Messaging queue type (e.g., "kafka").
• Returns:
  • string: The generated SQL query.

generateNetworkLatencyThroughputSQL

• Purpose: Generates a SQL query to retrieve network latency throughput metrics, grouped by service name, client ID, and service instance ID.
• Parameters:
  • start (int64): Start timestamp in nanoseconds.
  • end (int64): End timestamp in nanoseconds.
  • consumerGroup (string): Kafka consumer group.
  • partitionID (string): Kafka partition ID.
  • queueType (string): Messaging queue type (e.g., "kafka").
• Returns:
  • string: The generated SQL query.

onboardProducersSQL

• Purpose: Generates a SQL query to check if there is any data for producers based on certain conditions related to kafka.
• Parameters:
  • start (int64): Start timestamp in nanoseconds.
  • end (int64): End timestamp in nanoseconds.
  • queueType (string): Messaging queue type (e.g., "kafka").
• Returns:
  • string: The generated SQL query.

onboardConsumerSQL

• Purpose: Generates a SQL query to check if there is any data for consumers based on certain conditions related to kafka.
• Parameters:
  • start (int64): Start timestamp in nanoseconds.
  • end (int64): End timestamp in nanoseconds.
  • queueType (string): Messaging queue type (e.g., "kafka").
• Returns:
  • string: The generated SQL query.

3. Code Examples N/A

4. Clarity and Accuracy The documentation accurately reflects the code's functionality.

Include in Getting Started: NO