# Catatan Seekor Jaeger

Catatan Seekor: Jaeger

> <� **Quick Start**: Panduan lengkap distributed tracing menggunakan Jaeger untuk monitoring dan troubleshooting microservices architecture.

## =� Daftar Isi

* [<� Overview](#-overview)
* \[<� Architecture]\(#-architecture)
* \[=� Installation & Setup]\(#-installation--setup)
* [=' Instrumentation](#-instrumentation)
* [=� Data Collection](#-data-collection)
* [= Querying & Analysis](#-querying--analysis)
* [=� Performance Optimization](#-performance-optimization)
* [=' Integration](#-integration)
* [=� Best Practices](#-best-practices)
* [=� Resources](#-resources)

## <� Overview

Jaeger adalah open-source distributed tracing system yang dirancang untuk monitoring dan troubleshooting microservices-based architectures. Jaeger membantu mengidentifikasi performance bottlenecks, dependency analysis, dan error tracking dalam distributed systems.

### < **Mengapa Jaeger?**

* **= End-to-End Tracing**: Trace request flow melalui multiple services
* **=� Performance Analysis**: Identifikasi latency dan bottleneck issues
* **=' Root Cause Analysis**: Temukan penyebab masalah dengan cepat
* **< Multi-Language Support**: Instrumentasi untuk berbagai programming languages
* **=� Scalable Architecture**: Dapat handle large-scale distributed systems
* **= Ecosystem Integration**: Integrasi dengan Prometheus, Grafana, OpenTelemetry

## <� Architecture

### <� **Core Components**

```yaml
Jaeger Components:
  - Jaeger Client: Libraries untuk instrumentasi aplikasi
  - Jaeger Agent: Process yang menerima spans dari clients
  - Jaeger Collector: Centralized component untuk6� spans
  - Jaeger Query: Service untuk querying stored traces
  - Jaeger UI: Web interface untuk trace visualization
  - Storage: Backend storage untuk trace data
```

### = **Data Flow**

```
Application Spans � Jaeger Client � Jaeger Agent � Jaeger Collector � Storage
                                                          �
                                                Jaeger Query � Jaeger UI
```

### =� **Trace & Span Concepts**

```yaml
Trace: Tree of spans yang mewakili request path
Span: Single operation dalam trace dengan timing dan metadata
Trace ID: Unique identifier untuk seluruh trace
Span ID: Unique identifier untuk individual span
Parent Span ID: Identifier untuk parent-child relationship
```

## =� Installation & Setup

### =3 **Docker Installation**

```yaml
# docker-compose.yml
version: '3'
services:
  jaeger:
    image: jaegertracing/all-in-one:latest
    ports:
      - "16686:16686"  # Jaeger UI
      - "14268:14268"  # HTTP collector
      - "14250:14250"  # gRPC collector
    environment:
      - COLLECTOR_OTLP_ENABLED=true
      - SPAN_STORAGE_TYPE=memory
```

### 8 **Kubernetes Installation**

```yaml
# jaeger-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: jaeger
spec:
  replicas: 1
  selector:
    matchLabels:
      app: jaeger
  template:
    metadata:
      labels:
        app: jaeger
    spec:
      containers:
      - name: jaeger
        image: jaegertracing/all-in-one:latest
        ports:
        - containerPort: 16686
        - containerPort: 14268
        - containerPort: 14250
---
apiVersion: v1
kind: Service
metadata:
  name: jaeger
spec:
  selector:
    app: jaeger
  ports:
  - name: ui
    port: 16686
    targetPort: 16686
  - name: collector
    port: 14268
    targetPort: 14268
```

### =' **Production Setup with Elasticsearch**

```yaml
# jaeger-production.yml
version: '3'
services:
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.14.0
    environment:
      - discovery.type=single-node
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ports:
      - "9200:9200"

  jaeger-collector:
    image: jaegertracing/jaeger-collector:latest
    command: [
      "/go/bin/all-in-one-linux",
      "--es.server-urls=http://elasticsearch:9200",
      "--es.index-prefix=jaeger",
      "--collector.grpc-server.host-port=:14250",
      "--collector.http-server.host-port=:14268"
    ]
    depends_on:
      - elasticsearch
    ports:
      - "14250:14250"
      - "14268:14268"

  jaeger-query:
    image: jaegertracing/jaeger-query:latest
    command: [
      "/go/bin/all-in-one-linux",
      "--es.server-urls=http://elasticsearch:9200",
      "--es.index-prefix=jaeger",
      "--query.http-server.host-port=:16686"
    ]
    depends_on:
      - elasticsearch
    ports:
      - "16686:16686"
```

## =' Instrumentation

### =9 **Go Instrumentation**

```go
package main

import (
    "context"
    "io"
    "net/http"
    "time"

    "github.com/opentracing/opentracing-go"
    "github.com/opentracing/opentracing-go/ext"
    "github.com/uber/jaeger-client-go"
    jaegercfg "github.com/uber/jaeger-client-go/config"
    jaegerlog "github.com/uber/jaeger-client-go/log"
)

func initJaeger(service string) (opentracing.Tracer, io.Closer) {
    cfg := jaegercfg.Configuration{
        ServiceName: service,
        Sampler: &jaegercfg.SamplerConfig{
            Type:  jaeger.SamplerTypeConst,
            Param: 1,
        },
        Reporter: &jaegercfg.ReporterConfig{
            LogSpans: true,
            LocalAgentHostPort: "localhost:6831",
        },
    }

    tracer, closer, err := cfg.NewTracer(
        jaegercfg.Logger(jaegerlog.StdLogger),
    )
    if err != nil {
        panic(fmt.Sprintf("ERROR: cannot init Jaeger: %v\n", err))
    }

    return tracer, closer
}

func httpHandler(w http.ResponseWriter, r *http.Request) {
    // Extract span context from incoming request
    spanCtx, _ := tracer.Extract(opentracing.HTTPHeaders, opentracing.HTTPHeadersCarrier(r.Header))

    // Create new span for this handler
    span := tracer.StartSpan("http-handler", ext.RPCServerOption(spanCtx))
    defer span.Finish()

    // Add tags
    span.SetTag("http.method", r.Method)
    span.SetTag("http.url", r.URL.Path)
    span.SetBaggageItem("user-id", "12345")

    // Business logic
    time.Sleep(100 * time.Millisecond)
    w.Write([]byte("Hello, World!"))
}

func main() {
    tracer, closer := initJaeger("example-service")
    defer closer.Close()
    opentracing.SetGlobalTracer(tracer)

    http.HandleFunc("/", httpHandler)
    http.ListenAndServe(":8080", nil)
}
```

### =

**Python Instrumentation**

```python
from flask import Flask
from jaeger_client import Config
import opentracing
from opentracing.ext import tags

app = Flask(__name__)

def init_tracer(service_name='flask-app'):
    config = Config(
        config={
            'sampler': {
                'type': 'const',
                'param': 1,
            },
            'logging': True,
            'local_agent': {
                'reporting_host': 'localhost',
                'reporting_port': '6831',
            },
        },
        service_name=service_name,
    )
    return config.initialize_tracer()

tracer = init_tracer()

@app.route('/')
def hello_world():
    # Extract span context from headers
    span_ctx = tracer.extract(opentracing.Format.HTTP_HEADERS, dict(request.headers))

    # Start new span
    span = tracer.start_span('flask-handler', child_of=span_ctx)
    span.set_tag(tags.HTTP_METHOD, request.method)
    span.set_tag(tags.HTTP_URL, request.url)
    span.set_tag('custom.tag', 'custom-value')

    try:
        # Business logic here
        with tracer.start_span('database-query', child_of=span):
            # Simulate database operation
            time.sleep(0.1)

        with tracer.start_span('external-api-call', child_of=span):
            # Simulate external API call
            time.sleep(0.2)

        span.set_tag(tags.HTTP_STATUS_CODE, 200)
        return 'Hello, World!'

    except Exception as e:
        span.set_tag(tags.ERROR, True)
        span.log_kv({'event': 'error', 'error.object': e})
        return 'Error occurred', 500
    finally:
        span.finish()

if __name__ == '__main__':
    app.run(debug=True)
```

### =� **Java (Spring Boot) Instrumentation**

```java
@SpringBootApplication
@RestController
public class JaegerDemoApplication {

    private final Tracer tracer;

    public JaegerDemoApplication(Tracer tracer) {
        this.tracer = tracer;
    }

    @GetMapping("/hello")
    public String hello(HttpServletRequest request) {
        // Extract span context from incoming request
        SpanContext spanContext = tracer.extract(
            Format.Builtin.HTTP_HEADERS,
            new TextMapExtractAdapter(
                Collections.list(request.getHeaderNames())
                    .stream()
                    .collect(Collectors.toMap(h -> h, request::getHeader))
            )
        );

        // Start new span
        Span span = tracer.buildSpan("hello-endpoint")
                .asChildOf(spanContext)
                .withTag(Tags.SPAN_KIND, Tags.SPAN_KIND_SERVER)
                .withTag(Tags.HTTP_METHOD, request.getMethod())
                .withTag(Tags.HTTP_URL, request.getRequestURL().toString())
                .start();

        try (Scope scope = tracer.scopeManager().activate(span)) {
            // Business logic
            span.setTag("custom.tag", "custom-value");

            // Create child span for database operation
            Span dbSpan = tracer.buildSpan("database-query")
                    .asChildOf(span)
                    .start();
            try {
                // Simulate database operation
                Thread.sleep(100);
                dbSpan.setTag("db.statement", "SELECT * FROM users");
                dbSpan.setTag("db.type", "sql");
            } catch (InterruptedException e) {
                Tags.ERROR.set(dbSpan, true);
                dbSpan.log(Map.of("event", "error", "error.object", e));
            } finally {
                dbSpan.finish();
            }

            Tags.HTTP_STATUS.set(span, 200);
            return "Hello, World!";
        } catch (Exception e) {
            Tags.ERROR.set(span, true);
            span.log(Map.of("event", "error", "error.object", e));
            return "Error occurred";
        } finally {
            span.finish();
        }
    }

    public static void main(String[] args) {
        SpringApplication.run(JaegerDemoApplication.class, args);
    }
}
```

## =� Data Collection

### <� **Span Attributes**

```yaml
Required Attributes:
  - operation.name: Name of the operation being traced
  - service.name: Name of the service generating the span
  - span.kind: Type of span (server, client, producer, consumer)

Recommended Attributes:
  - http.method: HTTP method for web requests
  - http.url: Full URL for web requests
  - http.status_code: HTTP response status
  - db.type: Database type (sql, mongodb, redis)
  - db.statement: Database query or statement
  - messaging.system: Message broker system
  - messaging.destination: Queue/topic name
```

### =� **Baggage Items**

```go
// Setting baggage items in Go
span.SetBaggageItem("user.id", "12345")
span.SetBaggageItem("request.id", "req-abc-123")
span.SetBaggageItem("tenant.id", "tenant-xyz")

// Getting baggage items
if userID := span.BaggageItem("user.id"); userID != "" {
    // Use user ID for business logic
}
```

### = **Trace Context Propagation**

```javascript
// HTTP Client with Trace Context
function makeTracedRequest(url) {
    const span = tracer.startSpan('http-request', {
        tags: {
            'http.method': 'GET',
            'http.url': url,
            'span.kind': 'client'
        }
    });

    const headers = {};
    tracer.inject(span, opentracing.FORMAT_HTTP_HEADERS, headers);

    fetch(url, { headers })
        .then(response => {
            span.setTag('http.status_code', response.status);
            span.finish();
            return response;
        })
        .catch(error => {
            span.setTag('error', true);
            span.log({ event: 'error', 'error.object: error });
            span.finish();
        });
}
```

## =

Querying & Analysis

### < **Jaeger UI Features**

```yaml
Search Capabilities:
  - Service-based filtering
  - Operation-based filtering
  - Tag-based filtering
  - Time range selection
  - Duration filtering
  - Trace ID lookup

Visualization Features:
  - Gantt chart visualization
  - Service dependency graph
  - Span details view
  - Log correlation
  - Performance metrics
```

### =

**Query API Examples**

```bash
# Get trace by ID
curl "http://localhost:16686/api/traces/{trace-id}"

# Search traces
curl -X POST "http://localhost:16686/api/traces" \
  -H "Content-Type: application/json" \
  -d '{
    "service": "user-service",
    "operation": "GetUser",
    "start": "2023-01-01T00:00:00Z",
    "end": "2023-01-01T23:59:59Z",
    "limit": 100
  }'

# Get services
curl "http://localhost:16686/api/services"

# Get operations for a service
curl "http://localhost:16686/api/services/user-service/operations"
```

### =� **Trace Analysis Patterns**

```yaml
Performance Analysis:
  1. Identify slow traces (high duration)
  2. Locate bottleneck spans
  3. Analyze service dependencies
  4. Compare normal vs error traces

Error Analysis:
  1. Filter traces with error tags
  2. Examine error logs and stack traces
  3. Identify error propagation patterns
  4. Correlate with system metrics
```

## =� Performance Optimization

### � **Sampling Strategies**

```yaml
Sampling Types:
  - Constant: Sample all or none traces
  - Probabilistic: Sample percentage of traces
  - Rate Limiting: Sample up to N traces per second
  - Adaptive: Dynamic sampling based on load

Best Practices:
  - Use higher sampling for critical services
  - Implement adaptive sampling for production
  - Consider business-critical operations
  - Balance cost vs. observability
```

### =' **Performance Tuning**

```yaml
Collector Optimization:
  - Increase collector replicas
  - Use appropriate storage backend
  - Optimize batch processing
  - Configure appropriate timeouts

Storage Optimization:
  - Choose storage based on scale
  - Implement data retention policies
  - Use indexing for efficient queries
  - Monitor storage performance
```

### =� **Monitoring Jaeger Itself**

```yaml
Metrics to Monitor:
  - Collector throughput
  - Storage query latency
  - Agent connectivity
  - UI response times
  - Memory usage
  - CPU utilization

Alerting Rules:
  - High error rates
  - Increasing latency
  - Storage capacity issues
  - Component failures
```

## =' Integration

### = **OpenTelemetry Integration**

```yaml
Migration Benefits:
  - Vendor-neutral observability
  - Unified instrumentation
  - Future-proof technology
  - Enhanced functionality

Migration Path:
  1. Deploy OpenTelemetry Collector
  2. Update instrumentation to OpenTelemetry
  3. Configure Jaeger as backend
  4. Validate trace collection
  5. Retire old instrumentation
```

### =� **Prometheus Integration**

```yaml
Metrics Integration:
  - Jaeger metrics in Prometheus
  - Correlate traces with metrics
  - Unified observability dashboard
  - Alerting based on trace data

Configuration:
  jaeger:
    metrics-backend: prometheus
    metrics-http-route: /metrics
```

### =� **Grafana Integration**

```yaml
Dashboard Components:
  - Trace search interface
  - Service dependency graphs
  - Performance metrics
  - Error rate visualization
  - Latency heatmaps

Data Sources:
  - Jaeger for trace data
  - Prometheus for metrics
  - Loki for logs
```

## =� Best Practices

### <� **Instrumentation Best Practices**

```yaml
Span Naming:
  - Use descriptive, consistent names
  - Include operation type and resource
  - Avoid high cardinality values
  - Follow naming conventions

Tag Usage:
  - Use standard semantic conventions
  - Include relevant context
  - Avoid sensitive data
  - Keep tag values concise
```

### =' **Production Readiness**

```yaml
Security:
  - Secure Jaeger endpoints
  - Implement authentication
  - Use TLS for communication
  - Audit access logs

Scalability:
  - Horizontal scaling of collectors
  - Appropriate storage sizing
  - Load balancing configuration
  - Monitoring resource usage
```

### =� **Operational Excellence**

```yaml
Monitoring:
  - Component health checks
  - Performance metrics
  - Error rate tracking
  - Capacity planning

Maintenance:
  - Regular backups
  - Software updates
  - Storage cleanup
  - Configuration review
```

## =� Resources

### =� **Official Documentation**

* [Jaeger Documentation](https://www.jaegertracing.io/docs/)
* [OpenTelemetry Documentation](https://opentracing.io/docs/)
* [Jaeger GitHub Repository](https://github.com/jaegertracing/jaeger)

### =' **Client Libraries**

* [Jaeger Client Libraries](https://www.jaegertracing.io/docs/#client-libraries)
* [OpenTelemetry Instrumentation](https://opentracing.io/docs/instrumentation/)
* [Spring Cloud Sleuth](https://spring.io/projects/spring-cloud-sleuth)

### =� **Learning Resources**

* [Distributed Tracing Concepts](https://opentracing.io/docs/overview/)
* [Observability Best Practices](https://opentracing.io/docs/best-practices/)
* [Microservices Monitoring](https://opentracing.io/docs/microservices/)

***

## =� **Implementation Checklist**

### **Setup & Configuration**

* [ ] Deploy Jaeger infrastructure
* [ ] Configure appropriate storage backend
* [ ] Set up sampling strategy
* [ ] Implement monitoring and alerting
* [ ] Verify component connectivity

### **Instrumentation**

* [ ] Install client libraries
* [ ] Implement tracing in critical services
* [ ] Add context propagation
* [ ] Include relevant span attributes
* [ ] Test trace collection

### **Operations**

* [ ] Configure data retention policies
* [ ] Set up performance monitoring
* [ ] Implement security measures
* [ ] Create operational procedures
* [ ] Document architecture and processes

***

*Catatan Seekor: Jaeger - Comprehensive distributed tracing guide for microservices observability*